Deploy shiny with shinyproxy - no app showing - shiny

I've developed a shiny app and i'm trying to do a first lightweight deploy using shinyproxy.
All installation seems fine.
I've installed docker, java.
I thought that building a package that wraps the app and other function would be a good idea.
So I developed a package (CI) and CI::launch_application is basically a wrapper around RunApp function of shiny package. This is the code:
launch_application <- function(launch.browser = interactive(), ...) {
runApp(
appDir = system.file("app", package = "CI"),
launch.browser = launch.browser,
...
)
}
I succesfully built the docker image with this Dockerfile
FROM rocker/r-base:latest
## Install required dependencies
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
## for R package 'curl'
libcurl4-gnutls-dev \
apt-utils \
## for R package 'xml2'
libxml2-dev \
## for R package 'openssl'
libssl-dev \
zlib1g-dev \
default-jdk \
default-jre \
&& apt-get clean \
&& R CMD javareconf \
&& rm -rf /var/lib/apt/lists/
## Install major fixed R dependencies
# - they will always be needed and we want them in a dedicated layer,
# as opposed to getting them dynamically via `remotes::install_local()`
RUN install2.r --error \
shiny \
dplyr \
devtools \
rJava \
RJDBC
# copy the app to the image
RUN mkdir /root/CI
COPY . /root/CI
# Install CI
RUN install2.r --error remotes \
&& R -e "remotes::install_local('/root/CI')"
# Set host and port
RUN echo "options(shiny.port = 80, shiny.host = '0.0.0.0')" >> /usr/local/lib/R/Rprofile.site
EXPOSE 80
CMD ["R", "-e", "CI::launch_application()"]
This is my application.yml file
proxy:
title:
logo-url: http://www.openanalytics.eu/sites/www.openanalytics.eu/themes/oa/logo.png
landing-page: /
heartbeat-rate: 10000
heartbeat-timeout: 60000
port: 8080
admin-groups: scientists
users:
- name: jack
password: password
groups: scientists
- name: jeff
password: password
groups: mathematicians
authentication: simple
# Docker configuration
docker:
cert-path: /home/none
url: http://localhost:2375
port-range-start: 20000
specs:
- id: home
display-name: Customer Intelligence
description: Segment your customer
container-cmd: ["R", "-e", "CI::launch_application()"]
container-image: company/image
access-groups: scientist
logging:
file:
shinyproxy.log
When I launch java shinyproxy.jar and i visited the url with the port I exposed, I see a login mask.
I logged in with simple authentication ( login is successful from shinyproxy.log) but neither an app is showing nor a list of app.
When I launch the app locally everything is fine.
Thanks

There is a misprint in the allowed user group in application.yml (should be scientists over scientist):
access-groups: scientists

Dzimitry is right. It was a typo error: scientists over scientist.

Related

Google Cloud Storage File System, Python Package Error: AttributeError: 'GCSFile' object has no attribute 'gcsfs'

I am trying to run a python code which will download and stream chunks of data from source URL to destination cloud storage blob.
It is working fine in standalone pc, local function and so on.
But when i try same with GCP Cloud RUN it is throwing weird error.
AttributeError: 'GCSFile' object has no attribute 'gcsfs'
Complete error:
Traceback (most recent call last):
File "/home/<user>/.local/lib/python3.9/site-packages/fsspec/spec.py", line 1683, in __del__
self.close()
File "/home/<user>/.local/lib/python3.9/site-packages/fsspec/spec.py", line 1661, in close
self.flush(force=True)
File "/home/<user>/.local/lib/python3.9/site-packages/fsspec/spec.py", line 1527, in flush
self._initiate_upload()
File "/home/<user>/.local/lib/python3.9/site-packages/gcsfs/core.py", line 1443, in _initiate_upload
self.gcsfs.loop,
AttributeError: 'GCSFile' object has no attribute 'gcsfs'
It consumed my week, any help or direction is highly appriciated, thanks in advance.
The actual code which has been used:
from flask import Flask, request
import os
import gcsfs
import requests
app = Flask(__name__)
#app.route('/urltogcs')
def urltogcs():
try:
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "secret.json"
gcp_file_system = gcsfs.GCSFileSystem(project='<project_id>')
session = requests.Session()
url = request.args.get('source', 'temp')
blob_path = request.args.get('destination', 'temp')
with session.get(url, stream=True) as r:
r.raise_for_status()
with gcp_file_system.open(blob_path, 'wb') as f_obj:
for chunk in r.iter_content(chunk_size=1024 * 1024):
f_obj.write(chunk)
return f'Successfully downloaded from {url} to {blob_path} :)'
except Exception as e:
print("Failure")
print(e)
return f'download failed for {url} :('
if __name__ == "__main__":
app.run(debug=True, host="0.0.0.0", port=int(os.environ.get("PORT", 8080)))
Your code (with the proposed changes) works for me:
main.py:
from flask import Flask, request
import os
import gcsfs
import requests
app = Flask(__name__)
project = os.getenv("PROJECT")
port = os.getenv("PORT", 8080)
#app.route('/urltogcs')
def urltogcs():
try:
gcp_file_system = gcsfs.GCSFileSystem(project=project)
session = requests.Session()
url = request.args.get('source', 'temp')
blob_path = request.args.get('destination', 'temp')
with session.get(url, stream=True) as r:
r.raise_for_status()
with gcp_file_system.open(blob_path, 'wb') as f_obj:
for chunk in r.iter_content(chunk_size=1024 * 1024):
f_obj.write(chunk)
return f'Successfully downloaded from {url} to {blob_path} :)'
except Exception as e:
print("Failure")
print(e)
return f'download failed for {url}
if __name__ == "__main__":
app.run(debug=True, host="0.0.0.0", port=int(port))
Note: The code requires project from the environment which isn't ideal. It would be better if gcsfs.GCSFileSystem didn't require project. Alternatively project could be obtained from Google's metadata service. For convenience (!), I'm setting it using the environment.
requirements.txt:
Flask==2.2.2
gcsfs==2022.7.1
gunicorn==20.1.0
Dockerfile:
FROM python:3.10-slim
ENV PYTHONUNBUFFERED True
ENV APP_HOME /app
WORKDIR $APP_HOME
COPY . ./
RUN pip install --no-cache-dir -r requirements.txt
CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 --timeout 0 main:app
Bash script:
BILLING="[YOUR-BILLING]"
PROJECT="[YOUR-PROJECT]"
REGION="[YOUR-REGION]"
BUCKET="[YOUR-BUCKET]"
# Create Project
gcloud projects create ${PROJECT}
# Associate with Billing Account
gcloud beta billing projects link ${PROJECT} \
--billing-account=${BILLING}
# Enabled services
SERVICES=(
"artifactregistry"
"cloudbuild"
"run"
)
for SERVICE in ${SERVICES[#]}
do
gcloud services enable ${SERVICE}.googleapis.com \
--project=${PROJECT}
done
# Create Bucket
gsutil mb -p ${PROJECT} gs://${BUCKET}
# Service Account
ACCOUNT=tester
EMAIL=${ACCOUNT}#${PROJECT}.iam.gserviceaccount.com
# Create Service Account
gcloud iam service-accounts create ${ACCOUNT} \
--project=${PROJECT}
# Create Service Account key
gcloud iam service-accounts keys create ${PWD}/${ACCOUNT}.json \
--iam-account=${EMAIL} \
--project=${PROJECT}
# Ensure Service Account can write to storage
gcloud projects add-iam-policy-binding ${PROJECT} \
--role=roles/storage.admin \
--member=serviceAccount:${EMAIL}
# Only needed for local testing
export GOOGLE_APPLICATION_CREDENTIALS=${PWD}/${ACCOUNT}.json
# Deploy Cloud Run service
# Run service as Service Account
NAME="urltogcs"
gcloud run deploy ${NAME} \
--source=${PWD} \
--set-env-vars=PROJECT=${PROJECT} \
--no-allow-unauthenticated \
--service-account=${EMAIL} \
--region=${REGION} \
--project=${PROJECT}
# Grab the Cloud Run service's endpoint
ENDPOINT=$(gcloud run services describe ${NAME} \
--region=${REGION} \
--project=${PROJECT} \
--format="value(status.url)")
# Cloud Run service requires auth
TOKEN=$(gcloud auth print-identity-token)
# This page
SRC="https://stackoverflow.com/questions/73393808/"
# Generate a GCS Object name by epoch
DST="gs://${BUCKET}/$(date +%s)"
curl \
--silent \
--get \
--header "Authorization: Bearer ${TOKEN}" \
--data-urlencode "source=${SRC}" \
--data-urlencode "destination=${DST}" \
--write-out '%{response_code}' \
--output /dev/null \
${ENDPOINT}/urltogcs
Yields OK:
200
And:
gsutil ls gs://${BUCKET}
gs://${BUCKET}/1660780270

Argo giving x509: cannot validate certificate for 127.0.0.1 because it doesn't contain any IP SANs error

I've installed Argo on a managed k8 service following the guidelines here.
When i launch the following example task i get an error (if you have argo installed you should be able to copy paster the below code):
# create a.yml
cat >> a.yml<<EOL
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: hello-world- # Name of this Workflow
spec:
entrypoint: whalesay # Defines "whalesay" as the "main" template
templates:
- name: whalesay # Defining the "whalesay" template
container:
image: docker/whalesay
command: [cowsay]
args: ["hello world"] # This template runs "cowsay" in the "whalesay" image with arguments "hello world"
EOL
# submit a.yml
argo --insecure-skip-tls-verify --insecure-skip-verify -n argo submit a.yml
# monitor
$ argo list
# NAME STATUS AGE DURATION PRIORITY
# hello-world-hxrcp Succeeded 4m 10s 0
argo watch --insecure-skip-tls-verify --insecure-skip-verify -v -n argo hello-world-hxrcp
# DEBU[2021-06-09T19:37:22.125Z] CLI version version="{v3.0.7 2021-05-25T18:57:09Z e79e7ccda747fa4487bf889142c744457c26e9f7 v3.0.7 clean go1.16.3 gc linux/amd64}"
# DEBU[2021-06-09T19:37:22.125Z] Client options opts="(argoServerOpts=(url=127.0.0.1:2746,path=,secure=true,insecureSkipVerify=true,http=true),instanceID=)"
# DEBU[2021-06-09T19:37:22.125Z] curl -H 'Accept: text/event-stream' -H 'Authorization: ******' 'https://127.0.0.1:2746/api/v1/workflow-events/argo?listOptions.fieldSelector=metadata.name%3Dhello-world-hxrcp&listOptions.resourceVersion=0'
# FATA[2021-06-09T19:37:22.536Z] Get "https://127.0.0.1:2746/api/v1/workflow-events/argo?listOptions.fieldSelector=metadata.name%3Dhello-world-hxrcp&listOptions.resourceVersion=0": x509: cannot validate certificate for 127.0.0.1 because it doesn't contain any IP SANs
Why am i seeing this error ?
The install process was this:
kubectl create namespace argo
kubectl apply -n argo -f https://raw.githubusercontent.com/argoproj/argo-workflows/stable/manifests/install.yaml
CLI (taken from the latest version here):
# Download the binary
curl -sLO https://github.com/argoproj/argo/releases/download/v3.0.7/argo-linux-amd64.gz
# Unzip
gunzip argo-linux-amd64.gz
# Make binary executable
chmod +x argo-linux-amd64
# Move binary to path
sudo mv ./argo-linux-amd64 /usr/local/bin/argo
# Test installation
argo version
# link with server
# recommended on user panel in interface
cat >> ~/.bashrc <<EOL
export ARGO_SERVER='127.0.0.1:2746'
export ARGO_HTTP1=true
export ARGO_SECURE=true
export ARGO_BASE_HREF=
export ARGO_TOKEN=''
export ARGO_NAMESPACE=argo
export ARGO_INSECURE_SKIP_VERIFY=true
EOL
# check it works:
argo list
Heyo, I ran into this issue when setting up with the argo helm chart on kind. The problem is that you have to disable tls verification for the executor (the thing that executes the workflow) using the ARGO_KUBELET_INSECURE env var. Here are the docs https://argoproj.github.io/argo-workflows/environment-variables/#executor
Sorry I don't have the exact code change you need for your setup, but I'm sure you can figure that out now that you know what the problem is ;).
Here's what my helm values.yaml file looks like in case that helps anyone else:
server:
serviceType: LoadBalancer
extraArgs:
- --auth-mode=server
controller:
containerRuntimeExecutor: k8sapi
executor:
env:
- name: ARGO_KUBELET_INSECURE
value: true

Gunicorn error 203 while setting up django project

I try setting up a django project with nginx and gunicorn but get a gunicorn error 203:
My installation is located at path /webapps/sarlex and all project files are owned by user "sarlex" and group "webapps".
The virtualenv is located at /webapps/sarlex/environment.
Gunicorn is located at /webapps/sarlex/environment/bin/ and owned by user sarlex.
Gunicorn configuration is set in /webapps/sarlex/gunicorn_start.sh:
NAME="sarlex" # Name of the application
DJANGODIR=/webapps/sarlex/ # Django project directory
SOCKFILE=/webapps/sarlex/run/gunicorn.sock # we will communicte using this unix socket
USER=sarlex # the user to run as
GROUP=webapps # the group to run as
NUM_WORKERS=9 # how many worker processes should Gunicorn spawn
DJANGO_SETTINGS_MODULE=sarlex.settings # which settings file should Django use
DJANGO_WSGI_MODULE=sarlex.wsgi # WSGI module name
echo "Starting $NAME as `whoami`"
# Activate the virtual environment
cd $DJANGODIR
source /webapps/sarlex/environment/bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH
# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
# Start your Django Unicorn
exec gunicorn ${DJANGO_WSGI_MODULE}:application \
--name $NAME \
--workers $NUM_WORKERS \
--user=$USER --group=$GROUP \
--bind=unix:$SOCKFILE \
--log-level=debug \
--log-file=-
Executing gunicorn_start.sh works with environment activated and deactivated.
/etc/systemd/system/gunicorn.service is owned by root and has this content:
[Unit]
Description=gunicorn daemon
After=network.target
[Service]
User=sarlex
Group=webapps
WorkingDirectory=/webapps/sarlex
ExecStart=/webapps/sarlex/gunicorn_start.sh
[Install]
WantedBy=multi-user.target
I tried to replace "User=Sarlex" with "User=root" but still get the same error.
The correct gunicorn it taken
Thanks for your help
What exactly is your question?
The export for DJANGO_SETTINGS_MODULE was awkward.
Enclose expressions and variable references in double quotes
NAME="sarlex" # Name of the application
DJANGODIR=/webapps/sarlex/ # Django project directory
SOCKFILE=/webapps/sarlex/run/gunicorn.sock # we will communicte using this unix socket
USER=sarlex # the user to run as
GROUP=webapps # the group to run as
NUM_WORKERS=9 # how many worker processes should Gunicorn spawn
DJANGO_SETTINGS_MODULE=sarlex.settings # which settings file should Django use
DJANGO_WSGI_MODULE=sarlex.wsgi # WSGI module name
echo "Starting $NAME as `whoami`"
# Activate the virtual environment
source /webapps/sarlex/environment/bin/activate
cd "$DJANGODIR"
export DJANGO_SETTINGS_MODULE
export PYTHONPATH="$DJANGODIR":$PYTHONPATH
# Create the run directory if it doesn't exist
RUNDIR="$(dirname $SOCKFILE)"
test -d "$RUNDIR" || mkdir -p "$RUNDIR"
# Start your Django Unicorn
exec gunicorn "${DJANGO_WSGI_MODULE}:application" \
--name "$NAME" \
--workers "$NUM_WORKERS" \
--user="$USER" --group="$GROUP" \
--bind="unix:$SOCKFILE" \
--log-level=debug \
--log-file=-
Please show /var/log/nginx/ error messages.
Django failed starting because in one of my scripts I wrote "admin" in a path instead of "Admin".
This didn't cause a problem on my windows PC where I am developing my code.

Gunicorn settings for virtualenvwrapper

I am trying to deploy a test site which made using Django and virtualenvwrapper. I want to use nginx for requests.I used Taskbuster's tutorial. So my project layer is similar as below :
--abctasarim **main folder
--manage.py **django manage file
----/yogavidya ** project folder
----/yogavidya/wsgi.py **wsgi file
----/yogavidya/settings/base.py ***settings
I prepared a script to use with gunicorn. I addressed virtualenv to virtualenvwrapper envs
#!/bin/bash
NAME="yogavidya" #Name of the application (*)
DJANGODIR=/home/ytsejam/public_html/abctasarim # Django project directory (*)
SOCKFILE=/home/ytsejam/public_html/abctasarim/run/gunicorn.sock # we will communicate using this unix socket (*)
USER=ytsejam # the user to run as (*)
GROUP=webdata # the group to run as (*)
NUM_WORKERS=1 # how many worker processes should Gunicorn spawn (*)
DJANGO_SETTINGS_MODULE=yogavidya.settings.base # which settings file should Django use (*)
DJANGO_WSGI_MODULE=yogavidya.wsgi # WSGI module name (*)
echo "Starting $NAME as `whoami`"
# Activate the virtual environment
cd $DJANGODIR
source /home/ytsejam/.virtualenvs/yv_dev/bin/activate
#export /home/ytsejam/.virtualenvs/yv_dev/bin/postactivate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH
# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
# Start your Django Unicorn
# Programs meant to be run under supervisor should not daemonize themselves (do not use --daemon)
exec /home/ytsejam/public_html/abctasarim/gunicorn \
--name $NAME \
--workers $NUM_WORKERS \
--env DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE \
--pythonpath $DJANGODIR \
--user $USER \
--bind=unix:$SOCKFILE yogavidya.wsgi:application
When I try to run it , I am getting an error for my service file :
...
ImportError: No module named ' '
...
How can I fix my script to serve the site correctly ?
Thanks
virtualenvwrapper is supposed to be what you use in development. You want to deploy using the package that virtualenvwrapper is built on—the virtualenv package. The best suggestion I have for you for how to make this work would be to try the steps that you would normally use to start your virtualenvwrapper environment, namely sourcing the shell script and then using workon:
NAME="yogavidya" #Name of the application (*)
DJANGODIR=/home/ytsejam/public_html/abctasarim # Django project directory (*)
SOCKFILE=/home/ytsejam/public_html/abctasarim/run/gunicorn.sock # we will communicate using this unix socket (*)
USER=ytsejam # the user to run as (*)
GROUP=webdata # the group to run as (*)
NUM_WORKERS=1 # how many worker processes should Gunicorn spawn (*)
DJANGO_SETTINGS_MODULE=yogavidya.settings.base # which settings file should Django use (*)
DJANGO_WSGI_MODULE=yogavidya.wsgi # WSGI module name (*)
echo "Starting $NAME as `whoami`"
# Activate the virtual environment
cd $DJANGO_DIR
source /path/to/virtualenvwrapper.sh
workon yv_dev
You should also just try invoking gunicorn from the command line after activating your virtualenv.
Here’s how you can do it with virtualenv:
cd /home/ytsejam/public_html/abctasarim
sudo pip install virtualenv
virtualenv .
. bin/activate
pip install -r requirements.txt
pip install gunicorn
gunicorn script:
NAME="yogavidya" #Name of the application (*)
DJANGODIR=/home/ytsejam/public_html/abctasarim # Django project directory (*)
SOCKFILE=/home/ytsejam/public_html/abctasarim/run/gunicorn.sock # we will communicate using this unix socket (*)
USER=ytsejam # the user to run as (*)
GROUP=webdata # the group to run as (*)
NUM_WORKERS=1 # how many worker processes should Gunicorn spawn (*)
cd $DJANGO_DIR
. bin/activate
# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
exec /home/ytsejam/public_html/abctasarim/bin/gunicorn \
--name $NAME \
--workers $NUM_WORKERS \
--user $USER \
--bind=unix:$SOCKFILE yogavidya.wsgi:application

Permission denied # rb_sysopen - log/application.log (Errno::EACCES)

Hi I am using docker to deploy my rails app using phusion/passenger image. Here is my Dockerfile:
FROM phusion/passenger-ruby22:0.9.19
# set correct environment variables
ENV HOME /root
ENV RAILS_ENV production
# Use baseimage-docker's init system.
CMD ["/sbin/my_init"]
# Expose Nginx HTTP service
EXPOSE 80
# Start Nginx / Passenger
RUN rm -f /etc/service/nginx/down
# Remove the default site
RUN rm /etc/nginx/sites-enabled/default
# Add the nginx site and config
ADD nginx.conf /etc/nginx/sites-enabled/nginx.conf
ADD rails-env.conf /etc/nginx/main.d/rails-env.conf
# Let ensure these packages are already installed
# otherwise install them anyways
RUN apt-get update && apt-get install -y build-essential \
nodejs \
libpq-dev
# bundle gem and cache them
WORKDIR /tmp
ADD Gemfile /tmp/
ADD Gemfile.lock /tmp/
RUN gem install bundler
RUN bundle install
# Add rails app
ADD . /home/app/webapp
WORKDIR /home/app/webapp
RUN touch log/delayed_job.log log/production.log log/
RUN chown -R app:app /home/app/webapp
RUN RAILS_ENV=production rake assets:precompile
# Clean up APT and bundler when done.
RUN apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
I am getting permission issue for tmp and log files.
web_1 | [ 2016-07-19 08:45:12.6653 31/7ff812726700 age/Cor/App/Implementation.cpp:304 ]: Could not spawn process for application /home/app/webapp: An error occurred while starting up the preloader.
web_1 | Error ID: 42930e85
web_1 | Error details saved to: /tmp/passenger-error-9DeJ86.html
web_1 | Message from application: Permission denied # rb_sysopen - log/logentries.log (Errno::EACCES)
web_1 | /usr/lib/ruby/2.2.0/logger.rb:628:in `initialize'
web_1 | /usr/lib/ruby/2.2.0/logger.rb:628:in `open'
web_1 | /usr/lib/ruby/2.2.0/logger.rb:628:in `open_logfile'
web_1 | /usr/lib/ruby/2.2.0/logger.rb:584:in `initialize'
web_1 | /usr/lib/ruby/2.2.0/logger.rb:318:in `new'
web_1 | /usr/lib/ruby/2.2.0/logger.rb:318:in `initialize'
web_1 | /var/lib/gems/2.2.0/gems/le-2.7.2/lib/le/host/http.rb:37:in `new'
web_1 | /var/lib/gems/2.2.0/gems/le-2.7.2/lib/le/host/http.rb:37:in `initialize'
I tried to give chmod -R 665/775/777 log/ and still didn't fixed the problem.
Thanks
Rearrange your line RUN RAILS_ENV=production rake assets:precompile first then RUN chown -R app:app /home/app/webapp(after your rake task) So, It should be something like this:
RUN RAILS_ENV=production rake assets:precompile
RUN chown -R app:app /home/app/webapp