I'm trying to run IPython on a production ubuntu server. I want to control it with upstart.
I have a bash script that properly invokes it in the foreground but it doesn't work when invoked through upstart. I'm not sure how to debug the problem other than piping the upstart script's output to a file, which just confirms that the IPython console dashboard properly shows up.
I'm using django-extensions with the following configuration:
IPYTHON_ARGUMENTS = [
'--ext', 'django_extensions.management.notebook_extension',
'--pylab=inline',
'--profile=myprofile',
]
My bash script is:
#!/bin/bash
set -e
cd /home/ubuntu/myproject
exec venv/bin/python /home/ubuntu/myproject/manage.py shell_plus --notebook
Any help is appreciated
No idea what can be the reason.
Did you had a look at Hydra that have been designed to launch multiple IPython server?
Related
I have a instance where I have some Flask web app. In order the app to start when the VM is booted I have included a startup script:
#!/bin/sh
cd documentai_webapp
cd docai_webapp_instance_gcp
sudo python3 server.py
However, this is not at all executed, anyone can help me?thanks!
PD: When I execute this script manually within the VM it works perfectly fine
As context it is necessary contemplate:
For Linux startup scripts, you can use bash or non-bash file. To use a non-bash file, designate the interpreter by adding a #! to the top of the file. For example, to use a Python 3 startup script, add #! /usr/bin/python3 to the top of the file.
If you specify a startup script by using one of the procedures in this document, Compute Engine does the following:
Copies the startup script to the VM
Sets run permissions on the startup script
Runs the startup script as the root user when the VM boots (missing step from #Andoni)
For information about the various tasks related to startup scripts and when to perform each one, see the Overview.
I wrote a bash script to create a scheduled task and run a django app. I am using Git Bash as my terminal, and I have been able to manually run the commands in the snippet posted below with success. However, when I run bash script with these same commands, the scheduled task and the django app are never run. Why is there a discrepancy in behavior and how can I correct this in my bash script?
#!/usr/bin/env bash
// Create scheduled task
echo schtasks //create //tn my-task //tr '"python app/manage.py loaddata /resources/output.json"' //sc daily //st 09:30 //ri 60 //et 16:00
// Run app
echo python app/manage.py runserver
echo "TERMINATED"
$SHELL
For a bash script to run from a CMD session (triggered by the Windows scheduler), you would need:
a script named git-xxx (replace xxx by the name of your chosing)
that script in your Windows PATH (as well as Git itself)
a schedule task running git xxx (note the space)
That would run git-xxx in a Git bash session.
The other option would be, still from a CMD (or a scheduled task) to run:
bash -c "/c/path/to/your/script"
In both instances, make sure bash is not the one from WSL if you are on Windows 10, and have activated that feature.
I need to update /etc/hosts for all instances in my EMR cluster (EMR AMI 4.3).
The whole script is nothing more than:
#!/bin/bash
echo -e 'ip1 uri1' >> /etc/hosts
echo -e 'ip2 uri2' >> /etc/hosts
...
This script needs to run as sudo or it fails.
From here: https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-bootstrap.html#bootstrapUses
Bootstrap actions execute as the Hadoop user by default. You can execute a bootstrap action with root privileges by using sudo.
Great news... but I can't figure out how to do this, and I can't find an example.
I've tried a bunch of things... including...
running as Hadoop and adding 'sudo' to each of the 'echo' statements in the script
using a shell script to copy and chmod the above ('echo' statements with no 'sudo') and running local copy using run-if bootstrap that calls 1=1 sudo bash /home/hadoop/myDir/myScript.sh
hard coding the whole script as a one-liner into a run-if bootstrap action
I consistently get:
On the master instance (i-xxx), bootstrap action 2 returned a non-zero return code
If i check the logs for the "Setup hadoop debugging" step, there's nothing there.
From here: https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-overview.html#emr-overview-cluster-lifecycle
summary emr setup (in order):
provisions ec2 instances
runs bootstrap actions
installs native applications... like hadoop, spark, etc.
So it seems like there's some risk that since I'm mucking around as user Hadoop before hadoop is installed, I could be messing something up there, but I can't imagine what.
I think it must be that my script isn't running as 'sudo' and it's failing to update /etc/hosts.
My question... how can I use bootstrap actions (or something else) on EMR to run a simple shell script as sudo? ...specifically to update /etc/hosts?
I've not had problems using sudo from within a shell script run as an EMR bootstrap action, so it should work. You can test that it works with a simple script that simply does "sudo ls /root".
Your script is trying to append to /etc/hosts by redirecting stdout with:
sudo echo -e 'ip1 uri1' >> /etc/hosts
The problem here is that while the echo is run with sudo, the redirection (>>) is not. It's run by the underlying hadoop user, who does not have permission to write to /etc/hosts. The fix is:
sudo sh -c 'echo -e "ip1 uri1" >> /etc/hosts'
This runs the entire command, including the stdout redirection, in a shell with sudo.
I have Flask application named as rest.py and I have dockerize but it is not running.
#!flask/bin/python
from flask import Flask, jsonify
app = Flask(__name__)
tasks = [
{
'id': 1,
'title': u'Buy groceries',
'description': u'Milk, Cheese, Pizza, Fruit, Tylenol',
'done': False
}
]
#app.route('/tasks', methods=['GET'])
def get_tasks():
return jsonify({'tasks': tasks})
if __name__ == '__main__':
app.run(debug=True)
Dockerfile is as follows
FROM ubuntu
RUN apt-get update -y
RUN apt-get install -y python-dev python-pip
COPY . /rest
WORKDIR /rest
RUN pip install -r Req.txt
ENTRYPOINT ["python"]
CMD ["rest.py"]
I have build it using this command...
$ docker build -t flask-sample-one:latest
...and when I run container...
$ docker run -d -p 5000:5000 flask-sample-one
returning the following output:
7d1ccd4a4471284127a5f4579427dd106df499e15b868f39fa0ebce84c494a42
What am I doing wrong?
The output you get is the container ID. Check with docker ps whether it keeps running.
Use docker logs [container-id] to figure out what's going on inside.
Some problems I can find in your question:
Change the app.run line to app.run(host='0.0.0.0', debug=True). From the point of view of the container, its services need to be externally available. So they need to run on the loopback interface, like you would run it if you'd set up a publicly available server on a host directly.
Make sure that Flask gets installed. Your docker image file requires all the commands to make it work from a blank Ubuntu installation.
Please do not forget to deactivate debug if you'd ever expose this service on your host. Debug mode in Flask makes it possible for visitors to run arbitrary code if they can trigger an exception (it's a feature, not a bug).
After that (and building the container again [1]), try curl http://127.0.0.1:5000/tasks on the host. Let me know if it works, if not there are other problems in your setup.
[1] You can improve the prototyping workflow with Flask's built-in reloader (which is enabled by default) if you use a volume mount in your docker container for the directory that contains your python files - this would allow you to change your script on the host, reload in the browser and directly see the result.
I believe that you need to reinforce your concepts about Docker, in order to understand how it works, and then you will achieve your objectives regarding "dockerizing" whatever application.
Here is an article which can give your some first steps.
An official HOWTO will also help you.
Some observations that might help you:
check if your Req.txt contains flask for installation
before dockerizing, check if your application is working
check your running containers with docker ps and see if your container is running
if it is running, test your application: curl http://127.0.0.1:5000/tasks
*
One more thing:
your JSON has an OBJECT with an ARRAY with just one ELEMENT
Is that what you want for your prototype?
Take a look on this doc, about the JSON standard.
I want to run my development django server at startup so I defined following cron job:
#reboot screen -d -m django-admin.py runserver 192.168.0.28:8000
But it didn't work.
What is really interesting, when I copy/paste directly to terminal and execute it works just fine.
I even tried something like this:
#reboot cd /home/ubuntu && /usr/bin/screen -d -m /usr/bin/python /usr/local/bin/django-admin.py runserver 192.168.0.28:8000 &> /home/ubuntu/cron.err
To be sure I'm not using some undefined commands in wrong location and examined contents of cron.err file but it's empty.
And (of course) when I fire this directly from the console it works immediately.
Please help.
Does it work if you try and run it from cron at a specific time? Eg:
50 12 2 8 * /usr/bin/screen -dmS set_from_cron