I am having difficulties running a series of sequential commands using the subprocess module, i need to do this so a python program can call in an installation of a cv virtualenv and then run another python program (that needs to be run within the virtualenv)
This is the command string i run from terminal, you can see it contains multiple commands that run in sequence until the creation of the cv virtual env:
sudo pip install virtualenv virtualenvwrapper && sudo rm -rf ~/.cache/pip && export WORKON_HOME=$HOME/.virtualenvs && source /usr/local/bin/virtualenvwrapper.sh && source ~/.bashrc && mkvirtualenv cv
Running this in the terminal returns me something like this:
(cv) name#computer:~$
from that i can run my python scripts that need the openCV
my code so far is this:
from subprocess import Popen, PIPE, STDOUT
cmd1 = 'sudo pip install virtualenv virtualenvwrapper'
cmd2 = 'sudo rm -rf ~/.cache/pip'
cmd3 = 'export WORKON_HOME=$HOME/.virtualenvs'
cmd4 = 'source /usr/local/bin/virtualenvwrapper.sh'
cmd5 = 'source ~/.bashrc'
cmd6 = 'mkvirtualenv cv'
cmd7 = 'cd /script path'
cmd8 = 'python novo.py'
final = Popen("{}; {}; {}; {}; {}; {}; {}; {}".format(cmd1, cmd2,cmd3, cmd4, cmd5, cmd6, cmd7, cmd8), shell=True, stdin=PIPE,
stdout=PIPE, stderr=STDOUT, close_fds=True)
stdout, nothing = final.communicate()
log = open('log', 'w')
log.write(stdout)
log.close()
And the errors in log look like this:
/bin/sh: 1: source: not found
/bin/sh: 1: source: not found
/bin/sh: 1: mkvirtualenv: not found
How can i achieve a terminal like execution ?
again, sequence is crucial.
/bin/sh: 1: source: not found
shell=True uses /bin/sh by default. source shell builtin hints at bash. Pass executable='/bin/bash'.
btw, you could use a multiline string literal:
#!/usr/bin/env python3
import sys
from subprocess import check_call, DEVNULL, STDOUT
with open('log', 'wb', 0) as file:
check_call("""set -e -x
{python} -mpip install --user virtualenv virtualenvwrapper
rm -rf ~/.cache/pip
export WORKON_HOME=$HOME/.virtualenvs
source /path/to/virtualenvwrapper.sh
source ~/.bashrc
mkvirtualenv cv
cd /script path
{python} novo.py
""".format(python=sys.executable),
shell=True, executable='/bin/bash',
stdin=DEVNULL, stdout=file, stderr=STDOUT, close_fds=True)
Or save the command into a separate bash script and run the script instead.
DEVNULL is Python 3 feature—it is easy to emulate it on Python 2 too: DEVNULL = open(os.devnull, 'r+b', 0).
Related
I want to automate these commands:
echo something
git config --global user.name "myUser"
git config --global user.email "myMail#mail.com"
I did some research and saw this library in Python subprocess. But the following commands don't work:
import subprocess
subprocess.call(["echo", "something"])
subprocess.call(["git", "config", "--global", "user.name", "\"myUser\""])
subprocess.call(["git", "config", "--global", "user.email", "myMail#mail.com"])
This is my ".bat" file, where I have the Python location and the script location:
#echo off
"C:\Python27\python.exe" -c "D:\Tools\Scripts\GIT_login.py"
pause
Unfortunately, when I run the ".bat" file, I get this message:
File "<string>", line 1
D:\Tools\Scripts\GIT_login.py
↑
SyntaxError: invalid syntax
If you are running the bat on git-bash, you need to give the python file path like this:
'D://Tools/Scripts/GIT_login.py'
I can list the root directory using a script file and command line.
%%writefile ShowDir.py
import os
import sys
from sys import argv
DIR = argv[1]
show = os.listdir(DIR)
print show
Then I ran in the terminal:
!echo mypassword | sudo -S python ShowDir.py /root
It worked, but I want a more graceful approach.
I also tried:
This will run files, make directories, remove files directories, but for some reason will not LIST them.
import os, subprocess
ret = subprocess.check_call("echo mypassword | sudo -S ls /root", shell=True)
ret
It just returned a 0, no directory listing
In one of my test files I call a Django management command:
def setUpModule():
management.call_command('loaddata', 'frontend/fixtures/chemicals.json',
verbosity=0)
management.call_command('create_indexes_and_matviews',
db_name, db_user, db_pass,
verbosity=2)
This test runs fine when I run it locally with manage.py test.
However, on Travis I get this error:
======================================================================
ERROR: setUpModule (frontend.tests.test_api_views)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/travis/build/.../frontend/tests/test_api_views.py", line 35, in setUpModule
verbosity=2)
File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/django/core/management/__init__.py", line 95, in call_command
raise CommandError("Unknown command: %r" % name)
CommandError: Unknown command: 'create_indexes_and_matviews'
How can I let Travis know about the command?
This is my Travis file:
language: python
python:
- "2.7"
addons:
postgresql: "9.3"
env:
- SECRET_KEY=test DB_NAME=dbtest DB_USER=test DB_PASS=test
before_install:
- export DJANGO_SETTINGS_MODULE=....settings.local
- export PYTHONPATH=$HOME/builds/...
install:
- pip install -r requirements.txt
- pip install -r requirements/local.txt
before_script:
- psql -U postgres -c 'CREATE DATABASE dbtest;'
- psql -U postgres -c "CREATE EXTENSION postgis" -d dbtest
- psql -U postgres -c "CREATE EXTENSION postgis_topology" -d dbtest
- psql -U postgres -c "CREATE USER test WITH CREATEUSER PASSWORD 'test';"
- psql -U postgres -c "GRANT ALL PRIVILEGES ON DATABASE dbtest to test;"
- psql -U postgres -c "ALTER USER test CREATEDB;"
- cd frontend && python manage.py migrate
script:
- python manage.py test
Is there something I should add so that it knows where to find management commands?
From my practice I know two reasons for such problem.
A. No the_app with create_indexes_and_matviews listed in settings.INSTALLED_APPS (it could be missed, excluded in if/else or try/except magic)
To check actual settings, try to add following command to the Travis file
echo "from django.conf import settings;print(settings.INSTALLED_APPS)" | python manage.py shell
B. Missed app dependencies. Try to get the actual error on travis with following command:
echo "from the_app.management.commands.create_indexes_and_matviews import Command" | python manage.py shell
Usually, real import error is descriptive enough to find the fix.
I want to know as title says: how to download a package using python-apt API? Only download it, don't to install it, just like does the command:
apt-get download ${package_name}
I'm using python v2.7.5-5ubuntu3 with python-apt v0.9.3.5 on Ubuntu 14.04.
You can use this script
#!/usr/bin/python
import apt
import os, sys
if len(sys.argv) != 2:
print 'Usage: apt-download pkg_name'
sys.exit()
cache = apt.Cache()
pkg = cache[sys.argv[1]]
cmd = 'wget ' + pkg.candidate.uri
print cmd
os.system(cmd)
It works well with Python 2.7.8 and python-apt 0.9.3.11 but it should also work with your configuration.
From a windows application written on C++ or python, how can I execute arbitrary shell commands?
My installation of Cygwin is normally launched from the following bat file:
#echo off
C:
chdir C:\cygwin\bin
bash --login -i
From Python, run bash with os.system, os.popen or subprocess and pass the appropriate command-line arguments.
os.system(r'C:\cygwin\bin\bash --login -c "some bash commands"')
The following function will run Cygwin's Bash program while making sure the bin directory is in the system path, so you have access to non-built-in commands. This is an alternative to using the login (-l) option, which may redirect you to your home directory.
def cygwin(command):
"""
Run a Bash command with Cygwin and return output.
"""
# Find Cygwin binary directory
for cygwin_bin in [r'C:\cygwin\bin', r'C:\cygwin64\bin']:
if os.path.isdir(cygwin_bin):
break
else:
raise RuntimeError('Cygwin not found!')
# Make sure Cygwin binary directory in path
if cygwin_bin not in os.environ['PATH']:
os.environ['PATH'] += ';' + cygwin_bin
# Launch Bash
p = subprocess.Popen(
args=['bash', '-c', command],
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
p.wait()
# Raise exception if return code indicates error
if p.returncode != 0:
raise RuntimeError(p.stderr.read().rstrip())
# Remove trailing newline from output
return (p.stdout.read() + p.stderr.read()).rstrip()
Example use:
print cygwin('pwd')
print cygwin('ls -l')
print cygwin(r'dos2unix $(cygpath -u "C:\some\file.txt")')
print cygwin(r'md5sum $(cygpath -u "C:\another\file")').split(' ')[0]
Bash should accept a command from args when using the -c flag:
C:\cygwin\bin\bash.exe -c "somecommand"
Combine that with C++'s exec or python's os.system to run the command.