Here's the context:
I am using python 2.7.5. And I would like to run UNIX commands as well as maven commands in a python script.
I was successful to do so, using os.system("cmd"), but I need to work on the result of the given command. After reading the doc and some threads in here, I decided to use the subprocess module to redirect the output to the stdout using PIPE. Unexpectedly, I am getting an OSError as shown in the attached image. Your help will be much appreciated.
In addition to the given sample in the attached image, I have tried:
p = os.popen("java -version")
result = subprocess.check_output(p, shell=True)
subprocess.call("ls /usr", shell=True)
p.s. Using shell=True is strongly discouraged (doc), since it can be dangerous when coupled with unsanitized input.
Also, I took a look at the given script in the error message /usr/lib64/python2.7/subprocess.py, line 711 adn 1327 but didn't learn more than what is mentionned in the error message: raise child_exception
Subprocess Terminal Output
You aren't using subprocess.check_output correctly. You're trying to pass a pipe file object (the return value of os.popen) to check_output but it's expecting a command argument or argument vector.
Also, the subprocess.call function won't capture the executed command's output, so you would only use that if you want the output of ls /usr (or whatever) to be seen by the user running the script interactively. (Which is pretty much the same as os.system.)
Try this instead (showing with and without the shell):
import subprocess
out1a = subprocess.check_output(['java', '-version'], stderr=subprocess.STDOUT)
print(out1a)
out1b = subprocess.check_output('java -version', stderr=subprocess.STDOUT, shell=True)
print(out1b)
out2a = subprocess.check_output(['ls', '/usr'])
print(out2a)
out2b = subprocess.check_output('ls /usr', shell=True)
print(out2b)
# Cannot capture output this way, but it will be visible to user
subprocess.call('ls /usr', shell=True)
Note that in the case of the java -version command, the version info gets printed to the command's standard error output so you must redirect that in order to capture it as the returned value of check_output (hence the stderr=subprocess.STDOUT).
Related
I have multiple python scripts, each with print statements and prompts for input. I run these scripts from a single python script as below.
os.system('python script1.py ' + sys.argv[1])
os.system('python script2.py ' + sys.argv[1]).....
The run is completed successfully, however, when I run all the scripts from a single file, I no longer see any print statements or prompts for input on the run console. Have researched and attempted many different ways to get this to work w/o success. Help would be much appreciated. Thanks.
If I understand correctly you want to run multiple python scripts synchronously, i.e. one after another.
You could use a bash script instead of python, but to answer your question of starting them from python...
Checkout out the subprocess module: https://docs.python.org/3.4/library/subprocess.html
In particular the call method, it accepts a stdin and stdout which you can pass sys.stdin and sys.stdout to.
import sys
import subprocess
subprocess.call(['python', 'script1.py', sys.argv[1]], stdin=sys.stdin, stdout=sys.stdout)
subprocess.call(['python', 'script2.py', sys.argv[1]], stdin=sys.stdin, stdout=sys.stdout)
^
This will work in python 2.7 and 3, another way of doing this is by importing your file (module) and calling the methods in it. The difference here is that you're no longer running the code in a separate process.
subroutine.py
def run_subroutine():
name = input('Enter a name: ')
print(name)
master.py
import subroutine
subroutine.run_subroutine()
I am running the following code in Python 2.7:
values = os.system("bazel build tensorflow/examples/image_retraining:"
"label_image && bazel-bin/tensorflow/examples/image_retraining/label_image "
"--graph=/tmp/output_graph.pb --labels=/tmp/output_labels.txt "
"--output_layer=final_result:0 --image=$HOME/Desktop/Image-3/image1.png")
print values
But for the values variable I am returned a 0. I believe this means that I am not getting any errors. How do I store the output into a variable or into a .txt file?
You can just redirect the output of the system call appending > output.txt to your command.
The output of the command will be in file output.txt in the directory where you invoke the command (likely the very same one you invoke your python script in).
Since I can't readily reproduce your command, I used a simple example - try to switch to Pyopen in the subprocess module:
from subprocess import Popen
proc = Popen(['ls', '-t'], stdout = open('/path/redir.txt', 'w'))
Here you run the command in square brackets and redirect the output from stdout i.e. the terminal to a file redir.txt.
I have a python snippet like this:
self.task = subprocess.Popen("bash ./FOLDER/script.sh", cwd=some_dir, shell=True)
self.task.wait()
from which an exception is raised, complaining that a 'NoneType' object has no method wait(). I guess it means the Popen call returns None ? What could be the reason for that. The documentation does not mention this possibility
I'm using python 2.7.13
Well, apparently self.task gives a NoneType respons meaning the subprocess.Popen() command is likely at fault.
First thing I notice is an incorrect syntax, since you did not wrap your command line in square brackets [] and you did not split the arguments.
Furthermore, the Python docs state (regarding the cwd option you used):
If cwd is not None, the child’s current directory will be changed to
cwd before it is executed. Note that this directory is not considered
when searching the executable, so you can’t specify the program’s path
relative to cwd.
So first thing to check would be if your script.sh is situated in some_dir/FOLDER/script.sh.
If that is indeed the case, please check if you inserted the cwd-argument with the right syntax, so as a string.. Meaning cwd="/path/to/some/dir".
Then, since the Python docs clearly state that:
Using shell=True can be a security hazard
I would remove that argument. This might mean you will have to use the full path to your bash. To find out the correct path, open a terminal and execute which bash. Or, to be sure, type bash.
Then, give this a try:
import subprocess
self.task = subprocess.Popen(["/path/to/your/bash", "./FOLDER/script.sh"], cwd="/path/to/some_dir", stdout=subprocess.PIPE, stderr=subprocess.PIPE) # This makes sure you will also catch any standard errors, so it allows for a bit more control.
output, errors = self.task.communicate() # This already encapsulates .wait()
print(output.decode()) # if you'd like to check the output.
Read the comments in code for some further explanation..
Is it possible to prevent subprocess.popen from showing prompts in the terminal?
Attempting to map a drive but would like to read the prompt for credentials in the script rather than display them to the terminal. The idea being I can carry out actions based on the response.
I am aware the use of shell is frowned upon when using string based commands (for obvious reasons), however I'm controlling the input so am happy with the risk for testing purposes.
I was under the impression that all stdout (interaction) would be parsed into the output_null variable. Instead I am still getting the prompt in the terminal (as illustrated below). I'm either miss understanding how the streams work or I'm missing something. Can anyone enlighten me please
command = "mount -t smbfs //{s}/SYSVOL {m}".format(s=server, m=temp_dir)
p = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
output_null = p.communicate()[0]
if "Password for" in output_null:
print 'awdaa'
Terminal Shows
Password for 192.168.1.111:
I would like to run LIWC (installed in my Mac) within a python 2.7 script.
I have been reading about subprocess (popen and check_output seem the way to go), but I do not get the syntax for:
opening the program;
getting a text file to be analysed;
running the program;
getting the output (analysis) and storing it in a text file.
This is my first approach to subprocess, is this possible?
I appreciate the suggestions.
EDIT
This is the closest to implementing a solution (still does not work):
I can open the application.
subprocess.call(['open', '/file.app'])
But cannot make it process the input file and get an output one.
subprocess.Popen(['/file.app', '-input', 'input.txt', '-output', 'output.txt'])
Nothing comes out of this code.
EDIT 2
After reading dozens of posts, I am still very confused about the syntax for the solution.
Following How do I pipe a subprocess call to a text file?
I came out with this code:
g = open('in_file.txt', 'rb', 0)
f = open('out_file.txt', 'wb')
subprocess.call(['open', "file.app"] stdin=g, stdout=f)
The output file comes out empty.
EDIT 3
Following http://www.cplusplus.com/forum/unices/40680/
When I run the following shell script on the Terminal:
cat input.txt | /Path/LIWC > output.txt
The output txt file is empty.
EDIT 4
When I run:
subprocess.check_call(['/PATH/LIWC', 'PATH/input.txt', 'PATH/output.txt'])
It opens LIWC, does not create an output file and freezes.
EDIT 5
When I run:
subprocess.call(['/PATH/LIWC', 'PATH/input.txt', 'PATH/output.txt'])
It runs LIWC, creates an empty output.txt file and freezes (the process does not end).
The problem with using 'open' in subprocess.call(['open', "file.app"] stdin=g, stdout=f) is that it requests that a file be opened through a service, and doesn't directly attach it to your python process. You'll need to instead use the path to LIWC. I'm not sure that it supports reading from stdin, though, so you might need to even pass in the path to the file you'd like it to open.