python subprocess popen returns None - python-2.7

I have a python snippet like this:
self.task = subprocess.Popen("bash ./FOLDER/script.sh", cwd=some_dir, shell=True)
self.task.wait()
from which an exception is raised, complaining that a 'NoneType' object has no method wait(). I guess it means the Popen call returns None ? What could be the reason for that. The documentation does not mention this possibility
I'm using python 2.7.13

Well, apparently self.task gives a NoneType respons meaning the subprocess.Popen() command is likely at fault.
First thing I notice is an incorrect syntax, since you did not wrap your command line in square brackets [] and you did not split the arguments.
Furthermore, the Python docs state (regarding the cwd option you used):
If cwd is not None, the child’s current directory will be changed to
cwd before it is executed. Note that this directory is not considered
when searching the executable, so you can’t specify the program’s path
relative to cwd.
So first thing to check would be if your script.sh is situated in some_dir/FOLDER/script.sh.
If that is indeed the case, please check if you inserted the cwd-argument with the right syntax, so as a string.. Meaning cwd="/path/to/some/dir".
Then, since the Python docs clearly state that:
Using shell=True can be a security hazard
I would remove that argument. This might mean you will have to use the full path to your bash. To find out the correct path, open a terminal and execute which bash. Or, to be sure, type bash.
Then, give this a try:
import subprocess
self.task = subprocess.Popen(["/path/to/your/bash", "./FOLDER/script.sh"], cwd="/path/to/some_dir", stdout=subprocess.PIPE, stderr=subprocess.PIPE) # This makes sure you will also catch any standard errors, so it allows for a bit more control.
output, errors = self.task.communicate() # This already encapsulates .wait()
print(output.decode()) # if you'd like to check the output.
Read the comments in code for some further explanation..

Related

Python subprocess module giving an OSError while running UNIX commands

Here's the context:
I am using python 2.7.5. And I would like to run UNIX commands as well as maven commands in a python script.
I was successful to do so, using os.system("cmd"), but I need to work on the result of the given command. After reading the doc and some threads in here, I decided to use the subprocess module to redirect the output to the stdout using PIPE. Unexpectedly, I am getting an OSError as shown in the attached image. Your help will be much appreciated.
In addition to the given sample in the attached image, I have tried:
p = os.popen("java -version")
result = subprocess.check_output(p, shell=True)
subprocess.call("ls /usr", shell=True)
p.s. Using shell=True is strongly discouraged (doc), since it can be dangerous when coupled with unsanitized input.
Also, I took a look at the given script in the error message /usr/lib64/python2.7/subprocess.py, line 711 adn 1327 but didn't learn more than what is mentionned in the error message: raise child_exception
Subprocess Terminal Output
You aren't using subprocess.check_output correctly. You're trying to pass a pipe file object (the return value of os.popen) to check_output but it's expecting a command argument or argument vector.
Also, the subprocess.call function won't capture the executed command's output, so you would only use that if you want the output of ls /usr (or whatever) to be seen by the user running the script interactively. (Which is pretty much the same as os.system.)
Try this instead (showing with and without the shell):
import subprocess
out1a = subprocess.check_output(['java', '-version'], stderr=subprocess.STDOUT)
print(out1a)
out1b = subprocess.check_output('java -version', stderr=subprocess.STDOUT, shell=True)
print(out1b)
out2a = subprocess.check_output(['ls', '/usr'])
print(out2a)
out2b = subprocess.check_output('ls /usr', shell=True)
print(out2b)
# Cannot capture output this way, but it will be visible to user
subprocess.call('ls /usr', shell=True)
Note that in the case of the java -version command, the version info gets printed to the command's standard error output so you must redirect that in order to capture it as the returned value of check_output (hence the stderr=subprocess.STDOUT).

How to use SublimeLinter's cmd attribute

I'm writing a SublimeLinter (a SublimeText plugin) plugin that uses luacheck for a custom syntax we use. So far I have it working with simply cmd = 'luacheck #', the # being apparently replaced by the filename when SublimeLinter calls luacheck. The problem is that with SublimeLinter on 'background' mode, the warnings don't actually update until the file is saved, e.g. if I remove a line containing a warning the warning will still be there, just highlighting a blank space (until I save, that is). I have a feeling that this is because I'm using the # and since this is replaced by the filename, luacheck won't update till the file is updated. However, the SublimeLinter documentation on cmd is not great, and I'm having trouble figuring out how to write one correctly. None of the plugins on their GitHub seem to use #, either. If I copy the default lua plugin (which uses cmd = 'luac -p * -') and use cmd = 'luacheck * -', luacheck executes but only returns an I/O error. Could someone maybe provide some more insight into how SublimeLinter's cmd attribute works?
EDIT: I was able to fix this issue by using tempfile_suffix = 'lua' in linter.py. According to the SublimeLinter docs, this is used for linters that don't use stdin, so I suppose my issue could have been with luacheck instead.
I was able to fix this issue by using tempfile_suffix = 'lua' in linter.py. According to the SublimeLinter docs, this is used for linters that don't use stdin, so I suppose my issue could have been with luacheck instead.

Using less as gdb pager

I noticed that in GDB, when issuing commands with long output like info variables, the output is displayed one page at time, pressing enter to go down and q to quit.
Is it possible to replace the default pager with another one, such as less, so that I can navigate up and down, quitting, searching, etc?
Is it possible to replace the default pager with another one
No: GDB doesn't call into external program to display the output, it simply pauses the output every screenfull (and you can make it not pause by set height 0).
In addtion to running inside emacs, you could also use screen or tmux (learning them will generally help you in a lot of other situations), or ask GDB to log output (set logging on) and then search in gdb.txt with any $PAGER you want.
Starting with version 9.1, GDB has a pipe command, so you can send a command's output to the pager of your choice. From the documentation:
pipe [command] | shell_command
Executes command and sends its output to shell_command. Note that no space is needed around |. If no command is provided, the last command executed is repeated.
run gdb inside of emacs and you should be able to use emacs' paging commands.
run emacs
type M-x gdb return (M stands for meta - alt key or option on Macs)
The Emacs message bar will now display the message:
Run gdb (like this): gdb
More information is available here: http://tedlab.mit.edu/~dr/gdbintro.html
HTH
you can put the following user-defined commands in ~/.gdbinit, then
% cat ~/.gdbinit
python import os
define less1
python os.popen("less","w").write(gdb.execute("$arg0",to_string=True))
end
define less2
python os.popen("less","w").write(gdb.execute("$arg0 $arg1",to_string=True))
end
...
% gdb
(gdb) less2 info var
...
(gdb) less1 disass
...
It is a bit old thread, but I think it is worth to add. #yichun gave a very nice idea here, but to be more practical it can be extended to any number of arguments:
define less
python import os
python os.popen("less","w").write(gdb.execute(' '.join(["$arg{0}".format(i) for i in range(0, argc)]), to_string=True))
end
Then it can also woth adding exceptions handling and waiting for processes to terminate to avoid keyboard glitches and we have something like that:
% cat ~/.gdbinit
define less
python argc = $argc
python
import os
f = None
try:
f = os.popen("less","w")
f.write(gdb.execute(' '.join(["$arg{0}".format(i) for i in range(0, argc)]), to_string=True))
except Exception as e:
if f:
f.write(str(e))
else:
print (str(e))
finally:
if f:
f.close()
end
end
In gdb 8.1.1 this code in .gdbinit adds the required functionality:
python
import os
class Less(gdb.Command):
def __init__(self):
super().__init__("less", gdb.COMMAND_USER, gdb.COMPLETE_COMMAND)
def invoke(self, argstr, from_tty):
with os.popen("less","w") as pipe:
try:
pipe.write(gdb.execute(argstr, to_string=True))
except Exception as e:
pipe.write(str(e))
Less()
end
Usage
(gdb) less info breakpoints
(gdb) less backtrace
Information: Commands In Python.

TypeError: abspath() takes exactly 1 argument (2 given)

I get this error while running a python script (called by ./waf --run):
TypeError: abspath() takes exactly 1 argument (2 given)
The problem is that it is indeed called with: obj.path.abspath(env).
This is not a python issue, because that code worked perfectly before, and it's part of a huge project (ns3) so I doubt this is broken.
However something must have changed in my settings, because this code worked before, and now it doesn't.
Can you help me to figure out why I get this error ?
Here is the python code: http://pastebin.com/EbJ50BBt. The error occurs line 61.
The documentation of the method Node.abspath() states it does not take an additional env parameter, and I confirmed that it never did by checking the git history. I suggest replacing
if not (obj.path.abspath().startswith(launch_dir)
or obj.path.abspath(env).startswith(launch_dir)):
continue
with
if not obj.path.abspath().startswith(launch_dir):
continue
If this code worked before, this is probably due to the fact that the first operator of the or expression happened to always be True, so the second operator was never executed. It seems to be a bug in your code anyway.
You should have a file name and line number in the traceback. Go to that file and line and find out was "obj" and "obj.path.abspath" are. A simple solution would be to put the offending line in a try/except block to print (or log) more informations, ie:
# your code here
try:
whatever = obj.path.abspath(env)
except Exception, e:
# if you have a logger
logger.exception("oops : obj is '%s' (%s)" % (obj, type(obj)))
# else
import sys
print >> sys.stderr, "oops, got %s on '%s' (%s)" % (e, obj, type(obj))
# if you can run this code directly from a shell,
# this will send you in the interactive debugger so you can
# inspect the offending objet and the whole call stack.
# else comment out this line
import pdb; pdb.set_trace()
# and re-raise the exception
raise
My bet is that "obj.path" is NOT the python 'os.path' module, and that "obj.path.abspath" is a an instance method that only takes "self" as argument.
The problem came from the fact that apparently waf doesn't like symlinks, the python code must not be prepared for such cases.
Problem solved, thanks for your help everybody

popen and system behaves unexpectedly with multiple quoted file paths

I am trying to execute a dos command from within my C++ program, however soon as I add quotes to the output filepath (of a redirection) the command no longer gets executed and returns instantly. I've shown an example below of a path without spaces, but since paths may have spaces and thus be quoted for the shell to understand it properly I need to solve this dilemma - and I'm trying to get the simplest case working first.
i.e.
The following WORKS:
sprintf(exec_cmd,"\"C:/MySQL Server 5.5/bin/mysqldump.exe\" -u%s -p%s %s > C:/backup.bak",user,password,db_name);
system(exec_cmd);
The following does NOT work (notice the quotes around the output):
sprintf(exec_cmd,"\"C:/MySQL Server 5.5/bin/mysqldump.exe\" -u%s -p%s %s > \"C:/backup.bak\"",user,password,db_name);
system(exec_cmd);
I'm guessing it is choking somewhere. I've tried the same "exec_cmd" in popen to no avail.
Any help/advice is greatly appreciated.
I don't think your shell (cmd.exe) allows redirection to a file name with spaces. I couldn't make my command.com from DOS 6.22 accept it (I don't have a cmd.exe nearby to test).
Anyway, you can use the --result-file option to pass the redirection to the command itself.
mysqldump ... --result-file="file name" ...