Python 2.7.3 with ubuntu:
Trying to run a program, (youtube-dl in this example) with subprocess.call with some arguments, I encounter the following issue. consider the following script:
try.py:
#!/usr/bin/python
from subprocess import call
url = "https://www.youtube.com/watch?v=8SbUC-UaAxE"
myArray = ['./youtube-dl {}'.format(url),'-x','--audio-format mp3']
#print the array before executing:
for item in myArray:
print item,
#execute:
call(myArray)
This script prints outputs:
oris#oris:~/Desktop/YouTube/backend$ ./try.py
./youtube-dl https://www.youtube.com/watch?v=8SbUC-UaAxE -x --audio-format mp3
Traceback (most recent call last):
File "./try.py", line 16, in <module>
call(myArray)
File "/usr/lib/python2.7/subprocess.py", line 493, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1259, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Strangly, On the first line of the output I see the script does output ./youtube-dl https://www.youtube.com/watch?v=8SbUC-UaAxE -x --audio-format mp3 This command runs perfectly from bash directly, but produces an OSError from python. I also thought maybe to try and supply the url as an argument like this:
myArray = ['./youtube-dl', url,'-x','--audio-format mp3']
but than youtube-dl has an error of incorrect usage:
oris#oris:~/Desktop/YouTube/backend$ ./try.py
Usage: youtube-dl [options] url [url...]
youtube-dl: error: no such option: --audio-format mp3
./youtube-dl https://www.youtube.com/watch?v=8SbUC-UaAxE -x --audio-format mp3
I've read youtube-dl source to see how it handles the supplied arguments with optparse. I don't see the url as an argument there, so I'm guessing i'm passing the arguments to subprocess.call incorrectly.
Side note: another thing I find odd is the fact the print here takes effect after the subprocess call, as opposed to their order on my script. Is something happening here asynchronously?
What am I missing here? Many thanks
--audio-format and mp3 should be passed as separated arguments:
myArray = ['./youtube-dl', url, '-x', '--audio-format', 'mp3']
Related
I need to use a file as argument to the command but subprocess.call returns following error
>>> with open('test.txt') as F:
... subprocess.call(['cat', F])
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "/usr/lib64/python2.7/subprocess.py", line 524, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
TypeError: execv() arg 2 must contain only strings
However when file is not stored as variable it works fine
>>> subprocess.call(['cat', 'test.txt'])
fruit=banana
fruit=mango
fruit=apple
0
Code with open('test.txt') as F opens the file and assigns its pointor to variable F.
If I understand correctly you wanted file's content. Since subprocess.call required a command like you execute from terminal/console(In your case cat test.txt). Here in the subprocess.call you are passing File Descriptor sometime called File Pointer instead of file itself.
Therefore from the code if you really want a file content, you can simply use F.read() or F.readlines(). For Example
with open('test.txt') as F:
print F.readlines()
Also note that subprocess.call expect arguments as string. In the first case its the file object i.e <open file 'test.txt', mode 'r' at 0x7fa64a6f5a50> that leads to a traceback whereas in the second case its string i.e test.txt that's why works without any issue.
I have been writing a command line programs with Argparse for some time now, and I am trying to write it in such a way that when the user supplies the following to the command line:
$python my_script.py -h
A help section (usage) will be printed out that prints out help section of the main parser, as well as brief overviews of the subparsers.
But right now, anytime I type in the previous line into my terminal, I receive no usage and instead get a massive traceback and the following error:
TypeError: expected string or buffer
This error has never occurred to me before with argparse-based command line programs. Furthermore, if I supply the name of one of the subparsers,
$python my_script.py subparserA -h
I get a print-out of the subparser's usage. The same holds true for other subparsers.
So why is it not possible for me to get the usage for the main parser? This worked for me before so I don't know why it's not working now. I really would like for the user to be able to look at an overview of the different subparsers available.
My basic code is currently set up in the following way:
import argparse
import sys
if __name__ == "__main__":
Parser = argparse.ArgumentParser(prog= "My_program")
Parser.description= "This program does A and B things."
subparsers= Parser.add_subparsers(help= "SubparserA does A things and SubparserB does B things", dest='mode')
subparserA= subparsers.add_parser("subparserA", help= "Additional explanation of what A things entail")
subparserA.add_arguments("-foo", required=True, help= "foo is needed for SubparserA to work")
subparserB= subparsers.add_parser("subparserB", help="Additional explanation of what B things entail")
subparserB.add_argument("-bar", required=True, help= "bar is needed for SubparserB to work")
args= Parser.parse_args()
if args.mode == "subparserA":
###do things pertinent to subparserA
elif args.mode== "subparserB":
###do things pertinent to subparserB
else:
argparse.print_help()
argparse.ArgumentError("too few arguments")
UPDATE
Here is the full traceback of the error:
Traceback (most recent call last):
File "my_program.py", line 164, in <module>
args= Parser.parse_args()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1701, in parse_args
args, argv = self.parse_known_args(args, namespace)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1733, in parse_known_args
namespace, args = self._parse_known_args(args, namespace)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1939, in _parse_known_args
start_index = consume_optional(start_index)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1879, in consume_optional
take_action(action, args, option_string)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1807, in take_action
action(self, namespace, argument_values, option_string)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 996, in __call__
parser.print_help()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 2340, in print_help
self._print_message(self.format_help(), file)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 2314, in format_help
return formatter.format_help()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 281, in format_help
help = self._root_section.format_help()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 211, in format_help
func(*args)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 485, in _format_text
return self._fill_text(text, text_width, indent) + '\n\n'
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 621, in _fill_text
text = self._whitespace_matcher.sub(' ', text).strip()
TypeError: expected string or buffer
You should be using
Parser.print_help()
Parser.error('too few arguments')
That is use methods of the existing Parser object.
When I run your script
1019:~/mypy$ python stack46754855.py
Traceback (most recent call last):
File "stack46754855.py", line 10, in <module>
subparserA= subparsers.add_parser("subparserA", help= "Additional explanation of what A things entail", dest= 'mode')
File "/usr/lib/python2.7/argparse.py", line 1066, in add_parser
parser = self._parser_class(**kwargs)
TypeError: __init__() got an unexpected keyword argument 'dest'
dest is a not a valid parameter for the add_parser method. It is a valid, and useful, parameter for add_subparsers.
subparsers= Parser.add_subparsers(dest='mode')
It also objects to the add_arguments method.
After correction those I get:
1022:~/mypy$ python stack46754855.py
usage: My_program [-h] {subparserA,subparserB} ...
My_program: error: too few arguments
In Py2, subparsers is a required argument. It is optional in Py3 (a bug), allowing the script to run to the invalid argparse.print_help call:
1022:~/mypy$ python3 stack46754855.py
Traceback (most recent call last):
File "stack46754855.py", line 27, in <module>
argparse.print_help()
AttributeError: module 'argparse' has no attribute 'print_help'
With the change I suggested above:
1025:~/mypy$ python3 stack46754855.py
usage: My_program [-h] {subparserA,subparserB} ...
This program does A and B things.
positional arguments:
{subparserA,subparserB}
SubparserA does A things and SubparserB does B things
subparserA Additional explanation of what A things entail
subparserB Additional explanation of what B things entail
optional arguments:
-h, --help show this help message and exit
usage: My_program [-h] {subparserA,subparserB} ...
My_program: error: too few arguments
The second usage comes from the Parser.error call.
I can't reproduce your
massive traceback and the following error:
TypeError: expected string or buffer
I need to see that traceback (or part of it) to see what exactly is raising the error. That's not a normal argparse error; certainly it isn't one that argparse traps and reroutes.
More on the required/not required subparser behavior at How to Set a Default Subparser using Argparse Module with Python 2.7
Use + instead of , for multi line help string in parser.add_argument. If you have split you argument help in multiple lines using ',' then, you will see this issue
parser.add_argument("xml",help=("long help here",
" long help second line"))
This will result in above exception
instead
parser.add_argument("xml",help=("long help here" +
" long help second line"))
from a Python script I want to call a few external programs and use their output inside the script. Now I found that this script is running on my local system, but gets problems on another one that uses another language. So instead of checking the output in different languages, I want to call these programs with a specified "LC_ALL=en_US.utf8" as first argument. But it seems that subprocess does not like this:
lc = "LC_ALL=en_US.utf8"
uptimedata = subprocess.check_output([lc, "/bin/uptime"])
When I run the script, I get:
Traceback (most recent call last):
File "./serverwatch.py", line 22, in <module>
uptimedata = subprocess.check_output([lc, "/bin/uptime"])
File "/usr/lib/python2.7/subprocess.py", line 537, in check_output
process = Popen(stdout=PIPE, *popenargs, **kwargs)
File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1259, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
I understand that subprocess.check_output() wants the first argument to be the command to be called. So how can I run the command with the changed LC?
This is Python 2.7.3 on Debian Wheezy. The other system is running the same versions.
Execute your command through bash:
>>> lc = "LC_ALL=en_US.utf8"
>>> command = ["/usr/bin/bash", "-c", "%s /bin/uptime" % lc]
>>> command
['/usr/bin/bash', '-c', 'LC_ALL=en_US.utf8 /bin/uptime']
>>> uptimedata = subprocess.check_output(command)
>>> uptimedata
b' 13:26:01 up 17:43, 1 user, load average: 0.36, 0.41, 0.44\n'
>>>
I am trying to mount to a CIFS share with space in my linux machine using python like below:
from subprocess import call
t = mount -t cifs -o username=kalair2 "//10.32.135.87/root/Singapore Lab/SYMM" /mnt/share
print t
if os.path.exists("/mnt/share"):
print "/mnt/share path already exists"
else:
call("mkdir /mnt/share")
call("chmod 777 /mnt/share")
print "mnt/share has been created"
call(t)
But ended up in the error... Traceback (most recent call last): File
"mounttest.py", line 12, in
call(t) File "/usr/lib64/python2.6/subprocess.py", line 478, in call
p = Popen(*popenargs, **kwargs) File "/usr/lib64/python2.6/subprocess.py", line 642, in init
errread, errwrite) File "/usr/lib64/python2.6/subprocess.py", line 1234, in _execute_child
raise child_exception OS Error: [Err no 2] No such file or directory
It works if i execute this mount command in shell with space. Can anyone help me with this?
Try escaping your space:
"//10.32.135.87/root/Singapore\ Lab/SYMM"
call should be called with an array of arguments - not a single string for the command with the arguments, as documented here: https://docs.python.org/2/library/subprocess.html
Your command should look like this:
call(["mkdir", "/mnt/share"])
i installed portia and got it to work i annotated some websites (looks really good)
but when i try to run the spiders i get some errors and nothing gets crawled
im running python 2.7.6 on win 7
C:\Python27\Scripts>python portiacrawl C:\portia\slyd\data\projects\new_project
Traceback (most recent call last):
File "portiacrawl", line 7, in <module>
execfile(__file__)
File "C:\portia\slybot\bin\portiacrawl", line 56, in <module>
main()
File "C:\portia\slybot\bin\portiacrawl", line 54, in main
subprocess.call(command_spec)
File "C:\Python27\lib\subprocess.py", line 522, in call
return Popen(*popenargs, **kwargs).wait()
File "C:\Python27\lib\subprocess.py", line 709, in __init__
errread, errwrite)
File "C:\Python27\lib\subprocess.py", line 957, in _execute_child
startupinfo)
WindowsError: [Error 2] O sistema nÒo conseguiu localizar o ficheiro especificado
I am troubleshooting portia on Windows 8.1 and encountered the same error, exactly.
Try running 'python portiacrawl' by itself to determine if there is a subsequent menu. You should be able to see Help info on 'portiacrawl'. I suspect that you need to name the [spider] & [options] as well as change the terminal directory to see the output from the crawler. I suggest trying the following but rename [spider] to actual name of your spider w/o brackets:
Enter into terminal: C:\portia\slyd\data\projects <------Change to proper directory in cmd
Make sure you are in the terminal directory "C:\portia\slyd\data\projects"
The Cmd propmpt should look like: C:\portia\slyd\data\projects> <----waiting for portia initiation.
Enter into terminal:
python portiacrawl C:\portia\slyd\data\projects\new_project [spider] -t csv -o test.csv; or,
python portiacrawl [spider] -t csv -o test.csv
Report back. I am curious as to the terminal response. Did it initiate portiacrawl & return "access is denied."