I am trying to mount to a CIFS share with space in my linux machine using python like below:
from subprocess import call
t = mount -t cifs -o username=kalair2 "//10.32.135.87/root/Singapore Lab/SYMM" /mnt/share
print t
if os.path.exists("/mnt/share"):
print "/mnt/share path already exists"
else:
call("mkdir /mnt/share")
call("chmod 777 /mnt/share")
print "mnt/share has been created"
call(t)
But ended up in the error... Traceback (most recent call last): File
"mounttest.py", line 12, in
call(t) File "/usr/lib64/python2.6/subprocess.py", line 478, in call
p = Popen(*popenargs, **kwargs) File "/usr/lib64/python2.6/subprocess.py", line 642, in init
errread, errwrite) File "/usr/lib64/python2.6/subprocess.py", line 1234, in _execute_child
raise child_exception OS Error: [Err no 2] No such file or directory
It works if i execute this mount command in shell with space. Can anyone help me with this?
Try escaping your space:
"//10.32.135.87/root/Singapore\ Lab/SYMM"
call should be called with an array of arguments - not a single string for the command with the arguments, as documented here: https://docs.python.org/2/library/subprocess.html
Your command should look like this:
call(["mkdir", "/mnt/share"])
Related
from a Python script I want to call a few external programs and use their output inside the script. Now I found that this script is running on my local system, but gets problems on another one that uses another language. So instead of checking the output in different languages, I want to call these programs with a specified "LC_ALL=en_US.utf8" as first argument. But it seems that subprocess does not like this:
lc = "LC_ALL=en_US.utf8"
uptimedata = subprocess.check_output([lc, "/bin/uptime"])
When I run the script, I get:
Traceback (most recent call last):
File "./serverwatch.py", line 22, in <module>
uptimedata = subprocess.check_output([lc, "/bin/uptime"])
File "/usr/lib/python2.7/subprocess.py", line 537, in check_output
process = Popen(stdout=PIPE, *popenargs, **kwargs)
File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1259, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
I understand that subprocess.check_output() wants the first argument to be the command to be called. So how can I run the command with the changed LC?
This is Python 2.7.3 on Debian Wheezy. The other system is running the same versions.
Execute your command through bash:
>>> lc = "LC_ALL=en_US.utf8"
>>> command = ["/usr/bin/bash", "-c", "%s /bin/uptime" % lc]
>>> command
['/usr/bin/bash', '-c', 'LC_ALL=en_US.utf8 /bin/uptime']
>>> uptimedata = subprocess.check_output(command)
>>> uptimedata
b' 13:26:01 up 17:43, 1 user, load average: 0.36, 0.41, 0.44\n'
>>>
i installed portia and got it to work i annotated some websites (looks really good)
but when i try to run the spiders i get some errors and nothing gets crawled
im running python 2.7.6 on win 7
C:\Python27\Scripts>python portiacrawl C:\portia\slyd\data\projects\new_project
Traceback (most recent call last):
File "portiacrawl", line 7, in <module>
execfile(__file__)
File "C:\portia\slybot\bin\portiacrawl", line 56, in <module>
main()
File "C:\portia\slybot\bin\portiacrawl", line 54, in main
subprocess.call(command_spec)
File "C:\Python27\lib\subprocess.py", line 522, in call
return Popen(*popenargs, **kwargs).wait()
File "C:\Python27\lib\subprocess.py", line 709, in __init__
errread, errwrite)
File "C:\Python27\lib\subprocess.py", line 957, in _execute_child
startupinfo)
WindowsError: [Error 2] O sistema nÒo conseguiu localizar o ficheiro especificado
I am troubleshooting portia on Windows 8.1 and encountered the same error, exactly.
Try running 'python portiacrawl' by itself to determine if there is a subsequent menu. You should be able to see Help info on 'portiacrawl'. I suspect that you need to name the [spider] & [options] as well as change the terminal directory to see the output from the crawler. I suggest trying the following but rename [spider] to actual name of your spider w/o brackets:
Enter into terminal: C:\portia\slyd\data\projects <------Change to proper directory in cmd
Make sure you are in the terminal directory "C:\portia\slyd\data\projects"
The Cmd propmpt should look like: C:\portia\slyd\data\projects> <----waiting for portia initiation.
Enter into terminal:
python portiacrawl C:\portia\slyd\data\projects\new_project [spider] -t csv -o test.csv; or,
python portiacrawl [spider] -t csv -o test.csv
Report back. I am curious as to the terminal response. Did it initiate portiacrawl & return "access is denied."
I have installed package envoy. I ran the script but a windows error occured .I commented envoy.run then the full script runs but when I remove the comment, error occurs.
import envoy
# This data is checked-in to the repository and is a compressed
# version of the output from Example 3
F = 'resources/ch06-mailboxes/data/enron.mbox.json.bz2'
r = envoy.run("bunzip2 %s" % (F,))
print r.std_out
print r.std_err
traceback of script:
Exception in thread Thread-9:
Traceback (most recent call last):
File "C:\Users\sachin\Anaconda\lib\threading.py", line 810, in __bootstrap_inner
self.run()
File "C:\Users\sachin\Anaconda\lib\threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "C:\Users\sachin\Anaconda\lib\site-packages\envoy\core.py", line 40, in target
bufsize=0,
File "C:\Users\sachin\Anaconda\lib\subprocess.py", line 709, in __init__
errread, errwrite)
File "C:\Users\sachin\Anaconda\lib\subprocess.py", line 957, in _execute_child
startupinfo)
WindowsError: [Error 2] The system cannot find the file specified
Please try this:
F = os.path.join(os.getcwd(), 'resources/ch06-mailboxes/data/enron.mbox.json.bz2')
I am trying to learn mapreduce program using python mrjob. I am getting following error:
Traceback:
dumping stdin to local file /tmp/pyes_mrjob.testuser.20131004.103251.998597/STDIN
Making directory hdfs:///user/testuser/tmp/mrjob/pyes_mrjob.user.20131004.103251.998597/files/ on HDFS
> /usr/lib/hadoop-mapreduce/bin/hadoop fs -mkdir hdfs:///user/testuser/tmp/mrjob/pyes_mrjob.testuser.20131004.103251.998597/files/
Traceback (most recent call last):
File "pyes_mrjob.py", line 34, in <module>
MRWordFrequencyCount.run()
File "/usr/local/lib/python2.7/dist-packages/mrjob/job.py", line 500, in run
mr_job.execute()
File "/usr/local/lib/python2.7/dist-packages/mrjob/job.py", line 518, in execute
super(MRJob, self).execute()
File "/usr/local/lib/python2.7/dist-packages/mrjob/launch.py", line 146, in execute
self.run_job()
File "/usr/local/lib/python2.7/dist-packages/mrjob/launch.py", line 207, in run_job
runner.run()
File "/usr/local/lib/python2.7/dist-packages/mrjob/runner.py", line 458, in run
self._run()
File "/usr/local/lib/python2.7/dist-packages/mrjob/hadoop.py", line 236, in _run
self._upload_local_files_to_hdfs()
File "/usr/local/lib/python2.7/dist-packages/mrjob/hadoop.py", line 263, in _upload_local_files_to_hdfs
self._mkdir_on_hdfs(self._upload_mgr.prefix)
File "/usr/local/lib/python2.7/dist-packages/mrjob/hadoop.py", line 271, in _mkdir_on_hdfs
self.invoke_hadoop(['fs', '-mkdir', path])
File "/usr/local/lib/python2.7/dist-packages/mrjob/fs/hadoop.py", line 81, in invoke_hadoop
proc = Popen(args, stdout=PIPE, stderr=PIPE)
File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
I executed the command manually its working fine there but when i try to execute my program its not working.
Since just started learning can someone suggest what library i have to choose. According to some blogs somelibraries has good documention and some libraries has better perfomance and .... I came across below post which looks older
http://blog.cloudera.com/blog/2013/01/a-guide-to-python-frameworks-for-hadoop/
But so many libraries got updates recently. So can some suggest me library i can start with..
i guess this problem is caused by the way how mrjob calls "hadoop fs -mkdir", if the parent dir of the targeted dir you want to make doesn't exist, -mkdir will fail. that means you have to use "hadoop fs -mkdir -p [path]". Ultimately, you will need to modify mrjob library manually in [path of mrjob install](mine is /usr/lib/python2.6/site-packages/mrjob)/hadoop.py at line 271:
self.invoke_hadoop(['fs', '-mkdir', path])
to
self.invoke_hadoop(['fs', '-mkdir', '-p', path])
Good Luck!
It looks like you set your HADOOP_HOME to "/usr/lib/hadoop-mapreduce". However, this is wrong and it should be set to "/usr/lib/hadoop".
Also, if you get an error saying that the hadoop-streaming.jar could not be found, create a symlink in "/usr/lib/hadoop" to this jar as follows:
sudo ln -s /usr/lib/hadoop-mapreduce/hadoop-streaming.jar /usr/lib/hadoop
Python 2.7.3 with ubuntu:
Trying to run a program, (youtube-dl in this example) with subprocess.call with some arguments, I encounter the following issue. consider the following script:
try.py:
#!/usr/bin/python
from subprocess import call
url = "https://www.youtube.com/watch?v=8SbUC-UaAxE"
myArray = ['./youtube-dl {}'.format(url),'-x','--audio-format mp3']
#print the array before executing:
for item in myArray:
print item,
#execute:
call(myArray)
This script prints outputs:
oris#oris:~/Desktop/YouTube/backend$ ./try.py
./youtube-dl https://www.youtube.com/watch?v=8SbUC-UaAxE -x --audio-format mp3
Traceback (most recent call last):
File "./try.py", line 16, in <module>
call(myArray)
File "/usr/lib/python2.7/subprocess.py", line 493, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1259, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Strangly, On the first line of the output I see the script does output ./youtube-dl https://www.youtube.com/watch?v=8SbUC-UaAxE -x --audio-format mp3 This command runs perfectly from bash directly, but produces an OSError from python. I also thought maybe to try and supply the url as an argument like this:
myArray = ['./youtube-dl', url,'-x','--audio-format mp3']
but than youtube-dl has an error of incorrect usage:
oris#oris:~/Desktop/YouTube/backend$ ./try.py
Usage: youtube-dl [options] url [url...]
youtube-dl: error: no such option: --audio-format mp3
./youtube-dl https://www.youtube.com/watch?v=8SbUC-UaAxE -x --audio-format mp3
I've read youtube-dl source to see how it handles the supplied arguments with optparse. I don't see the url as an argument there, so I'm guessing i'm passing the arguments to subprocess.call incorrectly.
Side note: another thing I find odd is the fact the print here takes effect after the subprocess call, as opposed to their order on my script. Is something happening here asynchronously?
What am I missing here? Many thanks
--audio-format and mp3 should be passed as separated arguments:
myArray = ['./youtube-dl', url, '-x', '--audio-format', 'mp3']