I'm running a wget in Python via os.system.
Is there anyways to hide the output? I tried
> /dev/null
and tried running the command with a $ in front of it.
Use the subprocess module instead.
Using subprocess.call (which is a helper function for some more advanced subprocess features), you can redirect stdout and stderr to file objects. If you open /dev/null (the os module has os.devnull which is a platform-independant path of the null device that can help), you can hand it to subprocess.call and suppress all output
import os
import subprocess
devnull = open(os.devnull, 'w')
subprocess.call([...], stdout=devnull, stderr=devnull)
Related
When I use the tilde character in the subprocess call I'm getting an error. Otherwise the program runs fine.
Program
#!/usr/local/bin/python
import subprocess
subprocess.call(["ls","-lrth","~"])
Error
ls: cannot access ~: No such file or directory
Tilde (~) is the shell shortcut for the user home directory. If you want to list all the files/directories in the user home directory you can expand the path using
os.path.expanduser.
#!/usr/bin/env python
import subprocess
import os
subprocess.call(["ls","-lrth",os.path.expanduser("~")])
I am creating a Python package, I anticipate that it will be called both by command line and from other scripts. Here is a simplified version of my file structure:
GREProject/
__init__.py
__main__.py
Parsing.py
Parseing.py contains a method, parse(), it takes two arguments, an input file and an output file. I am trying to figure out the proper code for "__main__.py" so that when the following is called from the command line or terminal the arguments will be passed to "parse()":
Python GREProject -i input.file -o output.file
I have tried this numerous ways but they have all met with failure, I do believe I need the "-m" flag for the interpreter but more than that I don't know. Example with the flag:
Python -m GREProject -i input.file -o output.file
When running the later command I receive the following error:
Import by filename is not supported.
Presumably from this line:
from . import Parsing
Ok, turns out this was a problem with my IDE, PyCharm. No idea why I recieved this error but I have setting that fixed it:
Import by filename is not supported.
For the record here are the options I set in my Pycharm project
Script:
GREProject
Script parameters:
-i .\GREProject\pr2.nyc1 -o .\GREProject\Test.pkl
Enviroment variables:
PYTHONUNBUFFERED=1
Python interpreter:
Python 2.7.11 (c:\Python27\python.exe)
Interpreter options:
-m
Working directory:
C:\Users\probert.dan\PycharmProjects
Here is an explanation of the options:
Script: This is the script to run, by default PyCharm will only insert absolute references to .py files, nothing prevents you from manually typing in a relative reference, in this case it is the GREProjects folder.
Script Parameters: These are passed onto the script itself, in this case I am telling my script that the input file is ".\GREProject\pr2.nyc1" which means, look the file "pr2.nyc1" in the "GREProject" directory below the current working directory.
Environment variables: This was set by PyCharm and left unchanged.
Python interpreter: My active interpreter.
Interpreter options: The option here tells python that we are calling a module, python then knows to access the "__main__.py" file.
Working directory: The directory the script is run from, I chose the directory above "GREProject"
For reference here is the contents of my "__main__.py file":
from . import Parsing
import argparse
parser = argparse.ArgumentParser(description='Parse flags.')
parser.add_argument('-i', help='Import file.')
parser.add_argument('-o', help='(Optional) Output file.')
arguments = parser.parse_args()
Parsing.parse(arguments.i, arguments.o)
It is also important to note that debugging in PyCharm is not possible like this. Here is the solution to debugging: Intellij/Pycharm can't debug Python modules
I have a file which contains XML elements in every line, which needs to be converted to JSON. I have written a Python script which does the conversion but runs in serial mode. I have two options to use Hadoop or GNU Parallel, I have tried Hadoop and want to see how GNU could help, will be simple for sure.
My Python code is as follows:
import sys
import json
import xmltodict
with open('/path/sample.xml') as fd:
for line in fd:
o=xmltodict.parse(line)
t=json.dumps(o)
with open('sample.json', 'a') as out:
out.write(t+ "\n")
So can I use GNU parallel to directly work on the huge file or do I need to split it?
Or is this right:
cat sample.xml | parallel python xmltojson.py >sample.json
Thanks
You need to change your Python code to a UNIX filter, i.e. a program that reads from standard input (stdin) and writes to standard output (stdout). Untested:
import fileinput
import sys
import json
import xmltodict
for line in fileinput.input():
o=xmltodict.parse(line)
t=json.dumps(o)
print t + "\n"
Then you use --pipepart in GNU Parallel:
parallel --pipepart -a sample.xml --block -1 python my_script.py
I am running an executable from cmd:
*.exe input.inp
I want to run it using python and tried following:
os.system('"*.exe"')
But don't know how to specify input file as well. Any suggestions?
import os
from subprocess import Popen, PIPE
p = Popen('fortranExecutable', stdin=PIPE) #NOTE: no shell=True here
p.communicate(os.linesep.join(["input 1", "input 2"]))
For more please refer to:
Using Python to run executable and fill in user input
I had to launch cmd window and specify input file location from within Python script. This page was really helpful in getting it done.
I used Popen(['cmd', '/K', 'command']) from above page and replaced '/K' with '/C' in it to run and close the cmd window.
import os
os.system(r'pathToExe.exe inputFileOrWhateverOtherCommand')
I am trying to get a prompt that will ask for my password but when I try to call getpass.getpass() it just freezes. I am running on Windows 7 64 bit using Python 2.7 on Canopy.
import sys
import getpass
p = getpass.getpass()
print p
It is correct that Python "effectively freezes because it can't receive the input from standard input", however, for windows you can prefix your command with winpty. Then password can be inputted correctly when started like:
winpty python fileToExecute.py
winpty provides a interface similar to a Unix pty-master in a way that communication is also possible from windows terminals.
Python "effectively freezes because it can't receive the input from standard input". See https://support.enthought.com/entries/22157050-Canopy-Python-prompt-QtConsole-Can-t-run-getpass-or-interactive-OS-shell-commands-or-Windows-process
The fix is to use a different interpreter. I switched to IDLE and fixed the issue.
Faced the same issue with getpass (mingw64) and found this simple solution.
os.system("stty -echo")
password = input('Enter Password:')
os.system("stty echo")
print("")
getpass() will freeze if python is unable to read properly from standard input.
This can happen on e.g. some Windows terminals, such as using git bash.
You can use the sys module to detect if this will happen, to avoid hanging:
import getpass
import sys
# ...
if not sys.stdin.isatty():
# notify user that they have a bad terminal
# perhaps if os.name == 'nt': , prompt them to use winpty?
return
else:
password = getpass.getpass()
# ...
I also had this on mac with both Jupyter Lab and Jupyter Notebook. For me the issue was caused by the variable name.
Naming the variable PG_REMOTEPASSWORD caused a hang but PG_PASSWORD & PG_ABCPass didn't. I don't know why that is an issue, there's nothing in docs about restrictions on what a variable can be called.
My setup up is Anaconda running Python 3.7.7
In pycharm RUN i had similar issue. When i opened files using Terminal "CMD" issue resolved.