In my python script, I am reading one text file. For that file, I am giving path to command line in UNIX as follows:
python My_script.py --d /fruit/apple/data1.txt
I am going to read one more file in same script. So I just wanted to know how to pass 2 arguments to get path to 2 files.
I have following code which is working perfectly for one argument.
parser=argparse.ArgumentParser()
parser.add_argument('--d', '--directory', required=True, action='store', dest='directory', default=False, help="provide directory name")
args=parser.parse_args()
file_apple=args.directory
A=open(file_apple)
file1=A.read()
so in my unix command line I write following and script runs successfully
python My_script.py --d /fruit/apple/data1.txt
Goal is to provide second argument as follows and want to read that file as the first one.
python My_script.py --d /fruit/apple/data1.txt --d /fruit/orange/data2.txt
I will appreciate your help on this.
You can make use of nargs.
parser=argparse.ArgumentParser()
parser.add_argument('-d', '--directory', nargs='+' required=True, action='store', dest='directory', default=False, help="provide directory name")
args=parser.parse_args()
file_apple=args.directory
print file_apple
...
I have given nargs value as + which means 1 or many arguments for that command. So, you have to give at least one file path argument.
If you are sure that you are going to have only two or some fixed number always, then you can specify that also like nargs = 3
Now file_apple will be a variable containing list of paths you passed.
$ python My_script.py -d /fruit/apple/data1.txt
['/fruit/apple/data1.txt']
and:
$ python My_script.py -d /fruit/apple/data1.txt /fruit/orange/data2.txt
['/fruit/apple/data1.txt', '/fruit/orange/data2.txt']
PS: conventionally single dash is used for single character flags and doubledash for multi characters. like -d or --directory
Related
I'm using Python 2.7 and I'm trying to accomplish a shell like behavior using argparse.
My issue, in general, that I cannot seem to find a way, in Python 2.7, to use argparse's subparsers as optional.
It's kind of hard to explain my issue so I'll describe what I require from my program.
The program has 2 modes of work:
Starting the program with a given command (each command has it's own
additional arguments) and additional arguments will run a specific
task.
Starting the program without a command will start a shell-like program that can take a line of arguments and process them as if the
program was called with the given line as it's arguments.
So, if for example my program supports 'cmd1' and 'cmd2' commands, I could use it like so:
python program.py cmd1 additional_args1
python program.py cmd2 additional_args2
or with shell mode:
python program.py
cmd1 additional_args1
cmd2 additional_args2
quit
In addition, I also want my program to be able to take optional global arguments that will effect all commands.
For that I'm using argparse like so (This is a pure example):
parser = argparse.ArgumentParser(description="{} - Version {}".format(PROGRAM_NAME, PROGRAM_VERSION))
parser.add_argument("-i", "--info", help="Display more information")
subparsers = parser.add_subparsers()
parserCmd1 = subparsers.add_parser("cmd1", help="First Command")
parserCmd1.set_defaults(func=cmd1)
parserCmd2 = subparsers.add_parser("cmd2", help="Second Command")
parserCmd2.add_argument("-o", "--output", help="Redirect Output")
parserCmd2.set_defaults(func=cmd2)
So I can call cmd1 (with no additional args) or cmd2 (with or without -o flag). And for both I can add flag -i to display even more information of the called command.
My issue is that I cannot activate shell mode, because I have to provide cmd1 or cmd2 as an argument (because of using subparsers which are mandatory)
Restrictions:
I cannot use Python 3 (I know it can be easily done there)
Because of global optional arguments I cannot check to see if I get no arguments to skip arg parsing.
I don't want to add a new command to call shell, it must be when providing no command at all
So how can I achieve This kind of behavior with argparse and python 2.7?
Another idea is to use a 2 stage parsing. One handles 'globals', returning strings it can't handle. Then conditionally handle the extras with subparsers.
import argparse
def cmd1(args):
print('cmd1', args)
def cmd2(args):
print('cmd2', args)
parser1 = argparse.ArgumentParser()
parser1.add_argument("-i", "--info", help="Display more information")
parser2 = argparse.ArgumentParser()
subparsers = parser2.add_subparsers(dest='cmd')
parserCmd1 = subparsers.add_parser("cmd1", help="First Command")
parserCmd1.set_defaults(func=cmd1)
parserCmd2 = subparsers.add_parser("cmd2", help="Second Command")
parserCmd2.add_argument("-o", "--output", help="Redirect Output")
parserCmd2.set_defaults(func=cmd2)
args, extras = parser1.parse_known_args()
if len(extras)>0 and extras[0] in ['cmd1','cmd2']:
args = parser2.parse_args(extras, namespace=args)
args.func(args)
else:
print('doing system with', args, extras)
sample runs:
0901:~/mypy$ python stack46667843.py -i info
('doing system with', Namespace(info='info'), [])
0901:~/mypy$ python stack46667843.py -i info extras for sys
('doing system with', Namespace(info='info'), ['extras', 'for', 'sys'])
0901:~/mypy$ python stack46667843.py -i info cmd1
('cmd1', Namespace(cmd='cmd1', func=<function cmd1 at 0xb74b025c>, info='info'))
0901:~/mypy$ python stack46667843.py -i info cmd2 -o out
('cmd2', Namespace(cmd='cmd2', func=<function cmd2 at 0xb719ebc4>, info='info', output='out'))
0901:~/mypy$
A bug/issue (with links) on the topic of 'optional' subparsers.
https://bugs.python.org/issue29298
Notice that this has a recent pull request.
With your script and the addition of
args = parser.parse_args()
print(args)
results are
1008:~/mypy$ python3 stack46667843.py
Namespace(info=None)
1009:~/mypy$ python2 stack46667843.py
usage: stack46667843.py [-h] [-i INFO] {cmd1,cmd2} ...
stack46667843.py: error: too few arguments
1009:~/mypy$ python2 stack46667843.py cmd1
Namespace(func=<function cmd1 at 0xb748825c>, info=None)
1011:~/mypy$ python3 stack46667843.py cmd1
Namespace(func=<function cmd1 at 0xb7134dac>, info=None)
I thought the 'optional' subparsers affected both Py2 and 3 versions, but apparently it doesn't. I'll have to look at the code to verify why.
In both languages, subparsers.required is False. If I set it to true
subparsers.required=True
(and add a dest to the subparsers definition), the PY3 error message is
1031:~/mypy$ python3 stack46667843.py
usage: stack46667843.py [-h] [-i INFO] {cmd1,cmd2} ...
stack46667843.py: error: the following arguments are required: cmd
So there's a difference in how the 2 versions test for required arguments. Py3 pays attention to the required attribute; Py2 (apparently) uses the earlier method of checking whether the positionals list is empty or not.
Checking for required arguments occurs near the end of parser._parse_known_args.
Python2.7 includes
# if we didn't use all the Positional objects, there were too few
# arg strings supplied.
if positionals:
self.error(_('too few arguments'))
before the iteration that checks action.required. That's what's catching the missing cmd and saying too few arguments.
So a kludge is to edit your argparse.py and remove that block so it matches the corresponding section of the Py3 version.
I am a new user of Python. I got to learn a way of changing value of a parameter in a single file. The script:
#####test.py##########
from sys import argv
script,filename,sigma = argv
file_data = open(filename,'r')
txt = file_data.read()
txt=txt.replace('3.7',sigma)
file_data = open(filename,'w')
file_data.write(txt)
file_data.close()
It's run in command line with test.txt as
test.py test.txt 2.
3.7 is replaced by 2 in test.txt, as a result.
Now if I want to do the same for all the .txt files in the directory e.g.
test.py *.txt 2
what are the suggested modifications?
Your suggestions are highly appreciated.
Hafiz.
bash (or whatever your shell is) will expand the *.txt (to test0.txt test1.txt ... or whatever the *.txt files in your current directory are called) before passing it to your python script. your python script will therefore get many arguments (and not just 2 as you expect). print sys.argv to inspect.
you could solve that in bash itself with something like
for name in *.txt; do test.py ${name} 2; done
otherwise you would need to treat sys.argv differently in python and allow for more than 2 arguments.
Importing glob solved that issue. But I've got some queries.
Query 1:
I'm rewriting my code as:
#####test.py##########
from sys import argv
script,filename,sigma = argv
file_data = open(filename,'r')
txt = file_data.read()
txt=txt.replace('3.7'|'3',sigma) #gives syntax error
file_data = open(filename,'w')
file_data.write(txt)
file_data.close()
I want to replace 3.7 or 3 by sigma. What will be the corrected code?
Query 2:
I'm rewriting it in the following manner:
#####test.py##########
from sys import argv
script,filename,sigma = argv
file_data = open(filename,'r')
txt = file_data.read()
txt=txt.replace('x="2"','x=sigma')
file_data = open(filename,'w')
file_data.write(txt)
file_data.close()
With
py test.py test.txt 3.
I get x=sigma, but I want to get x=3
What'd be the modification?
Regards,
Hafiz
I am new to GitPython and I am trying to get the content of a file within a commit. I am able to get each file from a specific commit, but I am getting an error each time I run the command. Now, I know that the file exist in GitPython, but each time I run my program, I am getting the following error:
returned non-zero exit status 1
I am using Python 2.7.6 and Ubuntu Linux 14.04.
I know that the file exist, since I also go directly into Git from the command line, check out the respective commit, search for the file, and find it. I also run the cat command on it, and the file contents are displayed. Many times when the error shows up, it says that the file in question does not exist. I am trying to go through each commit with GitPython, get every blob or file from each individual commit, and run an external Java program on the content of that file. The Java program is designed to return a string to Python. To capture the string returned from my Java code, I am also using subprocess.check_output. Any help will be greatly appreciated.
I tried passing in the command as a list:
cmd = ['java', '-classpath', '/home/rahkeemg/workspace/CSCI499_Java/bin/:/usr/local/lib/*:', 'java_gram.mainJava','absolute/path/to/file']
subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=False)
And I have also tried passing the command as a string:
subprocess.check_output('java -classpath /home/rahkeemg/workspace/CSCI499_Java/bin/:/usr/local/lib/*: java_gram.mainJava {file}'.format(file=entry.abspath.strip()), shell=True)
Is it possible to access the contents of a file from GitPython?
For example, say there is a commit and it has one file foo.java
In that file is the following lines of code:
foo.java
import java.io.FileInputStream;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;
public class foo{
public static void main(String[] args) throws Exception{}
}
I want to access everything in the file and run an external program on it.
Any help would be greatly appreciated. Below is a piece of the code I am using to do so
#! usr/bin/env python
__author__ = 'rahkeemg'
from git import *
import git, json, subprocess, re
git_dir = '/home/rahkeemg/Documents/GitRepositories/WhereHows'
# make an instance of the repository from specified path
repo = Repo(path=git_dir)
heads = repo.heads # obtain the different repositories
master = heads.master # get the master repository
print master
# get all of the commits on the master branch
commits = list(repo.iter_commits(master))
cmd = ['java', '-classpath', '/home/rahkeemg/workspace/CSCI499_Java/bin/:/usr/local/lib/*:', 'java_gram.mainJava']
# start at the very 1st commit, or start at commit 0
for i in range(len(commits) - 1, 0, -1):
commit = commits[i]
commit_num = len(commits) - 1 - i
print commit_num, ": ", commit.hexsha, '\n', commit.message, '\n'
for entry in commit.tree.traverse():
if re.search(r'\.java', entry.path):
current_file = str(entry.abspath.strip())
# add the current file or blob to the list for the command to run
cmd.append(current_file)
print entry.abspath
try:
# This is the scenario where I pass arguments into command as a string
print subprocess.check_output('java -classpath /home/rahkeemg/workspace/CSCI499_Java/bin/:/usr/local/lib/*: java_gram.mainJava {file}'.format(file=entry.abspath.strip()), shell=True)
# scenario where I pass arguments into command as a list
j_response = subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=False)
except subprocess.CalledProcessError as e:
print "Error on file: ", current_file
# Use pop on list to remove the last string, which is the selected file at the moment, to make place for the next file.
cmd.pop()
First of all, when you traverse the commit history like this, the file will not be checked out. All you get is the filename, maybe leading to the file or maybe not, but certainly it will not lead to the file from different revision than currently checked-out.
However, there is a solution to this. Remember that in principle, anything you could do with some git command, you can do with GitPython.
To get file contents from specific revision, you can do the following, which I've taken from that page:
git show <treeish>:<file>
therefore, in GitPython:
file_contents = repo.git.show('{}:{}'.format(commit.hexsha, entry.path))
However, that still wouldn't make the file appear on disk. If you need some real path for the file, you can use tempfile:
f = tempfile.NamedTemporaryFile(delete=False)
f.write(file_contents)
f.close()
# at this point file with name f.name contains contents of
# the file from path entry.path at revision commit.hexsha
# your program launch goes here, use f.name as filename to be read
os.unlink(f.name) # delete the temp file
I m trying to read two files and replace content of one file with content of other file in files present in folder which also has sub directories.
But its tell sub process not defined.
i'm new to python and shell script can anybody help me with this please?
import os
import sys
import os.path
f = open ( "file1.txt",'r')
g = open ( "file2.txt",'r')
text1=f.readlines()
text2=g.readlines()
i = 0;
for line in text1:
l = line.replace("\r\n", "")
t = text2[i].replace("\r\n", "")
args = "find . -name *.tml"
Path = subprocess.Popen( args , shell=True )
os.system(" sed -r -i 's/" + l + "/" + t + "/g' " + Path)
i = i + 1;
To specifically address your actual error, you need to import the subprocess module as you are making use of it (oddly) in your code:
import subprocess
After that, you will find more problems. I will try and keep it as simple as possible with my suggestions. Code first, then I will break it down. Keep in mind, there are more robust ways to accomplish this task. But I am doing my best to keep in mind your experience level and making it make your current approach as closely as possible.
import subprocess
import sys
# 1
results = subprocess.Popen("find . -name '*.tml'",
shell=True, stdout=subprocess.PIPE)
if results.wait() != 0:
print "error trying to find tml files"
sys.exit(1)
# 2
tml_files = []
for tml in results.stdout:
tml_files.append(tml.strip())
if not tml_files:
print "no tml files found"
sys.exit(0)
tml_string = " ".join(tml_files)
# 3
with open ("file1.txt") as f, open("file2.txt") as g:
while True:
# 4
f_line = f.readline()
if not f_line:
break
g_line = g.readline()
if not g_line:
break
f_line = f_line.strip()
g_line = g_line.strip()
if not f_line or not g_line:
continue
# 5
cmd = "sed -i -e 's/%s/%s/g' %s" % \
(f_line.strip(), g_line.strip(), tml_string)
ret = subprocess.Popen(cmd, shell=True).wait()
if ret != 0:
print "error doing string replacement"
sys.exit(1)
You do not need to read in your entire files at once. If they are large this could be a lot of memory. You can consume a line at a time, and you can also make use of what is called "context managers" when you open the files. This will ensure they close properly no matter what happens:
We start with a subprocess command that is run only once to find all your .tml files. Your version had the same command being run multiple times. If the search path is the same, then we only need it once. This checks the exit code of the command and quits if it failed.
We loop over stdout on the subprocess command, and add the stripped lines to a list. This is a more robust way of your replace("\r\n"). It removes whitespace. A "list comprehension" would be better suited here (down the line). If we didn't find any tml files, then we have no work to do, so we exit. Otherwise, we join them together in a space-separated string to be suitable for our command later.
This is called "context managers". You can open the file in a way that no matter what they will be closed properly. The file is open for the length of the context within that code block. We are going to loop forever, and break when appropriate.
We pull a line, one at a time, from each file. If either line is blank, we reached the end of the file and cannot do any more work, so we break out. We then strip the newlines, and if either string is empty (blank line) we still can't do any work, but we just continue to the next available line.
A modified version of your sed command. We construct the command string on each loop for the source and replacement strings, and tack on the tml file string. Bear in mind this is a very naive approach to the replacement. It really expects your replacement strings to be safe characters and not break the s///g sed format. But we run that with another subprocess command. The wait() simply waits for the return code, and we check it for an error. This approach replaces your os.system() version.
Hope this helps. Eventually you can improve this to do more checking and safe operations.
From a windows application written on C++ or python, how can I execute arbitrary shell commands?
My installation of Cygwin is normally launched from the following bat file:
#echo off
C:
chdir C:\cygwin\bin
bash --login -i
From Python, run bash with os.system, os.popen or subprocess and pass the appropriate command-line arguments.
os.system(r'C:\cygwin\bin\bash --login -c "some bash commands"')
The following function will run Cygwin's Bash program while making sure the bin directory is in the system path, so you have access to non-built-in commands. This is an alternative to using the login (-l) option, which may redirect you to your home directory.
def cygwin(command):
"""
Run a Bash command with Cygwin and return output.
"""
# Find Cygwin binary directory
for cygwin_bin in [r'C:\cygwin\bin', r'C:\cygwin64\bin']:
if os.path.isdir(cygwin_bin):
break
else:
raise RuntimeError('Cygwin not found!')
# Make sure Cygwin binary directory in path
if cygwin_bin not in os.environ['PATH']:
os.environ['PATH'] += ';' + cygwin_bin
# Launch Bash
p = subprocess.Popen(
args=['bash', '-c', command],
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
p.wait()
# Raise exception if return code indicates error
if p.returncode != 0:
raise RuntimeError(p.stderr.read().rstrip())
# Remove trailing newline from output
return (p.stdout.read() + p.stderr.read()).rstrip()
Example use:
print cygwin('pwd')
print cygwin('ls -l')
print cygwin(r'dos2unix $(cygpath -u "C:\some\file.txt")')
print cygwin(r'md5sum $(cygpath -u "C:\another\file")').split(' ')[0]
Bash should accept a command from args when using the -c flag:
C:\cygwin\bin\bash.exe -c "somecommand"
Combine that with C++'s exec or python's os.system to run the command.