When using the python logger, my program starts fine with logging but at some point the log output starts outputting lines twice, looking like this:
DEBUG:pluginbrowser:Scanning plugs
DEBUG:pluginbrowser:Doing good stuff
....
INFO:pluginbrowser:=== doing something ===
=== doing something ===
Currently all my python files contain the line
LOGGER = logging.getLogger(__name__)
LOGGER.setLevel(logging.DEBUG)
Thus, the doubled message comes from the pluginbrowser.py file. Examining this, I found out that at the beginning of my program, the same files output some log without double-ing the lines. I tried to find out at which point exactly it happens but I am somehow stuck here.
I also read log messages appearing twice with Python Logging but I am not using configure_logging at all.
Looks like you add an additional handler somewhere along the way.
I would search for addHandler in the code. You could also debug and watch logger.root.handlers
This is how I could reproduce your effects:
In [1]: import logging
In [2]: logging.basicConfig(level=logging.DEBUG)
In [3]: logger = logging.getLogger('pluginbrowser')
In [4]: logger.info('=== doing something ===')
INFO:pluginbrowser:=== doing something ===
In [5]: logging.root.addHandler(logging.StreamHandler())
In [6]: logger.info('=== doing something ===')
INFO:pluginbrowser:=== doing something ===
=== doing something ===
I am new to GitPython and I am trying to get the content of a file within a commit. I am able to get each file from a specific commit, but I am getting an error each time I run the command. Now, I know that the file exist in GitPython, but each time I run my program, I am getting the following error:
returned non-zero exit status 1
I am using Python 2.7.6 and Ubuntu Linux 14.04.
I know that the file exist, since I also go directly into Git from the command line, check out the respective commit, search for the file, and find it. I also run the cat command on it, and the file contents are displayed. Many times when the error shows up, it says that the file in question does not exist. I am trying to go through each commit with GitPython, get every blob or file from each individual commit, and run an external Java program on the content of that file. The Java program is designed to return a string to Python. To capture the string returned from my Java code, I am also using subprocess.check_output. Any help will be greatly appreciated.
I tried passing in the command as a list:
cmd = ['java', '-classpath', '/home/rahkeemg/workspace/CSCI499_Java/bin/:/usr/local/lib/*:', 'java_gram.mainJava','absolute/path/to/file']
subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=False)
And I have also tried passing the command as a string:
subprocess.check_output('java -classpath /home/rahkeemg/workspace/CSCI499_Java/bin/:/usr/local/lib/*: java_gram.mainJava {file}'.format(file=entry.abspath.strip()), shell=True)
Is it possible to access the contents of a file from GitPython?
For example, say there is a commit and it has one file foo.java
In that file is the following lines of code:
foo.java
import java.io.FileInputStream;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;
public class foo{
public static void main(String[] args) throws Exception{}
}
I want to access everything in the file and run an external program on it.
Any help would be greatly appreciated. Below is a piece of the code I am using to do so
#! usr/bin/env python
__author__ = 'rahkeemg'
from git import *
import git, json, subprocess, re
git_dir = '/home/rahkeemg/Documents/GitRepositories/WhereHows'
# make an instance of the repository from specified path
repo = Repo(path=git_dir)
heads = repo.heads # obtain the different repositories
master = heads.master # get the master repository
print master
# get all of the commits on the master branch
commits = list(repo.iter_commits(master))
cmd = ['java', '-classpath', '/home/rahkeemg/workspace/CSCI499_Java/bin/:/usr/local/lib/*:', 'java_gram.mainJava']
# start at the very 1st commit, or start at commit 0
for i in range(len(commits) - 1, 0, -1):
commit = commits[i]
commit_num = len(commits) - 1 - i
print commit_num, ": ", commit.hexsha, '\n', commit.message, '\n'
for entry in commit.tree.traverse():
if re.search(r'\.java', entry.path):
current_file = str(entry.abspath.strip())
# add the current file or blob to the list for the command to run
cmd.append(current_file)
print entry.abspath
try:
# This is the scenario where I pass arguments into command as a string
print subprocess.check_output('java -classpath /home/rahkeemg/workspace/CSCI499_Java/bin/:/usr/local/lib/*: java_gram.mainJava {file}'.format(file=entry.abspath.strip()), shell=True)
# scenario where I pass arguments into command as a list
j_response = subprocess.check_output(cmd, stderr=subprocess.STDOUT, shell=False)
except subprocess.CalledProcessError as e:
print "Error on file: ", current_file
# Use pop on list to remove the last string, which is the selected file at the moment, to make place for the next file.
cmd.pop()
First of all, when you traverse the commit history like this, the file will not be checked out. All you get is the filename, maybe leading to the file or maybe not, but certainly it will not lead to the file from different revision than currently checked-out.
However, there is a solution to this. Remember that in principle, anything you could do with some git command, you can do with GitPython.
To get file contents from specific revision, you can do the following, which I've taken from that page:
git show <treeish>:<file>
therefore, in GitPython:
file_contents = repo.git.show('{}:{}'.format(commit.hexsha, entry.path))
However, that still wouldn't make the file appear on disk. If you need some real path for the file, you can use tempfile:
f = tempfile.NamedTemporaryFile(delete=False)
f.write(file_contents)
f.close()
# at this point file with name f.name contains contents of
# the file from path entry.path at revision commit.hexsha
# your program launch goes here, use f.name as filename to be read
os.unlink(f.name) # delete the temp file
I am creating a log using the following statement:
module = sys.modules['__main__'].__file__
logging.basicConfig(stream=sys.stderr, level=logging.DEBUG,
format='%(asctime)s %(name)s (%(levelname)s): %(message)s')
log = logging.getLogger(module)
I then have an option on my script to write the output to a file using an argument -o. I think it's an issue with this statement that occurs when a person specifies that they want the log written to disk:
if arguments.o:
fh = RotatingFileHandler(arguments.o, mode='a', maxBytes=2*1024*1024,
backupCount=2, encoding=None, delay=0)
fh.setLevel(logging.DEBUG)
log.addHandler(fh)
Without adding an additional datetime statement to the text string, is there a way to do this using the format parameter?
Here is what is produced on print:
2015-11-06 07:27:13,592 C:\GIS\move_content\src\updatecontent.py (INFO): Reading Configuration file
Here is what is the configuration outputs:
Reading Configuration file
Thank you
Answer I found:
Based on my one response I was able to add this to my code, and the output now looks correct:
# create formatter and add it to the handlers
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
fh.setFormatter(formatter)
File handler uses different format string, you should set it manually to file handler too.
Example:
fh.setFormatter(formatter)
Cookbook for Python 3 (Python 2 is the same for this question)
Similar questions were caused by the logger getting called twice. So maybe in my case it is caused by the second getLogger(). Any ideas how I can fix it?
import logging
import logging.handlers
logger = logging.getLogger("")
logger.setLevel(logging.DEBUG)
handler = logging.handlers.RotatingFileHandler(
"the_log.log", maxBytes=3000000, backupCount=2)
formatter = logging.Formatter(
'[%(asctime)s] {%(filename)s:%(lineno)d} %(levelname)s - %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)
# This is required to print messages to the console as well as the log file.
logging.getLogger().addHandler(logging.StreamHandler())
Using a config file. e.g. logging.config.fileConfig('logging.ini')
[logger_root]
level=ERROR
handlers=stream_handler
[logger_webserver]
level=DEBUG
handlers=stream_handler
qualname=webserver
propagate=0
You have to set logger.propagate = 0 (Python 3 Docs) when you're configuring the root logger and using non-root-loggers at the same time.
I know this was asked a long time ago, but it's the top result on DuckDuckGo.
I'm trying to access the data within the logs that are printed. Here's how to set up logging:
import logging
import httplib as http_client
http_client.HTTPConnection.debuglevel = 1
#Initialize Logging
logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True
When I make an HTTP request using Python Requests,
r = requests.get(api+'download?name=XXXX),headers=header)
it prints out a bunch of lines of log. This is the one I'm interested in:
header: Location: http://09.bm-data-api.prod.XXXXXXX.net/download/XXXXX
How do I pass this header info back into a variable to use?
Thanks
Dave
The value you need has been logged (presumably to a file). So you'll need to read the contents of the logfile, find the line(s) you're interested in and perform the necessary work that you wish to do. You can read a file with:
with open(myfile) as f:
data = f.readlines()
and you can then iterate over data to find the lines that you're interested in.