I would like to have 2 loggers, one that logs a summary of what my code is doing, and one that logs specific details. I would like to have the loggers write to different files in different locations as well. However my loggers currently write to the same file.
# Create MAIN Log
logging.basicConfig(filename=main_log_path,level=logging.INFO)
logging.getLogger("requests").setLevel(logging.WARNING)
# Create Details logger
logging.basicConfig(filename= detailed_log_path,level=logging.INFO)
logging.getLogger("requests").setLevel(logging.WARNING)
main_logger = logging.getLogger('Log_Summary')
detailed_logger = logging.getLogger('Detailed_Summary')
main_logger_handler = logging.FileHandler(main_log_path)
main_logger.addHandler(main_logger_handler)
detailed_logger_handler = logging.FileHandler(main_log_path)
detailed_logger.addHandler(detailed_logger_handler)
main_logger.info("this is the main logger")
detailed_logger.info("this is the detailed logger")
detailed_logger_handler = logging.FileHandler(main_log_path)
detailed_logger.addHandler(detailed_logger_handler)
Change the main_log_path to something else. Right now both the loggers point to the same file. You need different paths to log to different files.
Related
I'm using Python 2.6 and PyGTK 2.22.6 from the all-in-one installer on Windows XP, trying to build a single-file executable (via py2exe) for my app.
My problem is that when I run my app as a script (ie. not built into an .exe file, just as a loose collection of .py files), it uses the native-looking Windows theme, but when I run the built exe I see the default GTK theme.
I know that this problem can be fixed by copying a bunch of files into the dist directory created by py2exe, but everything I've read involves manually copying the data, whereas I want this to be an automatic part of the build process. Furthermore, everything on the topic (including the FAQ) is out of date - PyGTK now keeps its files in C:\Python2x\Lib\site-packages\gtk-2.0\runtime\..., and just copying the lib and etc directories doesn't fix the problem.
My questions are:
I'd like to be able to programmatically find the GTK runtime data in setup.py rather than hard coding paths. How do I do this?
What are the minimal resources I need to include?
Update: I may have almost answered #2 by trial-and-error. For the "wimp" (ie. MS Windows) theme to work, I need the files from:
runtime\lib\gtk-2.0\2.10.0\engines\libwimp.dll
runtime\etc\gtk-2.0\gtkrc
runtime\share\icons\*
runtime\share\themes\MS-Windows
...without the runtime prefix, but otherwise with the same directory structure, sitting directly in the dist directory produced by py2exe. But where does the 2.10.0 come from, given that gtk.gtk_version is (2,22,0)?
Answering my own question here, but if anyone knows better feel free to answer too. Some of it seems quite fragile (eg. version numbers in paths), so comment or edit if you know a better way.
1. Finding the files
Firstly, I use this code to actually find the root of the GTK runtime. This is very specific to how you install the runtime, though, and could probably be improved with a number of checks for common locations:
#gtk file inclusion
import gtk
# The runtime dir is in the same directory as the module:
GTK_RUNTIME_DIR = os.path.join(
os.path.split(os.path.dirname(gtk.__file__))[0], "runtime")
assert os.path.exists(GTK_RUNTIME_DIR), "Cannot find GTK runtime data"
2. What files to include
This depends on (a) how much of a concern size is, and (b) the context of your application's deployment. By that I mean, are you deploying it to the whole wide world where anyone can have an arbitrary locale setting, or is it just for internal corporate use where you don't need translated stock strings?
If you want Windows theming, you'll need to include:
GTK_THEME_DEFAULT = os.path.join("share", "themes", "Default")
GTK_THEME_WINDOWS = os.path.join("share", "themes", "MS-Windows")
GTK_GTKRC_DIR = os.path.join("etc", "gtk-2.0")
GTK_GTKRC = "gtkrc"
GTK_WIMP_DIR = os.path.join("lib", "gtk-2.0", "2.10.0", "engines")
GTK_WIMP_DLL = "libwimp.dll"
If you want the Tango icons:
GTK_ICONS = os.path.join("share", "icons")
There is also localisation data (which I omit, but you might not want to):
GTK_LOCALE_DATA = os.path.join("share", "locale")
3. Piecing it together
Firstly, here's a function that walks the filesystem tree at a given point and produces output suitable for the data_files option.
def generate_data_files(prefix, tree, file_filter=None):
"""
Walk the filesystem starting at "prefix" + "tree", producing a list of files
suitable for the data_files option to setup(). The prefix will be omitted
from the path given to setup(). For example, if you have
C:\Python26\Lib\site-packages\gtk-2.0\runtime\etc\...
...and you want your "dist\" dir to contain "etc\..." as a subdirectory,
invoke the function as
generate_data_files(
r"C:\Python26\Lib\site-packages\gtk-2.0\runtime",
r"etc")
If, instead, you want it to contain "runtime\etc\..." use:
generate_data_files(
r"C:\Python26\Lib\site-packages\gtk-2.0",
r"runtime\etc")
Empty directories are omitted.
file_filter(root, fl) is an optional function called with a containing
directory and filename of each file. If it returns False, the file is
omitted from the results.
"""
data_files = []
for root, dirs, files in os.walk(os.path.join(prefix, tree)):
to_dir = os.path.relpath(root, prefix)
if file_filter is not None:
file_iter = (fl for fl in files if file_filter(root, fl))
else:
file_iter = files
data_files.append((to_dir, [os.path.join(root, fl) for fl in file_iter]))
non_empties = [(to, fro) for (to, fro) in data_files if fro]
return non_empties
So now you can call setup() like so:
setup(
# Other setup args here...
data_files = (
# Use the function above...
generate_data_files(GTK_RUNTIME_DIR, GTK_THEME_DEFAULT) +
generate_data_files(GTK_RUNTIME_DIR, GTK_THEME_WINDOWS) +
generate_data_files(GTK_RUNTIME_DIR, GTK_ICONS) +
# ...or include single files manually
[
(GTK_GTKRC_DIR, [
os.path.join(GTK_RUNTIME_DIR,
GTK_GTKRC_DIR,
GTK_GTKRC)
]),
(GTK_WIMP_DIR, [
os.path.join(
GTK_RUNTIME_DIR,
GTK_WIMP_DIR,
GTK_WIMP_DLL)
])
]
)
)
I'm trying to find out how I can modify the way a custom TensorFlow estimator creates event files for Tensorboard. Currently, I have the impression that, by default, a summary (containing the values of all the things (like typically accuracy) I'm following with tf.summary.scalar(...) ) is created every 100 steps in my model directory. The names of the event files later used by tensorboard look like
events.out.tfevents.1531418661.nameofmycomputer.
I found a routine online to change this behaviour and create directories for each run with the date and time of the computation, but it uses TensorFlow basic APIs:
logdir = "tensorboard/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S") + "/"
writer = tf.summary.FileWriter(logdir, sess.graph)
Is it possible to do something similar with a TF custom estimator?
It is possible to specify a directory for each evaluation run using name argument of the evaluate method of tf.estimator.Estimator e.g.:
estimator = tf.estimator.Estimator(
model_fn=model_fn,
model_dir=model_dir
)
eval_results = estimator.evaluate(
input_fn=eval_input_fn,
name=eval_name
)
The event files for this evaluation will be saved in the directory inside model_dir named "eval_" + eval_name.
Summary Writers are not needed for TensorFlow Estimators. The summary log of the model is written to the designated folder location using the model_dir attribute of tf.Estimator function when the tf.Estimator.fit() method is called.
In the example below, the selected directory to store the training logs is './my_model'.
tf.estimator.DNNClassifier(
model_fn,
model_dir='./my_model',
config=None,
params=None,
warm_start_from=None
)
Launch TensorBoard by running tensorboard --logdir=./my_model from the terminal.
My application uses log4j but OkHttpClient uses java util logging. So apart from log4j.properties, I created a logging.properties file with the following contents:
handlers=java.util.logging.FileHandler
.level=FINE
okhttp3.internal.http2.level=FINE
java.util.logging.FileHandler.pattern = logs/%hjava%u.log
java.util.logging.FileHandler.limit = 50000
java.util.logging.FileHandler.count = 1
java.util.logging.FileHandler.formatter = java.util.logging.XMLFormatter
java.util.logging.ConsoleHandler.level = FINE
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
I then added this to jvm params used for starting the application -Djava.util.logging.config.file="file://${BASE_DIR}/logging.properties"
But I don't see any new folders being created as indicated by the Filehandler. Any one know why?
But I don't see any new folders being created as indicated by the Filehandler. Any one know why?
The FileHandler will not create any new folders. A directory must be created before the FileHandler will create a file.
The system property requires a path to file that is located on the filesystem It will not expand system properties or environment variables by using the dollar sign syntax.
You can use a relative path based off of the working directory or you have to use an absolute path to the logging.properties. The logging properties can not be packaged inside of an archive.
If you want to work around this limitation then you want to create a custom config class and use the java.util.logging.config.class property in conjunction with the java.util.logging.config.file property. You then write a class that reads the file://${BASE_DIR}/logging.properties and performs the needed transformation into a path to a file. Then update the configuration if you are using JDK9 or newer. On older versions you need to use readConfiguration and add code to work work around limitations of the LogManager
I'm trying to use the bazel restricted_to attribute for a test.
I want the test to only run on a specific cpu = build.
To make this somewhat more complicated, the cpu type is defined in our
/tools/cpp/CROSSTOOL file (cpu=armhf-debian).
I've had no luck with guessing the syntax of the restricted_to parameter
(my first guess was //cpu:armhf-debian, which just looked for a cpu package)
Any Suggestions?
There's not a lot of documentation on restricted_to, and the other rules it works with, environment and environment_group. Mostly this is because the use case they are for is very specific to Google's repository setup, and we're in the process of replacing them with a more flexible system.
To use restricted_to, you would need to define several environment rules, and an environment_group to contain them, and then specify which environment the test is restricted to, and finally always use the "--target_environment" flag to specify the current environment group. That would look something like this:
environment(name = "x86")
environment(name = "ppc")
environment_group(
name = "cpus",
defaults = [":x86"],
environments = [
":x86",
":ppc",
])
cc_test(
name = "test",
other config
restricted_to = [":ppc"],)
You could then run the test as so:
bazel test --target_environment=//:ppc //:test
to get the environment checking.
This isn't terribly useful, as whoever is running the test has to also remember to set "--target_environment" properly.
A better way to disable the test, using currently supported code, is to use config_setting and select, like this:
config_setting(
name = "k8",
values = {"cpu": "k8"})
config_setting(
name = "ppc",
values = {"cpu":, "ppc")
cc_test(
name = "test",
other config
srcs = [other sources] +
select({
"//:x86": ["x86_test_src.cpp"],
"//:ppc": ["ppc_test_src.cpp"],
"//conditions:default": ["default_test_src.cpp"],
})
config_setting will take a value based on the current "--cpu" flag. By changing the files included in the select, you can control what files are included in the test for each cpu setting.
Obviously, these don't have to be in the same package, and the usual Bazel visibility rules apply. See Bazel's src/BUILD for an example of config_setting, and src/test/cpp/BUILD for an example of using it in select.
We're working hard on platforms, which is a better way to describe and query Bazel's execution environment, and we'll make sure to post documentation and a blog post when that's ready for people to test.
I need to get every definitions from a django config file without executing/sourcing the file.
Ideally I want to be able to get:
# common.py
DJANGO_ROOT = ...
DEBUG = ...
Return value :
-> ("DJANGO_ROOT", "DEBUG", ...)
Through this I want to compare different config file to check if all variable are defined first in common.
I am currently trying with the compiler.parseFile method but I saw that the package is deprecated.
Is there another way to do?