Running GeoDjango in a Docker container - have added additional libraries via pip in the Dockerfile, and am now experiencing unwanted console output whenever I invoke any of the migrations commands, e.g. manage.py showmigrations/makemigrations/migrate.
The output is as follows:
user#host:/src$ ./manage.py showmigrations
CONFIGDIR=/home/django/.config/matplotlib
(private) matplotlib data path: /usr/local/lib/python3.7/site-packages/matplotlib/mpl-data
matplotlib data path: /usr/local/lib/python3.7/site-packages/matplotlib/mpl-data
loaded rc file /usr/local/lib/python3.7/site-packages/matplotlib/mpl-data/matplotlibrc
matplotlib version 3.2.1
interactive is False
platform is linux
loaded modules: ['sys', 'builtins', '_frozen_importlib', '_imp', '_thread', '_warnings', '_weakref', 'zipimport', '_frozen_importlib_external', '_io', 'marshal', 'posix', 'encodings', 'codecs', '_codecs', ...
Comprehensive modules listing snipped, it continues:
Using fontManager instance from /home/django/.cache/matplotlib/fontlist-v310.json
Loaded backend qt5agg version unknown.
Loaded backend tkagg version unknown.
Loaded backend agg version unknown.
Loaded backend agg version unknown.
Found GEOS DLL: <CDLL '/usr/local/lib/python3.7/site-packages/shapely/.libs/libgeos_c-5031f9ac.so.1.13.1', handle 5608f64e4c40 at 0x7f22a5aaaf10>, using it.
Trying `CDLL(libc.so.6)`
Library path: 'libc.so.6'
DLL: <CDLL 'libc.so.6', handle 7f22c4809000 at 0x7f22aef3b650>
GDAL_DATA not found in environment, set to '/usr/local/lib/python3.7/site-packages/fiona/gdal_data'.
PROJ data files are available at built-in paths
Entering env context: <fiona.env.Env object at 0x7f22a0798450>
Starting outermost env
No GDAL environment exists
New GDAL environment <fiona._env.GDALEnv object at 0x7f22a0798490> created
Logging error handler pushed.
All drivers registered.
GDAL_DATA found in environment: '/usr/local/lib/python3.7/site-packages/fiona/gdal_data'.
PROJ data files are available at built-in paths
Started GDALEnv <fiona._env.GDALEnv object at 0x7f22a0798490>.
Updated existing <fiona._env.GDALEnv object at 0x7f22a0798490> with options {}
Entered env context: <fiona.env.Env object at 0x7f22a0798450>
Exiting env context: <fiona.env.Env object at 0x7f22a0798450>
Cleared existing <fiona._env.GDALEnv object at 0x7f22a0798490> options
Stopping GDALEnv <fiona._env.GDALEnv object at 0x7f22a0798490>.
Error handler popped.
Stopped GDALEnv <fiona._env.GDALEnv object at 0x7f22a0798490>.
Exiting outermost env
Exited env context: <fiona.env.Env object at 0x7f22a0798450>
Could not import boto3, continuing with reduced functionality.
PROJ data files are available at built-in paths
Finally, the normal migrations output is displayed:
admin
[X] 0001_initial
[X] 0002_logentry_remove_auto_add
[snipped ...]
user#host:/src$
It's on a production system with gunicorn, NOT running in DEBUG mode. On the development system, with the same libraries set up, but in DEBUG mode, the output is normal. Dev and production dockerfiles are almost identical, stemming from python:3.7.4-buster.
On a first glance, it looks like a "chatty" library that is printing all that when it is loaded? Not sure if there is something broken or if this is normal? There are no signs of problems in the gunicorn error log. This also seems to only affect the migrations commands, no other manage.py commands.
Any hints appreciated!
Looking at the source for Fiona we find
log.debug("Entering env context: %r", self)
Since that's a debug level message, it wouldn't be visible by default.
This is a clue that logging has been configured to log debug-level messages.
This could happen via your Django config (which uses logging.dictConfig() under the hood), or by e.g. some module having run logging.basicConfig(level=logging.DEBUG).
Also, given how the .pth files that some packages use to hook themselves up to your import path work (they're just Python), this could inadvertently happen even if the package itself isn't imported.
I'd suggest removing some of those packages one by one until you find which one causes extra chattiness.
Related
So, I'm working on a robotframework test project, and the goal is to run several test suites in parallel. For this purpose, pabot was chosen as the solution. I am trying to implement it, but with little success.
My issue is: after installing Pabot (which, I might say, I did by cloning the project and running "setup.py install", instead of using pip, since the corporate proxy I'm behind has proven an obstacle I can't overcome), I created a new directory in the project tree, moved some suites there, and ran:
pabot --processes 2 --outputdir pabot_results Login*.robot
Doing so results in the following error message:
2018-10-10 10:27:30.449000 [PID:9676] [0] EXECUTING Suites.LoginAdmin
2018-10-10 10:27:30.449000 PID:400 EXECUTING Suites.LoginUser
2018-10-10 10:27:30.777000 PID:400 FAILED Suites.LoginUser
2018-10-10 10:27:30.777000 [PID:9676] [0] FAILED Suites.LoginAdmin
WARN: No output files in "pabot_results\pabot_results"
Output:
[ ERROR ] Reading XML source '' failed: invalid mode ('rb') or filename
Try --help for usage information.
Elapsed time: 0 minutes 0.578 seconds
Upon inspecting the stderr file that was generated, I have this message:
Traceback (most recent call last):
File "C:\Python27\Lib\site-packages\robotframework-3.1a2.dev1-py2.7.egg\robot\running\runner.py", line 22, in
from .context import EXECUTION_CONTEXTS
ValueError: Attempted relative import in non-package
Apparently, this has to do with something from the runner.py script, which, if I'm not mistaken, came with the installation of robotframework. Since manually modifying that script does not seem to me the optimal solution, my question is, what am I missing here? Did I forget to do anything while setting this up? Or is this an issue of compatibility between versions?
This project is using Maven as the tool to manage dependencies. The version I am running is 3.5.4. I am using a Windows 10, 64bit system; I have Python 2.7.14, and Robot Framework 3.1a2.dev1. The Pabot version is 0.44. Obviously, I added C:\Python27 and C:\Python27\Scripts to the PATH environment variable.
Edit: I am also using robotframework-maven-plugin version 1.4.0.8, if that happens to be relevant.
Edit 2: added the error messages in text format.
I believe I've come across an issue similar when setting up parallel execution on my machine. Firstly I would confirm that pabot is installed using pip show robotframework-pabot.
Then you should define the directory your results are going to using -d.
I then modified the name of the -o to Output.xml to make it easy to identify.
This is a copy of the code I use. Runs optimally with 8 processes
pabot --processes 8 -d results -o Output.xml Tests
Seems that you stumbled on a bug in the prerelease version of robot framework (3.1a2.dev1).
Please install a release version of robot framework. For example 3.0.4.
Just in case anyone happens to stumble upon this issue in the future:
Since I can't use pip, and I tried a good deal of workarounds that eventually made things more unstable, I ended up saving my project and removing everything Python-related from my system, so as to allow me to install everything from scratch. In a Windows 10, 64bit system, I used:
Python 2.7.14
wxPython 2.8.12.1, win64, unicode, for py27
setuptools 40.2.0 (to allow me to use the easy_install command)
Robot Framework 3.0.4
robotremoteserver 1.1
Selenium2Library 3.0.0
and Pabot version 0.45.
I might add that, when installing the Selenium2Library the way I described above, it eventually tries to download some things from the pip repositories - which, if you have a proxy, will cause you trouble. I solved this problem by browsing https://pypi.org/simple/selenium/, manually downloading the 2.53.6 .tar.gz file, then extracting it and running setup.py install on the command line.
PS: Ideally, though, anyone should be able to use proxy settings from the command line (--proxy http://user:password#server:port) to get pip and then use it; however, for some reason, probably related to network security configurations that I didn't want to lose time with, this didn't work in my case.
Good day all. First time stack overflow has not previously answered an issue that I have. My problem is exactly like the one posted here, except I am running windows:
Google App Engine Launcher is not running my hello world for Python Mac
The only thing the log gives me is this
2016-08-18 13:39:13 Running command:
"['C:\Users\Kesi\Desktop\Documents\Computer
Science\Udacity\Intro to backend\hello-udacity\main.py',
'C:\Program Files (x86)\Google\google_appengine\dev_appserver.py',
'--skip_sdk_update_check=yes', '--port=8080', '--admin_port=8000',
'C:\Users\Kesi\Desktop\Documents\Computer Science\Udacity\Intro
to backend\hello-udacity']"
Any suggestions are greatly appreciated.
Converting comments to an answer.
The 1st argument in the command list from your message is 'C:\Users\Kesi\Desktop\Documents\Computer Science\Udacity\Intro to backend\hello-udacity\main.py', preceeding the dev_appserver.py suggests a configuration problem.
In the google app engine preferences section there is a place to specify the path to executable python file. That should not be set to your app's python file (as you have it), but to the actual python executable (from your python installation) - which executes dev_appserver.py (the 2nd arg in the command list) which in turn loads the subsequent command list args including one which is the one specifying your app code's location - the last one in your case.
The python executable could be a python.exe or pythonw.exe according to the Executing scripts section of Using Python on Windows doc:
Python scripts (files with the extension .py) will be executed by
python.exe by default. This executable opens a terminal, which stays open even if the program uses a GUI. If you do not want this to
happen, use the extension .pyw which will cause the script to be
executed by pythonw.exe by default (both executables are located
in the top-level of your Python installation directory). This
suppresses the terminal window on startup.
Am new to Flask development and am using an IDE like Pycharm for the first time, so please pardon the ignorance.
So while playing around with a project that I am working on to understand Flask, I created a virtual environment by the name venv. So the tree of my project would look something like :
my_project_directory
Project_specific_directories_and_files
requirements.txt
venv
To activate the virtual env, I used to do venv/bin/activate and my project would run as needed.
I came across virtualevnwrapper later and decided to use that. After installing it and setting it up, I moved over to the virtualenvwrapper way of working with virtual envs and completely removed(deleted) the venv directory from my project structure. The new project structure after deleting venv is as :
my_project_directory
Project_specific_directories_and_files
requirements.txt
Everything works fine. The project runs as expected.
However, now after the venv directory was removed, when I open my project in Pycharm, for my import statements like :
from flask import Flask, render_template, redirect, url_for, request, session, flash # , g
from flask_sqlalchemy import SQLAlchem
all the packages that I try importing gets underlined as error in the Pycharm IDE. I opened the Python console, in Pycharm, trying to debug the error and it gives the following error :
Error:Cannot run program "/Users/my_user_name/Desktop/some_parent_directory/my_project_directory/venv/bin/python" (in directory "/Users/my_user_name/Desktop/some_parent_directory/my_project_directory"): error=2, No such file or directory
Now I thought that apparently Pycharm is still using it's cache. So I tried invalidating the cache and restarting Pycharm as explained here, but the problem was still there.
Now I have 2 questions:
Why is Pycharm behaving this way? I removed the venv directory completely etc. Why is it still referencing the old (venv) location?
If it really is not able to resolve the dependencies, how exactly then is my project running all fine? Shouldn't it break the app?
Ok. After some further digging now, I was able to get the errors flagged in Pycharm rectified. I went to Pycharm -> Preferences -> Project: my_project_directory -> Project Interpreter & found that it was 'for some reason' still pointing to the older venv directory. From the drop down of the 'Project Interpreter:' section, I chose the path to my virtualenv created by virtualenvwrapper & saved the changes by Apply->Ok. This removed the error indications from the code in Pycharm. Although solved, the mystery of what/why was this happening is still unsolved for me. – qre0ct Jul 12 at 17:13
Moreover, I also removed the unused/deleted interpreters from the list of project interpreters by going to the 'more' section in the 'Project Interpreter' part and using the '-' at the bottom to remove the ones not needed.
the issue: running Python's matplotib from a Windows virtual environment as reported & discussed Github ticket 93 and a relevant suggested fix here.
Code (myp.py):
import matplotlib.pyplot as pl
import Tkinter as tk
# insert code to read/load mydata
pl.plot(mydata['foo'],mydata['bar'],'*')
Tcl Error:
_tkinter.TclError: Can't find a usable init.tcl in the following directories:
C:/Python27/lib/tcl8.5 C:/Users/Remi/MyVirtEnv/lib/tcl8.5....(ect)
Fix attempt 1: as suggested in ticket above, I added the following env. variables to path in batch file activate.bat located in C:/Users/Remi/MyVirtEnv/Scripts/:
set "TCL_LIBRARY=C:\Python27\tcl\tcl8.5"
set "TK_LIBRARY=C:\Python27\tcl\tk8.5"
Not resolved this way. Potentially relevant info: using PyCharm IDE with the following configuration:
Environment variables: PYTHONUNBUFFERED=1
Python interpreter: python 2.7.10 virtualenv at C:\Users\Remi\MyVirtEnv
"buttons checked": Add content roots and Add source roots to PYTHONPATH
Any help I could get would be much appreciated.
After adding the following environment variables to path in batch file activate.bat located in the virtual environment, e.g.: C:/Users/Remi/MyVirtEnv/Scripts/:
set "TCL_LIBRARY=C:\Python27\tcl\tcl8.5"
set "TK_LIBRARY=C:\Python27\tcl\tk8.5"
it is able to launch IDLE in virtual environment.
Make sure the path is valid, e.g. "C:\Python27\tcl\tcl8.5"
I did my first development in a Win7 machine, then switched to a Mac when I got that, which was great due to the speed of building with ember server running. I've been keeping everything in a Git repo. Everything works fine running ember server from my Mac.
Now, when I do a fresh clone from the repo on my Win7 computer, then do npm install and bower install (both of which complete fine), when I run ember server next, I get a bunch of errors saying ENOTEMPTY, directory not empty (path, blah blah blah).
It's complaining about some folder in the tmp directory, which the server command is creating and putting stuff into all by itself, so I don't know what it's complaining about.
I've been finagling with this about 5 different times now, but without success.
Ember CLI version is 0.1.1
Ember version is 1.6
Here's an example of one of the errors, but they're all similar (and similarly unhelpful):
ENOTEMPTY, directory not empty 'c:\Dev\star\tmp\remover-tmp_dest_dir-sRc08q1q.tm
p\templates'
Error: ENOTEMPTY, directory not empty 'c:\Dev\star\tmp\remover-tmp_dest_dir-sRc0
8q1q.tmp\templates'
at Object.fs.rmdirSync (fs.js:623:18)
at rmkidsSync (c:\Dev\star\node_modules\ember-cli\node_modules\rimraf\rimraf
.js:247:11)
at rmdirSync (c:\Dev\star\node_modules\ember-cli\node_modules\rimraf\rimraf.
js:237:7)
at fixWinEPERMSync (c:\Dev\star\node_modules\ember-cli\node_modules\rimraf\r
imraf.js:150:5)
at Function.rimrafSync [as sync] (c:\Dev\star\node_modules\ember-cli\node_mo
dules\rimraf\rimraf.js:216:26)
at Remover._remove (c:\Dev\star\node_modules\ember-cli\node_modules\broccoli
-file-remover\index.js:46:10)
at c:\Dev\star\node_modules\ember-cli\node_modules\broccoli-file-remover\ind
ex.js:60:14
at Array.forEach (native)
at c:\Dev\star\node_modules\ember-cli\node_modules\broccoli-file-remover\ind
ex.js:59:12
at $$$internal$$tryCatch (c:\Dev\star\node_modules\ember-cli\node_modules\rs
vp\dist\rsvp.js:470:16)
file added bootstrap.min.js
file added favicon.ico
file added sb-admin-2.js
Okay, I managed to do something that worked. I had forgotten I needed to disable Windows Indexing on my tmp folder at least (I did it for the whole cloned project):
Windows Key, type "Indexing Options"
Click the drive, then Modify
Uncheck places you don't want indexed (like the tmp folder)
If you use Sublime, go to Perferences --> Settings - User and add the following:
"folder_exclude_patterns":
[
"tmp/*",
],
Finally, I just had to keep doing ember server, stopping it, and doing it again until it build successfully (took around 5 tries).