Django manage.py syncdb can't find my settings file on CentOS - django

I'm deploying my django application onto a CentOS 5.5 server, with django-1.1.4 over python-2.6.5.
I have multiple settings files inside the myapp/settings/ folder.
I would like to run the syncdb; here's what I do (with myapp inside myproject folder):
$> cd /var/www/apps/myproject
$> export PYTHONPATH=/var/www/apps/myproject
$> export DJANGO_SETTINGS_MODULE=myapp.settings.my_serverconfig
$> python26 myapp/manage.py syncdb
Django then issues an error like this :
Error: Can't find the file 'settings.py' in the directory containing 'myapp/manage.py'. It appears you've customized things.
You'll have to run django-admin.py, passing it your settings module.
(If the file settings.py does indeed exist, it's causing an ImportError somehow.)
Traceback (most recent call last):
File "emon/manage.py", line 17, in <module>
execute_manager(settings)
File "/usr/lib/python2.6/site-packages/django/core/management/__init__.py", line 360, in execute_manager
setup_environ(settings_mod)
File "/usr/lib/python2.6/site-packages/django/core/management/__init__.py", line 343, in setup_environ
project_module = import_module(project_name)
File "/usr/lib/python2.6/site-packages/django/utils/importlib.py", line 35, in import_module
__import__(name)
ImportError: No module named my_serverconfig
In the myapp.wsgi file, os.path is appended with myproject path, and the os.environ['DJANGO_SETTINGS_MODULE'] is set also. Apache (through mod_wsgi) can start the app with no such error.
Finally, this works under Windows, where I run python-2.6.6 with django-1.1.1.
$> d:
$> cd d:\Code\myproject
$> export PYTHONPATH=d:\Code\myproject
$> export DJANGO_SETTINGS_MODULE=myapp.settings.dev_settings
$> python.exe myapp/manage.py syncdb
I know the versions are not the same, but I'm not sure that the minor differences may cause all my woe. Moreover I don't seem to find the exact same python version for Windows.
Any thoughts? Thanks a lot for reading.
O.
EDIT: added the manage.py content
#!/usr/bin/env python
from django.core.management import execute_manager
import os
if __name__ == "__main__":
settings = None
try:
if os.environ.has_key('LOCAL_SERVER_SETTINGS'):
os.environ['DJANGO_SETTINGS_MODULE'] = 'myapp.settings.%s' % os.environ['LOCAL_SERVER_SETTINGS']
if os.environ.has_key('DJANGO_SETTINGS_MODULE'):
settings = __import__(os.environ['DJANGO_SETTINGS_MODULE'])
if settings is None:
import settings
execute_manager(settings)
except ImportError:
import sys
sys.stderr.write("Error: Can't find the file 'settings.py' in the directory containing %r. It appears you've customized things.\nYou'll have to run django-admin.py, passing it your settings module.\n(If the file settings.py does indeed exist, it's causing an ImportError somehow.)\n" % __file__)
import traceback
traceback.print_exc()
sys.exit(1)
EDIT : more on what happens in the myapppackage
I patch some django functions/classes from within the myapp.__init__ module. I was thinking the import django part in this module was causing a circular reference. The code is executed when I load myapp.settings.[any_config] and could have caused the crash. But then, how come the correct settings module is loaded with no error by WSGI, and that it works fine also on Windows? More : after commenting out the said code parts, the ImportError is still there.

If you move your settings file, you need to modify manage.py to tell it where to find it. The default is for it to be in the same directory as manage.py, but if you move into another python module(folder with __init__.py) then you need to update the reference in manage.py.
Look in manage.py where it imports your settings module, and import settings.dev_settings in your case instead. Look for 2 important lines
imp.find_module('settings') # Assumed to be in the same directory.
...
import settings
Change these to reference where you moved the settings file to(assuming you moved to settings.dev_settings):
imp.find_module('settings.dev_settings') # Assumed to be in the same directory.
...
from settings import dev_settings as settings
You can also use the command line option --settings to specify which settings file to use.

Related

neovim: no module named __future__

When I try to open certain python files in neovim, I get an error:
"pool.py" 667L, 25276C
function provider#python#Call[9]..remote#host#Require[10]..provider#pythonx#Require, line 15
Vim(if):ch 1 was closed by the client
Traceback (most recent call last):
File "/home/user/.pyenv/versions/neovim2/lib/python2.7/site.py", line 67, in <module>
import os
File "./os.py", line 44, in <module>
from __future__ import absolute_import
ImportError: No module named __future__
Failed to load python host. You can try to see what happened by starting nvim with $NVIM_PYTHON_LOG_FILE set and opening the generated log file. Also
, the host stderr is available in messages.
Press ENTER or type command to continue
This happens any time I open a python file in a directory that contains an os.py or os.pyc file. It looks like neovim is trying to import the local os.py file instead of the one in the virtualenv.
What can I do about this?
EDIT: turns out it's not when I open a file in the same directory as an os.py file, it's when I open a file anywhere while the current working directory has an os.py file. Basically, it looks like python is checking the local directory for imports before checking the python libs.
I figured it out. The problem was with my $PYTHONPATH. I had in my .bashrc file this:
export PYTHONPATH="$PYTHONPATH:~/.local/lib/python"
The problem was that, when that line is executed, $PYTHONPATH is empty, leading to the string starting with a :. I'm not 100% sure why, but that resulted in python checking the local directory for a module BEFORE checking the python libraries.
I changed it to
if [ -z "$PYTHONPATH" ]; then
export PYTHONPATH="~/.local/lib/python"
else
export PYTHONPATH="$PYTHONPATH:~/.local/lib/python"
fi
And now it works.

PyInstaller runs fine but exe file errors: No module named, Failed to Execute Script

I am running the following code:
pyinstaller --onefile main.py
main.py looks like:
import sys
import os
sys.path.append(r'C:\Model\Utilities')
from import_pythonpkg import *
......
import_pythonpkg.py looks like:
from astroML.density_estimation import EmpiricalDistribution
import calendar
import collections
from collections import Counter, OrderedDict, defaultdict
import csv
....
By running the pyinstaller on main.py, main.exe file is created successfully.
But when I run main.exe it gives error with astroML. If I move astroML to main.py from import_pythonpkg.py, there is no error with astroML. Now I get error with csv.
i.e. if I change my main.py to look as:
import sys
from astroML.density_estimation import EmpiricalDistribution
import os
sys.path.append(r'C:\Model\Utilities')
from import_pythonpkg import *
......
The astroML error is no longer present when I run main.exe.
There is no error with import calendar line in import_pythonpkg.py at all.
I am not sure how to handle this random error with packages when running main.exe after pyinstaller run.
import_pythonpkg is located at r'C:\Model\Utilities'
Edit:
Error with main.exe looks as following even though the original main.py runs fine. Pyinstaller was even able to let me create the main.exe without error.
Traceback (most recent call last):
File "main.py", line 8, in <module>
File "C:\Model\Utilities\import_pythonpkg.py", line 1, in <module>
from astroML.density_estimation import EmpiricalDistribution
ImportError: No module named astroML.density_estimation
[29180] Failed to execute script main
I believe PyInstaller is not seeing import_pythonpkg. In my experience, when adding to the path or dealing with external modules and dlls, PyInstaller will not go searching for that, you have to explicitly tell it to do so. It will compile down to an .exe properly because it just ignores it, but then won't run. Check to see if there are any warnings about missing packages or modules when you run your PyInstaller command.
But how to fix it...If indeed this is the issue (which I am not sure that it is) you can try 3 things:
1) move that package into your working directory and avoid using sys.path.append. Then compile with PyInstaller to so see if this works, then you know the issue is that pyinstaller is failing to find import_pythonpkg. You can stop there if this works.
2) explicitly tell PyInstaller to look there. You can use the hidden-import tag when compiling with PyInstaller to let it know (give it the full pathname).
--hidden-import=modulename
for more info, check here: How to properly create a pyinstaller hook, or maybe hidden import?
3) If you use the spec file that PyInstaller creates, you can try adding a variable call pathex to tell PyInstaller to search there for things:
block_cipher = None
a = Analysis(['minimal.py'],
pathex=['C:\\Program Files (x86)\\Windows Kits\\10\\example_directory'],
binaries=None,
datas=None,
hiddenimports=['path_to_import', 'path_to_second_import'],
hookspath=None,
runtime_hooks=None,
excludes=None,
cipher=block_cipher)
pyz = PYZ(a.pure, a.zipped_data,
cipher=block_cipher)
exe = EXE(pyz,... )
coll = COLLECT(...)
for more information on spec files: https://pyinstaller.readthedocs.io/en/stable/spec-files.html
(notice you can also add hiddenimports here)
This answer may also prove helpful: PyInstaller - no module named
It is about to module which loaeded on your computer. If your IDE is different from your environment, you have to load same modules on your device via pip. Check the modules on CMD screen and complete the missing modules.
Sometimes you must load the modules all IDEs on your device. In my case, there were two IDEs (pycharm and anaconda). I used pycharm but pyinstaller used anaconda's modules so i unistalled anaconda and tried again. now it works..

Unable To Run IPython nbconvert From Python2.7 Virtual Environment

I have a virtual environment of Python 2.7 with ipython installed (Ubuntu 16.04.2 (Xenial) LTS.)
When I’m working in the virtual environment (after running source venv/bin/activate in bash shell while being in the parent directory of the virtual environment) I have no problem executing conversion of my jupiter’s notebook from bash shell like so:
ipython nbconvert --to html --execute my_notes.ipynb --stdout > /tmp/report.html
But when I’m trying to call that command from fabric’s task using subprocess:
command = ['ipython', 'nbconvert', '--to', 'html', '--execute', notebook_path, '--stdout']
output = subprocess.check_output(command,
cwd=os.environ['PYTHONPATH'],
env=os.environ.copy())
It always fails with this exception I cannot find a reason for it:
Traceback (most recent call last):
File "/opt/backend/venv/bin/ipython", line 7, in <module>
from IPython import start_ipython
File "/opt/backend/venv/local/lib/python2.7/site-packages/IPython/__init__.py", line 48, in <module>
from .core.application import Application
File "/opt/backend/venv/local/lib/python2.7/site-packages/IPython/core/application.py", line 25, in <module>
from IPython.core import release, crashhandler
File "/opt/backend/venv/local/lib/python2.7/site-packages/IPython/core/crashhandler.py", line 28, in <module>
from IPython.core import ultratb
File "/opt/backend/venv/local/lib/python2.7/site-packages/IPython/core/ultratb.py", line 119, in <module>
from IPython.core import debugger
File "/opt/backend/venv/local/lib/python2.7/site-packages/IPython/core/debugger.py", line 46, in <module>
from pdb import Pdb as OldPdb
File "/usr/lib/python2.7/pdb.py", line 59, in <module>
class Pdb(bdb.Bdb, cmd.Cmd):
AttributeError: 'module' object has no attribute 'Cmd'
More info to save your time.
I’ve tried:
Using same paths for PYTHONPATH as I got from PyCharm run/debug configuration.
Using nbconvert as python library from this documentation.
Tried os.system("ipython nbconvert…").
Wrapped working command (ipython nbconvert…) with a shell script and used it in subprocess.check_output and os.system.
Get drunk and bang my head on a brick wall.
And always end-up with that cursed exception.
Reposting as an answer for completeness:
There was a file called cmd.py somewhere where Python was finding it as an importable module. This was shadowing the cmd module in the standard library, which is used by pdb, which IPython imports. When pdb tried to subclass a class from cmd, that class wasn't there. Moving cmd.py out of the way lets it find the cmd module it needs.
This is an unfortunate annoyance with Python - lots of short words are already used as module names, and using them yourself produces crashes, with a wide range of different errors.

Error when using django.template

I'm a beginner in django and I got a lot of errors when using template module from django.
The following works from the python shell:
from django import template
t = template.Template('My name is {{ name }}.')
When i use this code , i get the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/site-packages/django/template/base.py", line 123, in __init__
if settings.TEMPLATE_DEBUG and origin is None:
File "/usr/lib/python2.7/site-packages/django/conf/__init__.py", line 53, in __getattr__
self._setup(name)
File "/usr/lib/python2.7/site-packages/django/conf/__init__.py", line 46, in _setup
% (desc, ENVIRONMENT_VARIABLE))
django.core.exceptions.ImproperlyConfigured: Requested setting TEMPLATE_DEBUG,but
settings are not configured. You must either define the environment variable
DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
Dose anyone have an idea about this error?
You can't access and run django projects from python shell. Django doesn't know what project you want to work on.
You have to do one of these things:
1. python manage.py shell
2. Set DJANGO_SETTINGS_MODULE environment variable in your OS to mysite.settings
3. Use setup_environ in the python interpreter:
from django.core.management import setup_environ
from mysite import settings
setup_environ(settings)
The first one is easiest and best method. Run your code in the django shell.
If you want to use the template system without the django shell, you can do:
from django.conf import settings
settings.configure()
and after this your code
t = template.Template('My name is {{ name }}.')
I also had the same problem.
The main point is that in the django book 2, this piece of code is not supposed for users to put into a Python IDLE started from your python.exe ( presumably in C:\Program Files\Python33 on Windows).
Rather, it is just showing how the piece of code will look like. The solution is simply run 'cmd' and change the directory to your 'mysite', and run
python manage.py shell
which ensure that the 'mysite.settings.py' is used.
Of course the #Sudipta's answer works correctly but (just for the future) for the explanation you can also visit the "Creating Template Objects" -> "A special Python prompt" section in Django book
To fix this issue, you need to run it in python manage.py shell instead of just python.
Reason: Well, many parts of Django rely on settings and you won't be able to use them until it knows which settings file to use. When you run it on python manage.py shell, Django knows which settings file to use and does the job for you.
The error message states that you need to set the DJANGO_SETTINGS_MODULE variable.
Running export DJANGO_SETTINGS_MODULE=mysite.settings, worked for me.

pycharm - how to "run" a project with add_to_builtins?

i am using pycharm with django. When i do the runserver command, my project starts up and everything is fine.
if i use the pycharm run command - that green arrow at the top - then i get problems.
The problems are:
runnerw.exe C:\development\python\python.exe manage.py runserver 127.0.0.1:8000
Traceback (most recent call last):
File "manage.py", line 11, in
import settings
File "C:\development\PycharmProjects\dumpstown\settings.py", line 185, in
add_to_builtins('gravatar.templatetags.gravatar')
File "C:\development\python\lib\site-packages\django\template\base.py", line 1017, in add_to_builtins
builtins.append(import_library(module))
File "C:\development\python\lib\site-packages\django\template\base.py", line 963, in import_library
raise InvalidTemplateLibrary("ImportError raised loading %s: %s" % (taglib_module, e))
django.template.base.InvalidTemplateLibrary: ImportError raised loading
gravatar.templatetags.gravatar: Settings cannot be imported, because environment variable DJANGO_SETTINGS_MODULE is undefined.
Process finished with exit code 1
And stem from my use of add_to_builtins here: (this is in the settings.py file)
#gravatar stuff here.
add_to_builtins('gravatar.templatetags.gravatar')
I know this is the problem, because if i remove this line in the settings.py file? everything works fine.
Is there a way to remedy this problem for pycharm?
You need to set your Django settings module in Settings | Django Support | Settings
http://www.jetbrains.com/pycharm/webhelp/django-support.html
UPDATED
The problem with django-gravatar is that it's templatetags import django.contrib.auth.models.User which relies on settings module while setting module is loaded in Django by DJANGO_SETTINGS_MODULE environment variable, which is set internally by execute_manager function call in manage.py, which is executed AFTER import of settings. So when you use undocumented add_to_builtins feature just in settings.py at that point you have no DJANGO_SETTINGS_MODULE env variable set. So that is not a problem of PyCharm, but a problem of unset environment variable and usage of undocumented Django feature add_to_builtin.
When I ran the same project from Unix console I get the same error.
Probably you have DJANGO_SETTINGS_MODULE set in your environment if it works from console.
So to make it work in PyCharm you need to set up the variable in Django run configuration.
You can read here about that (see Environment variable section).
As i answered here: https://stackoverflow.com/a/11299516/1061426
pycharm is broken and doesn't work with add_to_builtins. The two obvious solutions are:
don't use pycharm, use some free django plugins for eclipse, or old school text editing?
use pycharm, just don't use add_to_builtins. This is the route i've gone down - it is annoying fixing all the template's to import a module, but it was a lot simpler in my case than the hassle of porting across to a new IDE.