Django loaddata settings error - django

When trying to use loaddata on my local machine (win/sqlite):
python django-admin.py loaddata dumpdata.json
I get the following error:
raise ImportError("Settings cannot be imported, because environment variable %s is undefined." % ENVIRONMENT_VARIABLE) ImportError: Settings cannot be imported, because environment variable DJANGO_SETTINGS_MODULE is undefined.
I am using djangoconfig app if that helps:
"""
Django-config settings loader.
"""
import os
CONFIG_IDENTIFIER = os.getenv("CONFIG_IDENTIFIER")
if CONFIG_IDENTIFIER is None:
CONFIG_IDENTIFIER = 'local'
# Import defaults
from config.base import *
# Import overrides
overrides = __import__(
"config." + CONFIG_IDENTIFIER,
globals(),
locals(),
["config"]
)
for attribute in dir(overrides):
if attribute.isupper():
globals()[attribute] = getattr(overrides, attribute)
projects>python manage.py loaddata dumpdata.json --settings=config.base
WARNING: Forced to run environment with LOCAL configuration.
Problem installing fixture 'dumpdata.json': Traceback
(most recent call last):
File "loaddata.py", line 174, in handle
obj.save(using=using)
...
File "C:\Python26\lib\site-packages\django\db\backends\sqlite3\base.py", line
234, in execute
return Database.Cursor.execute(self, query, params)
IntegrityError: columns app_label, model are not unique

Don't use django-admin.py for anything other than setting up an initial project. Once you have a project, use manage.py instead - it sets up the reference to your settings file.

syncdb will load content_types, you need to clear that table before loading data. Something like this:
c:\> sqlite3 classifier.db
sqlite> delete from django_content_type;
sqlite> ^Z
c:\> python django-admin.py loaddata dumpdata.json
Also, make sure you do not create a superuser, or any user, when you syncdb, as those are likely to also collide with your data fixture ...

There are two standard ways to provide your settings to Django.
Using set (or export on Unix) set DJANGO_SETTINGS_MODULE=mysite.settings
Alternatively as an option with django-admin.py --settings=mysite.settings
Django-config does things differently because it allows you to have multiple settings files. Django-config works with manage.py to specify which to use. You should use manage.py whenever possible; it sets up the environment. In your case try this where --settings points to the specific .py file you want to use from django-config's config folder.
django-admin.py loaddata dumpdata.json --settings=<config/settings.py>
Actually --settings wants python package syntax so maybe <mysite>.config.<your settings>.py

Related

How do I store environment variables both locally and not to have to change code when deploying on Heroku in Django

I have a Django project I have been working on offline and now I have hosted it on Heroku and it works well on Heroku but fails on my local machine with this error.
File "/usr/lib/python3.9/os.py", line 679, in __getitem__
raise KeyError(key) from None
KeyError: 'DEBUG'
and I think it is because I used environment variables like this.
from boto.s3.connection import S3Connection
import os
DEBUG = S3Connection(os.environ['DEBUG'], os.environ['DEBUG'])
I also have a .env file in my root(project folder) with the environment variables like this.
export JWT_SECRET_KEY = "dfge..."
export DEBUG = 1
What is the right way to store the environment variables on my local machine?
I have local file secret.py added to .gitignore with all keys, env values needed:
#secret.py
DEBUG = 1
Then in settings.py:
# settings.py
try:
import secret
DEBUG = secret.DEBUG
except ModuleNotFoundError:
DEBUG = S3Connection(os.environ['DEBUG'], os.environ['DEBUG'])

How do I use "discover" to run tests in my "tests" directory?

Using DJango/Python 3.7. I read here -- How do I run all Python unit tests in a directory? that I could use a "discover" command to find tests in a specified directory. I want to have a "tests" folder, so I cretaed one and then ran
(venv) localhost:myproject davea$ python -m unittest discover tests
Traceback (most recent call last):
File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/__main__.py", line 18, in <module>
main(module=None)
File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/main.py", line 100, in __init__
self.parseArgs(argv)
File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/main.py", line 124, in parseArgs
self._do_discovery(argv[2:])
File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/main.py", line 244, in _do_discovery
self.createTests(from_discovery=True, Loader=Loader)
File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/main.py", line 154, in createTests
self.test = loader.discover(self.start, self.pattern, self.top)
File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/loader.py", line 344, in discover
raise ImportError('Start directory is not importable: %r' % start_dir)
ImportError: Start directory is not importable: 'tests'
This is odd to me because I have an (empty) init file ...
(venv) localhost:myproject davea$ ls web/tests/
__init__.py model_tests.py
What else do I need to do to get my test directory recognized?
Edit: Below are the contents of model_tests.py ...
from django.conf import settings
from django.test import TestCase
from django.core import management
def setup():
print("setup")
management.call_command('loaddata', 'test_data.yaml', verbosity=0)
def teardown():
management.call_command('flush', verbosity=0, interactive=False)
class ModelTest(TestCase):
# Verify we can correctly calculate the amount of taxes when we are working
# with a state whose tax rates are defined in our test data
def test_calculate_tax_rate_for_defined_state(self):
state = "MN"
income = 30000
taxes = IndividualTaxBracket.objects.get_taxes_owed(state, income)
print(taxes)
self.assertTrue(taxes > 0, "Failed to calucate taxes owed properly.")
I think you are having some confusion about discover command. According to docs.
Unittest supports simple test discovery. In order to be compatible
with test discovery, all of the test files must be modules or packages
(including namespace packages) importable from the top-level directory
of the project (this means that their filenames must be valid
identifiers).
It means all the test files must be importable from the directory from which you are running the command (directory that holds your web directory). It make this sure, all test files must be in valid python packages (directories containing __init__.py).
Secondly you are running the command python -m unittest discover tests which is wrong. You don't have to add tests at the end. unittests with discover command support 4 options. You can read more about it here.
I have following directory structure.
web
├── __init__.py
└── tests
├── __init__.py
└── test_models.py
And I am running following command.
python3 -m unittest discover
With following results.
...
----------------------------------------------------------------------
Ran 3 tests in 0.000s
OK
First things first: Having an __init__.py is not unusual, because the __init__.py tells python that the directory is a module; Its usual to have an empty __init__.py file. I had the same error, and fixed it by renaming my directory ..
Should a file named tests.py exist as a sibling of tests module, that would probably cause the mentioned ImportError, and removing test.py should fix it.
If still unit tests are not discovered, a couple of question are in order:
1) does the test module contain at least a class derived from django.test.TestCase ?
2) and in that case, does that class contain at least one method whose name starts with "test_"
Please note that the name of any file containing a unit test should start with "test".
So model_test.py will not work; is is generally used to setup some fake Models, but unit tests should reside elsewhere.
You can discover and run tests with this management command:
python manage.py test
or
python manage.py test appname
Is there any particular reason for using python -m unittest discover instead ? I think that could work either, but then you'll have to manually bootstrap the django environment
For completion ...
You already know that form here:
The names of your tests and files have to match a specific pattern in order to be discoverable by discover().
But then you got this error:
"django.core.exceptions.ImproperlyConfigured: Requested settings, but settings are not configured"
That means Django wasn't able to find its settings while running your tests. You can tell where to find settings using an environment variable:
DJANGO_SETTINGS_MODULE='myproyect.settings' python3 -m unittest discover
Reference: https://docs.djangoproject.com/en/2.2/topics/settings/#designating-the-settings
On the other hand ...
You should be running your Django tests with
./manage.py tests
this will search tests automatically using the same mechanism than discover(), and since you would be running a Django command, you will have some benefits against running the Django tests directly.
#Nafees Anwar asked: How does setting environment variable configure settings?
At the very beginning of the model_tests.py file there is the line from django.conf import settings, while creating the settings LazyObject instance, Django will search for that environment variable. Read the code for more detail.
I'll post here a snippet from that code for illustration.
# django.conf module.
ENVIRONMENT_VARIABLE = "DJANGO_SETTINGS_MODULE"
class LazySettings(LazyObject):
"""
A lazy proxy for either global Django settings or a custom settings object.
The user can manually configure settings prior to using them. Otherwise,
Django uses the settings module pointed to by DJANGO_SETTINGS_MODULE.
"""
def _setup(self, name=None):
"""
Load the settings module pointed to by the environment variable. This
is used the first time we need any settings at all, if the user has not
previously configured the settings manually.
"""
settings_module = os.environ.get(ENVIRONMENT_VARIABLE)
if not settings_module:
desc = ("setting %s" % name) if name else "settings"
raise ImproperlyConfigured(
"Requested %s, but settings are not configured. "
"You must either define the environment variable %s "
"or call settings.configure() before accessing settings."
% (desc, ENVIRONMENT_VARIABLE))
self._wrapped = Settings(settings_module)
So if you do:
from django.conf import settings
having that environment variable settled, the statement
settings.configure()
will fail with RuntimeError('Settings already configured.')

How can I perform Django's `syncdb --noinput` with call_command?

>>> from django.core.management import call_command
>>> call_command('syncdb')
executes the syncdb management command from within a python script. However, I want to run the equivalent of
$ python manage.py syncdb --noinput
from within a python shell or script. How can I do that?
The following lines don't work without interrupting me with the question whether I want to create a super user.
>>> call_command('syncdb', noinput = True) # asks for input
>>> call_command('syncdb', 'noinput') # raises an exception
I use Django 1.3.
call_command('syncdb', interactive = False)
EDIT:
I found the answer in the source code. The source code for all management commands can be found in a python module called management/commands/(command_name).py
The python module where the syncdb command resides is django.core.management.commands.syncdb
To find the source code of the command you can do something like this:
(env)$ ./manage.py shell
>>> from django.core.management.commands import syncdb
>>> syncdb.__file__
'/home/user/env/local/lib/python2.7/site-packages/django/core/management/commands/syncdb.pyc'
>>>
Of course, check the contents of syncdb.py, and not syncdb.pyc.
Or looking at the online source, the syncdb.py script contains:
make_option('--noinput', action='store_false', dest='interactive', default=True,
help='Tells Django to NOT prompt the user for input of any kind.'),
that tells us that instead of --noinput on the command line, we should use interactive if we want to automate commands with the call_command function.

Django timer thread

I would like to compute some information in my Django application on regular basis.
I need to select and insert data each second and want to use Django ORM.
How can I do this?
In a shell script, set the DJANGO_SETTINGS_MODULE variable and call a python script
export DJANGO_SETTINGS_MODULE=yourapp.settings
python compute_some_info.py
In compute_some_info.py, set up django and import your modules (look at how the manage.py script sets up to run Django)
#!/usr/bin/env python
import sys
try:
import settings # Assumed to be in the same directory.
except ImportError:
sys.stderr.write("Error: Can't find the file 'settings.py'")
sys.exit(1)
sys.path = sys.path + ['/yourapphome']
from yourapp.models import YourModel
YourModel.compute_some_info()
Then call your shell script in a cron job.
Alternatively -- you can just keep running and sleeping (better if it's every second) -- you would still want to be outside of the webserver and in your own process that is set up this way.
One way to do it would be to create a custom command, and invoke python manage.py your_custom_command from cron or windows scheduler.
http://docs.djangoproject.com/en/dev/howto/custom-management-commands/
For example, create myapp/management/commands/myapp_task.py which reads:
from django.core.management.base import NoArgsCommand
class Command(NoArgsCommand):
def handle_noargs(self, **options):
print 'Doing task...'
# invoke the functions you need to run on your project here
print 'Done'
Then you can run it from cron like this:
export DJANGO_SETTINGS_MODULE=myproject.settings; export PYTHONPATH=/path/to/project_parent; python manage.py myapp_task

Programmatically using Django's loaddata

I'd like to call the equivalent of manage.py loaddata from a Django view. I'd like to be able to specify where to load the data from and which application to load it into.
Any ideas?
Each django-admin.py (manage.py) command, as seen in the documentation, you can call from your code with:
from django.core.management import call_command
call_command('loaddata', 'myapp')
Where first param is the command name, all other position params are the same as command line position params and all keyword params are options.