Avoiding circular imports in Django Models (Config class) - django

I've created a Configuration model in django so that the site admin can change some settings on the fly, however some of the models are reliant on these configurations. I'm using Django 2.0.2 and Python 3.6.4.
I created a config.py file in the same directory as models.py.
Let me paracode (paraphase the code? Real Enum has many more options):
# models.py
from .config import *
class Configuration(models.Model):
starting_money = models.IntegerField(default=1000)
class Person(models.Model):
funds = models.IntegarField(default=getConfig(ConfigData.STARTING_MONEY))
# config.py
from .models import Configuration
class ConfigData(Enum):
STARTING_MONEY = 1
def getConfig(data):
if not isinstance(data, ConfigData):
raise TypeError(f"{data} is not a valid configuration type")
try:
config = Configuration.objects.get_or_create()
except Configuration.MultipleObjectsReturned:
# Cleans database in case multiple configurations exist.
Configuration.objects.exclude(Configuration.objects.first()).delete()
return getConfig(data)
if data is ConfigData.MAXIMUM_STAKE:
return config.max_stake
How can I do this without an import error? I've tried absolute imports

You can postpone loading the models.py by loading it in the getConfig(data) function, as a result we no longer need models.py at the time we load config.py:
# config.py (no import in the head)
class ConfigData(Enum):
STARTING_MONEY = 1
def getConfig(data):
from .models import Configuration
if not isinstance(data, ConfigData):
raise TypeError(f"{data} is not a valid configuration type")
try:
config = Configuration.objects.get_or_create()
except Configuration.MultipleObjectsReturned:
# Cleans database in case multiple configurations exist.
Configuration.objects.exclude(Configuration.objects.first()).delete()
return getConfig(data)
if data is ConfigData.MAXIMUM_STAKE:
return config.max_stake
We thus do not load models.py in the config.py. We only check if it is loaded (and load it if not) when we actually execute the getConfig function, which is later in the process.

Willem Van Onsem's solution is a good one. I have a different approach which I have used for circular model dependencies using django's Applications registry. I post it here as an alternate solution, in part because I'd like feedback from more experienced python coders as to whether or not there are problems with this approach.
In a utility module, define the following method:
from django.apps import apps as django_apps
def model_by_name(app_name, model_name):
return django_apps.get_app_config(app_name).get_model(model_name)
Then in your getConfig, omit the import and replace the line
config = Configuration.objects.get_or_create()
with the following:
config_class = model_by_name(APP_NAME, 'Configuration')
config = config_class.objects.get_or_create()

Related

how to keep data created in the ready method? Django production vs test database

As you know django give you clear database in testing, but I have a ready() method that create some data for me and I need to query these data in my tests.
class YourAppConfig(AppConfig):
default_auto_field = 'django.db.models.AutoField'
name = 'Functions.MyAppsConfig'
def ready(self):
from django.contrib.auth.models import Permission
from django import apps
from django.contrib.contenttypes.models import ContentType
try:
Permission.objects.get_or_create(....)
MyOtherModel.objects.get_or_create(....)
except:
pass
class TestRules(APITestCase):
def test_my_model(self):
....
x = MyOtherModel.objects.filter(....).first()
# x = None # <=========== problem is here ========= I need to see the data that I created in the ready method
....
You can use the fixtures for that, in each Test case you can fixtures to it as stated documentation example is
class Test(TransactionTestCase):
fixtures = ['user-data.json']
def setUp():
…
Django will load the fixtures before under test case

Testing Django Rest Framework: how to test hyperlink relations?

I'm trying to create a true unit test for a customized DjangoRestFramework Hyperlinked related field. But I cannot seem to get around this error:
django.core.exceptions.ImproperlyConfigured: Could not resolve URL for hyperlinked relationship using view name "relatedtestmodel-detail". You may have failed to include the related model in your API, or incorrectly configured the `lookup_field` attribute on this field.
And here is the unit test, stripped down to simplify the example:
from django.conf.urls import url
from django.test import TestCase, override_settings
from api_tests.models import APITestModel, RelatedTestModel
from api_tests.serializers import APITestModelSerializer
def dummy_view(request, pk):
pass
urlpatterns = [
url(r'/path/is/irrelevant/', dummy_view, name='relatedtestmodel-detail')
]
#override_settings(ROOT_URLCONF='tests.test_relations')
class HyperlinkedRelatedFieldTestCase(TestCase):
def setUp(self):
self.parent = APITestModel.objects.create()
self.child = RelatedTestModel.objects.create(parent=self.parent)
assert self.child.parent.id == self.parent.id
def test_to_internal_value_correct_error_message(self):
queryset = APITestModel.objects.all()
serializer = APITestModelSerializer(queryset, many=True, context={'request': None})
expected = [{'foo': 'bar'}]
self.assertEqual(serializer.data, expected)
I more or less lifted the test from https://github.com/encode/django-rest-framework/blob/master/tests/test_relations_hyperlink.py, because I figured who knows best how to unit test DRF than the makers of DRF? But as it stands, my test refuses to run. The error is raised during the test, when I attempt to access serializer.data for the assert.
Notice in particular that I override the settings with a custom urlpatterns (which is this same file, hence the urlpatterns at the top). So I don't understand why DRF thinks that url name doesn't exist - I can clearly see that my url conf in fact has ONLY that view name! I've even gone so far as to edit my actual URL conf and replace it with the single, named, dummy url pattern shown here, and removedthe settings override, just to be sure that it wasn't that the override_settings simply wasn't working, but even then I get the same error.
To my eye, the dummy url pattern is exactly the same as how DRF did it in their tests. Anyone have any ideas what is going on?
A bit more requested context:
api_tests.models:
from django.db import models
class APITestModel(models.Model):
pass
class RelatedTestModel(models.Model):
parent = models.ForeignKey(
APITestModel,
related_name='children',
related_query_name='child'
)
I do not have access to the traceback at this time, but I can confirm it did not pass through any of my code - it was all isolated to the DjangoRestFramework code, basically exclusively relations.py
Preamble
A few things this question is lacking
No definition of APITestModelSerializer
RelatedTestModel is not used in the test and therefore irrelevant for the example
No error stacktrace
No "useful" expected dict for the asserts
APITestModel has no fields so it can't be serialized (your test shouldn't even have run)
Minor things but still relevant
You are creating specific instances of APITestModel and RelatedTestModel in the setUp but in the test you serialize all instances of APITestModel
The line assert self.child.parent.id == self.parent.id should not be in the setUp. It should be in a separate test
My changes
I deleted all irrelevant information for this question mentioned above
I added an integer field to APITestModel
I changed the urlpatterns element from url(...) to path(...)
I added a regex to the relative path
The serializer is a subclass of HyperlinkedModelSerializer and includes fields "url" and "year"
My project and app urls.py files are the "stock" ones (not shown here) to emphasize that this test resolves the path in isolation.
Changed #override_settings(ROOT_URLCONF='tests.test_relations') to #override_settings(ROOT_URLCONF=__name__)
Code
models.py
from django.db import models
class APITestModel(models.Model):
year = models.IntegerField(null=False)
serializers.py
from rest_framework import serializers
from api.models import APITestModel
class APITestModelSerializer(serializers.HyperlinkedModelSerializer):
class Meta:
model = APITestModel
fields = ["url", "year"]
test_serializers.py
from django.test import TestCase, override_settings
from django.urls import path
from api.models import APITestModel
from api.serializers import APITestModelSerializer
urlpatterns = [
path('whateveryouwant/<int:pk>/', lambda request: None, name='apitestmodel-detail'),
]
#override_settings(ROOT_URLCONF=__name__)
class HyperlinkedRelatedFieldTestCase(TestCase):
def setUp(self):
# Populate db with APITestModel instances
_ = APITestModel.objects.create(year=1960)
_ = APITestModel.objects.create(year=1961)
_ = APITestModel.objects.create(year=1962)
def test_to_internal_value_correct_error_message(self):
queryset = APITestModel.objects.all()
serializer = APITestModelSerializer(queryset, many=True, context={'request': None})
expected = [
{'url': '/whateveryouwant/1/', 'year': 1960},
{'url': '/whateveryouwant/2/', 'year': 1961},
{'url': '/whateveryouwant/3/', 'year': 1962},
]
self.assertEqual(serializer.data, expected)
The other files in the project are the default ones created automatically by django + djangorestframework.
For future readers, I created a github project with this working code and can be found here: https://github.com/Alechan/drf_test_hyperlink_relations

Django test script to pre-populate DB

I'm trying to pre-populate the database with some test data for my Django project. Is there some easy way to do this with a script that's "outside" of Django?
Let's say I want to do this very simple task, creating 5 test users using the following code,
N = 10
i = 0
while i < N:
c = 'user' + str(i) + '#gmail.com'
u = lancer.models.CustomUser.objects.create_user(email=c, password="12345")
i = i + 1
The questions are,
WHERE do I put this test script file?
WHAT IMPORTS / COMMANDS do I need to put at the beginning of the file so it has access to all the Django environment & resources as if I were writing this inside the app?
I'm thinking you'd have to import and set up the settings file, and import the app's models, etc... but all my attempts have failed one way or another, so would appreciate some help =)
Thanks!
Providing another answer
The respondes below are excellent answers. I fiddled around and found an alternative way. I added the following to the top of the test data script,
from django.core.management import setup_environ
from project_lancer import settings
setup_environ(settings)
import lancer.models
Now my code above works.
I recommend you to use fixtures for these purposes:
https://docs.djangoproject.com/en/dev/howto/initial-data/
If you still want to use this initial code then read:
If you use south you can create migration and put this code there:
python manage.py schemamigration --empty my_data_migration
class Migration(SchemaMigration):
no_dry_run = False
def forwards(self, orm):
# more pythonic, you can also use bulk_insert here
for i in xrange(10):
email = "user{}#gmail.com".format(i)
u = orm.CustomUser.objects.create_user(email=email, password='12345)
You can put it to setUp method of your TestCase:
class MyTestCase(TestCase):
def setUp(self):
# more pythonic, you can also use bulk_insert here
for i in xrange(10):
email = "user{}#gmail.com".format(i)
u = lancer.models.CustomUser.objects.create_user(email=email,
password='12345')
def test_foo(self):
pass
Also you can define your BaseTestCase in which you override setUp method then you create your TestCase classes that inherit from BaseTestCase:
class BaseTestCase(TestCase):
def setUp(self):
'your initial logic here'
class MyFirstTestCase(BaseTestCase):
pase
class MySecondTestCase(BaseTestCase):
pase
But I think that fixtures is the best way:
class BaseTestCase(TestCase):
fixtures = ['users_for_test.json']
class MyFirstTestCase(BaseTestCase):
pase
class MySecondTestCase(BaseTestCase):
fixtures = ['special_users_for_only_this_test_case.json']
Updated:
python manage.py shell
from django.contrib.auth.hashers import make_password
make_password('12312312')
'pbkdf2_sha256$10000$9KQ15rVsxZ0t$xMEKUicxtRjfxHobZ7I9Lh56B6Pkw7K8cO0ow2qCKdc='
You can also use something like this or this to auto-populate your models for testing purposes.

Django with Pluggable MongoDB Storage troubles

I'm trying to use django, and mongoengine to provide the storage backend only with GridFS. I still have a MySQL database.
I'm running into a strange (to me) error when I'm deleting from the django admin and am wondering if I am doing something incorrectly.
my code looks like this:
# settings.py
from mongoengine import connect
connect("mongo_storage")
# models.py
from mongoengine.django.storage import GridFSStorage
class MyFile(models.Model):
name = models.CharField(max_length=50)
content = models.FileField(upload_to="appsfiles", storage=GridFSStorage())
creation_time = models.DateTimeField(auto_now_add=True)
last_update_time = models.DateTimeField(auto_now=True)
I am able to upload files just fine, but when I delete them, something seems to break and the mongo database seems to get in an unworkable state until I manually delete all FileDocument.objects. When this happens I can't upload files or delete them from the django interface.
From the stack trace I have:
/home/projects/vector/src/mongoengine/django/storage.py in _get_doc_with_name
doc = [d for d in docs if getattr(d, self.field).name == name] ...
▼ Local vars
Variable Value
_[1]
[]
d
docs
Error in formatting: cannot set options after executing query
name
u'testfile.pdf'
self
/home/projects/vector/src/mongoengine/fields.py in __getattr__
raise AttributeError
Am I using this feature incorrectly?
UPDATE:
thanks to #zeekay's answer I was able to get a working gridfs storage plugin to work. I ended up not using mongoengine at all. I put my adapted solution on github. There is a clear sample project showing how to use it. I also uploaded the project to pypi.
Another Update:
I'd highly recommend the django-storages project. It has lots of storage backed options and is used by many more people than my original proposed solution.
I think you are better off not using MongoEngine for this, I haven't had much luck with it either. Here is a drop-in replacement for mongoengine.django.storage.GridFSStorage, which works with the admin.
from django.core.files.storage import Storage
from django.conf import settings
from pymongo import Connection
from gridfs import GridFS
class GridFSStorage(Storage):
def __init__(self, host='localhost', port=27017, collection='fs'):
for s in ('host', 'port', 'collection'):
name = 'GRIDFS_' + s.upper()
if hasattr(settings, name):
setattr(self, s, getattr(settings, name))
for s, v in zip(('host', 'port', 'collection'), (host, port, collection)):
if v:
setattr(self, s, v)
self.db = Connection(host=self.host, port=self.port)[self.collection]
self.fs = GridFS(self.db)
def _save(self, name, content):
self.fs.put(content, filename=name)
return name
def _open(self, name, *args, **kwars):
return self.fs.get_last_version(filename=name)
def delete(self, name):
oid = fs.get_last_version(filename=name)._id
self.fs.delete(oid)
def exists(self, name):
return self.fs.exists({'filename': name})
def size(self, name):
return self.fs.get_last_version(filename=name).length
GRIDFS_HOST, GRIDFS_PORT and GRIDFS_COLLECTION can be defined in your settings or passed as host, port, collection keyword arguments to GridFSStorage in your model's FileField.
I referred to Django's custom storage documenation, and loosely followed this answer to a similar question.

Sphinx documentation on Django templatetags

I am using Sphinx with autodoc for documenting my Django project.
Design guys want to have documentation page(s) about all the template tags that are defined in the project. Of course, I can make such a page by enumerating all the template processing functions by hand, but I think it is not DRY, isn't it? In fact, template tag processing functions are all marked with #register.inclusion_tag decorator. So it seems to be possible and natural for some routine to collect them all and put into documentation.
The same about filter functions.
I've googled it, searched Django documentation but in vein. I can hardly believe that such a natural functionality hasn't been implemented by someone.
I didn't stop at this point and implemented a Sphinx autodoc extension.
Snippet 2. Sphinx autodoc extension
"""
Extension of Sphinx autodoc for Django template tag libraries.
Usage:
.. autotaglib:: some.module.templatetags.mod
(options)
Most of the `module` autodoc directive flags are supported by `autotaglib`.
Andrew "Hatter" Ponomarev, 2010
"""
from sphinx.ext.autodoc import ModuleDocumenter, members_option, members_set_option, bool_option, identity
from sphinx.util.inspect import safe_getattr
from django.template import get_library, InvalidTemplateLibrary
class TaglibDocumenter(ModuleDocumenter):
"""
Specialized Documenter subclass for Django taglibs.
"""
objtype = 'taglib'
directivetype = 'module'
content_indent = u''
option_spec = {
'members': members_option, 'undoc-members': bool_option,
'noindex': bool_option,
'synopsis': identity,
'platform': identity, 'deprecated': bool_option,
'member-order': identity, 'exclude-members': members_set_option,
}
#classmethod
def can_document_member(cls, member, membername, isattr, parent):
# don't document submodules automatically
return False
def import_object(self):
"""
Import the taglibrary.
Returns True if successful, False if an error occurred.
"""
# do an ordinary module import
if not super(ModuleDocumenter, self).import_object():
return False
try:
# ask Django if specified module is a template tags library
# and - if it is so - get and save Library instance
self.taglib = get_library(self.object.__name__)
return True
except InvalidTemplateLibrary, e:
self.taglib = None
self.directive.warn(unicode(e))
return False
def get_object_members(self, want_all):
"""
Decide what members of current object must be autodocumented.
Return `(members_check_module, members)` where `members` is a
list of `(membername, member)` pairs of the members of *self.object*.
If *want_all* is True, return all members. Else, only return those
members given by *self.options.members* (which may also be none).
"""
if want_all:
return True, self.taglib.tags.items()
else:
memberlist = self.options.members or []
ret = []
for mname in memberlist:
if mname in taglib.tags:
ret.append((mname, self.taglib.tags[mname]))
else:
self.directive.warn(
'missing templatetag mentioned in :members: '
'module %s, templatetag %s' % (
safe_getattr(self.object, '__name__', '???'), mname))
return False, ret
def setup(app):
app.add_autodocumenter(TaglibDocumenter)
This extension defines Sphinx directive autotaglib which behaves much like automodule, but enumerates only tag implementing functions.
Example:
.. autotaglib:: lib.templatetags.bfmarkup
:members:
:undoc-members:
:noindex:
For the record, Django has an automatic documentation system (add django.contrib.admindocs to your INSTALLED_APPS).
This will give you extra views in the admin (usually at /admin/docs/) that represent your models, views (based on the URL), template tags and template filters.
More documentation for this can be found in the admindocs section.
You can take a look at that code to include it in your documentation or at the extensions for the Django documentation.
I solved the problem and would like to share my snippets - in case they will be useful to someone.
Snippet 1. Simple documenter
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'project.settings'
from django.template import get_library
def show_help(libname):
lib = get_library(libname)
print lib, ':'
for tag in lib.tags:
print tag
print lib.tags[tag].__doc__
if __name__ == '__main__':
show_help('lib.templatetags.bfmarkup')
Before running this script you should setup PYTHONPATH environment variable.