Django's callable model field default are broken. When I set a model field default equal to a callable (see migration file below), all models get the same value even though the callable returns different values each time it is called.
This worked on previous fields, so I'm confused why Django would be failing on this field. Everything works up until the point I migrate, when inspecting the database column reveals all values to be the same.
Migration file:
# -*- coding: utf-8 -*-
# Generated by Django 1.10.1 on 2018-05-10 14:53
from __future__ import unicode_literals
from django.db import migrations, models
import screen.models
class Migration(migrations.Migration):
dependencies = [
('screen', '0064_employer_enable_show_question_template_tags'),
]
operations = [
migrations.AddField(
model_name='question',
name='key',
field=models.TextField(default=screen.models.generate_question_key),
),
]
Model
def generate_question_key():
# return a random string 64 characters long
return generate_random_hash(64)
class Question(Model):
name = TextField()
points = PositiveIntegerField() # how much it's worth
time_spent = PositiveIntegerField(default = 0) # seconds
score = PositiveIntegerField(default = 0)
key = TextField(default = generate_question_key)
The default returns different values:
>>> import screen
>>> screen.models.generate_question_key()
'JpZzloZkiLyvPLrDZ9764VTWkNUon1FD08mGKODa2uiqW1nV422HXVvt78MsW7aR'
>>> screen.models.generate_question_key()
'NHyTwPDA2cAAsTeIR77INLMM6Ik14EQ6vTlrTv4ZwV56nt6jGEtR8bKn8iyWDeMA'
>>> screen.models.generate_question_key()
'q2aALA7WmvtiKLiGXfNEStpKhOFcNpMDrJ8Y9sv6mwWNsUU6mdgMlgaW5yJJ1yEI'
>>>
It looks like this is a limitation of Django: https://docs.djangoproject.com/en/1.11/howto/writing-migrations/#migrations-that-add-unique-fields
Related
As you know django give you clear database in testing, but I have a ready() method that create some data for me and I need to query these data in my tests.
class YourAppConfig(AppConfig):
default_auto_field = 'django.db.models.AutoField'
name = 'Functions.MyAppsConfig'
def ready(self):
from django.contrib.auth.models import Permission
from django import apps
from django.contrib.contenttypes.models import ContentType
try:
Permission.objects.get_or_create(....)
MyOtherModel.objects.get_or_create(....)
except:
pass
class TestRules(APITestCase):
def test_my_model(self):
....
x = MyOtherModel.objects.filter(....).first()
# x = None # <=========== problem is here ========= I need to see the data that I created in the ready method
....
You can use the fixtures for that, in each Test case you can fixtures to it as stated documentation example is
class Test(TransactionTestCase):
fixtures = ['user-data.json']
def setUp():
…
Django will load the fixtures before under test case
I'm trying to create a true unit test for a customized DjangoRestFramework Hyperlinked related field. But I cannot seem to get around this error:
django.core.exceptions.ImproperlyConfigured: Could not resolve URL for hyperlinked relationship using view name "relatedtestmodel-detail". You may have failed to include the related model in your API, or incorrectly configured the `lookup_field` attribute on this field.
And here is the unit test, stripped down to simplify the example:
from django.conf.urls import url
from django.test import TestCase, override_settings
from api_tests.models import APITestModel, RelatedTestModel
from api_tests.serializers import APITestModelSerializer
def dummy_view(request, pk):
pass
urlpatterns = [
url(r'/path/is/irrelevant/', dummy_view, name='relatedtestmodel-detail')
]
#override_settings(ROOT_URLCONF='tests.test_relations')
class HyperlinkedRelatedFieldTestCase(TestCase):
def setUp(self):
self.parent = APITestModel.objects.create()
self.child = RelatedTestModel.objects.create(parent=self.parent)
assert self.child.parent.id == self.parent.id
def test_to_internal_value_correct_error_message(self):
queryset = APITestModel.objects.all()
serializer = APITestModelSerializer(queryset, many=True, context={'request': None})
expected = [{'foo': 'bar'}]
self.assertEqual(serializer.data, expected)
I more or less lifted the test from https://github.com/encode/django-rest-framework/blob/master/tests/test_relations_hyperlink.py, because I figured who knows best how to unit test DRF than the makers of DRF? But as it stands, my test refuses to run. The error is raised during the test, when I attempt to access serializer.data for the assert.
Notice in particular that I override the settings with a custom urlpatterns (which is this same file, hence the urlpatterns at the top). So I don't understand why DRF thinks that url name doesn't exist - I can clearly see that my url conf in fact has ONLY that view name! I've even gone so far as to edit my actual URL conf and replace it with the single, named, dummy url pattern shown here, and removedthe settings override, just to be sure that it wasn't that the override_settings simply wasn't working, but even then I get the same error.
To my eye, the dummy url pattern is exactly the same as how DRF did it in their tests. Anyone have any ideas what is going on?
A bit more requested context:
api_tests.models:
from django.db import models
class APITestModel(models.Model):
pass
class RelatedTestModel(models.Model):
parent = models.ForeignKey(
APITestModel,
related_name='children',
related_query_name='child'
)
I do not have access to the traceback at this time, but I can confirm it did not pass through any of my code - it was all isolated to the DjangoRestFramework code, basically exclusively relations.py
Preamble
A few things this question is lacking
No definition of APITestModelSerializer
RelatedTestModel is not used in the test and therefore irrelevant for the example
No error stacktrace
No "useful" expected dict for the asserts
APITestModel has no fields so it can't be serialized (your test shouldn't even have run)
Minor things but still relevant
You are creating specific instances of APITestModel and RelatedTestModel in the setUp but in the test you serialize all instances of APITestModel
The line assert self.child.parent.id == self.parent.id should not be in the setUp. It should be in a separate test
My changes
I deleted all irrelevant information for this question mentioned above
I added an integer field to APITestModel
I changed the urlpatterns element from url(...) to path(...)
I added a regex to the relative path
The serializer is a subclass of HyperlinkedModelSerializer and includes fields "url" and "year"
My project and app urls.py files are the "stock" ones (not shown here) to emphasize that this test resolves the path in isolation.
Changed #override_settings(ROOT_URLCONF='tests.test_relations') to #override_settings(ROOT_URLCONF=__name__)
Code
models.py
from django.db import models
class APITestModel(models.Model):
year = models.IntegerField(null=False)
serializers.py
from rest_framework import serializers
from api.models import APITestModel
class APITestModelSerializer(serializers.HyperlinkedModelSerializer):
class Meta:
model = APITestModel
fields = ["url", "year"]
test_serializers.py
from django.test import TestCase, override_settings
from django.urls import path
from api.models import APITestModel
from api.serializers import APITestModelSerializer
urlpatterns = [
path('whateveryouwant/<int:pk>/', lambda request: None, name='apitestmodel-detail'),
]
#override_settings(ROOT_URLCONF=__name__)
class HyperlinkedRelatedFieldTestCase(TestCase):
def setUp(self):
# Populate db with APITestModel instances
_ = APITestModel.objects.create(year=1960)
_ = APITestModel.objects.create(year=1961)
_ = APITestModel.objects.create(year=1962)
def test_to_internal_value_correct_error_message(self):
queryset = APITestModel.objects.all()
serializer = APITestModelSerializer(queryset, many=True, context={'request': None})
expected = [
{'url': '/whateveryouwant/1/', 'year': 1960},
{'url': '/whateveryouwant/2/', 'year': 1961},
{'url': '/whateveryouwant/3/', 'year': 1962},
]
self.assertEqual(serializer.data, expected)
The other files in the project are the default ones created automatically by django + djangorestframework.
For future readers, I created a github project with this working code and can be found here: https://github.com/Alechan/drf_test_hyperlink_relations
In my Django site there are some permissions entries linked to applications that I've removed. For example I have permissions entries linked to "Dashboard" and "Jet" applications. How can you remove them?
Permissions have foreign keys to content types under the hood, so removing the content types for the models that no longer exist will also remove the permissions for those models.
Fortunately, Django also provides a manage.py command to remove old content types: remove_stale_contenttypes. Running that command will list the content types that no longer exist and the related objects (including permissions) that will be deleted, allowing you to review the changes and approve them.
$ manage.py remove_stale_contenttypes
Some content types in your database are stale and can be deleted.
Any objects that depend on these content types will also be deleted.
The content types and dependent objects that would be deleted are:
- Content type for stale_app.removed_model
- 4 auth.Permission object(s)
This list doesn't include any cascade deletions to data outside of Django's
models (uncommon).
Are you sure you want to delete these content types?
If you're unsure, answer 'no'.
Type 'yes' to continue, or 'no' to cancel:
To start, make an empty migration file:
python manage.py makemigrations --empty yourappname
Change the migration (this is an example, adjust to your needs):
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import migrations
def add_permissions(apps, schema_editor):
pass
def remove_permissions(apps, schema_editor):
"""Reverse the above additions of permissions."""
ContentType = apps.get_model('contenttypes.ContentType')
Permission = apps.get_model('auth.Permission')
content_type = ContentType.objects.get(
model='somemodel',
app_label='yourappname',
)
# This cascades to Group
Permission.objects.filter(
content_type=content_type,
codename__in=('add_somemodel', 'change_somemodel', 'delete_somemodel'),
).delete()
class Migration(migrations.Migration):
dependencies = [
('yourappname', '0001_initial'),
]
operations = [
migrations.RunPython(remove_permissions, add_permissions),
]
I did it this way:
import re
for perm in Permission.objects.all():
if re.match( r".+modelname.+permissionname.+",str(perm)):
print(perm)
perm.delete()
If you have custom or model based (default) permissions you wish to remove you could write a command like this to accomplish this task:
from django.conf import settings
from django.contrib.auth.models import Permission
from django.core.management.base import BaseCommand
import django.apps
class Command(BaseCommand):
help = 'Remove custom permissions that are no longer in models'
def handle(self, *args, **options):
# get the db name needed for removal...
database_name = input('Database Name: ')
default_perm_names = list()
# are real perms in db, may not be accurate
db_custom_perm_names = list()
# will be used to ensure they are correct.
meta_custom_perm_names = list()
default_and_custom_perms = list()
for model in django.apps.apps.get_models():
# add to models found to fix perms from removed models
app_label = model._meta.app_label
lower_model_name = model._meta.model_name
all_model_permissions = Permission.objects.using(database_name).filter(content_type__app_label=app_label, content_type__model=lower_model_name)
default_and_custom_perms.extend([x for x in all_model_permissions])
# get the custom meta permissions, these should be in the meta of the class
# will be a list or tuple or list, [0=codename, 1=name]
meta_permissions = model._meta.permissions
if meta_permissions:
for perm in all_model_permissions:
# will be the model name from the content type, this is how django makes default perms
# we are trying to remove them so now we can figure out which ones are default by provided name
model_name_lower = perm.content_type.name
# default_perms = ['add', 'change', 'view', 'delete', 'undelete']
# append them to the list of default names
default_perm_names.append(f'Can add {model_name_lower}')
default_perm_names.append(f'Can change {model_name_lower}')
default_perm_names.append(f'Can view {model_name_lower}')
default_perm_names.append(f'Can delete {model_name_lower}')
default_perm_names.append(f'Can undelete {model_name_lower}')
# will mean this is a custom perm...so add it
if not perm.name in default_perm_names:
db_custom_perm_names.append(perm.name)
# the perms to ensure are correct...
for model_perm in meta_permissions:
# get the meta perm, will be a list or tuple or list, [0=codename, 1=name]
custom_perm = Permission.objects.using(database_name).get(codename=model_perm[0], name=model_perm[1])
meta_custom_perm_names.append(custom_perm.name)
perms_to_remove = [perm for perm in db_custom_perm_names if perm not in meta_custom_perm_names]
if not perms_to_remove:
print('There are no stale custom permissions to remove.')
# print(perms_to_remove)
# now remove the custom permissions that were removed from the model
for actual_permission_to_remove in Permission.objects.using(database_name).filter(name__in=perms_to_remove):
# print(actual_permission_to_remove)
actual_permission_to_remove.delete(using=database_name)
print(actual_permission_to_remove, '...deleted')
for perm in [x for x in Permission.objects.using(database_name)]:
# loop all perms...if it is not in the model perms it does not exist...
if perm.content_type.model not in [x.content_type.model for x in default_and_custom_perms]:
perm.delete(using=database_name)
print(perm, 'regular permission...deleted')
If you also wish to ensure that the default permissions are added from Django you can add this snippet in the command:
from django.apps import apps
from django.contrib.auth.management import create_permissions
from apps.client.models import ClientInformation
# add all permissions the django way
# get the db name needed from settings.py
database_name = 'default' # or whatever DB you are looking for
print(f'adding all permissions if not there to {database_name}')
for app_config in apps.get_app_configs():
# print(app_config)
app_config.models_module = True
create_permissions(app_config, using=database_name)
app_config.models_module = None
Then call via python manage.py fix_permissions if you name your command file fix_permissions.py
I've reworked #ViaTech's code to use Django's contrib.auth.management._get_all_permissions() functions which makes it more straight forward:
from typing import List, Set, Tuple
import django.apps
# noinspection PyProtectedMember
from django.contrib.auth.management import _get_all_permissions
from django.contrib.auth.models import Permission
from django.contrib.contenttypes.models import ContentType
from django.core.management.base import BaseCommand
from django.db import DEFAULT_DB_ALIAS
class Command(BaseCommand):
help = "Remove custom permissions that are no longer defined in models"
def add_arguments(self, parser):
parser.add_argument(
"--database",
default=DEFAULT_DB_ALIAS,
help=f'Specifies the database to use. Default is "{DEFAULT_DB_ALIAS}".',
)
parser.add_argument(
"--dry",
action="store_true",
help="Do a dry run not actually deleting any permissions",
)
def handle(self, *args, **options) -> str:
using = options["database"]
# This will hold the permissions that models have defined,
# i.e. default permissions plus additional custom permissions:
# (content_type.pk, codename)
defined_perms: List[Tuple[int, str]] = []
for model in django.apps.apps.get_models():
ctype = ContentType.objects.db_manager(using).get_for_model(
model, for_concrete_model=False
)
# noinspection PyProtectedMember
for (codename, _) in _get_all_permissions(model._meta):
defined_perms.append((ctype.id, codename))
# All permissions in current database (including stale ones)
all_perms = Permission.objects.using(using).all()
stale_perm_pks: Set[int] = set()
for perm in all_perms:
if (perm.content_type.pk, perm.codename) not in defined_perms:
stale_perm_pks.add(perm.pk)
self.stdout.write(f"Delete permission: {perm}")
# Delete all stale permissions
if options["dry"]:
result = f"DRY RUN: {len(stale_perm_pks)} stale permissions NOT deleted"
else:
if stale_perm_pks:
Permission.objects.filter(pk__in=stale_perm_pks).delete()
result = f"{len(stale_perm_pks)} stale permissions deleted"
return result
We are making a Pc builder compare website, so you can compare pc components prices and make a cheap pc. We are using external data with Scrapy that saves it into MongoDB, we use Django as framework.
I have a 'prijs' Array (price) of an object as unicode, this object is in the QuerySet. I want that each 'prijs' in that array is a Float instead of unicode. Is there some way to convert the unicode array to a normal float array. I Googled alot but no solution for me, I hope you can help me with our problem.
Sorry for my bad English, thanks in advance!
Here is one object of MongoDB:
> db.processoren.findOne()
{
"_id" : ObjectId("547db39af2125f612cb8a1e5"),
"info" : [
"Quad-core, Inclusief koeler"
],
"sku" : [
"BX80646I54590"
],
"prijs" : [
"188.00000000000000000000"
],
"categorie" : "processoren",
"herkomst" : "paradigit",
"naam" : [
"Intel Core i5-4590 - 3.3GHz - Socket 1150"
],
"link" : "http://www.paradigit.nl/intel-core-i5-4590-3-3ghz-socket-1150/80015736/details.aspx",
"stock" : [
"Op dit moment niet beschikbaar"
]
}
models.py
from django.db import models
from mongoengine import *
from APc.settings import DBNAME
connect(DBNAME)
class Processoren(Document):
herkomst = StringField(max_length=200)
categorie = StringField(max_length=120)
naam = StringField(max_length=500)
subnaam = StringField(max_length=500)
info = StringField(max_length=500)
prijs = FloatField()
stock = StringField(max_length=500)
ean = StringField(max_length=200)
sku = StringField(max_length=200)
herkomst = StringField(max_length=200)
link = StringField(max_length=200)
views.py
from django.shortcuts import render_to_response
from django.template.loader import render_to_string
from django.shortcuts import HttpResponseRedirect
from django.core.paginator import Paginator, EmptyPage, PageNotAnInteger
from django.template import RequestContext
from django.template import loader
from django.http import HttpResponse
from bson.json_util import dumps
from pcbuilder.compatibility import *
from pcbuilder.filters import *
import json as simplejson
from models import Processoren, Moederborden, Koeling, Behuizingen, Grafische, Harde, Dvd, Geheugen, Voeding
from itertools import chain
import json
def processoren(request):
processorenlijst = Processoren.objects
print type(processorenlijst[0].prijs[0])
This is the output of the print
<type 'unicode'>
You can easily convert a list of strings to floats by doing something like below.
unicode_list = [u'10', u'20', u'123.1']
[float(val) for val in unicode_list]
I've never used MongoDB but I know in db backends like MySQL and PSQL you enforce types. The ORM will then return the values as the correct types. Perhaps the real issue is the structure of your data.
Desciption:
I have a generic function
def gen(model_name,model_type):
objects = model_name.objects.all()
for object in objects:
object.model_type = Null (Or some activity)
object.save()
How Can I achieve the above ? Is it possible?
I would use get_model:
from django.db.models import get_model
mymodel = get_model('some_app', 'SomeModel')
As of Django 1.7 the django.db.models.loading is deprecated (to be removed in 1.9) in favor of the the new application loading system. The 1.7 docs give us the following instead:
$ python manage.py shell
Python 2.7.6 (default, Mar 5 2014, 10:59:47)
>>> from django.apps import apps
>>> User = apps.get_model(app_label='auth', model_name='User')
>>> print User
<class 'django.contrib.auth.models.User'>
>>>
if you pass in 'app_label.model_name' you could use contenttypes e.g.
from django.contrib.contenttypes.models import ContentType
model_type = ContentType.objects.get(app_label=app_label, model=model_name)
objects = model_type.model_class().objects.all()
The full answer for Django 1.5 is:
from django.db.models.loading import AppCache
app_cache = AppCache()
model_class = app_cache.get_model(*'myapp.MyModel'.split('.',1))