After upgrading from Django 1.8 to 1.11 I've been looking at a means of merging some records - some models have multiple entries with the same name field, for example. There's an answer here that would appear to have what I would need:
https://stackoverflow.com/a/41291137/1195207
I tried it with models like this:
class GeneralType(models.Model):
#...
domains = models.ManyToManyField(Domain, blank=True)
#...
class Domain(models.Model):
name = models.TextField(blank=False)
#...
...where Domain has various records with duplicate names. But, it fails at the point indicated:
def merge(primary_object, alias_objects=list(), keep_old=False):
"""
Use this function to merge model objects (i.e. Users, Organizations, Polls,
etc.) and migrate all of the related fields from the alias objects to the
primary object. This does not look at GenericForeignKeys.
Usage:
from django.contrib.auth.models import User
primary_user = User.objects.get(email='good_email#example.com')
duplicate_user = User.objects.get(email='good_email+duplicate#example.com')
merge(primary_user, duplicate_user)
"""
# ...snip....
for alias_object in alias_objects:
for related_object in alias_object._meta.related_objects:
related_name = related_object.get_accessor_name()
if related_object.field.many_to_one:
#...snip...
elif related_object.field.one_to_one:
#...snip...
elif related_object.field.many_to_many:
related_name = related_name or related_object.field.name
for obj in getattr(alias_object, related_name).all():
getattr(obj, related_name).remove(alias_object) # <- fails here
getattr(obj, related_name).add(primary_object)
The problem is apparently that 'GeneralType' object has no attribute 'generaltype_set'. Adding a related_name to GeneralType doesn't fix this - the script fails in the same manner but quoting the name I've now given it. I'm not quite sure what Django is up to here so any suggestions would be welcome.
Edit:
In a Django shell I can successfully reference GeneralType from Domain, so it's something about the script above that I'm not getting. Example:
>>> d = Domain.objects.first()
>>> d
<Domain: 16s RNA>
>>> d.generaltype_set
<django.db.models.fields.related_descriptors.ManyRelatedManager object at 0x11175ba90>
>>> d.generaltype_set.first()
<GeneralType: Greengenes>
>>> getattr(d,'generaltype_set')
<django.db.models.fields.related_descriptors.ManyRelatedManager object at 0x10aa38250>
I managed to come up with a workaround. It seems that everything would function if I referenced generaltype.domains in the getattr(obj, related_name) part of the script, so I modified it as follows just before the line marked as failing in the question above:
if obj.__class__.__name__ == 'GeneralType':
related_name = 'domains'
Everything ran as it should after that, it seems.
Related
I got a ValueError while trying to add model instances with a many-to-many relationship.
ValueError: "(Idea: hey)" needs to have a value for field "id" before this many-to-many relationship can be used.
A lot of responses were given here, but none was helpful.My (idea) solution was to "manually" input the "id" values.
>>> import django
>>> django.setup()
>>> from myapp1.models import Category, Idea
# Notice that I manually add an "id"
>>> id2=Idea.objects.create(
... title_en='tre',
... subtitle_en='ca',
... description_en='mata',
... id=5,
... is_original=True,
... )
>>> id2.save()
>>> cat22=Category(title_en='yo')
>>> cat22.save()
>>> id2.categories.add(cat22)
>>> Idea.objects.all()
<QuerySet [<Idea: tre>]>
>>> exit()
How do i command django to auto-add the "id" field?
Note: I tried adding autoField but failed, thanks
#python_2_unicode_compatible
class Idea(UrlMixin, CreationModificationDateMixin, MetaTagsMixin):
id = models.IntegerField(primary_key=True,)
title = MultilingualCharField(_("Title"), max_length=200,)
subtitle = MultilingualCharField(_("Subtitle"), max_length=200, blank=True,)
description = MultilingualTextField(_("Description"), blank=True,)
is_original = models.BooleanField(_("Original"), default=False,)
categories = models.ManyToManyField(Category,
You're confusing two things here:
With many-to-many relationships, when connecting two objects, both objects must already be saved to the database (have a primary key), because under the hoods, Django creates a third object that points at the two objects to connect them. It can only do that if both have an id, assuming id is the primary key.
When creating an object, you don't have to explicitly set the id (actually you shouldn't). By default, a django Model will have id set as an auto field and as a primary key (you can override that by specifying your own pk, but in general there's no need to). The id is automatically created when the model is saved the first time.
You saw the error because probably one of the objects (idea or category) wasn't saved to the database before you connected them. In your code sample, you don't have to pass id=5, it will work without it, because you save id2 and category before connecting them.
I'd like to update multiple integer fields at once in following model.
class Foo(models.Model):
field_a = models.PositiveIntegerField()
field_b = models.PositiveIntegerField()
field_c = models.PositiveIntegerField()
Originally, it can be done like following code with two queries.
foo = Foo.objects.get(id=1)
foo.field_a += 1
foo.field_b -= 1
foo.field_c += 2
foo.save()
I'd like make it more simpler with update in one query.
However, following attempts raised error.
# 1st attempt
Foo.objects.filter(id=1).update(
field_a=F('field_a')+1,
field_b=F('field_a')-1,
field_c=F('field_a')+2)
# 2nd attempt
Foo.objects.filter(id=1).\
update(field_a=F('field_a')+1).\
update(field_b=F('field_b')-1) ).\
update(field_c=F('field_c')+2)
How can I solve this ?
Form the django docs:
Calls to update can also use F expressions to update one field based on the value of another field in the model. This is especially useful for incrementing counters based upon their current value. For example, to increment the pingback count for every entry in the blog:
>>> from django.db.models import F
>>> Entry.objects.all().update(n_pingbacks=F('n_pingbacks') + 1)
You have to have an instance of Foo or a queryset before you can update. You should do something like this:
Foo.objects.get(id=1)update(field_a=F('field_a')+1).\
update(field_b=F('field_b')-1) ).\
update(field_c=F('field_c')+2)
or
Foo.objects.filter(id__in=[1,3,6,7]).update(field_a=F('field_a')+1).\
update(field_b=F('field_b')-1) ).\
update(field_c=F('field_c')+2)
Reference: https://docs.djangoproject.com/en/1.8/topics/db/queries/#updating-multiple-objects-at-once
If save() is passed a list of field names in keyword argument update_fields, only the fields named in that list will be updated. This may be desirable if you want to update just one or a few fields on an object. There will be a slight performance benefit from preventing all of the model fields from being updated in the database. For example:
product.name = 'Name changed again'
product.save(update_fields=['name'])
see more docs [here]:https://docs.djangoproject.com/en/dev/ref/models/instances/#specifying-which-fields-to-save
ForeignKeys on django have the attribute on_delete to specify the behavior when the referenced object is deleted. Is there any way to get something similar for ManyToManyField?
Suppose I have the following model
class House(models.Model):
owners = models.ManyToManyField(Person)
The default behavior is to cascade, so if I delete a person that happens to own a house, it just vanishes from owners (that is, obviously, it no longer owns any houses). What I'd like to have is that if a person is an owner, it can not be deleted. That is, I want on_delete=models.PROTECT. Is this possible?
I know internally ManyToManyField is translated to another model with two ForeignKeys (in this case one to house and one to person), so it should be possible to achieve this. Any ideas how to? I'd like to avoid setting the through attribute to a new model, because this would result in a new table (I'd like to keep the old one).
Edit: I've tracked where django creates the appropriate m2m model:
def create_many_to_many_intermediary_model(field, klass):
from django.db import models
# ...
# Construct and return the new class.
return type(name, (models.Model,), {
'Meta': meta,
'__module__': klass.__module__,
from_: models.ForeignKey(klass,
related_name='%s+' % name,
db_tablespace=field.db_tablespace),
to: models.ForeignKey(to_model,
related_name='%s+' % name,
db_tablespace=field.db_tablespace)
})
The relevant line is
to: models.ForeignKey(to_model,
related_name='%s+' % name,
db_tablespace=field.db_tablespace)
I'd like it to be
to: models.ForeignKey(to_model,
related_name='%s+' % name,
db_tablespace=field.db_tablespace,
on_delete=models.PROTECT)
Any way to do this other than monkey patching the whole thing and creating a new class for ManyToManyField?
I think the smartest thing to do is use an explicit through table. I realise that you've stated you would prefer not to "because this would result in a new table (I'd like to keep the old one)."
I suspect your concern is over losing the data you have. If you're using South, you can easily "convert" your existing, automatic intermediate table to an explicit one OR, you can create a completely new one, then migrate your existing data to the new table before dropping your old one.
Both of these methods are explained here: Adding a "through" table to django field and migrating with South?
Considering the change you'd like to make to its definition, I'd probably go with the option of creating a new table, then migrating your data over. Test to make sure all your data is still there (and that your change does what you want), then drop the old intermediate table.
Considering that these tables will both only hold 3 integers per row, this is likely to be a very manageable exercise even if you have a lot of houses and owners.
If I understand you want, this is similar to what I need some time ago.
Your problem: you need to protect a record that is used in another table from accidental deletion.
I solved it from this way (tested on Django 2 and Django 3).
Imagine, you have:
TABLE1 and TABLE 2, and they are under M2M relationship where TABLE1 has ManyToManyField.
I put the main keys to you understand at uppercase, you will need to adjust to what you want.
Look at views.py that use the exists() method and rise the exception are crucial.
models.py
class TABLE1(models.Model):
FIELD_M2M = models.ManyToManyField(
TABLE2,
blank=False,
related_name='FIELD_M2M',
)
#put here your code
models.py
class TABLE2(models.Model):
#Put here your code
views.py
# Delete
#login_required
def delete(request, pk=None):
try: # Delete register selected
if TABLE1.objects.filter(FIELD_M2M=pk).exists():
raise IntegrityError
register_to_delete = get_object_or_404(TABLE2, pk=pk)
# register_to_delete.register_to_delete.clear() // Uncomment this, if you need broken relationship M2M before delete
register_to_delete.delete()
except IntegrityError:
message = "The register couldn't be deleted!"
messages.info(request, message)
That is a ugly solution, but it works.
Posting my own solution as requested by #Andrew Fount. Quite an ugly hack just to change a single line.
from django.db.models import ManyToManyField
from django.db.models.fields.related import ReverseManyRelatedObjectsDescriptor, add_lazy_relation, create_many_to_many_intermediary_model, RECURSIVE_RELATIONSHIP_CONSTANT
from django.utils import six
from django.utils.functional import curry
def create_many_to_many_protected_intermediary_model(field, klass):
from django.db import models
managed = True
if isinstance(field.rel.to, six.string_types) and field.rel.to != RECURSIVE_RELATIONSHIP_CONSTANT:
to_model = field.rel.to
to = to_model.split('.')[-1]
def set_managed(field, model, cls):
field.rel.through._meta.managed = model._meta.managed or cls._meta.managed
add_lazy_relation(klass, field, to_model, set_managed)
elif isinstance(field.rel.to, six.string_types):
to = klass._meta.object_name
to_model = klass
managed = klass._meta.managed
else:
to = field.rel.to._meta.object_name
to_model = field.rel.to
managed = klass._meta.managed or to_model._meta.managed
name = '%s_%s' % (klass._meta.object_name, field.name)
if field.rel.to == RECURSIVE_RELATIONSHIP_CONSTANT or to == klass._meta.object_name:
from_ = 'from_%s' % to.lower()
to = 'to_%s' % to.lower()
else:
from_ = klass._meta.object_name.lower()
to = to.lower()
meta = type('Meta', (object,), {
'db_table': field._get_m2m_db_table(klass._meta),
'managed': managed,
'auto_created': klass,
'app_label': klass._meta.app_label,
'db_tablespace': klass._meta.db_tablespace,
'unique_together': (from_, to),
'verbose_name': '%(from)s-%(to)s relationship' % {'from': from_, 'to': to},
'verbose_name_plural': '%(from)s-%(to)s relationships' % {'from': from_, 'to': to},
})
# Construct and return the new class.
return type(name, (models.Model,), {
'Meta': meta,
'__module__': klass.__module__,
from_: models.ForeignKey(klass, related_name='%s+' % name, db_tablespace=field.db_tablespace),
### THIS IS THE ONLY LINE CHANGED
to: models.ForeignKey(to_model, related_name='%s+' % name, db_tablespace=field.db_tablespace, on_delete=models.PROTECT)
### END OF THIS IS THE ONLY LINE CHANGED
})
class ManyToManyProtectedField(ManyToManyField):
def contribute_to_class(self, cls, name):
# To support multiple relations to self, it's useful to have a non-None
# related name on symmetrical relations for internal reasons. The
# concept doesn't make a lot of sense externally ("you want me to
# specify *what* on my non-reversible relation?!"), so we set it up
# automatically. The funky name reduces the chance of an accidental
# clash.
if self.rel.symmetrical and (self.rel.to == "self" or self.rel.to == cls._meta.object_name):
self.rel.related_name = "%s_rel_+" % name
super(ManyToManyField, self).contribute_to_class(cls, name)
# The intermediate m2m model is not auto created if:
# 1) There is a manually specified intermediate, or
# 2) The class owning the m2m field is abstract.
# 3) The class owning the m2m field has been swapped out.
if not self.rel.through and not cls._meta.abstract and not cls._meta.swapped:
self.rel.through = create_many_to_many_protected_intermediary_model(self, cls)
# Add the descriptor for the m2m relation
setattr(cls, self.name, ReverseManyRelatedObjectsDescriptor(self))
# Set up the accessor for the m2m table name for the relation
self.m2m_db_table = curry(self._get_m2m_db_table, cls._meta)
# Populate some necessary rel arguments so that cross-app relations
# work correctly.
if isinstance(self.rel.through, six.string_types):
def resolve_through_model(field, model, cls):
field.rel.through = model
add_lazy_relation(cls, self, self.rel.through, resolve_through_model)
Thanks for taking time to read my question.
I have a django app with the following model:
class UserProfile(models.Model):
user = models.OneToOneField(User)
...
class Visit(models.Model):
profile = models.ForeignKey(UserProfile)
date = models.DateField(auto_now_add=True, db_index=True)
ip = models.IPAddressField()
class Meta:
unique_together = ('profile', 'date', 'ip')
In a view:
profile = get_object_or_404(Profile, pk = ...)
get, create = Visit.objects.get_or_create(profile=profile, date=now.date(), ip=request.META['REMOTE_ADDR'])
if create: DO SOMETHING
Everything works fine, except that the Postgres Logs are full with duplicate key errors:
2012-02-15 14:13:44 CET ERROR: duplicate key value violates unique constraint "table_visit_profile_id_key"
2012-02-15 14:13:44 CET STATEMENT: INSERT INTO "table_visit" ("profile_id", "date", "ip") VALUES (1111, E'2012-02-15', E'xx.xx.xxx.xxx') RETURNING "table_visit"."id"
Tried different solution e.g.
from django.db import transaction
from django.db import IntegrityError
#transaction.commit_on_success
def my_get_or_create(prof, ip):
try:
object = Visit.objects.create(profile=prof, date=datetime.now().date(), ip=ip)
except IntegrityError:
transaction.commit()
object = Visit.objects.get(profile=prof, date=datetime.now().date(), ip=ip)
return object
....
created = my_get_or_create(prof, request.META['REMOTE_ADDR'])
if created: DO SOMETHING
This only helps for MySQL? Does anyone know how to avaid the duplicate key value errors for postgres?
Another possible reason for these errors in get_or_create() is data type mismatch in one of the search fields - for example passing False instead of None into a nullable field. The .get() inside .get_or_create() will not find it and Django will continue with new row creation - which will fail due to PostgreSQL constraints.
I had issues with get_or_create when using postgres. In the end I abandoned the boilerplate code for traditional:
try:
jobInvite = Invite.objects.get(sender=employer.user, job=job)
except Invite.DoesNotExist:
jobInvite = Invite(sender=employer.user, job=job)
jobInvite.save()
# end try
Have you at some point had unique=True set on Visit's profile field?
It looks like there's been a unique constraint generated for postgres that's still in effect. "table_visit_profile_id_key" is what it's auto generated name would be, and naturally it would cause those errors if you're recording multiple visits for a user.
If this is the case, are you using South to manage your database changes? If you aren't, grab it!
PostgreSQL behaves somewhat differently in some subtle queries, which results in IntegrityError errors, especially after you switch to Django 1.6. Here's the solution - you need to add select_on_save option to each failing model:
class MyModel(models.Model):
...
class Meta:
select_on_save = True
It's documented here: Options.select_on_save
I've defined a User class which (ultimately) inherits from models.Model. I want to get a list of all the fields defined for this model. For example, phone_number = CharField(max_length=20). Basically, I want to retrieve anything that inherits from the Field class.
I thought I'd be able to retrieve these by taking advantage of inspect.getmembers(model), but the list it returns doesn't contain any of these fields. It looks like Django has already gotten a hold of the class and added all its magic attributes and stripped out what's actually been defined. So... how can I get these fields? They probably have a function for retrieving them for their own internal purposes?
Django versions 1.8 and later:
You should use get_fields():
[f.name for f in MyModel._meta.get_fields()]
The get_all_field_names() method is deprecated starting from Django
1.8 and will be removed in 1.10.
The documentation page linked above provides a fully backwards-compatible implementation of get_all_field_names(), but for most purposes the previous example should work just fine.
Django versions before 1.8:
model._meta.get_all_field_names()
That should do the trick.
That requires an actual model instance. If all you have is a subclass of django.db.models.Model, then you should call myproject.myapp.models.MyModel._meta.get_all_field_names()
As most of answers are outdated I'll try to update you on Django 2.2
Here posts- your app (posts, blog, shop, etc.)
1) From model link: https://docs.djangoproject.com/en/stable/ref/models/meta/
from posts.model import BlogPost
all_fields = BlogPost._meta.fields
#or
all_fields = BlogPost._meta.get_fields()
Note that:
all_fields=BlogPost._meta.get_fields()
Will also get some relationships, which, for ex: you can not display in a view.
As in my case:
Organisation._meta.fields
(<django.db.models.fields.AutoField: id>, <django.db.models.fields.DateField: created>...
and
Organisation._meta.get_fields()
(<ManyToOneRel: crm.activity>, <django.db.models.fields.AutoField: id>, <django.db.models.fields.DateField: created>...
2) From instance
from posts.model import BlogPost
bp = BlogPost()
all_fields = bp._meta.fields
3) From parent model
Let's suppose that we have Post as the parent model and you want to see all the fields in a list, and have the parent fields to be read-only in Edit mode.
from django.contrib import admin
from posts.model import BlogPost
#admin.register(BlogPost)
class BlogPost(admin.ModelAdmin):
all_fields = [f.name for f in Organisation._meta.fields]
parent_fields = BlogPost.get_deferred_fields(BlogPost)
list_display = all_fields
read_only = parent_fields
The get_all_related_fields() method mentioned herein has been deprecated in 1.8. From now on it's get_fields().
>> from django.contrib.auth.models import User
>> User._meta.get_fields()
I find adding this to django models quite helpful:
def __iter__(self):
for field_name in self._meta.get_all_field_names():
value = getattr(self, field_name, None)
yield (field_name, value)
This lets you do:
for field, val in object:
print field, val
This does the trick. I only test it in Django 1.7.
your_fields = YourModel._meta.local_fields
your_field_names = [f.name for f in your_fields]
Model._meta.local_fields does not contain many-to-many fields. You should get them using Model._meta.local_many_to_many.
It is not clear whether you have an instance of the class or the class itself and trying to retrieve the fields, but either way, consider the following code
Using an instance
instance = User.objects.get(username="foo")
instance.__dict__ # returns a dictionary with all fields and their values
instance.__dict__.keys() # returns a dictionary with all fields
list(instance.__dict__.keys()) # returns list with all fields
Using a class
User._meta.__dict__.get("fields") # returns the fields
# to get the field names consider looping over the fields and calling __str__()
for field in User._meta.__dict__.get("fields"):
field.__str__() # e.g. 'auth.User.id'
def __iter__(self):
field_names = [f.name for f in self._meta.fields]
for field_name in field_names:
value = getattr(self, field_name, None)
yield (field_name, value)
This worked for me in django==1.11.8
A detail not mentioned by others:
[f.name for f in MyModel._meta.get_fields()]
get, for example
['id', 'name', 'occupation']
and
[f.get_attname() for f in MyModel._meta.get_fields()]
get
['id', 'name', 'occupation_id']
If
reg = MyModel.objects.first()
then
reg.occupation
get, for example
<Occupation: Dev>
and
reg.occupation_id
get
1
MyModel._meta.get_all_field_names() was deprecated several versions back and removed in Django 1.10.
Here's the backwards-compatible suggestion from the docs:
from itertools import chain
list(set(chain.from_iterable(
(field.name, field.attname) if hasattr(field, 'attname') else (field.name,)
for field in MyModel._meta.get_fields()
# For complete backwards compatibility, you may want to exclude
# GenericForeignKey from the results.
if not (field.many_to_one and field.related_model is None)
)))
Just to add, I am using self object, this worked for me:
[f.name for f in self.model._meta.get_fields()]
At least with Django 1.9.9 -- the version I'm currently using --, note that .get_fields() actually also "considers" any foreign model as a field, which may be problematic. Say you have:
class Parent(models.Model):
id = UUIDField(primary_key=True)
class Child(models.Model):
parent = models.ForeignKey(Parent)
It follows that
>>> map(lambda field:field.name, Parent._model._meta.get_fields())
['id', 'child']
while, as shown by #Rockallite
>>> map(lambda field:field.name, Parent._model._meta.local_fields)
['id']
So before I found this post, I successfully found this to work.
Model._meta.fields
It works equally as
Model._meta.get_fields()
I'm not sure what the difference is in the results, if there is one. I ran this loop and got the same output.
for field in Model._meta.fields:
print(field.name)
In sometimes we need the db columns as well:
def get_db_field_names(instance):
your_fields = instance._meta.local_fields
db_field_names=[f.name+'_id' if f.related_model is not None else f.name for f in your_fields]
model_field_names = [f.name for f in your_fields]
return db_field_names,model_field_names
Call the method to get the fields:
db_field_names,model_field_names=get_db_field_names(Mymodel)
Combined multiple answers of the given thread (thanks!) and came up with the following generic solution:
class ReadOnlyBaseModelAdmin(ModelAdmin):
def has_add_permission(self, request):
return request.user.is_superuser
def has_delete_permission(self, request, obj=None):
return request.user.is_superuser
def get_readonly_fields(self, request, obj=None):
return [f.name for f in self.model._meta.get_fields()]
Why not just use that:
manage.py inspectdb
Example output:
class GuardianUserobjectpermission(models.Model):
id = models.IntegerField(primary_key=True) # AutoField?
object_pk = models.CharField(max_length=255)
content_type = models.ForeignKey(DjangoContentType, models.DO_NOTHING)
permission = models.ForeignKey(AuthPermission, models.DO_NOTHING)
user = models.ForeignKey(CustomUsers, models.DO_NOTHING)
class Meta:
managed = False
db_table = 'guardian_userobjectpermission'
unique_together = (('user', 'permission', 'object_pk'),)