Given a Django model, I'm trying to list all of its fields. I've seen some examples of doing this using the _meta model attribute, but doesn't the underscore in front of meta indicate that the _meta attribute is a private attribute and shouldn't be accessed directly? ... Because, for example, the layout of _meta could change in the future and not be a stable API?
Is _meta an exception to this rule? Is it stable and ready to use or is it considered bad practice to access it? Or is there a function or some other way to introspect the fields of a model without using the _meta attribute? Below is a list of some links showing how to do this using the _meta attribute
Any advice is much appreciated.
django object get/set field
http://www.djangofoo.com/80/get-list-model-fields
How to introspect django model fields?
_meta is private, but it's relatively stable. There are efforts to formalise it, document it and remove the underscore, which might happen before 1.3 or 1.4. I imagine effort will be made to ensure things are backwards compatible, because lots of people have been using it anyway.
If you're particularly concerned about compatibility, write a function that takes a model and returns the fields. This means if something does change in the future, you only have to change one function.
def get_model_fields(model):
return model._meta.fields
I believe this will return a list of Field objects. To get the value of each field from the instance, use getattr(instance, field.name).
Update: Django contributors are working on an API to replace the _Meta object as part of a Google Summer of Code. See:
- https://groups.google.com/forum/#!topic/django-developers/hD4roZq0wyk
- https://code.djangoproject.com/wiki/new_meta_api
I know this post is pretty old, but I just cared to tell anyone who is searching for the same thing that there is a public and official API to do this: get_fields() and get_field()
Usage:
fields = model._meta.get_fields()
my_field = model._meta.get_field('my_field')
https://docs.djangoproject.com/en/3.2/ref/models/meta/#retrieving-all-field-instances-of-a-model
get_fields() returns a tuple and each element is a Model field type, which can't be used directly as a string. So, field.name will return the field name
my_model_fields = [field.name for field in MyModel._meta.get_fields()]
The above code will return a list conatining all fields name
Example
In [11]: from django.contrib.auth.models import User
In [12]: User._meta.get_fields()
Out[12]:
(<ManyToOneRel: admin.logentry>,
<django.db.models.fields.AutoField: id>,
<django.db.models.fields.CharField: password>,
<django.db.models.fields.DateTimeField: last_login>,
<django.db.models.fields.BooleanField: is_superuser>,
<django.db.models.fields.CharField: username>,
<django.db.models.fields.CharField: first_name>,
<django.db.models.fields.CharField: last_name>,
<django.db.models.fields.EmailField: email>,
<django.db.models.fields.BooleanField: is_staff>,
<django.db.models.fields.BooleanField: is_active>,
<django.db.models.fields.DateTimeField: date_joined>,
<django.db.models.fields.related.ManyToManyField: groups>,
<django.db.models.fields.related.ManyToManyField: user_permissions>)
In [13]: [field.name for field in User._meta.get_fields()]
Out[13]:
['logentry',
'id',
'password',
'last_login',
'is_superuser',
'username',
'first_name',
'last_name',
'email',
'is_staff',
'is_active',
'date_joined',
'groups',
'user_permissions']
Now there is special method - get_fields()
>>> from django.contrib.auth.models import User
>>> User._meta.get_fields()
It accepts two parameters that can be used to control which fields are returned:
include_parents
True by default. Recursively includes fields defined on parent classes. If set to False, get_fields() will only search for fields declared directly on the current model. Fields from models that directly inherit from abstract models or proxy classes are considered to be local, not on the parent.
include_hidden
False by default. If set to True, get_fields() will include fields that are used to back other field’s functionality. This will also include any fields that have a related_name (such as ManyToManyField, or ForeignKey) that start with a “+”
This is something that is done by Django itself when building a form from a model. It is using the _meta attribute, but as Bernhard noted, it uses both _meta.fields and _meta.many_to_many. Looking at django.forms.models.fields_for_model, this is how you could do it:
opts = model._meta
for f in sorted(opts.fields + opts.many_to_many):
print '%s: %s' % (f.name, f)
fields = [f"{f.name}_id" if f.is_relation else f.name for f in model._meta.fields]
The model fields contained by _meta are listed in multiple locations as lists of the respective field objects. It may be easier to work with them as a dictionary where the keys are the field names.
In my opinion, this is most irredundant and expressive way to collect and organize the model field objects:
def get_model_fields(model):
fields = {}
options = model._meta
for field in sorted(options.concrete_fields + options.many_to_many + options.virtual_fields):
fields[field.name] = field
return fields
(See This example usage in django.forms.models.fields_for_model.)
How about this one.
fields = Model._meta.fields
If you need this for your admin site, there is also the ModelAdmin.get_fields method (docs), which returns a list of field name strings.
For example:
class MyModelAdmin(admin.ModelAdmin):
# extending change_view, just as an example
def change_view(self, request, object_id=None, form_url='', extra_context=None):
# get the model field names
field_names = self.get_fields(request)
# use the field names
...
As per the django documentation 2.2 you can use:
To get all fields: Model._meta.get_fields()
To get an individual field: Model._meta.get_field('field name')
ex. Session._meta.get_field('expire_date')
Another way is add functions to the model and when you want to override the date you can call the function.
class MyModel(models.Model):
name = models.CharField(max_length=256)
created = models.DateTimeField(auto_now_add=True)
modified = models.DateTimeField(auto_now=True)
def set_created_date(self, created_date):
field = self._meta.get_field('created')
field.auto_now_add = False
self.created = created_date
def set_modified_date(self, modified_date):
field = self._meta.get_field('modified')
field.auto_now = False
self.modified = modified_date
my_model = MyModel(name='test')
my_model.set_modified_date(new_date)
my_model.set_created_date(new_date)
my_model.save()
instance._meta.get_fields() returns a list of all the fields (i.e. columns) in a Django model.
This method is used to introspect the model's fields, their types, and their relationships with other models. The method returns a list of Field objects, which represent the individual fields in the model.
For example, suppose you have a Django model called MyModel. You can use instance._meta.get_fields() to get a list of all the fields in the model:
from myapp.models import MyModel
my_instance = MyModel.objects.get(id=1)
fields = my_instance._meta.get_fields()
The fields variable will now contain a list of all the fields in the MyModel model, including fields such as id, name, created_at, and any related fields (such as foreign keys). You can use this list to access and manipulate the individual fields in the model.
Related
Having query like SELECT *, 'hello' AS world FROM myApp_myModel I'd like to serialize it to json.
Doesn't seem like a big deal, and there are plenty of similar questions on SO but none seems to give straight answer.
So far I've tried:
data = myModel.objects.raw(query)
# gives: ModelState is not serializable
json.dumps([dict(r.__dict__) for r in data])
# doesn't serialize 'world' column, only model fields:
serializers.serialize('json', data)
#dear God:
for r in data:
for k in dict(r.__dict__):
print(getattr(r,k))
The issue:
Builtin django core serializers are not ready to include extra fields ( from raw neither from annotation expression) It just takes model fields from _meta.local_fields.
You can see it at django django/core/serializers/base.py source code:
concrete_model = obj._meta.concrete_model #obj is an object model
...
for field in concrete_model._meta.local_fields:
if field.serialize or field is pk_parent:
if field.remote_field is None:
if (self.selected_fields is None
or field.attname in self.selected_fields):
self.handle_field(obj, field)
else:
if (self.selected_fields is None
or field.attname[:-3] in self.selected_fields):
self.handle_fk_field(obj, field)
django rest framework at rescue:
To solve your issue you can use a non builtin functionality. You can include a REST package in your project. For example django rest framework can handle extra fields:
from django.db.models import F
from aula.apps.alumnes.models import MyModel
from rest_framework.renderers import JSONRenderer
data=MyModel.objects.annotate(dummy = F('some_field') )
class MyModelSerializer(serializers.ModelSerializer):
dummy = serializers.CharField()
class Meta:
model = MyModel
fields = ('some_other_field','dummy')
read_only_fields = (
'dummy',
)
m=MyModelSerializer(data, many=True)
JSONRenderer().render(m.data)
You can create a DRF searializer for the task:
http://www.django-rest-framework.org/api-guide/serializers/
i.e.
class MyModelSerializer(serializers.ModelSerializer):
world = serializers.ReadOnlyField()
class Meta:
model = MyModel
fields = (world, ...)
you can also use serializer inheritance etc - see the docs.
There is a clean way you can do this using Django Rest Framework
First off did you know You can also execute queries containing fields that aren’t defined on the model when doing a Raw query
for example ( REF )
>>> people = Person.objects.raw('SELECT *, age(birth_date) AS age FROM myapp_person')
>>> for p in people:
... print("%s is %s." % (p.first_name, p.age))
John is 37.
Jane is 42.
That means you can use a standard serializer. You just need to tell the serializer what to do with fields that were not originally on the model consider the below. Needed to join 3 tables to a user. The user, the company they belong to and the companies membership. If your table has thousands of users and you did the standard serialiser method field, it would result in thousands of queries to get the related companies membership each time. so instead here was the solution I used
# api.py
class UserSAMAExportListApiView(ListAPIView):
serializer_class = UserExportSerializer
model = User
def get_queryset(self):
q = User.objects.raw(
"""
SELECT
[users_user].[id],
[users_user].[email],
[companies_company].[title] AS company__title,
[companies_company].[registration_number] AS company__registration_number,
[memberships_membership].number AS company__membership__number
FROM [users_user]
LEFT OUTER JOIN [dbo].[companies_company]
ON ([users_user].[company_id] = [companies_company].[id])
LEFT OUTER JOIN [memberships_membership]
ON ([companies_company].[id] = [memberships_membership].[company_id])
WHERE ([memberships_membership].[expiry_date] >= %s)
"""
, [date.today(),]
)
return q
Then just tell your standard serialiser that there are some new fields you defined
# serializers.py
class UserExportSerializer(ModelSerializer):
class Meta:
model = User
fields = (
'id',
'email',
'company__title',
'company__registration_number',
'company__membership__number',
)
def build_unknown_field(self, field_name, model_class):
"""
Return a two tuple of (cls, kwargs) to build a serializer field with. For fields that werent originally on
The model
"""
return fields.CharField, {'read_only': True}
And that's it DRF will handle the rest in a standard way and do proper serialization for you
Note you have to override the build_unknown_fields method. This is simply saying convert all the non-standard model fields to Text, if you want you can check the field name and convert to other formats here.
I have a model currently defined like this:
class Category(models.Model):
ID = models.AutoField()
name = models.CharField()
desc = models.CharField()
Another model Subcategory has a ForeignKey defined on Category.
When I run:
Category._meta.get_fields()
I get:
(<ManyToOneRel: siteapp.subcategory>, <django.db.models.fields.AutoField: ID>, <django.db.models.fields.CharField: name>, <django.db.models.fields.CharField: desc>)
However, I don't want the ManyToOneRel fields; I just want the others.
Currently, I am doing something like this:
from django.db.models.fields.reverse_related import ManyToOneRel
field_list = []
for field in modelClass._meta.get_fields():
if not isinstance(field, ManyToOneRel):
field_list.append(field)
However, is there a better way to do this, with or without using the model _meta API?
You could use the concrete_fields property.
Category._meta.concrete_fields
However this is an internal Django API, and it may be better to use get_fields() with your own filtering, even though it may be a little more verbose.
I had the same issue creating a serializer mixin that treated only GenericRelaTion fields. Unfortunately, when you use the get_fields() the ManyToOneRel in somecases appear principally when you need to get attname of a field. Therefore I created a function that treates this issue skiping all the ManyToOneRel from fields:
def get_generic_relation_fields(self):
"""
This function returns all the GenericRelation
fields needed to return the values that are
related such as polymorphic models.
"""
other_field = [field for field in self.Meta.model._meta.get_fields()]
fields = [field.attname for field in self.Meta.model._meta.get_fields() if not isinstance(field, (ManyToOneRel))]
generic_relation_fields = []
for field in fields:
get_type = self.Meta.model._meta.get_field(field)
field_type = get_type.__class__.__name__
if field_type == "GenericRelation": #<----- change field name to filter different fields.
generic_relation_fields.append(field)
return generic_relation_fields
Usage:
class MyModel(models.Model):
. . .
first_name = GenericRelation(MyUserPolymorphic)
last_name = GenericRelation(MyUserPolymorphic)
whatever = GenericRelation(MyUserPolymorphic)
class MyModelSerializer(serializer.ModelSerializer)
def create(self, validated_data):
fields = [field for field in get_generic_relation_fields(self)]
print("FIELDS ---->", fields)
. . .
output on POST:
FIELDS ----> ['first_name', 'last_name', 'whatever']
However, there I added 'GenericRelation' field you can add other fields to be filtered and treated as you want.
I have been looking for a solution for this, and I've ended up writing my own script which while not the most clean pythonic code, works. I thought to put it here if someone stumbles upon it in the future.
[field for field in fields if str(type(field)) != "<class 'django.db.models.fields.related.ForeignKey'>"]
I have also used this script as follows to get a dict in the format of {field_name:field_value}
{
field.name: (
getattr(self, field.name)
if str(type(field))
!= "<class 'django.db.models.fields.related.ForeignKey'>"
else getattr(self, field.name).id
)
for field in fields
if str(type(field))
!= "<class 'django.db.models.fields.reverse_related.ManyToOneRel'>"}
I had to do an additional check for ForeignKey constraints as usually showing the field value didn't make sense in that case
views.py
I'm creating a queryset that I want to serialize and return as JSON. The queryset looks like this:
all_objects = Program.objects.all()
test_data = serializers.serialize("json", all_objects, use_natural_keys=True)
This pulls back everything except for the 'User' model (which is linked across two models).
models.py
from django.db import models
from django.contrib.auth.models import User
class Time(models.Model):
user = models.ForeignKey(User)
...
class CostCode(models.Model):
program_name = models.TextField()
...
class Program(models.Model):
time = models.ForeignKey(Time)
program_select = models.ForeignKey(CostCode)
...
Question
My returned data has Time, Program, and CostCode information, but I'm unable to query back the 'User' table. How can I get back say the 'username' (from User Table) in the same queryset?
Note: I've changed my queryset to all_objects = Time.objects.all() and this gets User info, but then it doesn't pull in 'CostCode'. My models also have ModelManagers that return the get_by_natural_key so the relevant fields appear in my JSON.
Ultimately, I want data from all four models to appear in my serialized JSON fields, I'm just missing 'username'.
Here's a picture of how the JSON object currently appears in Firebug:
Thanks for any help!
It seems a bit heavyweight at first glance but you could look at using Django REST Framework:
http://www.django-rest-framework.org/api-guide/serializers#modelserializer
You can define and use the serializer classes without having to do anything else with the framework. The serializer returns a python dict which can then be easily dumped to JSON.
To get all fields from each related model as nested dicts you could do:
class ProgramSerializer(serializers.ModelSerializer):
class Meta:
model = Program
depth = 2
all_objects = Program.objects.all()
serializer = ProgramSerializer(all_objects, many=True)
json_str = json.dumps(serializer.data)
To customise which fields are included for each model you will need to define a ModelSerializer class for each of your models, for example to output only the username for the time.user:
class UserSerializer(serializers.ModelSerializer):
class Meta:
model = User
fields = ('username', )
class TimeSerializer(serializers.ModelSerializer):
"""
specifying the field here rather than relying on `depth` to automatically
render nested relations allows us to specify a custom serializer class
"""
user = UserSerializer()
class Meta:
model = Time
class ProgramSerializer(serializers.ModelSerializer):
time = TimeSerializer()
class Meta:
model = Program
depth = 1 # render nested CostCode with default output
all_objects = Program.objects.all()
serializer = ProgramSerializer(all_objects, many=True)
json_str = json.dumps(serializer.data)
What you really want is a "deep" serialization of objects which Django does not natively support. This is a common problem, and it is discussed in detail here: Serializing Foreign Key objects in Django. See that question for some alternatives.
Normally Django expects you to serialize the Time, CostCode, Program, and User objects separately (i.e. a separate JSON array for each) and to refer to them by IDs. The IDs can either be the numeric primary keys (PKs) or a "natural" key defined with natural_key.
You could use natural_key to return any fields you want, including user.username. Alternatively, you could define a custom serializer output whatever you want there. Either of these approaches will probably make it impossible to load the data back into a Django database, which may not be a problem for you.
Basically, I want to filter out inactive users from a related field of a ModelSerializer. I tried Dynamically limiting queryset of related field as well as the following:
class MySerializer(serializers.ModelSerializer):
users = serializers.PrimaryKeyRelatedField(queryset=User.objects.filter(active=True), many=True)
class Meta:
model = MyModel
fields = ('users',)
Neither of these approaches worked for just filtering the queryset. I want to do this for a nested related Serializer class as a field (but couldn't even get it to work with a RelatedField).
How do I filter queryset for nested relation?
I'll be curious to see a better solution as well. I've used a custom method in my serializer to do that. It's a bit more verbose but at least it's explicit.
Some pseudo code where a GarageSerializer would filter the nested relation of cars:
class MyGarageSerializer(...):
users = serializers.SerializerMethodField('get_cars')
def get_cars(self, garage):
cars_queryset = Car.objects.all().filter(Q(garage=garage) | ...).select_related()
serializer = CarSerializer(instance=cars_queryset, many=True, context=self.context)
return serializer.data
Obviously replace the queryset with whatever you want. You don't always need the to give the context (I used it to retrieve some query parameters in the nested serializer) and you probably don't need the .select_related (that was an optimisation).
One way to do this is to create a method on the Model itself and reference it in the serializer:
#Models.py
class MyModel(models.Model):
#...
def my_filtered_field (self):
return self.othermodel_set.filter(field_a = 'value_a').order_by('field_b')[:10]
#Serialziers.py
class MyModelSerialzer(serializers.ModelSerializer):
my_filtered_field = OtherModelSerializer (many=True, read_only=True)
class Meta:
model = MyModel
fields = [
'my_filtered_field' ,
#Other fields ...
]
Another way to avoid the SerializerMethodField solution and therefore still allow writing to the serializer as well would be to subclass the RelatedField and do the filtering there.
To only allow active users as values for the field, the example would look like:
class ActiveUsersPrimaryKeyField(serializers.PrimaryKeyRelatedField):
def get_queryset(self):
return super().get_queryset().filter(active=True)
class MySerializer(serializers.ModelSerializer):
users = ActiveUsersPrimaryKeyField(many=True)
class Meta:
model = MyModel
fields = ('users',)
Also see this response.
Note that this only restricts the set of input values to active users, though, i.e. only when creating or updating model instances, inactive users will be disallowed.
If you also use your serializer for reading and MyModel already has a relation to a user that has become inactive in the meantime, it will still be serialized. To prevent this, one way is to filter the relation using django's Prefetch objects. Basically, you'll filter out inactive users before they even get into the serializer:
from django.db.models import Prefetch
# Fetch a model instance, eagerly prefetching only those users that are active
model_with_active_users = MyModel.objects.prefetch_related(
Prefetch("users", queryset=User.objects.filter(active=True))
).first()
# serialize the data with the serializer defined above and see that only active users are returned
data = MyModelSerializer(model_with_active_users).data
i am working on saving on the same form two tables - having a m2m relation.
I don't succeed, my error persists with something like: Cannot set values on a ManyToManyField which specifies an intermediary model. Use Membership's Manager instead
where Membership is my 'through table'.
my code :
def save_classroom(request):
classroom_instance = Classroom()
if request.method == 'POST':
form = ClassroomForm(request.POST, request.FILES, user = request.user)
if form.is_valid():
new_obj = form.save(commit=False)
new_obj.user = request.user
new_obj.save()
membership = Membership(member = request.user,classroom=new_obj)
membership.save()
form.save_m2m()
return HttpResponseRedirect('.')
else:
form = ClassroomForm(user = request.user)
return render_to_response('classroom/classroom_form.html', {
'form': form,
},
context_instance=RequestContext(request))
my models:
class Classroom(models.Model):
user = models.ForeignKey(User, related_name = 'classroom_creator')
classname = models.CharField(max_length=140, unique = True)
date = models.DateTimeField(auto_now=True)
open_class = models.BooleanField(default=True)
members = models.ManyToManyField(User,related_name="list of invited members", through = 'Membership')
class Membership(models.Model):
accept = models.BooleanField(default=False)
date = models.DateTimeField(auto_now = True)
classroom = models.ForeignKey(Classroom, related_name = 'classroom_membership')
member = models.ForeignKey(User, related_name = 'user_membership')
where am i wrong?
If you are allowed to modify class Membership, adding auto_created = True might solve your problem,
class Membership(models.Model):
class Meta:
auto_created = True
In Django 1.7, the error message is changed to "Cannot set values on a ManyToManyField which specifies an intermediary model". The solution is the same.
NOTE: This will remove your intermediate model entirely, and all the additional fields with it.
As seen on:
http://docs.djangoproject.com/en/dev/topics/db/models/#intermediary-manytomany
Unlike normal many-to-many fields, you can't use add, create, or assignment (i.e., beatles.members = [...]) to create relationships
I guess your code trips up on the line "form.save_m2m()", which is unnecessary since you already manually create a membership.
I had a similar error message on a different problem. I post it here just in case it helps others.
I've added a new ManyToManyField on an existing model. This model was used in a ModelForm built with an exclude field.
I fixed the problem by add the new field in the excluded ones.
As of the Django 2.2 release you can also specify a through_defaults on the field in order to use add, create and assignment:
The RelatedManager.add(), create(), remove(), set(), get_or_create(),
and update_or_create() methods are now allowed on many-to-many
relationships with intermediate models. The new through_defaults
argument is used to specify values for new intermediate model
instance(s).
In your case since all the fields already have defaults it might just work in 2.2.
As specified in https://docs.djangoproject.com/en/2.1/topics/db/models/
"Unlike normal many-to-many fields, you can’t use add(), create(), or set() to create relationships"
This means for django version 2.1 ,these methods are impossible. But the same page of django 2.2 documentation https://docs.djangoproject.com/en/2.2/topics/db/models/ tells that:
"You can also use add(), create(), or set() to create relationships, as long as you specify through_defaults for any required fields"
So just update the django to 2.2 or newer versions to use the above methods and while creating objects give the argument 'through_defaults' as a python dictionary with keys as your extra field names and values as their default values.