django model serialization - django

I want to serialize models so:
class Schedule(models.Model):
Title = models.CharField(max_length=512)
class ScheduleEvent1(models.Model):
ScheduleContent = models.ForeignKey(Schedule)
Description = models.TextField()
class ScheduleEvent2(models.Model):
ScheduleContent = models.ForeignKey(Schedule)
AnotherField = models.TextField()
ShortDescription = models.TextField()
make smth like serializers.serialize('json', Schedule.objects.all())
The result likes
[
{
"pk": 2,
"model": "Schedule",
"fields": {
"Title": "Some Title",
"ScheduleEvent1": [
{
"pk": 19,
"model": "ScheduleEvent1",
"fields": {
"Description": "Some Descr",
}
},
{
"pk": 20,
"model": "ScheduleEvent1",
"fields": {
"Description": "Some Descr2222",
}
}
],
"ScheduleEvent2": [
{
"pk": 15,
"model": "ScheduleEvent2",
"fields": {
"AnotherField": "Some text",
"ShortDescription" : "Some text ...",
}
}
]
}
}
]
In general. I have entity tree. And I need serialize this tree from root.
tnx for help.

http://docs.djangoproject.com/en/1.3/topics/serialization/#dependencies-during-serialization
Covers the essence of this. The resulting file is not precisely what's required, since the objects won't be nested. However it is what Django produces by default and it's what Django will use for deserialization.
If you use Piston you can easily define a Handler which will produce nested JSON, but it won't be precisely as shown because Piston's JSON emitter isn't precisely in the Django format.
Getting to precisely what's hoped-for leads to extending the Django serializer to produce a nested JSON object as the natural key. See http://code.djangoproject.com/browser/django/trunk/django/core/serializers/python.py. Lines 47-59
Or, you can define a natural_key method which uses Django's 'django.core.serializers.json .Serializer` to emit the JSON serialization of the object instead of some other natural key value. This is a little weird semantically, but it may do what you want.

Related

Django Rest Framework: Get unique list of values from nested structure

I want to be able to return a list of strings from a deeply nested structure of data. In this scenario, I have a API that manages a chain of bookstores with many locations in different regions.
Currently, I have an API endpoint that takes a region's ID and returns a nested JSON structure of details about the region, the individual bookstores, and the books that can be found in each store.
{
"region": [
{
"store": [
{
"book": {
"name": "Foo"
}
},
{
"book": {
"name": "Bar"
}
},
{
"book": {
"name": "Baz"
}
}
],
},
{
"store": [
{
"book": {
"name": "Foo"
}
},
{
"book": {
"name": "Bar"
}
}
],
},
{
"store": [
{
"book": {
"name": "Foo"
}
},
{
"book": {
"name": "Baz"
}
},
{
"book": {
"name": "Qux"
}
}
]
}
]
}
My models look like the following. I am aware these models don't make the most sense for this contrived example, but it does reflect my real world code:
class Book(TimeStampedModel):
name = models.CharField(default="", max_length=512)
class Bookstore(TimeStampedModel):
value = models.CharField(default="", max_length=1024)
book = models.ForeignKey(Book, on_delete=models.CASCADE)
class Region(TimeStampedModel):
stores = models.ManyToManyField(Bookstore)
class BookstoreChain(TimeStampedModel):
regions = models.ManyToManyField(Region)
The serializers I created for the above response look like:
class BookSerializer(serializers.ModelSerializer):
class Meta:
model = Book
fields = "__all__"
class BookstoreSerializer(serializers.ModelSerializer):
books = BookSerializer()
class Meta:
model = Bookstore
fields = "__all__"
class RegionSerializer(serializers.ModelSerializer):
stores = BookstoreSerializer(many=True)
class Meta:
model = Region
fields = "__all__"
class BookstoreChainSerializer(serializers.ModelSerializer):
regions = RegionSerializer(many=True)
class Meta:
model = BookstoreChain
fields = "__all__"
I'm not sure what my view or serializer for this solution need to look like. I'm more familiar with writing raw SQL or using an ORM/Linq to get a set of results.
While the above response is certainty useful, what I really want is an API endpoint to return a unique list of book names that can be found in a given region (Foo, Bar, Baz, Qux). I would hope my response to look like:
{
"books": [
"Foo",
"Bar",
"Baz",
"Qux"
]
}
My feeble attempt so far has a urls.py with the following path:
path("api/regions/<int:pk>/uniqueBooks/", views.UniqueBooksForRegionView.as_view(), name="uniqueBooksForRegion")
My views.py looks like:
class UniqueBooksForRegionView(generics.RetrieveAPIView):
queryset = Regions.objects.all()
serializer_class = ???
So you start from region you have to get the stores, so you can filter the books in the stores, here is a solution which will work.
Note:
Avoid using .get() in *APIView because it will trigger an error if the request does not have the ID, you can use get_object_or_404(), but then you cannot log your error in Sentry.
To get an element from an *APIView, use filter().
import logging as L
class UniqueBooksForRegionView(generics.RetrieveAPIView):
lookup_field = 'pk'
def get(self, *args, **kwargs)
regions = Region.objects.filter(pk=self.kwargs[self.lookup_field])
if regions.exists():
region = regions.first()
stores_qs = region.stores.all()
books_qs = Book.objects.filter(store__in=stores_qs).distinct()
# use your book serializer
serializer = BookSerializer(books_qs, many=True)
return Response(serializer.data, HTTP_200_OK)
else:
L.error(f'Region with id {self.kwargs[self.lookup_field]} not found.')
return Response({'detail':f'Region with id {self.kwargs[self.lookup_field]} not found.'}, HTTP_404_NOT_FOUND)
Note
Here is the flow, the code may need some tweaks, but I hope it helps you understand the flow

Django graphql graphene removing redundant queries from return value

The last few days I have read up on so much graphql that I can't see the trees from the forest anymore.
The results that this person got in the beginning is almost exactly what I want (his problem, not his solution), but it seems that a lot of the code is deprecated and I can't seem to get it working: link
I have a bunch of containers that I return. All the containers have the amounts for every day in them. I only want to return the amounts of a certain day.
At the moment, I do return these results (day), but all the other results (days) also return with a Null value.
Current behavior:
{
"data": {
"listProductcontainers": [
{
"id": "1",
"productid": {
"productid": "CBG2",
"processedstockamountsSet": [
{
"timeStampID": {
"id": "2"
},
"id": "77745",
"prodName": {
"productid": "CBG2"
}
},
{
"timeStampID": null, <--------
"id": "89645",
"prodName": {
"productid": "CBG2"
}
},
{
"timeStampID": null, <--------
"id": "89848",
"prodName": {
"productid": "CBG2"
}
},
// ...
Requested behavior: (All values with 'Null' should not return)
{
"data": {
"listProductcontainers": [
{
"id": "1",
"productid": {
"productid": "CBG2",
"processedstockamountsSet": [
{
"timeStampID": {
"id": "2"
}
My query that I am running looks like this:
query{
listProductcontainers{
id
productid{
productid
processedstockamountsSet{
timeStampID(id:2){
id
}
id
prodName{
productid
}
}
}
}
}
Here are the relevant code for the results:
class TimeStampType(DjangoObjectType):
class Meta:
model = TimeStamp
class ProcessedStockAmountsType(DjangoObjectType):
timeStampID = graphene.Field(TimeStampType, id=graphene.Int())
class Meta:
model = ProcessedStockAmounts
def resolve_timeStampID(self, info, **kwargs):
id = kwargs.get('id')
if self.timeStampID.id == id:
return self.timeStampID
class ProductcontainersType(DjangoObjectType):
class Meta:
model = Productcontainers
class ProductlistType(DjangoObjectType):
class Meta:
model = Productlist
class Query(graphene.ObjectType):
list_productcontainers = graphene.List(ProductcontainersType)
def resolve_list_productcontainers(self, context, **kwargs):
return Productcontainers.objects.all()
I have read almost everything in graphene by now, but if you even have a link that mirrors what I want to do I would really appreciate it.
My final option is to make two calls where I get all the container ids, and another call where I get all the amounts (with container id) for a certain date, and with 2 for loops I just add the amounts into the corresponding container... :(

Django: defining a generic manager in an Abstract model

Using the following code and defining suitable NATURAL_KEY per every class fails (instead of defining different mangers in every class, which replicate the same code):
class NexchangeManager(models.Manager):
def get_by_natural_key(self, param):
if param == "*":
return self.all()
lookup = {self.NATURAL_KEY: param}
return self.get(**lookup)
class NexchangeModel(models.Model):
class Meta:
abstract = True
objects = NexchangeManager()
Djagno complains about fields replication, although NexhcnageModel is an Abstract model.
Should I use a mixin instead?
error:
django.core.serializers.base.DeserializationError: Problem installing fixture '/Users/beoleg/dev/nexchange/core/fixtures/pairs_cross.json': 'NexchangeManager' object has no attribute 'NATURAL_KEY': (core.pair:pk=1) field_value was '['LTC']'
The purpose of this, a bit overcomplicated code at first glance is, to have something like this in my fixtures:
[
{
"model": "payments.paymentpreference",
"pk": 8,
"fields": {
"user": ["onit"],
"identifier": "paypal#nexchange.co.uk",
"payment_method": 12,
"comment": "Please send the funds as a personal payment (this is a precaution to prevent charge backs, payments for goods and services will be automatically declined)",
"currency": [
["*"]
],
"created_on":"2016-11-01T17:41:28+00:00",
"modified_on":"2016-11-01T17:41:28+00:00"
}
}
]
Instaed of:
[
{
"model": "payments.paymentpreference",
"pk": 8,
"fields": {
"user": ["onit"],
"identifier": "paypal#nexchange.co.uk",
"payment_method": 12,
"comment": "Please send the funds as a personal payment (this is a precaution to prevent charge backs, payments for goods and services will be automatically declined)",
"currency": [
["USD"],
["RUB"],
["EUR"],
["GBP"],
["JPY"],
["HRK"],
["CHF"],
["PLN"],
["RON"],
["BGN"],
["CZK"],
["AUD"],
["CAD"],
["NOK"],
["SEK"],
["DKK"],
["HUF"],
["TRY"],
["ZAR"],
["NZD"],
["BRL"],
["IDR"],
["ILS"],
["INR"],
["KRW"],
["MXN"],
["MYR"],
["PHP"],
["THB"]
],
"created_on":"2016-11-01T17:41:28+00:00",
"modified_on":"2016-11-01T17:41:28+00:00"
}
}
]
I don't understand the way you write your manager. If the NATURAL_KEY is an attribute of each model, I would write:
class NexchangeManager(models.Manager):
def get_by_natural_key(self, param):
qs = self.get_queryset()
if param == "*":
return qs.all()
lookup = {qs.model.NATURAL_KEY: param}
return qs.filter(**lookup)
Note: this answer is based on albars answer, but improved for allowing pk params.
manager:
class NexchangeManager(models.Manager):
def get_by_natural_key(self, param):
qs = self.get_queryset()
if param == "*":
return self.all()
lookup = {qs.model.NATURAL_KEY: param}
return self.get(**lookup)
Generic Model class:
class NexchangeModel(models.Model):
class Meta:
abstract = True
objects = NexchangeManager()

change json structure in Django

I've never had to change json structure before using Django models, so I may have made a completely stupid question.
I have some data in my database and want to parse it in a json format.
My view.py (a bit simplified) is:
def get_json(request):
pos = Pos.objects.filter(id=1).values('lat', 'lon','id')
return JsonResponse ({'position': list(pos)})
As a result I get this json object:
{ "position": [{"lat": "21", "id": 1, "lon": "21"}, {"lat": "22", "id": 1, "lon": "22"}, {"lat": "23", "id": 1, "lon": "23"}]}
In order to reduce the volume of unusefull data, I need to get a json structure like this:
{"id":"1", "position":{"lats":[21,22,23], "longs":[21,22,22]} }
I would be gratefull if sombody could help me
You'll need to process the data; you can do whatever you like within the view function.
pos = Pos.objects.filter(id=1).values('lat', 'lon','id')
data = {"lats": [], "longs": []}
for p in pos:
data["lats"].append(p['lat'])
data["longs"].append(p['long'])
return JsonResponse({'position': data})

Different results on query changing Q object order

I have a problem when making a queryset using Q objects. I'm getting different results depending on how i order some Q conditions. I will simplify my models a little so as to describe my problem in a clean way.
class D(models.Model):
one_attr = models.BooleanField()
s_attr = models.ManyToManyField(S, through='DRelatedToS')
d_to_p = models.ForeignKey(P)
class S(models.Model):
other_attr = models.BooleanField()
s_to_p = models.ForeignKey(P)
class DRelatedToS(models.Model):
to_s = models.ForeignKey(S)
to_d = models.ForeignKey(D)
date = models.DateField()
class P(models.Model):
the_last_attr = models.PositiveIntegerField()
Summary of the relations:
D <-- DRelatedToS --> S --> P
| ^
| |
-------->------->------>----^
With these models and relations, i get two different results depending on how i arrange Q conditions:
First query, that gives one result
D.objects.filter(
Q(one_attr=True, s_attr__s_to_p__the_last_attr=5)
|
Q(one_attr=False, d_to_p__the_last_attr=10)
)
Second query, giving another result, different from first query
D.objects.filter(
Q(one_attr=False, d_to_p__the_last_attr=10)
|
Q(one_attr=True, s_attr__s_to_p__the_last_attr=5)
)
My question is: why is this happening? Is there any problem on how i am doing my query?
When i watch the SQL statements derived from these queries, i get two different statements: one that make a LEFT OUTER JOIN and a lot of INNER JOINs and the second that makes all INNER JOINs. The one that actually return what i want is the one that makes a LEFT OUTER JOIN. This make me feel that all my queries can return bad results depending on how i arrange its conditions. Is this a bug or i am doing anything (or everything) wrong?
Different order should return equal result in your example.
Nevertheless I tested your code (using corrections I made in the question's codes) but can't generate the error you describe. Maybe you had introduced other errors and missed when simplified the code, could you post the sample data you used? (see my data below).
First your sample code is buggy I had edited your question to suggest following corrections to fix problems, simplify and improve it for tests, but I don't see the changes updated so I repeat here:
Correction 1: Model changes in a diff format:
3,4c6,10
< s_attr = models.ManyToMany(S, through='DRelatedToS')
< d_to_p = models.ForeignKey(P)
---
> s_attr = models.ManyToManyField('S', through='DRelatedToS')
> d_to_p = models.ForeignKey('P')
>
> def __unicode__(self):
> return 'D:(%s,%s,%s)' % (self.id, self.one_attr, self.d_to_p.the_last_attr)
8,9c14
< other_attr = models.BooleanField()
< s_to_p = models.ForeignKey(P)
---
> s_to_p = models.ForeignKey('P')
13d17
< to_p = models.ForeignKey(P)
15c19
< date = models.DateField()
---
> to_s = models.ForeignKey(S)
19a24
>
So after apply corrections models look like this:
class D(models.Model):
one_attr = models.BooleanField()
s_attr = models.ManyToManyField('S', through='DRelatedToS')
d_to_p = models.ForeignKey('P')
def __unicode__(self):
return 'D:(%s,%s,%s)' % (self.id, self.one_attr, self.d_to_p.the_last_attr)
class S(models.Model):
s_to_p = models.ForeignKey('P')
class DRelatedToS(models.Model):
to_d = models.ForeignKey(D)
to_s = models.ForeignKey(S)
class P(models.Model):
the_last_attr = models.PositiveIntegerField()
Correction 2: Your lookup fields in queries are wrong (Fixed in answer).
Following is what I did:
Create project and app named testso:
django-admin.py startproject marianobianchi
cd marianobianchi
python manage.py startapp testso
Add your models & adjust project settings (database settings, add testso to INSTALLED_APPS)
Add sample data:
mkdir testso/fixtures
cat > testso/fixtures/initial_data.json
[
{"pk": 1, "model": "testso.d", "fields": {"one_attr": true, "d_to_p": 3}},
{"pk": 2, "model": "testso.d", "fields": {"one_attr": true, "d_to_p": 4}},
{"pk": 3, "model": "testso.d", "fields": {"one_attr": false, "d_to_p": 5}},
{"pk": 4, "model": "testso.d", "fields": {"one_attr": false, "d_to_p": 5}},
{"pk": 1, "model": "testso.s", "fields": {"s_to_p": 1}},
{"pk": 2, "model": "testso.s", "fields": {"s_to_p": 2}},
{"pk": 3, "model": "testso.s", "fields": {"s_to_p": 3}},
{"pk": 1, "model": "testso.drelatedtos", "fields": {"to_d": 2, "to_s": 1}},
{"pk": 2, "model": "testso.drelatedtos", "fields": {"to_d": 1, "to_s": 2}},
{"pk": 3, "model": "testso.drelatedtos", "fields": {"to_d": 1, "to_s": 3}},
{"pk": 1, "model": "testso.p", "fields": {"the_last_attr": 5}},
{"pk": 2, "model": "testso.p", "fields": {"the_last_attr": 5}},
{"pk": 3, "model": "testso.p", "fields": {"the_last_attr": 3}},
{"pk": 4, "model": "testso.p", "fields": {"the_last_attr": 4}},
{"pk": 5, "model": "testso.p", "fields": {"the_last_attr": 10}}
]
python manage.py syncdb
python manage.py shell
In the shell:
>>> from testso.models import *
>>> from django.db.models import Q
>>> D.objects.filter(Q(one_attr=True, s_attr__s_to_p__the_last_attr=5) | Q(one_attr=False, d_to_p__the_last_attr=10))
[<D: D:(1,True,3)>, <D: D:(2,True,4)>, <D: D:(3,False,10)>, <D: D:(4,False,10)>]
>>> D.objects.filter(Q(one_attr=False, d_to_p__the_last_attr=10) | Q(one_attr=True, s_attr__s_to_p__the_last_attr=5))
[<D: D:(1,True,3)>, <D: D:(2,True,4)>, <D: D:(3,False,10)>, <D: D:(4,False,10)>]
Same result! ...as expected.
I cannot answer your question directly, but there may be another way of doing what you want to do which may yield more consistent results:
subset_a = D.objects.filter(one_attr=False, d_to_p__the_last_attr=10)
subset_b = D.objects.filter(one_attr=True, s_attr__p__the_last_attr=5)
union_set = subset_a | subset_b
union_set = union_set.distinct()
The | operator on two querysets does a union, and the distinct will make sure you do not get dupe rows. I would be interested in hearing whether the arrangement here matters as well.
Django seems to have a weak ORM implementation. I would suggest using "filter" or some other way to query the database and see if you get the same inconsistencies.
Or perhaps you should look for alternative ORM implementations, like peewee.