I need help to improve API performance. Below is my approach also i have tried to do other combination of fields with select_related or prefetch_related but still i am getting like 400sqls.
model.py
class VisVisits(models.Model):
visit_id = models.IntegerField(primary_key=True)
null=True)
class Meta:
managed = False
db_table = 'vis_visits'
def __str__(self):
return str(self.visit_id)
class VisVisitData(models.Model):
vdata_id = models.IntegerField(primary_key=True)
app_local_id = models.IntegerField(blank=True, null=True)
visit = models.ForeignKey('VisVisits', models.DO_NOTHING, blank=True, null=True, related_name='data')
class Meta:
managed = False
db_table = 'vis_visit_data'
def __str__(self):
return str(self.vdata_id)
Serializer
class VisVisitDataSerializer(serializers.ModelSerializer):
class Meta:
model = VisVisitData
field = '__all__'
class VisVisitsSerializer(serializers.ModelSerializer):
data = VisVisitDataSerializer(many=True)
class Meta:
model = VisVisits
fields = ('visit_id','data')
views.py
visit_data = VisVisits.objects.filter(is_valid=1,user_id=u).prefetch_related('school_program__school')
visit_data_serializer = VisVisitsSerializer(visit_data,context={'request':request},many=True)
To reduce the number of queries you must prefetch the data that you serialize in the VisVisitsSerializer, i.e. data. In this serializer you don't include school_program so it's not necessary prefetch this. You can use the django debug toolbar (https://django-debug-toolbar.readthedocs.io/en/latest/) to inspect the repeated queries and find out what to put in select_releted/prefetch_related
Related
I have two models in my models.py. I need to return a json response which includes data from two tables.
How should my view and serializer look like?
class Device(models.Model):
id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
device_name = models.CharField(max_length=200, null=True, blank=True )
created_at = models.DateTimeField(auto_now_add=True)
def __str__(self):
return str(self.device_name)
class StatusActivity(models.Model):
OFFLINE = 1
ONLINE = 2
STATUS = (
(OFFLINE, ('Offline')),
(ONLINE, ('Online')),
)
id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
device_id = models.ForeignKey(Device, related_name='StatusActivity', on_delete=models.CASCADE)
changed_to = models.PositiveSmallIntegerField(choices=STATUS)
modified_at = models.DateTimeField(auto_now_add=True)
def __str__(self):
return str(self.device_id)
Expected Response:
{
"device_id":"",
"device_name":"",
"changed_to":"",
"modified_at":"",
}
UPDATE:
I set my views.py and serializer.py as below. I am checking
Serializer.py
class DeviceSerializer(serializers.ModelSerializer):
class Meta:
model = Device
fields = '__all__'
class StatusActivitySerializer(serializers.ModelSerializer):
class Meta:
model = StatusActivity
fields = '__all__'
class ListSerializer(serializers.Serializer):
# devices = DeviceSerializer(many=True)
# activities = StatusActivitySerializer(many=True)
class Meta:
model = [Device, StatusActivity]
fields = ['device_id', 'device_name', 'changed_to', 'modified_at']
Views.py
class DeviceListView(generics.ListAPIView):
queryset = Device.objects.all()
serializer_class = ListSerializer
class StatusActivityListView(generics.ListAPIView):
queryset = StatusActivity.objects.all()
serializer_class = StatusActivitySerializer
Actually you don't need to have two separated views for this, because you can easily serialize relations from one serializer class.
Take a look at this useful answer: How do I include related model fields using Django Rest Framework?
For your case you can write something like this:
class StatusActivitySerializer(serializers.ModelSerializer):
device_name = serializers.CharField(source='device_id.device_name')
class Meta:
model = StatusActivity
fields = ('changed_to', 'modified_at', 'device_id', 'device_name')
Something that worth to note:
it's a good idea for ForeignKey field use device name instead of
device_id;
related_name arg should have a name for reverse access. Keep it
meaningful, e.g. status_activities is a good choice.
filter the status activity. you can refer the foreignkey.
eg:
items = []
for statusact in StatusActivity.objects.all():
items.append({
"device_id":statusact.device_id.id,
"device_name":statusact.device_id.device_name,
"changed_to":statusact.changed_to,
"modified_at":statusact.modified_at,
})
I have the following models
class STUser(AbstractBaseUser):
email = models.EmailField(unique=True)
name = models.CharField(max_length=255)
companyname = models.CharField(max_length=200, blank=True, null=True)
...
class VenuePermissions(models.Model):
user = models.ForeignKey(STUser, on_delete=models.CASCADE)
venue = models.ForeignKey(Venue, on_delete=models.CASCADE)
signupvaildatestring = models.CharField(max_length=200, blank=True, null=True)
...
I want to grab all the STUser objects and grab all their permissions.
So what I would like is to grab all the VenuePermissions objects. And grab the user and venue object of each venuePermission
Two ways I can do this. use the VenuePermissions_set attribute on STUser but then how do I grab the venue when its just going to be a pk value?
Or focus on the VenuePermissions objects and grab the user and venue from the pk values but how?
I remember nested queries, and I kinda did one in my browse code.
here is an example:
rooms = Room.objects.filter(venue=OuterRef('pk'), sixtyroundseatingoption= True)
venuelist = venuelist.annotate(sixtyrounds=Exists(rooms))
venuelist = venuelist.filter(Q(sixtyrounds = True) | Q(fullbuyoutsixtyroundseatingoption = True))
I've done the set objects in a serializer before
Example serializer:
class RoomAndImageSerializer(serializers.ModelSerializer):
roomimage_set = RoomImageSerializer(many=True, read_only=True)
class Meta:
model = Room
fields = ('pk','name')
any help with this query would be appreciated!
So this is what I am currently trying, I will post an answer if this works:
class VenueUserList(ListAPIView):
serializer_class = VenueUserListSerializer
queryset = VenuePermissions.objects.select_related('user').select_related('venue').filter(signupvaildatestring=None)
class VenueUserListSerializer(serializers.ModelSerializer):
user = UserSerializer()
venue = VenueSerializer()
class Meta:
model = VenuePermissions
fields = ('user', 'venue', 'isvenueviewer', 'isvenueeventplanner', 'isvenueadministrator')
Here is the answer. However I still need to group venues by user. Working on that.
class VenueUserList(ListAPIView):
serializer_class = VenueUserListSerializer
queryset = VenuePermissions.objects.select_related('user').select_related('venue').filter(signupvaildatestring=None)
class VenueUserListSerializer(serializers.ModelSerializer):
user = UserSerializer()
venue = VenueSerializer()
class Meta:
model = VenuePermissions
fields = ('user', 'venue', 'isvenueviewer', 'isvenueeventplanner', 'isvenueadministrator')
Preliminary note: this is a rather newbie question, though I have not found a sufficient answer on StackOverflow; many similar questions, but not this one. So I am asking a new question.
The problem: I'm having difficulty creating records where one field is a foreign key to an existing record, and I do not know what I'm doing wrong in my code.
In my app there are two models in question, a one-to-many relationship between Company and BalanceSheet:
models:
class Company(models.Model):
cik = models.IntegerField(default=0, unique=True)
symbol = models.CharField(max_length=4, unique=True)
name = models.CharField(max_length=255, unique=True)
def __str__(self):
return self.symbol
class BalanceSheet(models.Model):
company = models.ForeignKey(Company,
null=True,
on_delete=models.CASCADE,
related_name='balance_sheets',)
date = models.DateField()
profit = models.BigIntegerField()
loss = models.BigIntegerField()
class Meta:
unique_together = (('company', 'date'),)
def __str__(self):
return '%s - %s' % (self.company, self.date)
serializers:
class BalanceSheetSerializer(serializers.ModelSerializer):
company = serializers.StringRelatedField()
class Meta:
model = BalanceSheet
fields = ('company','date','profit','loss')
class CompanySerializer(serializers.ModelSerializer):
class Meta:
model = Company
fields = ('cik', 'symbol', 'name')
Views:
class BalanceSheetCreate(generics.CreateAPIView):
model = BalanceSheet
queryset = BalanceSheet.objects.all()
serializer_class = BalanceSheetSerializer
urls include:
url(r'^(?P<symbol>[A-Z]{1,4})/create-balance-sheet/$', views.BalanceSheetCreate.as_view(),
name='create_balance_sheet'),
To this point, I have zero problem reading data. However, when trying to create records, I get errors I don't understand:
curl http://localhost:8000/financials/AAPL/create-balance-sheet/ -X POST -d "company=AAPL&date=1968-04-17&profit=1&loss=1"
IntegrityError at /financials/AAPL/create-balance-sheet/
null value in column "company_id" violates not-null constraint
Dropping the company data from that curl command results in the same error.
How do I get around this error? I thought I was telling the api what company I'm interested in, both explicitly in the url and in the post data.
Using python3.6, django 1.11, and djangorestframework 3.7.7
You get the IntegrityError because your code will try to create a new BalanceSheet without a company. That's because StringRelatedField is read-only (see docs) and therefore it's not parsed when BalanceSheetSerializer is used in write mode.
SlugRelatedField is what you need here:
class BalanceSheetSerializer(serializers.ModelSerializer):
company = serializers.SlugRelatedField(slug_field='symbol')
class Meta:
model = BalanceSheet
fields = ('company','date','profit','loss')
To answer my own question, here's what I wound up with. Thanks again go to dukebody.
models:
class Company(models.Model):
cik = models.IntegerField(default=0)
symbol = models.CharField(max_length=4)
name = models.CharField(max_length=255)
def __str__(self):
return self.symbol
class BalanceSheet(models.Model):
company = models.ForeignKey(Company,
null=True,
on_delete=models.CASCADE,
related_name='balance_sheets',)
date = models.DateField()
profit = models.BigIntegerField()
loss = models.BigIntegerField()
class Meta:
unique_together = (('company', 'date'),)
def __str__(self):
return '%s - %s' % (self.company, self.date)
serializers:
class CompanySerializer(serializers.ModelSerializer):
class Meta:
model = Company
fields = ('cik', 'symbol', 'name')
class BalanceSheetSerializer(serializers.ModelSerializer):
company = CompanySerializer(many=False)
class Meta:
model = BalanceSheet
fields = ('company', 'date', 'profit', 'loss')
def create(self, validated_data):
company_data = validated_data['company']
company, created = Company.objects.get_or_create(**company_data)
validated_data['company'] = company
sheet = BalanceSheet.objects.create(**validated_data)
return sheet
I also had to include the full company data within my curl statement as a nested dict.
these are my models:
class ADS(models.Model):
advertiser = models.ForeignKey(User)
campaign = models.ForeignKey(Campaign, related_name="ads")
headline = models.CharField(max_length=50)
description_1 = models.TextField(blank=True)
description_2 = models.TextField(blank=True)
display_url = models.URLField(blank=True)
final_url = models.URLField(blank=True)
mobile_url = models.URLField(blank=True)
class Meta:
db_table = "ads"
and:
class AdsImages(models.Model):
ads = models.ForeignKey(ADS,blank=True, null=True)
image = models.ImageField(upload_to='images/',default='media/None/no-img.jpg')
class Meta:
db_table = "ads_images"
I have a nested serializer, as you can see the second model is a table with images, so I am uploading asynchronously the images via AngularJs. As you can see the ForeignKey(ADS, blank=True, null=True), so I am not filling it for the first time. My question is that how to update after that the ads field.
here are my serializers:
class AdsImagesSerializer(serializers.ModelSerializer):
image = serializers.ImageField(use_url=True,allow_empty_file=True)
class Meta:
model = AdsImages
fields = ('id','image','ads',)
class ADSerializer(serializers.ModelSerializer):
adsImages = AdsImagesSerializer(many=True)
class Meta:
model = ADS
fields = ('headline','description_1','description_2','display_url','final_url','mobile_url','advertiser','adsImages',)
Partial updates
By default, serializers must be passed values for all required fields or they will raise validation errors. You can use the partial argument in order to allow partial updates
# Update `comment` with partial data
serializer = CommentSerializer(comment, data={'content': u'foo bar'}, partial=True)
I'm building a serializer in django using the django rest framework. I need to filter the query set for a nested model.
I found How do you filter a nested serializer in Django Rest Framework?, which seemed to have the answer, but when I implemented it there was no change in my data. The only difference I can see is that the serializer referencing the filtered list serializer has other fields as well.
The models (abbreviated for clarity):
class GCUser(models.Model):
first_name = models.CharField(max_length=50)
last_name = models.CharField(max_length=50)
email = models.EmailField()
is_member = models.BooleanField(default=False)
age = models.SmallIntegerField(blank=True, null=True)
session_key = models.CharField(max_length=100, db_index=True, blank=True, null=True)
class Connection(models.Model):
creation_date = models.DateTimeField(auto_now_add=True)
user = models.ForeignKey(GCUser, related_name='user_connection')
event = models.ForeignKey(Event, related_name='event_connection')
role = models.CharField(max_length=8, choices=constants.Roles.ROLE_CHOICES,)
class Event(Game):
creation_date = models.DateTimeField(auto_now_add=True)
public = models.BooleanField(default=False)
start_time = models.DateTimeField(null=True, blank=True)
end_time = models.DateTimeField(null=True, blank=True)
gm_is_player = models.BooleanField(default=False,
help_text='Check if GM will be playing the game',
verbose_name='GM is a player')
gm_is_designer = models.BooleanField(default=False, help_text='Check if GM designed the game')
user_notes = models.TextField(blank=True, default='', verbose_name='Note to Scheduler')
scheduler_notes = models.TextField(blank=True, default='')
experience = models.CharField(max_length=3, choices=constants.ExpLevels.EXPERIENCE_CHOICES,
default=constants.ExpLevels.NOVICE,)
status = models.CharField(max_length=4, db_index=True,
choices=constants.Status.STATUS_CHOICES,)
Here's my code:
class FilteredListSerializer(serializers.ListSerializer):
def to_representation(self, data):
data = data.filter(status=constants.Status.ASSIGNED).order_by('start_time')
return super(FilteredListSerializer, self).to_representation(data)
class UserEventSerializer(serializers.ModelSerializer):
class Meta:
list_serializer_class = FilteredListSerializer
model = models.Event
fields = ('id', 'event_name', 'conflict_type', 'start_time', 'end_time')
class UserConnectionSerializer(serializers.ModelSerializer):
event = UserEventSerializer()
class Meta:
model = models.Connection
fields = ('get_role_display', 'conflict_type', 'event')
class GCUserSerializer(serializers.ModelSerializer):
user_connection = UserConnectionSerializer(many=True)
class Meta:
model = models.GCUser
fields = ('pk', 'first_name', 'last_name', 'email',
'is_member', 'age', 'user_connection')
PyCharm tells me that "class FilteredListSerializer must implement all abstract methods" but it doesn't actually throw an error. I put a breakpoint at the first line of the list serializer, but it doesn't get tripped.
I'm using Python 3.4 with django 1.7.
Thanks in advance for your help.
Edited to add: Looking into the serializer code, I realized what may be the key difference: my call has many=True, whereas the one from the previous post didn't. I tried taking out the model=, but as expected that threw an error, so apparently the "working" code in the earlier post didn't actually run as written.
So I am not sure how to use the method you are using, but, if I understand your question correctly, I believe you could do something like this:
class UserEventSerializer(serializers.ModelSerializer):
class Meta:
model = models.Event
fields = ('id', 'event_name', 'conflict_type', 'start_time', 'end_time')
class UserConnectionSerializer(serializers.ModelSerializer):
event = serializers.SerializerMethodField()
class Meta:
model = models.Connection
fields = ('get_role_display', 'conflict_type', 'event')
def get_event(self, obj):
if obj.event.status == constants.Status.ASSIGNED:
serializer = UserEventSerializer(obj.event)
return serializer.data
else:
serializer = UserEventSerializer(None)
return serializer.data
N.B. This assumes that you are trying to exclude Events from being serialized if their status is not assigned.
I hope this helps. If I didn't understand the problem, let me know.
it's not a bug or error. ModelSerializer has got already implemented all needed methods (most inhereted from Serializer class), but ListSerializer inherits from BaseSerializer and got implemented e.g.: .create() or .to_representation(), but not .update(). I've got some similar problems in PyCharm, when subclassing a Serializer. After implementing create, update and to_representation methods this issue was gone