Good afternoon,
I have my django server running with a REST api on top to serve my mobile devices. Now, at some point, the mobile device will communicate with Django.
Let's say the device is asking Django to add an object in the database, and within that object, I need to set a FK like this:
objectA = ObjectA.objects.create(title=title,
category_id = c_id, order = order, equipment_id = e_id,
info_maintenance = info_m, info_security = info_s,
info_general = info_g, alphabetical_notation = alphabetical_notation,
allow_comments = allow_comments,
added_by_id = user_id,
last_modified_by_id = user_id)
If the e_id and c_id is received from my mobile devices, should I check before calling this creation if they actually still exists in the DB? That is two extra queries... but if they can avoid any problems, I don't mind!
Thanks a lot!
It think that Django creates constraint on Foreign Key by default ( might depend on database though ). This means that if your foreign keys point to something that does not exist, then saving will fail ( resulting in Exception on Python side ).
You can reduce it to a single query (it should be a single query at least, warning I haven't tested the code):
if MyObject.objects.filter(id__in=[e_id, c_id]).distinct().count() == 2:
# create the object
ObjectA.objects.create(...)
else:
# objects corresponding e_id and c_id do not exist, do NOT create ObjectA
You should always validate any information that's coming from a user or that can be altered by a determined user. It wouldn't be difficult for someone to sniff the traffic and start constructing their own REST requests to your server. Always clean and validate external data that's being added to the system.
Related
I'm a big fan of Django-parler, but I've run into a problem when storing a translated model in two different databases.
My model is:
class InstrumentFamily(TranslatableModel):
primary_key = True
translations = TranslatedFields(
label=CharNullField(_('Label'), max_length=100, unique=False, null=True,)
I have 2 database aliases 'default' and 'test' and my database router directs my model to 'test'.
I insert models in both databases by doing this:
fam = InstrumentFamily(code=TEST_CODE)
with switch_language(fam, 'en'):
fam.label = "test_family_test EN"
with switch_language(fam, 'fr'):
fam.label = "test_family_test FR"
fam.save()
which stores the object and its translations in database 'test', or by doing this:
fam = InstrumentFamily(code="TEST_FAM")
with switch_language(fam, 'en'):
fam.label = "test_family_default_EN"
with switch_language(fam, 'fr'):
fam.label = "test_family_default_FR"
fam.save(using='default')
which saves the object and its translations to database 'default'. So far, so good.
But when accessing the object previously saved in 'default' by doing this (after properly clearing all caches to force a database read):
fam = InstrumentFamily.objects.using('default').get(code=TEST_CODE)
print(f" label: {fam.label}")
django-parler properly retrieves the object from database 'default', but looks for the translation from database 'test' ! (SQL trace below, see the very end of each line):
SELECT "orchestra_instrumentfamily"."id", "orchestra_instrumentfamily"."code" FROM "orchestra_instrumentfamily" WHERE "orchestra_instrumentfamily"."code" = 'TEST_FAM' LIMIT 21; args=('TEST_FAM',); alias=default
SELECT "orchestra_instrumentfamily_translation"."id", "orchestra_instrumentfamily_translation"."language_code", "orchestra_instrumentfamily_translation"."label", "orchestra_instrumentfamily_translation"."master_id" FROM "orchestra_instrumentfamily_translation" WHERE ("orchestra_instrumentfamily_translation"."master_id" = 34 AND "orchestra_instrumentfamily_translation"."language_code" = 'en') LIMIT 21; args=(34, 'en'); alias=test
I'm obviously missing something big... What am I supposed to do to have the 'using("default")' information propagated to the second query? I couldn't find anything in the documentation about storing TranslatableModels in more than one database. Am I trying to achieve something parler does not support?
Thanks in advance for enlightening me!
This looks like a bug in django-parler. It doesn't pass the using information to its internal queries that retrieve translation model data. You can file a bit in the GitHub repository so this can be addressed.
A workaround would be to implement a database-router that enforces using a particular database for this model.
Assume that from an external API call, we get the following response:
resp = ['123', '67283', '99829', '786232']
These are external_id fields for our objects, defined in our Article model. Some of which may already exist in database, while others don't.
Before returning a response, we need to check whether each external_id corresponds to a record in our database, and if not, we need to create it and fetch additional info from another, third, source.
What is the most efficient way to do this? Right now I can't think of something better than:
for external_id in resp:
if not Article.objects.filter(external_id=external_id).exists():
# item doesn't exist, go fetch more data and create object
else:
# already exists, do something else
But there must be a better way..?
You can use sets for this task. Following code will issue only one database call:
expected_ids = set(int(pk) for pk in resp)
exist_ids = set(Article.objects.filter(external_id__in=resp)
.values_list('external_id', flat=True))
not_exist_ids = list(expected_ids - exist_ids)
I'm building a Django app that will periodically take information from an external source, and use it to update model objects.
What I want to to be able to do is create a QuerySet which has all the objects which might match the final list. Then check which model objects need to be created, updated, and deleted. And then (ideally) perform the update in the fewest number of transactions. And without performing any unnecessary DB operations.
Using create_or_update gets me most of the way to what I want to do.
jobs = get_current_jobs(host, user)
for host, user, name, defaults in jobs:
obj, _ = Job.upate_or_create(host=host, user=user, name=name, defaults=defaults)
The problem with this approach is that it doesn't delete anything that no longer exists.
I could just delete everything up front, or do something dumb like
to_delete = set(Job.objects.filter(host=host, user=user)) - set(current)
(Which is an option) but I feel like there must already be an elegant solution that doesn't require either deleting everything, or reading everything into memory.
You should use Redis for storage and use this python package in your code. For example:
import redis
import requests
pool = redis.StrictRedis('localhost')
time_in_seconds = 3600 # the time period you want to keep your data
response = requests.get("url_to_ext_source")
pool.set("api_response", response.json(), ex=time_in_seconds)
It is a very specific question regarding Flask-appbuilder. During my development, I found FAB's ModelView is suitable for admin role, but need more user logic handlers/views for complex designs.
There is a many to many relationship between devices and users, since each device could be shared between many users, and each user could own many device. So there is a secondary table called accesses, describes the access control between devices and users. In this table, I add "isHost" to just if the user owns the device. Therefore, we have two roles: host and (regular) user. However, these roles are not two roles defined as other applications, since one man can be either host or user in same time. In a very simple application, enforce the user to switch two roles are not very convinient. That makes things worse.
Anyway, I need design some custom handlers with traditional Flask/Jinja2 templates. For example:
class PageView(ModelView):
# FAB default URL: "/pageview/list"
datamodel = SQLAInterface(Page)
list_columns = ['name', 'date', 'get_url']
#expose("/p/<string:url>")
def p(self, url):
title = urllib.unquote(url)
r = db.session.query(Page).filter_by(name = title).first()
if r:
md = r.markdown
parser = mistune.Markdown()
body = parser(md)
return self.render_template('page.html', title = title, body = body)
else:
return self.render_template('404.html'), 404
Above markdown page URL is simple, since it is a seperate UI. But if I goes to DeviceView/AccountView/AccessView for list/show/add/edit operations. I realized that I need a unique styles of UI.
So, now how can I reuse the existing templates/widgets of FAB with custom sqlalchemy queries? Here is my code for DeviceView.
class DeviceView(ModelView):
datamodel = SQLAInterface(Device)
related_views = [EventView, AccessView]
show_template = 'appbuilder/general/model/show_cascade.html'
edit_template = 'appbuilder/general/model/edit_cascade.html'
#expose('/host')
#has_access
def host(self):
base_filters = [['name', FilterStartsWith, 'S'],]
#if there is not return, FAB will throw error
return "host view:{}".format(repr(base_filters))
#expose('/my')
#has_access
def my(self):
# A pure testing method
rec = db.session.query(Access).filter_by(id = 1).all()
if rec:
for r in rec:
print "rec, acc:{}, dev:{}, host:{}".format(r.account_id, r.device_id, r.is_host)
return self.render_template('list.html', title = "My Accesses", body = "{}".format(repr(r)))
else:
return repr(None)
Besides sqlalchemy code with render_template(), I guess base_filters can also help to define custom queries, however, I have no idea how to get query result and get them rendered.
Please give me some reference code or example if possible. Actually I have grep keywords of "db.session/render_template/expoaw"in FAB's github sources. But no luck.
I have a database of exhibition listings related by foreign key to a database of venues where they take place. Django templates access the venue information in the query results through listing.venue.name, listing.venue.url, and so on.
However, some exhibitions take place in temporary venues, and that information is stored in the same database, in what would be listing.temp_venue_url and such. Because it seems wasteful and sad to put conditionals all over the templates, I want to move the info for temporary venues to where the templates are expecting info for regular venues. This didn't work:
def transfer_temp_values(listings):
for listing in listings:
if listing.temp_venue:
listing.venue = Venue
listing.venue.name = listing.temp_venue
listing.venue.url = listing.temp_venue_url
listing.venue.state = listing.temp_venue_state
listing.venue.location = listing.temp_venue_location
The error surprised me:
ValueError at /[...]/
Cannot assign "<class 'myproject.gsa.models.Venue'>": "Exhibition.venue" must be a "Venue" instance.
I rather thought it was. How do I go about accomplishing this?
The error message is because you have assigned the class Venue to the listing, rather than an instance of it. You need to call the class to get an instance:
listing.venue = Venue()