I am using ndb Structured property in my app. The models look like this:
Resource External Integration:
class ResourceExternalIntegration(ndb.Model):
integration_type = ndb.StringProperty()
external_resource_type = ndb.StringProperty()
external_resource_id = ndb.IntegerProperty()
external_group_id = ndb.IntegerProperty()
external_resource_name = ndb.StringProperty()
Resouce Model:
class Resource(ndb.Model):
owner = ndb.IntegerProperty()
name = ndb.StringProperty()
type = ndb.StringProperty()
external_integrations = ndb.StructuredProperty(ResourceExternalIntegration, repeated=True)
Note that i have structured property as repeated=True
Issue:
I have a function in Resource class which formats/serializes the data extracted from DB. It looks like this:
def serialize(self):
external_integrations_list = []
if self.external_integrations:
for external_integration in self.external_integrations:
external_integration_dict = dict()
external_integration_dict['integration_type'] = external_integration.integration_type,
external_integration_dict['external_resource_type'] = external_integration.external_resource_type,
external_integration_dict['external_resource_id'] = external_integration.external_resource_id
external_integration_dict['external_group_id'] = external_integration.external_group_id
external_integration_dict['external_resource_name'] = external_integration.external_resource_name
external_integrations_list.append(external_integration_dict)
resource_data.update(dict(
owner=self.owner,
name=self.name,
type=self.type,
external_integrations=external_integrations_list
))
return resource_data
Now, in the resource_data the attribute external_integrations should be an array and every element in it should also be an array i.e. external_resource_id, external_resource_type etc should also be an array. It is because of the fact that structured property was set as repeated=True. But, the resource_data does not contain this expected result. It looks like:
{'name': u'Scissors lift', 'type': u'Scissors', 'external_integrations': [{'external_resource_type': (u'FLEET',), 'integration_type': (u'ABC',), 'external_resource_id': 212017856321402L, 'external_resource_name': u"Test 1", 'external_group_id': 5000}],'owner': 5629490125014563L}
And, on browser it looks like this:
external_group_id: 5000
external_resource_id: 212017856321402
external_resource_name: "Test 1"
external_resource_type: ["FLEET"]
integration_type: ["ABC"]
i.e. the external_group_id, external_resource_id, external_resource_name does not appear as array, but they were expected as arrays.
I also have another model in which the structured property does not exists as repeated=True. It looks like:
External Integration
class ExternalIntegration(ndb.Model):
access_token = ndb.StringProperty()
group_ids = ndb.IntegerProperty(repeated=True)
Profile
class profile(ndb.Model):
name = ndb.StringProperty()
integration_info = ndb.StructuredProperty(ExternalIntegration)
Here, the serialize function of profile model show result as:
{'integration_info': {'group_ids': ([5000],), 'access_token': (u'blahblahblahblah',)}, ''name: 'Test'}
And, on browser the result looks like:
access_token: ["blahblahblahblah"]
group_ids: [[5000]]
I am unable to understand why access_token appears as an array and why groups_ids is an array of array.
Can anyone please help me understand such behavior of ndb structured property? Specifically the cases i explained above.
There are two questions here:
Regarding the profile model:
integration_info contains two elements defined in ExternalIntegration class. It is saved as a dictionary containing two elements: access_token and group_ids. group_ids is defined with repeated=True, which takes a list of values of the underlying type, which creates a list. To summarize:
access_token appears as a string.
group_ids appears as a list, because repeated is set to True.
Regarding the resource model:
external_integrations appear as a list, because you defined repeated=True.
There is only one element on the list because the whole dictionary was appended in a single operation, instead of element per element.
external_group_id, external_resource_id and external_resource_name don't appear as an array because they were not defined with repeated=True. The definition is applied one level above, to external_integrations, not to each of its contained strings.
Try to redefine the elements which should be repeated and the ones that shouldn't.
Related
I am trying to "DRY" my code and would like to create a function to make my query dynamic.
The code that I currently use is :
rightone = []
for item in taglist: #taglist is a list of model instances, not relevant here
content = content.filter(tags__title=item.title) #tags here is a M2M key : my problem, content is a query
rightone.append(content)
tagproofquery = rightone[-1]
and I would like to convert it to:
def uniquetogether(queryset,data, model):
rightone = []
for item in queryset:
content = data.filter(tags__title=item.title) # <--- problem is here with tags
rightone.append(content)
tagproofquery = rightone[-1]
return tagproofquery
I have no idea how to replace my M2M "tags" as in tags__title=item.title with the "model" parameter of my function. I tried f strings but it failed miserably (of course).
Is there a way to do this? Many thanks
I have 2 large postgres tables which have an index so that I can perform a full text search on each.
Typically, they look like:
class Post_1(db.Model):
query_class = PostQuery
id = db.Column(db.Integer, primary_key=True)
title = db.Column(db.String)
content = db.Column(db.Text)
datestamp = db.Column(db.Float)
search_vector = db.Column(TSVectorType('title', 'content'))
and
class Post_2(db.Model):
query_class = PostQuery
id = db.Column(db.Integer, primary_key=True)
title = db.Column(db.String)
content = db.Column(db.Text)
datestamp = db.Column(db.Float)
search_vector = db.Column(TSVectorType('title', 'content'))
In my flask application, to get the documents which have a specific keyword in one of the tables, I would do:
Post_1.query.search(keyword).\
order_by(Post_1.datestamp.desc()).limit(1)
Since I want to run the same search simultaneously on both tables, I wanted to use flask-executor and wrote the following code:
from flask_executor import Executor
executor = Executor(app)
futures=[]
keyword = "covid"
future=executor.submit(Post_1.query.search(keyword).\
order_by(Post_1.datestamp.desc()).limit(1))
futures.append(future)
future = executor.submit(Post_2.query.search(keyword).\
order_by(Post_2.datestamp.desc()).limit(1))
futures.append(future)
This does not work and I get the following error:
RuntimeError: This decorator can only be used at local scopes when a request context is on the stack. For instance within view functions.
Could anyone help me please?
The error you're getting is because flask-executor is intended to run tasks inside view functions - that is, as part of a request from a user. You're running your code outside of a view (i.e. outside of a scope that would normally be in place when your user is interacting with your application).
Do you need to do this, or is this simply part of a test? If you do something like this, just to test it out:
#app.route('/test')
def testroute():
future=executor.submit(Post_1.query.search(keyword).\
order_by(Post_1.datestamp.desc()).limit(1))
futures.append(future)
future = executor.submit(Post_2.query.search(keyword).\
order_by(Post_2.datestamp.desc()).limit(1))
futures.append(future)
Then you should no longer get the error about running outside of a request context, because the code will be running as part of a request (i.e. inside a view function).
As a side note, the SQLAlchemy tasks you're submitting aren't callable - they're not function objects. Executors, whether the one created by Flask-Executor or the vanilla ones you can get via concurrent.futures, expect you to pass a "callable". I suspect your code still wouldn't work unless it was something like:
query = Post_1.query.search(keyword).\
order_by(Post_1.datestamp.desc()).limit(1).all
future = executor.submit(query)
Notice the lack of brackets at the end, because I want to use the callable object itself, not the result it will return
The executor would then "call" the object that had been passed:
executor.submit(Post_1.query.search(keyword).\
order_by(Post_1.datestamp.desc()).limit(1).all()
key = modelobj.__class__.__name__
keyval = modelobj.pk
sets = ['cat_set', 'dog_set']
for x in set:
test = eval(key).objects.get(pk=keyval).eval(x).values()
print(test)
The entries in the 'sets' list have a ManytoOne relationship (already defined in the model file) with the 'key'. When I run this script I get the error: "AttributeError: 'Activities' object has no attribute 'eval'" for the "eval(x)" part (the first eval(key) works fine).
For instance, if I change the line to the following code, it runs fine but I need to be less explicit:
test = eval(key).objects.get(pk=keyval).cat_set.values()
Use getattr and refer the model directly (why would you need to get the model name and run eval?):
model = modelobj.__class__
keyval = modelobj.pk
sets = ['cat_set', 'dog_set']
for x in set:
test = getattr(model.objects.get(pk=keyval), x).values()
print(test)
I'm trying to get a unique value for a field (unique within the db column).
my code (other model fields omitted):
class PlatformUserChildren(models.Model):
dashboard = models.CharField('dashboard URL', max_length=64, unique=True, default=createDashboardCode(self))
def createDashboardCode(self):
stringCheck = False
while stringCheck is False:
newString = str(uuid.uuid4())[:32]
doesStringExist = newString in self.dashboard
if not doesStringExist:
stringCheck = True
return newString
I'm getting name 'self' is not defined as an error.
What should I be passing to the function so that I can check the db column to ensure the value is unique -or- is there a built-in way of doing this?
What I've already tried or looked at:
setting unique=True for the field and using default=uuid.uuid4 - that gives me duplicate values and generates a validation error (SO link)
I'm aware of Django 1.8's UUID field, however i'm on 1.7
The problem lies in the following line (indented for better readability) as you already know and mentioned before:
dashboard = models.CharField(
'dashboard URL',
max_length=64,
unique=True,
default=createDashboardCode(self)
)
In this part:
default=createDashboardCode(self)
you're calling the method createDashboardCode with the argument self. This is wrong, because you never pass self to a method as it is passed by Python. Whenever you call the method createDashboardCode you should do it this way:
createDashboardCode()
That's it, you're not passing the argument self explicitly.
You're getting an error "name 'self' is not defined" because self is not defined. There is no variable self in your code that you can pass to the method.
Now we're one step further, but your problem won't be solved if you just apply this slight change to your code.
The return value from the method createDashboardCode will be assigned to default. That's not what you really want. You have to assign a reference of the method to default:
default = createDashboardCode
Pay attention to the missing brackets. This will call the method every time a new instance of the model is created
Define a function:
def my_function():
print "hello"
and run it in the Python interpreter:
# once like this
my_function()
# and again like this
my_function
and you'll see the difference. That should help you to better comprehend this issue.
I'm trying to render a form with a combo that shows related entities. Therefore I'm using a ModelChoiceField.
This approach works well, until I needed to limit which entities to show. If I use a simple query expression it also works well, but things break if I use a raw SQL query.
So my code that works, sets the queryset to a filter expression.
class ReservationForm(forms.Form):
location_time_slot = ModelChoiceField(queryset=LocationTimeSlot.objects.all(), empty_label="Select your prefered time")
def __init__(self,*args,**kwargs):
city_id = kwargs.pop("city_id") # client is the parameter passed from views.py
super(ReservationForm, self).__init__(*args,**kwargs)
# TODO: move this to a manager
self.fields['location_time_slot'].queryset = LocationTimeSlot.objects.filter(city__id = city_id )
BUT, if I change that to a raw query I start having problems. Code that does not work:
class ReservationForm(forms.Form):
location_time_slot = ModelChoiceField(queryset=LocationTimeSlot.objects.all(), empty_label="Select your prefered time")
def __init__(self,*args,**kwargs):
city_id = kwargs.pop("city_id") # client is the parameter passed from views.py
super(ReservationForm, self).__init__(*args,**kwargs)
# TODO: move this to a manager
query = """SELECT ts.id, ts.datetime_to, ts.datetime_from, ts.available_reserves, l.name, l.'order'
FROM reservations_locationtimeslot AS ts
INNER JOIN reservations_location AS l ON l.id = ts.location_id
WHERE l.city_id = %s
AND ts.available_reserves > 0
AND ts.datetime_from > datetime() """
time_slots = LocationTimeSlot.objects.raw(query, [city_id])
self.fields['location_time_slot'].queryset = time_slots
The first error I get when trying to render the widget is: 'RawQuerySet' object has no attribute 'all'
I could solve that one thanks to one of the commets in enter link description here, by doing:
time_slots.all = time_slots.__iter__ # Dummy fix to allow default form rendering with raw SQL
But now I'm getting something similar when posting the form:
'RawQuerySet' object has no attribute 'get'
Is there a proper way to prepare a RawQuerySet to be used by ModelChoiceField?
Thanks!
Are you sure you actually need a raw query there? Just looking at that query, I can't see any reason you can't just do it with filter(location__city=city_id, available_reserves__gte=0, datetime_from__gt=datetime.datetime.now()).
Raw query sets are missing a number of methods that are defined on conventional query sets, so just dropping them in place isn't likely to work without writing your own definitions for all those methods.
I temporarily fixed the problem adding the missing methods.
The way I'm currently using the ModelChoiceField I only needed to add the all() and get() methods, but in different scenarios you might need to add some other methods as well. Also this is not a perfect solution because:
1) Defining the get method this way migth produce incorrect results. I think the get() method is used to validate that the selected option is within the options returned by all(). The way I temporarily implemented it only validates that the id exists in the table.
2) I guess the get method is less performant specified this way.
If anyone can think of a better solution, please let me know.
So my temporary solution:
class LocationTimeSlotManager(models.Manager):
def availableSlots(self, city_id):
query = """SELECT ts.id, ts.datetime_to, ts.datetime_from, ts.available_reserves, l.name, l.'order'
FROM reservations_locationtimeslot AS ts
.....
.....
MORE SQL """
time_slots = LocationTimeSlot.objects.raw(query, [city_id])
# Dummy fix to allow default form rendering with raw SQL
time_slots.all = time_slots.__iter__
time_slots.get = LocationTimeSlot.objects.get
return time_slots