I am facing a problem while filtering a record from the database. In my database I have records which contain a dictfield.
for eg
region is a dictionary which contains other dictionaries like
region = {'eastus': {'percentage_change': 0}, 'westus': {'percentage_change': 0}}
and I want to fetch eastus or westus.
I have tried something like this, but I am getting an empty list.
MyModel.objects.filter(region__eq='eastus')
any suggestions for this problem?
I believe you can achieve this with the exists operator (http://docs.mongoengine.org/guide/querying.html#query-operators)
MyModel.objects.filter(region__eastus__exists=True)
Regarding the OR, you should use the Q class (http://docs.mongoengine.org/guide/querying.html#advanced-queries), thus:
MyModel.objects.filter(Q(region__eastus__exists=True) | Q(region__westus__exists=True))
If you have keys that aren't simple (e.g including fancy character or simply a dash), you can use the __raw__ operator:
MyModel.objects.filter(__raw__={'region.a-fancy-key': {'$exists': True}})
Related
I have a jsonfield thats an array of dicts all with the same structure. I want to filter that checks whether a given record has all values in a list of params at a specific key, somewhere in the list.
(jsonfield) array = [{'dict_field': 1..}, {'dict_field': 2}, {'dict_field': 3}]
filter = [1,2,3]
return queryset.filter('array__dict_field__contains' = filter)
(1 result)
filter2 = [1,2,4]
return queryset.filter('array__dict_field__contains' = filter2)
(0 results; as it doesn't satisfy all filters)
I'm using the filter __contains as a placeholder but unfortunately its not contains - what should it be? Thanks
I tried array__dict_field__in (example schema above) but it doesn't seem to be working. It doesn't error out but doesn't match any records. The PSQL query it generates is as follows:
SELECT * FROM "table" WHERE ("table"."array" -> dict_field) IN (2)
This returns an error when I run it in PSQL also; it says column role doesn't exist. I thought it was because role has no quotes on it, but I added them and it persists. Perhaps its because my data is an array of dicts (as opposed to dict) as so:
[{"role": 2, "years_experience": 3}, {"role": 3, "years_experience": 5}]
Importantly in my case, I can only show a record if it satisfies all filter criteria in one of the keys, I was thinking __in might not be the right one here, but was also unsure about __contains as it needs to check role in across various dicts, not just one.
I see solutions here and here for filtering lists of arrays by one value, I'm unsure how to remix this to filter a dict list by a list and ensure all list entries are covered.
I have the following situation.
The Flight model (flights) has a field named 'airlines_codes' (TextField) in which I store data in JSON array like format:
["TB", "IR", "EP", "XX"]
I need to filter the flights by 2-letter airline code (IATA format), for example 'XX', and I achieve this primitively but successfully like this:
filtered_flights = Flight.objects.filter(airlines_codes__icontains='XX')
This is great but actually not.
I have flights where airlines_codes look like this:
["TBZ", "IR", "EP", "XXY"]
Here there are 3-letter codes (ICAO format) and obviously the query filter above will not work.
PS. I cannot move to PostgreSQL, also I cannot alter in anyway the database. This has to be achieved only by some query.
Thanks for any idea.
Without altering the database in any way you need to filter the value as a string. Your best bet might be airlines_codes__contains. Here's what I would recommend assuming your list will always be cleaned exactly as you represent it.
Flight.objects.filter(airlines_codes__contains='"XX"')
As of Django 3.1 JSONField is supported on a wider array of databases. Ideally, for someone else building a similar system from the ground up, this field would be a preferable approach.
I have a Project and when searching a query I need to split the data (not search query) in to words and apply searching.
for example:
my query is : 'bot' (typing 'bottle')
but if I use meta_keywords__icontains = query the filter will also return queries with 'robot'.
Here meta_keywords are keywords that can be used for searching.
I won't be able to access data if the data in meta_keywords is 'water bottle' when I use meta_keywords__istartswith is there any way I can use in this case.
what I just need is search in every words of data with just istartswith
I can simply create a model for 'meta_keywords' and use the current data to assign values by splitting and saving as different data. I know it might be the best way. I need some other ways to achieve it.
You can search the name field with each word that istartswith in variable query.
import re
instances = Model.objects.filter(Q(name__iregex=r'[[:<:]]' + re.escape(query)))
Eg: Hello world can be searched using the query 'hello' and 'world'. It don't check the icontains
note: It works only in Python3
I have a model with some HStoreField attributes and I can't seem to use Django's ORM HStoreField to query those values using LIKE.
When doing Model.objects.filter(hstoreattr__values__contains=['text']), the queryset only contains rows in which hstoreattr has any value that matches text exactly.
What I'm looking for is a way to search by, say, te instead of text and those same rows be returned as well. I'm aware this is possible in a raw PostgreSQL query but I'm looking for a solution that uses Django ORM.
If you want to check value of particular key in every object if it contains 'te', you can do:
Model.objects.filter(hstoreattr__your_key__icontains='te')
If you want to check if any key in your hstore field contains 'te', you will need to create your own lookup in django, because by default django won't do such thing. Refer to custom lookups in django docs for more info.
As far as I can remember, you cannot filter in values. If you want to filter in values, you have to pass a column and value you are referencing to. When you want it to be case insensitive use __icontains.
Although you cannot filter by all values, you can filter by all keys. Just like you showed in your code.
If you want to search for 'text' in all objects in key named let's say 'fo' - just do smth like this:
Model.objects.filter(hstoreattr__icontains={'fo': 'text'})
I have documents in couchdb. The schema looks like below:
userId
email
personal_blog_url
telephone
I assume two users are actually the same person as long as they have
email or
personal_blog_url or
telephone
be identical.
I have 3 views created, which basically maps email/blog_url/telephone to userIds and then combines the userIds into the group under the same key, e.g.,
_view/by_email:
----------------------------------
key values
a_email#gmail.com [123, 345]
b_email#gmail.com [23, 45, 333]
_view/by_blog_url:
----------------------------------
key values
http://myblog.com [23, 45]
http://mysite.com/ss [2, 123, 345]
_view/by_telephone:
----------------------------------
key values
232-932-9088 [2, 123]
000-111-9999 [45, 1234]
999-999-0000 [1]
My questions:
How can I merge the results from the 3 different views into a final user table/view which contains no duplicates?
Or whether it is a good practice to do such deduplication in couchdb?
Or what would be a good way to do a deduplication in couch then?
ps. in the finial view, suppose for all dupes, we only keep the smallest userId.
Thanks.
Good question. Perhaps you could listen to _changes and search for the fields you want to be unique for the real user in the views you suggested (by_*).
Merge the views into one (emit different fields in one map):
function (doc) {
if (!doc.email || !doc.personal_blog_url || !doc.telephone) return;
emit([1, doc.email], [doc._id]);
emit([2, doc.personal_blog_url], [doc._id]);
emit([3, doc.telephone], [doc._id]);
}
Merge the lists of id's in reduce
When new doc in changes feed arrives, you can query the view with keys=[[1, email], [2, personal_blog_url], ...] and merge the three lists. If its minimal id is smaller then the changed doc, update the field realId, otherwise update the documents in the list with the changed id.
I suggest using different document to store { userId, realId } relation.
You can't create new documents by just using a view. You'd need a task of some sort to do the actual merging.
Here's one idea.
Instead of creating 3 views, you could create one view (that indexes the data if it exists):
Key Values
--- ------
[userId, 'phone'] 777-555-1212
[userId, 'email'] username#example.com
[userId, 'url'] favorite.url.example.com
I wouldn't store anything else except the raw value, as you'd end up with lots of unnecessary duplication of data (if you stored the full object for example).
Then, to query, you could do something like:
...startkey=[userId]&endkey=[userId,{}]
That would give you all of the duplicate information as a series of docs for that user Id. You'd still need to parse it apart to see if there were duplicates. But, this way, the results would be nicely merged into a single CouchDB call.
Here's a nice example of using arrays as keys on StackOverflow.
You'd still probably load the original "user" document if it had other data that wasn't part of the de-duplication process.
Once discovered, you could consider cleaning up the data on the fly and prevent new duplicates from occurring as new data is entered into your application.