I have a django model with three fields and I'm trying to find the duplicates. If I run:
cls.objects.values('institution','person','title').annotate(records=Count('person')).filter(records__gt=1).count() I get 152 as the output.
However, if I attempt to see what those records are and run the same query without the count() cls.objects.values('institution','person','title').annotate(records=Count('person')).filter(records__gt=1)
I get <QuerySet []>.
Any idea what's going on? If I add a .first() I get null, and a [0] gives me an out of range error, however the count continues to return 152. Running SQL directly on the database shows there are in fact 152 matching entries so the count is correct, but I can't seem to get the queryset (or any elements in it) to return.
The startdate injection into the GROUP BY mentioned by #JoVi in the comment turned out to be the issue here. It was being added due to it being a default sort in the django model definition. Adding an empty .order_by() to the query removed that and resulted in the query working as expected.
I'm trying to add a scripted field to my query from django-elasticsearch-dsl
I've tried to add a test field like this:
result = ModelDocument.search().script_fields(test="doc['field_name'].value + 10")
for hit in result:
print(hit.to_dict())
but it returns 10 empty dictionary
and if I try it without adding "script_fields":
result = ModelDocument.search()
for hit in result:
print(hit.to_dict())
it returns 10 dictionary filled with my model instance data.
can you please tell me what is the problem and how can i fix it?
I have a app that lets the user search a database of +/- 100,000 documents for keywords / sentences.
I am using Django 1.11 and the Postgres FullTextSearch features described in the documentation
However, I am running into the following problem and I was wondering if someone knows a solution:
I want to create a SearchQuery object for each word in the supplied queryset like so:
query typed in by the user in the input field: ['term1' , 'term2', 'term3']
query = SearchQuery('term1') | SearchQuery('term2') | SearchQuery('term3')
vector = SearchVector('text')
Document.objects.annotate(rank=SearchRank(vector, query)).order_by('-rank').annotate(similarity=TrigramSimilarity(vector, query).filter(simularity__gt=0.3).order_by('-simularity')
The problem is that I used 3 terms for my query in the example, but I want that number to be dynamic. A user could also supply 1, or 10 terms, but I do not know how to add the relevant code to the query assignment.
I briefly thought about having the program write something like this to an empty document:
for query in terms:
file.write(' | (SearchQuery( %s )' % query ))
But having a python program writing python code seems like a very convoluted solution. Does anyone know a better way to achieve this?
Ive never used it, but to do a dynamic query you can just loop and add.
compound_statement = SearchQuery(list_of_words[0])
for term in list_of_words[1:]:
compound_statement = compound_statement | SearchQuery(term)
But the documentation tells us that
By default, all the words the user provides are passed through the stemming algorithms, and then it looks for matches for all of the resulting terms.
are you sure you need this?
I have a Django project powered by a MySQL database. I fed the database a CSV file (via raw SQL statements) with some 600 records. That went smoothly (almost, there was one error of Field 'customer_id' doesn't have a default value - no idea why). The problem is that these records are not showing up in the webapp itself.
For example, the CSV file contained a list of about 600 records describing client contact info. When I fire up the Django test server and go to the client contact page (which should list all the contact records -- the 600) nothing is there. Further, when I go into the Admin section and view the client contact records, there is nothing there -- However it shows that there is a total of 600 records right beside the pagination buttons. Also, there are 7 pages (7 pagination buttons) of records -- of course, there is nothing on any page.
What the heck?!
EDIT for More Details
The file I am trying to import is called subset_ressy_esc.csv and a couple of lines of it look like this:
R0138,Y,1432 MyRoad Ct,MyCity, MyProv,H0H 0H0,N,100.00,0,1,1
R0140,Y,268 MyStreet Link,MyCity,MyProv,H0H 0H0,N,100.00,0,1,1
R0142,Y,10994 123 St,MyCity,MyProv,H0H 0H0,N,0.00,1,0,0
And the fields they represent (in the same order shown above) are:
systemID (pk), isRessy, systemAddress, systemCity, systemProvince, systemPostalCode, isHoseBibb, servicePreauthAmt, noWorkRequired, SUAuthorized, BDAuthorized
Now the isRessy and isHoseBibb fields are select style fields, where the user selects from a drop down, while the noWorkReqd, SUAth, and BDAuth are booleans. The serviceAmt is a decimal (dollar) amt.
To import this data, I go into the mySql interpreter mysql -u garfonzo -p and run the following command:
mysql> load data local infile '/home/garfonzo/subset_ressy_esc.csv' into table systems_system fields terminated by ',' lines terminated by '\n' (systemID, isRessy, systemAddress, systemCity, systemProvince, systemPostalCode, isHoseBibb, servicePreauthAmt, noWorkRequired, SUAuthorized, BDAuthorized);
Query OK, 684 rows affected, 1 warning (0.01 sec)
Records: 684 Deleted: 0 Skipped: 0 Warnings: 0
mysql> show warnings;
+---------+------+-----------------------------------------------------+
| Level | Code | Message |
+---------+------+-----------------------------------------------------+
| Warning | 1364 | Field 'systemOwner_id' doesn't have a default value |
+---------+------+-----------------------------------------------------+
When I do a SELECT * FROM systems_system I spits out all 684 records, with all the right data. When I open the django project, nothing is listed but the admin has 7 pages of systems, without a single system actually present -- just 7 pages of nothing.
Any ideas?
Interesting. The fact that the admin shows the correct record count means that the table names match (that was my first thought). So it sees the app, the table, and the record count. My guess is one of two things:
You did not define a __str__() or __unicode__() method for the class, or
You did not set up the matching admin.py classes correctly and you are display a blank field.
My money is on number one.
Are you sure that you have actually saved those records via .save() method?
What does console show? I mean have you tried to manually iterate in shell? (manage.py shell)
And lastly have you tried to restart the server?
Admin should not require __str__ nor __unicode__.
Looking at the warning of Field systemOwner_id doesn't have a default value didn't really bother me. But it suddenly dawned on my that I am not importing a value for that field, and that field cannot be empty. So, (for testing purposes) I changed my model so that that field systemOwner_id can be null, ran the same MySQL import and, voila, all systems are present. Nice!!
Thanks for the help -- working through this on this Q&A has helped solve my problem.
Cheers!
I'm trying to order a list of items in django by the number of comments they have. However, there seems to be an issue in that the Count function doesn't take into account the fact that django comments also uses a content_type_id to discern between comments for different objects!
This gives me a slight problem in that the comment counts for all objects are wrong using the standard methods; is there a 'nice' fix or do I need to drop back to raw sql?
Code to try and ge the correct ordering:
app_list = App.objects.filter(published=True)
.annotate(num_comments=Count('comments'))
.order_by('-num_comments')
Sample output from the query (note no mention of the content type id):
SELECT "apps_app"."id", "apps_app"."name",
"apps_app"."description","apps_app"."author_name", "apps_app"."site_url",
"apps_app"."source_url", "apps_app"."date_added", "apps_app"."date_modified",
"apps_app"."published", "apps_app"."published_email_sent", "apps_app"."created_by_id",
"apps_app"."rating_votes", "apps_app"."rating_score", COUNT("django_comments"."id") AS
"num_comments" FROM "apps_app" LEFT OUTER JOIN "django_comments" ON ("apps_app"."id" =
"django_comments"."object_pk") WHERE "apps_app"."published" = 1 GROUP BY
"apps_app"."id", "apps_app"."name", "apps_app"."description", "apps_app"."author_name",
"apps_app"."site_url", "apps_app"."source_url", "apps_app"."date_added",
"apps_app"."date_modified", "apps_app"."published", "apps_app"."published_email_sent",
"apps_app"."created_by_id", "apps_app"."rating_votes", "apps_app"."rating_score" ORDER
BY num_comments DESC LIMIT 4
Think I found the answer: Django Snippet