jsonpath: mix two paths - facebook-graph-api

I am querying facebook for fullsize profilepictures of users that commented on posted pictures or who like posted pictures by the /me user, using the graph api.
I am trying to either:
do it in one query, where i am having trouble getting data from deep enough.
or minimize the amount of subqueries
In one query: this does not work (it does not give details of the users that commented):
me/photos/uploaded?fields=from,images,comments.picture,likes.picture&limit=500
and specifying the dimensions throws an error:
me/photos/uploaded?fields=from,images,comments.picture,likes.picture.type(large)&limit=500
There seems to be a limit as to how deep you can specify, though I havent found anything explicit in the documentation. (which is so terse that is hardly a big surprise)
so, resolving to batched queries with a dependency
myphotos => me/photos/uploaded?fields=created_time,name,id,from,comments.summary(1).limit(100),images,likes.summary(1).limit(100)&limit=500
userinfo_photocomments => ?ids={result=myphotos:$.data.*.comments.data.*.from.id}&fields=id,name,updated_time,picture.redirect(0).type(large).height(2000).width(2000)
userinfo_photolikes => ?ids={result=myphotos:$.data.*.likes.data.*.id}&fields=id,name,updated_time,picture.redirect(0).type(large).height(2000).width(2000)
though this works, the fact that i cant find how to do it in one or max 2 queries annoys me, especially as the results of the last 2 subqueries will most likely overlap.
I would like to find a way to mix the jsonpath anchors, such that i can combine the last two queries.
The problem seems to be that the userid's of the commentors and the likers are on different depths in the trees.
I would be gratefull for any insights.

I think you should be able to use the result from the first query twice, resulting in only two queries.
I tested the ?ids= endpoint if it filters out duplicate user_id like the following:
GET /?ids={user_id1},{user_id2},{user_id1}
which results in only getting two distinct user_ids back.
So, this yields in the following batch query:
curl \
-F 'access_token={access_token}' \
-F 'batch=[{ "method":"GET","name":"myphotos","relative_url":"me/photos/uploaded?fields=created_time,name,id,from,comments.summary(1).limit(100),images,likes.summary(1).limit(100)&limit=500",},{"method":"GET","relative_url":"?ids={result=myphotos:$.data.*.comments.data.*.from.id},{result=myphotos:$.data.*.likes.data.*.id}&fields=id,name,updated_time,picture.redirect(0).type(large).height(2000).width(2000)"}]' \
https://graph.facebook.com/
Just replace {access_token} with an actual access token to test this.
Basically, I just concatenated the user_id lists for comments and likes as
GET ?ids={result=myphotos:$.data.*.comments.data.*.from.id},{result=myphotos:$.data.*.likes.data.*.id}&fields=...

Related

How can I dynamically create multi-level hierarchical forms in Django?

I'm building an advanced search page for a scientific database using Django. The goal is to be able to allow some dynamically created sophisticated searches, joining groups of search terms with and & or.
I got part-way there by following this example, which allows multiple terms anded together. It's basically a single group of search terms, dynamically created, that I can either and-together or or-together. E.g.
<field1|field2> <is|is not|contains|doesn't contain> <search term> <->
<+>
...where <-> will remove a search term row and <+> will add a new row.
But I would like the user to be able to either add another search term row, or add an and-group and an or-group, so that I'd have something like:
<and-group|or-group> <->
<field1|field2> <is|is not|contains|doesn't contain> <search term> <->
<+term|+and-group|_or-group>
A user could then add terms or groups. The result search might end up like:
and-group
compound is lysine
or-group
tissue is brain
tissue is spleen
feeding status is not fasted
Thus the resulting filter would be like the following.
Data.objects.filter(Q(compound="lysine") & (Q(tissue=brain) | Q(tissue="spleen")) & ~Q(feeding_status="fasted"))
Note - I'm not necessarily asking how to get the filter expression below correct - it's just the dynamic hierarchical construction component that I'm trying to figure out. Please excuse me if I got the Q and/or filter syntax wrong. I've made these queries before, but I'm still new to Django, so getting it right off the top of my head here is pretty much guaranteed to be zero-chance. I also skipped the model relationships I spanned here, so let's assume these are all fields in the same model, for simplicity.
I'm not sure how I would dynamically add parentheses to the filter expression, but my current code could easily join individual Q expressions with and or or.
I'm also not sure how I could dynamically create a hierarchal form to create the sub-groups. I'm guessing any such solution would have to be a hack and that there are not established mechanisms for doing something like this...
Here's a screenshot example of what I've currently got working:
UPDATE:
I got really far following this example I found. I forked that fiddle and got this proof of concept working before incorporating it into my Django project:
http://jsfiddle.net/hepcat72/d42k38j1/18/
The console spits out exactly the object I want. And there are no errors. Clicking the search button works for form validation. Any fields I leave empty causes a prompt to fill in the field. Here's a demo gif:
Now I need to process the POST input to construct the query (which I think I can handle) and restore the form above the results - which I'm not quite sure how to accomplish - perhaps a recursive function in a custom tag?
Although, is there a way to snapshot the form and restore it when the results load below it? Or maybe have the results load in a different frame?
I don't know if I'm teaching a grandmother to suck eggs, but in case not, one of the features of the Python language may be useful.
foo( bar = 27, baz = None)
can instead be coded
args = {}
a1, a2 = 'bar', 'baz'
d[a1] = 27
d[a2] = None
foo( **args )
so an arbitrary Q object specified by runtime keys and values can be constructed q1 = Q(**args)
IIRC q1 & q2 and q1 | q2 are themselves Q objects so you can build up a filter of arbitrary complexity.
I'll also include a mention of Django-filter which is usually my answer to filtering questions like this one, but I suspect in this case you are after greater power than it easily provides. Basically, it will "and" together a list of filter conditions specified by the user. The built-in ones are simple .filter( key=value), but by adding code you can create custom filters with complex Q expressions related to a user-supplied value.
As for the forms, a Django form is a linear construct, and a formset is a list of similar forms. I think I might resort to JavaScript to build some sort of tree representing a complex query in the browser, and have the submit button encode it as JSON and return it through a single text field (or just pick it out of request.POST without using a form). There may be some Javascript out there already written to do this, but I'm not aware of it. You'd need to be sure that malicious submission of field names and values you weren't expecting doesn't result in security issues. For a pure filtering operation, this basically amounts to being sure that the user is entitled to get all data in database table in any case.
There's a form JSONField in the Django PostgreSQL extensions, which validates that user-supplied (or Javascript-generated) text is indeed JSON, and supplies it to you as Python dicts and lists.

What's wrong with django queryset? I get different answer when access the same indice

I found that the objects could be duplicate in a queryset. However, when I try to access each of the object and do nothing, it changes and seems to be right.
Here are the commands I have typed into the shell
At first I gained a queryset orderby the field 'receiveTime'. Then it seems that ds[1996] equals to ds[1997]. And I try to use the loop:
for d in ds:
pass
Then the ds[1996] isn't equal to ds[1997], but what have I done?
Maybe it is a feature of the lazy search?
plus 1:I have reproduced it just now. I didn't do any inserting or deleting just now.
These are the commands I just typed into the shell.
plus 2:I have seen the raw sql queries when I call the ds[0] and ds[1] which I have shown in the picture 2. The sql queries are correct but the answer seems to be wrong. I think maybe the reason is that the sorting parameter receiveTime of two objects are the same, which lead to the disorder of the objects?
Here are the raw sql queries
Replace order_by("receive_time") with order_by("receive_time", "id"). PostgreSQL uses qsort which is an unstable sort. Given only receive_time, if values are the same, the order is not guaranteed.
Don't post code or logs in images. Ever.

How to build query form to request AWS CloudSearch?

I have a SearchDomain on AWS CloudSearch. I know all the defined facets names.
I would like to build a web query form to use it, but I want to add my categories values (facets) on the side, like it is done on Amazon webstore
The only way I have to get facets values is to make a query (params query) and in the answer will contain facets linked to my query results.
Is there a way to fetch all the facet.FIELD possible values to build the query form ?
If not (as read here), how to design a form using facets ?
You could also use the matchall keyword in a structured query. Also, since you don't need the results you can pass size=0 so you only get the facets which will reduce latency.
/2013-01-01/search?q=matchall&q.parser=structured&size=0&facet.field={sort:'count',size:100}

Musicbrainz querying artist and release

I am trying to get an artist and their albums. So reading this page https://musicbrainz.org/doc/Development/XML_Web_Service/Version_2 i created the following query to get Michael Jackson's albums
http://musicbrainz.org/ws/2/artist/?query=artist:michael%20jackson?inc=releases+recordings
My understanding is to add ?inc=releases+recordings at the end of the URL which should return Michael Jackson's albums however this doesnt seem to return the correct results or i cant seem to narrow down the results? I then thought to use the {MBID} but again thats not returned in the artists query (which is why im trying to use inc in my query)
http://musicbrainz.org/ws/2/artist/?query=artist:michael%20jackson
Can anyone suggest where im going wrong with this?
You're not searching for the correct Entity. What you want is to get the discography, not artist's infos. Additionally, query fields syntax is not correct (you must use Lucene Search Syntax).
Here is what you're looking for:
http://musicbrainz.org/ws/2/release-group/?query=artist:"michael jackson" AND primarytype:"album"
We're targeting the release-group entity to get the albums, searching for a specific artist and filtering the results to limit them to albums. (accepted values are: album, single, ep, other)
There are more options to fit your needs, for example you can filter the type of albums using the secondarytype parameter. Here is the query to retrieve only live albums:
http://musicbrainz.org/ws/2/release-group/?query=artist:"michael jackson" AND primarytype:"album" AND secondarytype="live"
Here is the doc:
https://musicbrainz.org/doc/Development/XML_Web_Service/Version_2/Search
Note that to be able to use MB's API you need to understand how it is structured, especially, the relations between release_group, release and medium.

What is the best way to use query with a list and keep the list order? [duplicate]

This question already has answers here:
Django: __in query lookup doesn't maintain the order in queryset
(6 answers)
Closed 8 years ago.
I've searched online and could only find one blog that seemed like a hackish attempt to keep the order of a query list. I was hoping to query using the ORM with a list of strings, but doing it that way does not keep the order of the list.
From what I understand bulk_query only works if you have the id's of the items you want to query.
Can anybody recommend an ideal way of querying by a list of strings and making sure the objects are kept in their proper order?
So in a perfect world I would be able to query a set of objects by doing something like this...
Entry.objects.filter(id__in=['list', 'of', 'strings'])
However, they do not keep order, so string could be before list etc...
The only work around I see, and I may just be tired or this may be perfectly acceptable I'm not sure is doing this...
for i in listOfStrings:
object = Object.objects.get(title=str(i))
myIterableCorrectOrderedList.append(object)
Thank you,
The problem with your solution is that it does a separate database query for each item.
This answer gives the right solution if you're using ids: use in_bulk to create a map between ids and items, and then reorder them as you wish.
If you're not using ids, you can just create the mapping yourself:
values = ['list', 'of', 'strings']
# one database query
entries = Entry.objects.filter(field__in=values)
# one trip through the list to create the mapping
entry_map = {entry.field: entry for entry in entries}
# one more trip through the list to build the ordered entries
ordered_entries = [entry_map[value] for value in values]
(You could save yourself a line by using index, as in this example, but since index is O(n) the performance will not be good for long lists.)
Remember that ultimately this is all done to a database; these operations get translated down to SQL somewhere.
Your Django query loosely translated into SQL would be something like:
SELECT * FROM entry_table e WHERE e.title IN ("list", "of", "strings");
So, in a way, your question is equivalent to asking how to ORDER BY the order something was specified in a WHERE clause. (Needless to say, I hope, this is a confusing request to write in SQL -- NOT the way it was designed to be used.)
You can do this in a couple of ways, as documented in some other answers on StackOverflow [1] [2]. However, as you can see, both rely on adding (temporary) information to the database in order to sort the selection.
Really, this should suggest the correct answer: the information you are sorting on should be in your database. Or, back in high-level Django-land, it should be in your models. Consider revising your models to save a timestamp or an ordering when the user adds favorites, if that's what you want to preserve.
Otherwise, you're stuck with one of the solutions that either grabs the unordered data from the db then "fixes" it in Python, or constructing your own SQL query and implementing your own ugly hack from one of the solutions I linked (don't do this).
tl;dr The "right" answer is to keep the sort order in the database; the "quick fix" is to massage the unsorted data from the database to your liking in Python.
EDIT: Apparently MySQL has some weird feature that will let you do this, if that happens to be your backend.