I have a table that contains values saved as a dictionary.
FIELD_NAME: extra_data
VALUE:
{"code": null, "user_id": "103713616419757182414", "access_token": "ya29.IwBloLKFALsddhsAAADlliOoDeE-PD_--yz1i_BZvujw8ixGPh4zH-teMNgkIA", "expires": 3599}
I need to retrieve the user_id value from the field "extra_data" only not the dictionnary like below.
event_list = Event.objects.filter(season_id=season_id, event_status_id=2).value('extra_data')
If you are storing a dictionary as text in the code you can easily convert it to a python dictionary using eval - although I don't know why you'd want to as it opens you to all sorts of potential malicious code injections.
event_list = eval(Event.objects.filter(season_id=season_id, event_status_id=2).value('extra_data'))
user_id = event_list['user_id']
print user_id
Would give:
"103713616419757182414"
Edit:
On deeper inspection , thats not a Python dictionary, you could import a JSON library to import this, or declare what null is like so:
null = None
event_list = eval(Event.objects.filter(season_id=season_id, event_status_id=2).value('extra_data'))
user_id = event_list['user_id']
Either way, the idea of storing any structured data in a django textfield is fraught with danger that will come back to bite you. The best solution is to rethink your data structures.
This method worked for me. However, this works with a json compliant string
import json
json_obj = json.loads(event_list)
dict1 = dict(json_obj)
print dict1['user_id']
Related
I have a list of object of this kind of structure returned in my api
SomeCustomModel => {
itemId: "id",
relatedItem: "id",
data: {},
created_at: "data string"
}
I want to return a list that contains only unique relatedItemIds, filtered by the one that was created most recently.
I have written this and it seems to work
id_tracker = {}
query_set = SomeCustomModel.objects.all()
for item in query_set:
if item.relatedItem.id not in id_tracker:
id_tracker[item.relatedItem.id] = 1
else:
query_set = query_set.exclude(id=item.id)
return query_set
This works by I am wondering if there is cleaner way of writing this using only django aggregations.
I am using Mysql so the distinct("relatedItem") aggregation is not supported.
You should try to do this within sql. You can use Subquery to accomplish this. Here's the example from the django docs.
from django.db.models import OuterRef, Subquery
newest = Comment.objects.filter(post=OuterRef('pk')).order_by('-created_at')
Post.objects.annotate(newest_commenter_email=Subquery(newest.values('email')[:1]))
Unfortunately, I haven't found anything that can replace distict() in a django-esque manner. However, you could do something along the lines of:
list(set(map(lambda x: x.['relatedItem_id'], query_set.order_by('created_at').values('relatedItem_id'))))
or
list(set(map(lambda x: x.relatedItem_id, query_set.order_by('created_at'))))
which are a bit more Pythonic.
However, you are saying that you want to return a list yet your function returns a queryset. Which is the valid one?
I can add data to my db this way:
a = Model_tbl_name("3", "da", "3", "eheeee", "", "", "", "", "", "", "", "", func.now(), func.now())
db_session.add(a)
db_session.commit()
But i can't do it this way:
data = Model_tbl_name.insert().values({"title_hr":request.form['title_hr'],"text_hr":request.form['text_hr']})
I tried similar, but no help:
data = db_session.Model_tbl_name.insert().execute({"title_hr":request.form['title_hr'],"text_hr":request.form['text_hr']})
My initial motivation is to pass all form data like JSON, i would like to have it like this to work:
data = db_session.Model_tbl_name.insert().execute(json.loads(new_request_form))
In documentation, it is stated it can be done:
http://docs.sqlalchemy.org/en/rel_0_9/core/dml.html?highlight=insert%20values#sqlalchemy.sql.expression.Insert.values
like this:
users.insert().values({"name": "some name"})
But no help, I just can't get it. How can i make it to work, must i provide all JSON data in values() method? How should I write that command to make it work?
Second, how can I grab that error, because, I get no error in Flask, only stops working. I can figure out how to display errors when working with SQLAlchemy declarative way.
P.S. I am using Flask framework, SQLAlchemy, and Python version is 3.4
Well, I don't think you can do db_session.Class_name.
This is what you can do with it: http://docs.sqlalchemy.org/en/rel_1_0/orm/session_basics.html
If your problem is the default values, you can define relevant default values in the Model, and then you can construct an object with less fields.
a = Model_tbl_name(title_hr=request.form['title_hr'],
text_hr=request.form['text_hr'])
If you still want to pass the dictionary, you can do the following:
dataDict = json.loads(new_request_form)
a = Model_tbl_name(**dataDict)
db_session.add(a)
db_session.commit()
IMHO - the first method is more readable and reliable (you can check for validity of the params - that they match the object) -> at the end of the day, more secure...
if you need the insert way (let's say because you want to pass .prefix_with('IGNORE'))
you can do it like this
session.execute(Model_tbl_name.__table__.insert().values(
col1=val1, col2=val2).prefix_with('IGNORE'))
it works good with Connection class:
import sqlalchemy as sa
conn.execute(
sa.insert(Model_tbl_name).values(
{
Model_tbl_name.field1: 'value1',
Model_tbl_name.field2: 'value2',
Model_tbl_name.field3: 'value3',
}
)
)
I have a json field in my database which is like
jsonfield = {'username':'chingo','reputation':'5'}
how can i write a query so that i can find if a user name exists. something like
username = 'chingo'
query = User.objects.get(jsonfield['username']=username)
I know the above query is a wrong but I wanted to know if there is a way to access it?
If you are using the django-jsonfield package, then this is simple. Say you have a model like this:
from jsonfield import JSONField
class User(models.Model):
jsonfield = JSONField()
Then to search for records with a specific username, you can just do this:
User.objects.get(jsonfield__contains={'username':username})
Since Django 1.9, you have been able to use PostgreSQL's native JSONField. This makes search JSON very simple. In your example, this query would work:
User.objects.get(jsonfield__username='chingo')
If you have an older version of Django, or you are using the Django JSONField library for compatibility with MySQL or something similar, you can still perform your query.
In the latter situation, jsonfield will be stored as a text field and mapped to a dict when brought into Django. In the database, your data will be stored like this
{"username":"chingo","reputation":"5"}
Therefore, you can simply search the text. Your query in this siutation would be:
User.objects.get(jsonfield__contains='"username":"chingo"')
2019: As #freethebees points out it's now as simple as:
User.objects.get(jsonfield__username='chingo')
But as the doc examples mention you can query deeply, and if the json is an array you can use an integer to index it:
https://docs.djangoproject.com/en/2.2/ref/contrib/postgres/fields/#querying-jsonfield
>>> Dog.objects.create(name='Rufus', data={
... 'breed': 'labrador',
... 'owner': {
... 'name': 'Bob',
... 'other_pets': [{
... 'name': 'Fishy',
... }],
... },
... })
>>> Dog.objects.create(name='Meg', data={'breed': 'collie', 'owner': None})
>>> Dog.objects.filter(data__breed='collie')
<QuerySet [<Dog: Meg>]>
>>> Dog.objects.filter(data__owner__name='Bob')
<QuerySet [<Dog: Rufus>]>
>>> Dog.objects.filter(data__owner__other_pets__0__name='Fishy')
<QuerySet [<Dog: Rufus>]>
Although this is for postgres, I believe it works the same in other DBs like MySQL
Postgres: https://docs.djangoproject.com/en/2.2/ref/contrib/postgres/fields/#querying-jsonfield
MySQL: https://django-mysql.readthedocs.io/en/latest/model_fields/json_field.html#querying-jsonfield
This usage is somewhat anti-pattern. Also, its implementation is not going to have regular performance, and perhaps is error-prone.
Normally don't use jsonfield when you need to look up through fields. Use the way the RDBMS provides or MongoDB(which internally operates on faster BSON), as Daniel pointed out.
Due to the deterministic of JSON format,
you could achieve it by using contains (regex has issue when dealing w/ multiple '\' and even slower), I don't think it's good to use username in this way, so use name instead:
def make_cond(name, value):
from django.utils import simplejson
cond = simplejson.dumps({name:value})[1:-1] # remove '{' and '}'
return ' ' + cond # avoid '\"'
User.objects.get(jsonfield__contains=make_cond(name, value))
It works as long as
the jsonfield using the same dump utility (the simplejson here)
name and value are not too special (I don't know any egde-case so far, maybe someone could point it out)
your jsonfield data is not corrupt (unlikely though)
Actually I'm working on a editable jsonfield and thinking about whether to support such operations. The negative proof is as said above, it feels like some black-magic, well.
If you use PostgreSQL you can use raw sql to solve problem.
username = 'chingo'
SQL_QUERY = "SELECT true FROM you_table WHERE jsonfield::json->>'username' = '%s'"
User.objects.extra(where=[SQL_EXCLUDE % username]).get()
where you_table is name of table in your database.
Any methods when you work with JSON like with plain text - looking like very bad way.
So, also I think that you need a better schema of database.
Here is the way I have found out that will solve your problem:
search_filter = '"username":{0}'.format(username)
query = User.objects.get(jsonfield__contains=search_filter)
Hope this helps.
You can't do that. Use normal database fields for structured data, not JSON blobs.
If you need to search on JSON data, consider using a noSQL database like MongoDB.
I am trying to output a set of database records in JSON as follows:
def json_dbtable(request, p):
t = MyModel.objects.filter({some query})
s = serializers.get_serializer("json")()
re = s.serialize(t, ensure_ascii=False)
return HttpResponse(re, mimetype="application/json")
However, one of the fields i'm trying to return needs to change if it is null, and to remedy this the model has a definition that is used as a property .e.g:
name = property(_get_useful_name)
So, to get to the crux of the question. How can I include this "name" property in my json serialization as well as the raw field data?
The short answer is no, the long answer, is you could serialize your MyModel instance yourself:
simplejson.dumps([{'pk': m.pk, 'name': m.name} for m in MyModel.objects.filter(...)])
I have written a serialization framework for Python called any2any
which include (de)serializers for Django
and which allows you to do that easily.
It will be way cleaner than the DIY way.
Hope that helps !
I want to serialize a QuerySet that contains an extra statement:
region_list = Region.objects.extra(select={ 'selected': 'case when id = %s then 1 else 0 end' % (new_region.id)}).all()
I use the statement below to serialize
return HttpResponse(serializers.serialize('json', region_list), mimetype='application/json')
But when I obtain the json results in the browser, only the fields of the Region model appears, the selected field dissapear.
How can I fix that?
One slightly longwinded solution would be to to dump the objects to JSON via django-piston's JSONEmitter class. When you register your Region model with piston, you can say what fields to include, and mention 'selected' there, and then use your annotation to make sure that the queryset used in the piston handler contains all the info you want.
Or just look at how piston does it and, if you don't want all of piston, just mimic the bits you do.