I can add data to my db this way:
a = Model_tbl_name("3", "da", "3", "eheeee", "", "", "", "", "", "", "", "", func.now(), func.now())
db_session.add(a)
db_session.commit()
But i can't do it this way:
data = Model_tbl_name.insert().values({"title_hr":request.form['title_hr'],"text_hr":request.form['text_hr']})
I tried similar, but no help:
data = db_session.Model_tbl_name.insert().execute({"title_hr":request.form['title_hr'],"text_hr":request.form['text_hr']})
My initial motivation is to pass all form data like JSON, i would like to have it like this to work:
data = db_session.Model_tbl_name.insert().execute(json.loads(new_request_form))
In documentation, it is stated it can be done:
http://docs.sqlalchemy.org/en/rel_0_9/core/dml.html?highlight=insert%20values#sqlalchemy.sql.expression.Insert.values
like this:
users.insert().values({"name": "some name"})
But no help, I just can't get it. How can i make it to work, must i provide all JSON data in values() method? How should I write that command to make it work?
Second, how can I grab that error, because, I get no error in Flask, only stops working. I can figure out how to display errors when working with SQLAlchemy declarative way.
P.S. I am using Flask framework, SQLAlchemy, and Python version is 3.4
Well, I don't think you can do db_session.Class_name.
This is what you can do with it: http://docs.sqlalchemy.org/en/rel_1_0/orm/session_basics.html
If your problem is the default values, you can define relevant default values in the Model, and then you can construct an object with less fields.
a = Model_tbl_name(title_hr=request.form['title_hr'],
text_hr=request.form['text_hr'])
If you still want to pass the dictionary, you can do the following:
dataDict = json.loads(new_request_form)
a = Model_tbl_name(**dataDict)
db_session.add(a)
db_session.commit()
IMHO - the first method is more readable and reliable (you can check for validity of the params - that they match the object) -> at the end of the day, more secure...
if you need the insert way (let's say because you want to pass .prefix_with('IGNORE'))
you can do it like this
session.execute(Model_tbl_name.__table__.insert().values(
col1=val1, col2=val2).prefix_with('IGNORE'))
it works good with Connection class:
import sqlalchemy as sa
conn.execute(
sa.insert(Model_tbl_name).values(
{
Model_tbl_name.field1: 'value1',
Model_tbl_name.field2: 'value2',
Model_tbl_name.field3: 'value3',
}
)
)
Related
I am experimenting using elasticsearch in a dummy project in django. I am attempting to make a search page using django-elasticsearch-dsl. The user may provide a title, summary and a score to search for. The search should match all the information given by the user, but if the user does not provide any info about something, this should be skipped.
I am running the following code to search for all the values.
client = Elasticsearch()
s = Search().using(client).query("match", title=title_value)\
.query("match", summary=summary_value)\
.filter('range', score={'gt': scorefrom_value, 'lte': scoreto_value})
When I have a value for all the fields then the search works correctly, but if for example I do not provide a value for the summary_value, although I am expecting the search to continue searching for the rest of the values, the result is that it comes up with nothing as a result.
Is there some value that the fields should have by default in case the user does not provide a value? Or how should I approach this?
UPDATE 1
I tried using the following, but it returns every time no matter the input i am giving the same results.
s = Search(using=client)
if title:
s.query("match", title=title)
if summary:
s.query("match", summary=summary)
response = s.execute()
UPDATE 2
I can print using the to_dict().
if it is like the following then s is empty
s = Search(using=client)
s.query("match", title=title)
if it is like this
s = Search(using=client).query("match", title=title)
then it works properly but still if i add s.query("match", summary=summary) it does nothing.
You need to assign back into s:
if title:
s = s.query("match", title=title)
if summary:
s = s.query("match", summary=summary)
I can see in the Search example that django-elasticsearch-dsl lets you apply aggregations after a search so...
How about "staging" your search? I can think if the following:
#first, declare the Search object
s = Search(using=client, index="my-index")
#if parameter1 exists
if parameter1:
s.filter("term", field1= parameter1)
#if parameter2 exists
if parameter2:
s.query("match", field=parameter2)
Do the same for all your parameters (with the needed method for each) so only the ones that exist will appear in your query. At the end just run
response = s.execute()
and everything should work as you want :D
I would recommend you to use the Python ES Client. It lets you manage multiple things related to your cluster: set mappings, health checks, do queries, etc.
In its method .search(), the body parameter is where you send your query as you normally would run it ({"query"...}). Check the Usage example.
Now, for your particular case, you can have a template of your query stored in a variable. You first start with, let's say, an "empty query" only with filter, just like:
query = {
"query":{
"bool":{
"filter":[
]
}
}
}
From here, you now can build your query from the parameters you have.
This is:
#This would look a little messy, but it's useful ;)
#if parameter1 is not None or emtpy
#(change the if statement for your particular case)
if parameter1:
query["query"]["bool"]["filter"].append({"term": {"field1": parameter1}})
Do the same for all your parameters (for strings, use "term", for ranges use "range" as usual) and send the query in the .search()'s body parameter and it should work as you want.
Hope this is helpful! :D
I have a list of object of this kind of structure returned in my api
SomeCustomModel => {
itemId: "id",
relatedItem: "id",
data: {},
created_at: "data string"
}
I want to return a list that contains only unique relatedItemIds, filtered by the one that was created most recently.
I have written this and it seems to work
id_tracker = {}
query_set = SomeCustomModel.objects.all()
for item in query_set:
if item.relatedItem.id not in id_tracker:
id_tracker[item.relatedItem.id] = 1
else:
query_set = query_set.exclude(id=item.id)
return query_set
This works by I am wondering if there is cleaner way of writing this using only django aggregations.
I am using Mysql so the distinct("relatedItem") aggregation is not supported.
You should try to do this within sql. You can use Subquery to accomplish this. Here's the example from the django docs.
from django.db.models import OuterRef, Subquery
newest = Comment.objects.filter(post=OuterRef('pk')).order_by('-created_at')
Post.objects.annotate(newest_commenter_email=Subquery(newest.values('email')[:1]))
Unfortunately, I haven't found anything that can replace distict() in a django-esque manner. However, you could do something along the lines of:
list(set(map(lambda x: x.['relatedItem_id'], query_set.order_by('created_at').values('relatedItem_id'))))
or
list(set(map(lambda x: x.relatedItem_id, query_set.order_by('created_at'))))
which are a bit more Pythonic.
However, you are saying that you want to return a list yet your function returns a queryset. Which is the valid one?
I'm getting an unexpected result using icontains in my get_or_create call.
Take the following example:
>>>team_name = "Bears"
>>>Team.objects.get(name__icontains=team_name) # returns DoesNotExist as expected
>>>team, created = Team.objects.get_or_create(name__icontains=team_name)
>>>print(created) # Prints True as expected
>>>print(team.name) # Prints an empty string!
Why does this create a team with a blank name rather than "Bears"? The reason I'm using get_or_create here is that if a subsequent user posts something like "BearS" I want to get the correct team, not create a duplicate team with incorrect capitalization.
I think here you should split the get() and create() functionalities instead of using get_or_create(), because the __icontains lookup works for get() only.
Try doing something like this:
>>> team_name = 'Bears'
>>> teams = Team.objects.filter(name__icontains=team_name)
# This will filter the teams with this name
>>> team = teams.first() if teams.exists() else Team.objects.create(name=team_name)
# Now your team is the first element of your previous query (it returns a QuerySet with single element) if it exists
# Otherwise, you create a new Team.
Another option besides wencakisa's answer is to include the defaults parameter in get_or_create, because Django strips lookups containing the __ separator. See answers to this question.
The code would be:
Team.objects.get_or_create(
name__icontains=team_name,
defaults = {
"name": team_name
}
)
The right way to do it is using Django's function get_or_create(). But instead of "icontains", you should use "iexact" (), unless you want an exact match, in wich case you should use just "exact":
Team.objects.get_or_create(
name__iexact=team_name,
defaults = {
"name": team_name
}
)
Outside "defaults" you should put your search terms. If the objects doesn't exist, you should write your creation terms inside 'defaults'
I have a table that contains values saved as a dictionary.
FIELD_NAME: extra_data
VALUE:
{"code": null, "user_id": "103713616419757182414", "access_token": "ya29.IwBloLKFALsddhsAAADlliOoDeE-PD_--yz1i_BZvujw8ixGPh4zH-teMNgkIA", "expires": 3599}
I need to retrieve the user_id value from the field "extra_data" only not the dictionnary like below.
event_list = Event.objects.filter(season_id=season_id, event_status_id=2).value('extra_data')
If you are storing a dictionary as text in the code you can easily convert it to a python dictionary using eval - although I don't know why you'd want to as it opens you to all sorts of potential malicious code injections.
event_list = eval(Event.objects.filter(season_id=season_id, event_status_id=2).value('extra_data'))
user_id = event_list['user_id']
print user_id
Would give:
"103713616419757182414"
Edit:
On deeper inspection , thats not a Python dictionary, you could import a JSON library to import this, or declare what null is like so:
null = None
event_list = eval(Event.objects.filter(season_id=season_id, event_status_id=2).value('extra_data'))
user_id = event_list['user_id']
Either way, the idea of storing any structured data in a django textfield is fraught with danger that will come back to bite you. The best solution is to rethink your data structures.
This method worked for me. However, this works with a json compliant string
import json
json_obj = json.loads(event_list)
dict1 = dict(json_obj)
print dict1['user_id']
I have a json field in my database which is like
jsonfield = {'username':'chingo','reputation':'5'}
how can i write a query so that i can find if a user name exists. something like
username = 'chingo'
query = User.objects.get(jsonfield['username']=username)
I know the above query is a wrong but I wanted to know if there is a way to access it?
If you are using the django-jsonfield package, then this is simple. Say you have a model like this:
from jsonfield import JSONField
class User(models.Model):
jsonfield = JSONField()
Then to search for records with a specific username, you can just do this:
User.objects.get(jsonfield__contains={'username':username})
Since Django 1.9, you have been able to use PostgreSQL's native JSONField. This makes search JSON very simple. In your example, this query would work:
User.objects.get(jsonfield__username='chingo')
If you have an older version of Django, or you are using the Django JSONField library for compatibility with MySQL or something similar, you can still perform your query.
In the latter situation, jsonfield will be stored as a text field and mapped to a dict when brought into Django. In the database, your data will be stored like this
{"username":"chingo","reputation":"5"}
Therefore, you can simply search the text. Your query in this siutation would be:
User.objects.get(jsonfield__contains='"username":"chingo"')
2019: As #freethebees points out it's now as simple as:
User.objects.get(jsonfield__username='chingo')
But as the doc examples mention you can query deeply, and if the json is an array you can use an integer to index it:
https://docs.djangoproject.com/en/2.2/ref/contrib/postgres/fields/#querying-jsonfield
>>> Dog.objects.create(name='Rufus', data={
... 'breed': 'labrador',
... 'owner': {
... 'name': 'Bob',
... 'other_pets': [{
... 'name': 'Fishy',
... }],
... },
... })
>>> Dog.objects.create(name='Meg', data={'breed': 'collie', 'owner': None})
>>> Dog.objects.filter(data__breed='collie')
<QuerySet [<Dog: Meg>]>
>>> Dog.objects.filter(data__owner__name='Bob')
<QuerySet [<Dog: Rufus>]>
>>> Dog.objects.filter(data__owner__other_pets__0__name='Fishy')
<QuerySet [<Dog: Rufus>]>
Although this is for postgres, I believe it works the same in other DBs like MySQL
Postgres: https://docs.djangoproject.com/en/2.2/ref/contrib/postgres/fields/#querying-jsonfield
MySQL: https://django-mysql.readthedocs.io/en/latest/model_fields/json_field.html#querying-jsonfield
This usage is somewhat anti-pattern. Also, its implementation is not going to have regular performance, and perhaps is error-prone.
Normally don't use jsonfield when you need to look up through fields. Use the way the RDBMS provides or MongoDB(which internally operates on faster BSON), as Daniel pointed out.
Due to the deterministic of JSON format,
you could achieve it by using contains (regex has issue when dealing w/ multiple '\' and even slower), I don't think it's good to use username in this way, so use name instead:
def make_cond(name, value):
from django.utils import simplejson
cond = simplejson.dumps({name:value})[1:-1] # remove '{' and '}'
return ' ' + cond # avoid '\"'
User.objects.get(jsonfield__contains=make_cond(name, value))
It works as long as
the jsonfield using the same dump utility (the simplejson here)
name and value are not too special (I don't know any egde-case so far, maybe someone could point it out)
your jsonfield data is not corrupt (unlikely though)
Actually I'm working on a editable jsonfield and thinking about whether to support such operations. The negative proof is as said above, it feels like some black-magic, well.
If you use PostgreSQL you can use raw sql to solve problem.
username = 'chingo'
SQL_QUERY = "SELECT true FROM you_table WHERE jsonfield::json->>'username' = '%s'"
User.objects.extra(where=[SQL_EXCLUDE % username]).get()
where you_table is name of table in your database.
Any methods when you work with JSON like with plain text - looking like very bad way.
So, also I think that you need a better schema of database.
Here is the way I have found out that will solve your problem:
search_filter = '"username":{0}'.format(username)
query = User.objects.get(jsonfield__contains=search_filter)
Hope this helps.
You can't do that. Use normal database fields for structured data, not JSON blobs.
If you need to search on JSON data, consider using a noSQL database like MongoDB.