Saving data in django model don't actually saves - django

How should I understand this? And how this can happens?
Go into django shell:
>>> from customauth.models import Profile
>>> p = Profile.objects.get(user_id=1)
>>> p.status
u'34566'
>>> p.status = 'qwerty'
>>> p.status
'qwerty'
>>> p.save()
>>> p.status
'qwerty'
>>> p = Profile.objects.get(user_id=1)
>>> p.status
u'qwerty'
>>>
Exit and go into django shell again:
>>> from customauth.models import Profile
>>> p = Profile.objects.get(user_id=1)
>>> p.status
u'qwerty'
>>>
Everything seems OK. But go into dbshell now:
mysql> select user_id, status from customauth_profile where user_id=1;
+---------+--------+
| user_id | status |
+---------+--------+
| 1 | 34566 |

I was having the same problem, I'm using Django and Mongo... the data wasn't persisted after object.save()... Them, I used it to solve:
object.save_base(force_update=True)
Now is working for me, hope can help.

If the data persists after closing the shell, it is very unlikely that the data was not saved into the database. Can you check your settings.py and make sure you're saving to the correct mysql database ?

The shell session is a single database transaction. Connections outside that won't see the changes, because of transaction isolation. You'll need to commit the transaction - the easiest way would be to quit the shell and restart.
In a normal request, Django commits automatically at the end of the request, so this behaviour isn't an issue.

Problem was in caching module. Strange, but there is no caching middleware was plugged and cache doesn't used in saving methods.
Was used django.core.cache.backends.locmem.LocMemCache module, may be its broken or has some strange features, I don't know. Will try memcached module, it should (I wish) fix problem without swithing off cache on site.

Related

Django psql full text search not matching un-stemmed word

I'm running Django 1.10.1 against Postgres 9.4. My staging server and dev environments have psql servers at version 9.4.9 and production is an RDS instance at 9.4.7.
It seems like my SearchVectorField is not storing the search configuration given in production, though it is in staging and dev, and it seems to be either a version thing (unlikely, given the version difference and that it also worked on 9.3 in staging/dev) or the fact that production is on RDS instead of local on the server.
I'm using a custom configuration for full-text search called unaccent, which looks like this:
Token | Dictionaries
-----------------+-----------------------
asciihword | english_stem
asciiword | english_stem
email | simple
file | simple
float | simple
host | simple
hword | unaccent,english_stem
hword_asciipart | english_stem
hword_numpart | simple
hword_part | unaccent,english_stem
int | simple
numhword | simple
numword | simple
sfloat | simple
uint | simple
url | simple
url_path | simple
version | simple
word | unaccent,english_stem
Unaccent is installed in both environments, and works in both environments.
I'm storing the search data in a django.contrib.postgres.search.SearchVectorField on my Writer model:
class Writer(models.Model):
#...
search = SearchVectorField(blank=True)
That column is updated with the following search vector:
writer_search_vector = (SearchVector('first_name', 'last_name', 'display_name',
config='unaccent', weight='A') +
SearchVector('raw_search_data', config='unaccent', weight='B'))
by the following statement, which runs periodically:
Writer.objects.update(search=search_utils.writer_search_vector)
And, for some reason, the configuration is storing successfully on my staging server and in dev, but not in production. E.g., this code returns the same results in all environments:
In [3]: Writer.objects.annotate(searchy=SearchVector('last_name')).filter(searchy='kostenberger')
Out[3]: <QuerySet []>
In [4]: Writer.objects.annotate(searchy=SearchVector('last_name', config='unaccent')).filter(searchy='kostenberger')
Out[4]: <QuerySet [<Writer: Andreas J. Köstenberger>, <Writer: Margaret Elizabeth Köstenberger>]>
But in staging, I get the following correct result if I use the stored vector:
In [5]: Writer.objects.filter(search='kostenberger')
Out[5]: <QuerySet [<Writer: Andreas J. Köstenberger>, <Writer: Margaret Elizabeth Köstenberger>]>
while in production, against the RDS instance, I get the following, incorrect result:
In [5]: Writer.objects.filter(search='kostenberger')
Out[5]: <QuerySet []>
and yet, in production still, the unaccent works but the english_stem does not, in that it will match the stemmed version of the text (below), but not the original version (above):
In [6]: Writer.objects.filter(search='kostenberg')
Out[6]: <QuerySet [<Writer: Margaret Elizabeth Köstenberger>, <Writer: Andreas J. Köstenberger>]>
Note that the database tables for Writer in the two environments are identical for this test.
Any ideas why the stored vector isn't working in production with the correct config, while if I create the vector on the fly it will work?
On RDS Postgres, you aren't allowed to change the default_text_search_config parameter. So, you have to configure the text search with each query:
from django.contrib.postgres.search import SearchRank, SearchQuery
…
search_query = SearchQuery(value='kostenberger', config='unaccent')
Writer.objects.filter(search=search_query)

Discovering keys using h5py in python3

In python2.7, I can analyze an hdf5 files keys use
$ python
>>> import h5py
>>> f = h5py.File('example.h5', 'r')
>>> f.keys()
[u'some_key']
However, in python3.4, I get something different:
$ python3 -q
>>> import h5py
>>> f = h5py.File('example.h5', 'r')
>>> f.keys()
KeysViewWithLock(<HDF5 file "example.h5" (mode r)>)
What is KeysViewWithLock, and how can I examine my HDF5 keys in Python3?
From h5py's website (http://docs.h5py.org/en/latest/high/group.html#dict-interface-and-links):
When using h5py from Python 3, the keys(), values() and items()
methods will return view-like objects instead of lists. These objects
support containership testing and iteration, but can’t be sliced like
lists.
This explains why we can't view them. The simplest answer is to convert them to a list:
>>> list(for.keys())
Unfortunately, I run things in iPython, and it uses the command 'l'. That means that approach won't work.
In order to actually view them, we need to take advantage of containership testing and iteration. Containership testing means we'd have to already know the keys, so that's out. Fortunately, it's simple to use iteration:
>>> [key for key in f.keys()]
['mins', 'rects_x', 'rects_y']
I've created a simple function that does this automatically:
def keys(f):
return [key for key in f.keys()]
Then you get:
>>> keys(f)
['mins', 'rects_x', 'rects_y']

Django get/filter method default exact lookup not working

I am trying to write a query to search model of myapp but. As stated in the official Django documentation and here at this link the default behavior of __exact lookup if not working as it should.
For instance:
>>> from django.db import models
>>> from girvi.models import State
>>> State.objects.all()
[]
>>> s = State.objects.create(name='Uttar pradesh')
>>> State.objects.get(name='uttar pradesh')
<State: Uttar pradesh>
The above query should not work. It should return [] because I am looking for field with name='uttar pradesh' against fieldname='Uttar pradesh'`
>>> State.objects.get(name__exact='uttar pradesh')
<State: Uttar pradesh>
>>> State.objects.get(name__iexact='Uttar Pradesh')
<State: Uttar pradesh>
>>> State.objects.get(name__exact='Uttar Pradesh')
<State: Uttar pradesh>
Again the same behavior.
Please can someone explain it to me. What i am doing wrong.
What is the database table's collation used? Even if you use __exact, and your table's collation is set to a case-insensitive variant, then __exact would still behave like __iexact.
Django's documentation states
In MySQL, a database table’s “collation” setting determines whether exact comparisons are case-sensitive. This is a database setting, not a Django setting. It’s possible to configure your MySQL tables to use case-sensitive comparisons, but some trade-offs are involved. For more information about this, see the collation section in the databases documentation.
The Answer above is true, cause I've tried here and look what happened
>>> a = Postagem.objects.get(titulo = "Snippets Sublime text")
>>> a
<Postagem: 13 - Snippets Sublime text, 2014-09-17 00:37:08.268915+00:00>
>>> a = Postagem.objects.get(titulo = "snippets sublime text")
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/home/gpzim98/deploy/virtualenvs/BlogGP/local/lib/python2.7/site-ackages/django/db/models/manager.py", line 151, in get
return self.get_queryset().get(*args, **kwargs)
File "/home/gpzim98/deploy/virtualenvs/BlogGP/local/lib/python2.7/site-packages/django/db/models/query.py", line 307, in get
self.model._meta.object_name)
DoesNotExist: Postagem matching query does not exist.
Soon, it will depend of your collation. I'm use Postgre with collation pg_catalog."default", that is case-sensitive.

Importing CSV to Django and settings not recognised

So i'm getting to grips with Django, or trying to. I have some code that isn't dependent on being called by the webpage - it's designed to populate the database with information. Eventually it will be set up as a cron job to run overnight. This is the first crack at it, which is to do an initial population (once I have that working, I'll move to an add structure, where only new records are pushed.) I'm using Python 2.7, Django 1.5 and Sqlite3. When I run this code, I get
Requested setting DATABASES, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
That seems fairly obvious, but I've spent a couple of hours now trying to work out how to adjust that setting. How do I call / open a connection / whatever the right terminology is here? I have a number of functions like this that will be scheduled jobs, and this has been frustrating me all afternoon.
import urllib2
import csv
import requests
from django.db import models
from gmbl.models import Match
master_data_file = urllib2.urlopen("http://www.football-data.co.uk/mmz4281/1213/E0.csv", "GET")
data = list(tuple(rec) for rec in csv.reader(master_data_file, delimiter=','))
for row in data:
current_match = Match(matchdate=row[1],
hometeam=row[2],
awayteam = row [3],
homegoals = row [4],
awaygoals = row[5],
homeshots = row[10],
awayshots = row[11],
homeshotsontarget = row[12],
awayshotsontarget = row[13],
homecorners = row[16],
awaycorners = row[17])
current_match.save()
I had originally started out with http://django-csv-importer.readthedocs.org/en/latest/ but I had the same error, and the documentation doesn't make much sense trying to debug it. When I tried calling settings.configure in the function, it said it didn't exist; presumably I had to import it, but couldn't make that work.
Make sure Django, and your project are in PYTHONPATH then you can do:
import urllib2
import csv
import requests
from django.core.management import setup_environ
from django.db import models
from yoursite import settings
setup_environ(settings)
from gmbl.models import Match
master_data_file = urllib2.urlopen("http://www.football-data.co.uk/mmz4281/1213/E0.csv", "GET")
data = list(tuple(rec) for rec in csv.reader(master_data_file, delimiter=','))
# ... your code ...
Reference: http://www.b-list.org/weblog/2007/sep/22/standalone-django-scripts/
Hope it helps!

haystack whoosh no results - rebuild_index shows Indexing [number] <django.utils.functional.__proxy__ object at [memory location] >

when I run ./manage.py rebuild_index I get the readout for example:
Indexing 4574 <django.utils.functional.__proxy__ object at at 0x1aab690> .
Having seen other users' readouts, this should show the name of the search index/model instead and I am wondering if this could be part of the explanation as to why I have been experiencing no search results on the website and no objects appear to be indexed when performing:
>>> from haystack.query import SearchQuerySet
>>> sqs = SearchQuerySet().all()
>>> sqs.count()
I did not initially have a
def _unicode_self():
return self.name
on the models I am indexing but then I added it and nothing seemed to change even after doing rebuild_index
This was GitHub pull request #746 for Django Haystack, which has now been merged.
I was seeing this same issue on my local (dev) setup. Updating solved the "functional proxy" placeholder issue for me.
I ran the following command:
pip install -e git+git://github.com/toastdriven/django-haystack.git#master#egg=django-haystack
You may need to tweak the command to suit your own needs and/or environment.