After much hardship, I have managed to convert my django project, that previously ran with sqlite, to run with MongoDB.
This is great, beside from the fact that my old version had a massive initial_data.json file, which now fails to work with the new db when running django's syncdb command.
EDIT:
this is an example of the initial_data.json file :
[{"pk":1,
"model": "vcb.dishtype",
"fields": {
"photo": "images/dishes/breakfast.jpg",
"name": "Breakfast"
}
},
{"pk":2,
"model": "vcb.dishtype",
"fields": {
"photo": "images/dishes/bread_and_pastry.jpg",
"name": "Bread and pastry"
}
}]
and after running the syncdb I get:
DeserializationError: Problem installing fixture
'C:\Users..\initial_data.json' : u'pk'
It seems to be a problem with the MongoDB objectId and how I defined the initial_data file.
I tried to remove all the pks fields from the file, but still the same error.
EDIT
I tried putting just two fixtures, if I don't set the pk, I get the same error as above. If I do set it, I get :
"DatabaseErroe: Problem installing fixture 'C:..\initial_data.json':
could not load vcb.dishtype(pk=1): AutoField (default primary key)
values must be strings representing an ObjectId on MongoDB (got u'1'
instead)".
which is a similar problem I had with the django Site, that was solved with the help of this thread: Django MongoDB Engine error when running tellsiteid
This raises my suspicion that there's a different way to set the fixtures in this infrastructure. Maybe syncdb isn't the way to go, and there should be a sort of dump maybe?
I've searched google, and nothing seems to tackle this problem. I'm obviously asking the wrong questions.
what should I do, to create fixtures in my altered project?
thanks a lot,
Nitzan
From your error message, I assume you are using Django MongoDB Engine?
Your pk values must be valid ObjectIds, try using values like:
'000000000000000000000001'
'000000000000000000000002'
etc
You can get ObjectIds or check that you have correct values:
>>> from bson.objectid import ObjectId
>>> ObjectId()
> ObjectId('52af59bac38f8420734d064d')
>>> ObjectId('000000000000000000000001')
> ObjectId('000000000000000000000001')
>>> ObjectId('bad')
*error*
Related
A strange (to the uninitiated lol) issue with models using a custom CharField for a primary key/id:
id = models.CharField(max_length=10, primary_key=True)
Feeling I know what I'm doing, I've created the following (json) fixtures file:
[
{
"model": "products.product",
"id": "am11",
"fields": {
"title": "Test Product A",
}
},
{
"model": "products.product",
"id": "am22",
"fields": {
"title": "Test Product B",
}
}
]
, and proceeded with loading it:
✗ python manage.py loaddata fixtures/products.json
Installed 2 object(s) from 1 fixture(s)
Well, it kinda lied. A check on the admin page or in a shell shows that there's only one Product in the database - the last one in the fixture's list. Curiously enough, attempts to delete this Product via the admin page silently fail, only via the shell can it actually be deleted. Further investigation (in the shell) revealed an interesting problem - that (single) Product created has pk/id set to an empty string(?!..but explains why admin fails to delete it). If I manually create another, either on admin page or in the shell, the new product appears without any issues, both id and pk set to the string given. But loaddata with fixture fails on this. Originally discovered this problem when a basic test failed - given the same fixture, in failed the assertion on the number of products in the queryset, claiming there's just one.
Now, I was able to "fix" the problem by renaming 'id' to 'pk' in the fixture file. I say 'fix', because I don't understand what bit me here. Any clue will be appreciated. Thanks!
I am deploying my Django project on Windows Azure. I was using SQLite as a database server and everything was ok. When I have deployed My project, I decided to connect it with an SQL Azure Database but it seems that this solution created some errors. I am no longer able to edit users profiles. I get always this error :
AttributeError at /admin/auth/user/1/
'unicode' object has no attribute 'tzinfo'
This error happens when your database contains date-time stamps like this:
0000-00-00 00:00:00.000000
(this can happen in MySQL if you delete or overwrite a previous date with MySQLWorkbench)
When you try to retrieve these records in a Django model object, you will get an exception from the pytz timezone library:
AttributeError 'unicode' object has no attribute 'tzinfo'
You should edit these dates in your database first, and set them to more recent dates, like 2018-01-01 00:00:00.000000 or set to NULL (but not blank).
See:
https://groups.google.com/forum/#!topic/django-users/Jg_9fQ3jMcU
See also:
#1292 - Incorrect date value: '0000-00-00'
Error in mysql when setting default value for DATE or DATETIME
How to store NULL values in datetime fields in MySQL?
I had same issue trying to use django-pyodbc-azure database backend with Django (1.5.1): by default it stores DateTimeField fields in your DB as datetime2(7), which looks to be still unsupported in Django.
In my case I added the option 'use_legacy_datetime' : True in settings.py, like below:
DATABASES = {
'default': {
'ENGINE' : 'sql_server.pyodbc',
'NAME' : '<MYDBNAME>',
'USER': '<MYDBUSER>',
'PASSWORD': '<MYDBPWD>',
'HOST': '<MYHOST>',
'OPTIONS': {
'use_mars': True,
'use_legacy_datetime' : True, # I added this line
},
},
}
I found my solution here.
I don't know Azure platform well, so I don't know if this is exactly your case, if not you can still modify your database replacing datetime2(N) fields with good old datetime ones.
Hope it helps.
I have a model:
class mymodel(models.Model):
order_closed = models.BooleanField(default=False)
I added this field to my development mysqllite db manually since its a new field for a model/table that already existed. I then tried:
mymodel.objects.filter(order_closed=False) #and with True
and its producing incorrect or unpredictable results. I saw some mention that is could be a sqllite thing but I'm not sure? The templates seem to understand whether its a true or false value but python code doesn't. To clarify with some examples:
{{mymodel.order_closed}} will print 0 after I set the default to 0 in sqllite. but using .filter(order_closed=value) will still return every record.
I think you make some mistake in wrirting SQL . If you have db which have some important informations use
south
http://south.aeracode.org/
When you will have it you can easly upgrate/edit your database .
If you dont wanna install new 'plugins' try that .
1. Delete this field from DB manualy .
2. write : python manage.py sql 'name of app'
It will return the CREATE TABLE SQL statements for the app.
Then you can upr your database manualy with some of CReate command.
I recently upgraded to Django 1.4 from 1.1. I have been running WadofStuff Django Serializers 1.0.0. After upgrade, I noticed that dates from my django model get serialized with a 'T' character inserted:
{"pk": 7, "model": "ao.message", "fields": {"content_file": "bar.wav", "date": "2012-07-04T10:58:46", "summary_file": "foo.wav"}}
What's up with that 'T'? Can/should it be removed? Is there a way to specify my desired output date format to the serializer (say, if I didn't want it to return with a 'T')?
Thanks
A single point in time can be represented by concatenating: a complete date expression, the letter T as a delimiter, and a valid time expression. For example "2007-04-05T14:30" (Wikipedia Link )
For further details regarding this T insertion in datetime format as far as python is concerned, you may go to this link: "Python Datetime Representations". The first example specifically illustrates your problem and suggests its solutions too.
I have a json field in my database which is like
jsonfield = {'username':'chingo','reputation':'5'}
how can i write a query so that i can find if a user name exists. something like
username = 'chingo'
query = User.objects.get(jsonfield['username']=username)
I know the above query is a wrong but I wanted to know if there is a way to access it?
If you are using the django-jsonfield package, then this is simple. Say you have a model like this:
from jsonfield import JSONField
class User(models.Model):
jsonfield = JSONField()
Then to search for records with a specific username, you can just do this:
User.objects.get(jsonfield__contains={'username':username})
Since Django 1.9, you have been able to use PostgreSQL's native JSONField. This makes search JSON very simple. In your example, this query would work:
User.objects.get(jsonfield__username='chingo')
If you have an older version of Django, or you are using the Django JSONField library for compatibility with MySQL or something similar, you can still perform your query.
In the latter situation, jsonfield will be stored as a text field and mapped to a dict when brought into Django. In the database, your data will be stored like this
{"username":"chingo","reputation":"5"}
Therefore, you can simply search the text. Your query in this siutation would be:
User.objects.get(jsonfield__contains='"username":"chingo"')
2019: As #freethebees points out it's now as simple as:
User.objects.get(jsonfield__username='chingo')
But as the doc examples mention you can query deeply, and if the json is an array you can use an integer to index it:
https://docs.djangoproject.com/en/2.2/ref/contrib/postgres/fields/#querying-jsonfield
>>> Dog.objects.create(name='Rufus', data={
... 'breed': 'labrador',
... 'owner': {
... 'name': 'Bob',
... 'other_pets': [{
... 'name': 'Fishy',
... }],
... },
... })
>>> Dog.objects.create(name='Meg', data={'breed': 'collie', 'owner': None})
>>> Dog.objects.filter(data__breed='collie')
<QuerySet [<Dog: Meg>]>
>>> Dog.objects.filter(data__owner__name='Bob')
<QuerySet [<Dog: Rufus>]>
>>> Dog.objects.filter(data__owner__other_pets__0__name='Fishy')
<QuerySet [<Dog: Rufus>]>
Although this is for postgres, I believe it works the same in other DBs like MySQL
Postgres: https://docs.djangoproject.com/en/2.2/ref/contrib/postgres/fields/#querying-jsonfield
MySQL: https://django-mysql.readthedocs.io/en/latest/model_fields/json_field.html#querying-jsonfield
This usage is somewhat anti-pattern. Also, its implementation is not going to have regular performance, and perhaps is error-prone.
Normally don't use jsonfield when you need to look up through fields. Use the way the RDBMS provides or MongoDB(which internally operates on faster BSON), as Daniel pointed out.
Due to the deterministic of JSON format,
you could achieve it by using contains (regex has issue when dealing w/ multiple '\' and even slower), I don't think it's good to use username in this way, so use name instead:
def make_cond(name, value):
from django.utils import simplejson
cond = simplejson.dumps({name:value})[1:-1] # remove '{' and '}'
return ' ' + cond # avoid '\"'
User.objects.get(jsonfield__contains=make_cond(name, value))
It works as long as
the jsonfield using the same dump utility (the simplejson here)
name and value are not too special (I don't know any egde-case so far, maybe someone could point it out)
your jsonfield data is not corrupt (unlikely though)
Actually I'm working on a editable jsonfield and thinking about whether to support such operations. The negative proof is as said above, it feels like some black-magic, well.
If you use PostgreSQL you can use raw sql to solve problem.
username = 'chingo'
SQL_QUERY = "SELECT true FROM you_table WHERE jsonfield::json->>'username' = '%s'"
User.objects.extra(where=[SQL_EXCLUDE % username]).get()
where you_table is name of table in your database.
Any methods when you work with JSON like with plain text - looking like very bad way.
So, also I think that you need a better schema of database.
Here is the way I have found out that will solve your problem:
search_filter = '"username":{0}'.format(username)
query = User.objects.get(jsonfield__contains=search_filter)
Hope this helps.
You can't do that. Use normal database fields for structured data, not JSON blobs.
If you need to search on JSON data, consider using a noSQL database like MongoDB.