Querying with mongoengine and django - django

I have a database "tumblelog" (using mongoengine) in which I added some data in the "user" collection with a "User" model:
db.user.find()
...
{ "_id" : ObjectId("4fb0c9494ca88402dd000000"), "_types" : [ "User" ], "first_name" : "John", "last_name" : "Doe", "_cls" : "User", "email" : "jdoe#example.com" }
{ "_id" : ObjectId("4fb0cb9d4ca88402ec000000"), "_types" : [ "User" ], "first_name" : "Joe30", "last_name" : "Foe", "_cls" : "User", "email" : "hu#huu.com" }
When I try User.objects in a django shell, I get the following error:
Traceback (most recent call last):
File "<console>", line 1, in <module>
File ".../mongoengine/queryset.py", line 1127, in __getitem__
return self._document._from_son(self._cursor[key])
File ".../mongoengine/base.py", line 1018, in _from_son
obj = cls(**data)
TypeError: __init__() keywords must be strings
Same thing when I try
for user in User.objects:
print user.first_name
---- Edit ----
I tried this
>>> users = User.objects
>>> users.count()
7
>>> users.first()
...
TypeError: __init__() keywords must be strings
---- Edit 2 ----
I installed my project this way :
> virtualenv Test
> source Test/bin/activate
> pip install Django
> pip install mongoengine
> cd Test/
> django-admin.py startproject db
Then I added the lines
from mongoengine import *
connect('tumblelog')
in settings.py
then I created this simple model
from mongoengine import *
class User(Document):
email = StringField(required=True)
name = StringField(max_length=50)
then I run the server
> python manage.py runserver
And in the shell (python manage.py shell) I can save data if I import my model class but I can't read it, I always have the same TypeError: init() keywords must be strings !
-----Switching to django-mongodb engine----
I didn't find any solution so I will use django-mongodb-engine. I did not find any comparison, but I tried both and it's very similar.
I just regret django-mongodb-engine doesn't handle inheritance principle.
What am I doing wrong ?
Thanks in advance!

we had the exact same issue using mongoengine 0.6.9. I am not suggesting this as an ideal solution but downgrading to 0.6.5 resolved this issue for us.

use the all() method
for user in User.objects.all():
print user.first_name

Related

django dumpdata command throws doesnot exist error

I am trying to add some default/fixed values to postgres database and came across fixtures.
My model is like this
app/models.py
class Category(models.Model):
category = models.CharField(max_length=255)
def __str__(self):
return self.category
app/fixtures/category.json
[
{
"model": "core.category",
"pk": 1,
"fields": {
"category": "foo"
}
]
However, I am getting the following error when I run manage.py dumpdata
[CommandError: Unable to serialize database: cursor "_django_curs_139823939079496_sync_1" does not exist
It looks like you have json file already (serialize, means that your django models data has been translated already to json format) you just need to to migrate your data into a new database(you say postgres), try to run these few lines of codes NOTE: I used python3 because I am using macbook pro:-
python3 manage.py migrate
OR
python3 manage.py migrate --run -syncdb
Then, enter your django shell and delete all contenttypes:
>>> from django.contrib.contenttypes.models import ContentType
>>> c = ContentType.objects.all()
>>> c.delete()
(55, {'auth.Permission': 44, 'contenttypes.ContentType': 11})
>>> exit()
python3 manage.py loaddata datadump.json
After being notified that all contenttypes has been deleted, You need to ensure that those data will be displayed in your templates. Simply by loading them from from your json file as shown above.*

SolrError: Solr responded with an error (HTTP 400): [Reason: ERROR: [doc=auth.user.23] unknown field 'firstname']

I am trying to implement Autocomplete on the 'first_name' field of the 'User' model using Django-Haystack-Solr. But I am facing this "Unknown field" error.
search_indexes.py:
from haystack import indexes
from django.contrib.auth.models import User
class UserIndex(indexes.SearchIndex, indexes.Indexable):
text = indexes.CharField(document=True, use_template=True)
firstname = indexes.EdgeNgramField(model_attr='first_name')
def get_model(self):
return User
def index_queryset(self, using=None):
return self.get_model().objects.all()
views.py
results = SearchQuerySet().autocomplete(firstname=request.POST.get("search_text",''))
I performed the following:
Ran the command: $ python manage.py build_solr_schema
Saved new schema to schema.xml
Ran the command: $ python manage.py rebuild_index ---> Throws the Error
Error:
Indexing 5 users
/usr/local/lib/python2.7/dist-packages/haystack/fields.py:137: RemovedInDjango110Warning: render() must be called with a dict, not a Context.
return t.render(Context({'object': obj}))
Failed to add documents to Solr: Solr responded with an error (HTTP 400): [Reason: ERROR: [doc=auth.user.23] unknown field 'firstname']
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/haystack/backends/solr_backend.py", line 72, in update
self.conn.add(docs, commit=commit, boost=index.get_field_weights())
File "/usr/local/lib/python2.7/dist-packages/pysolr.py", line 860, in add
return self._update(m, commit=commit, softCommit=softCommit, waitFlush=waitFlush, waitSearcher=waitSearcher)
File "/usr/local/lib/python2.7/dist-packages/pysolr.py", line 462, in _update
return self._send_request('post', path, message, {'Content-type': 'text/xml; charset=utf-8'})
File "/usr/local/lib/python2.7/dist-packages/pysolr.py", line 394, in _send_request
raise SolrError(error_message % (resp.status_code, solr_message))
SolrError: Solr responded with an error (HTTP 400): [Reason: ERROR: [doc=auth.user.23] unknown field 'firstname']
Also if somebody can point me in the right direction about how to implement Autocomplete for multiple fields. Like in this case for 'first_name', 'last_name' and 'email', how can I do that?
Solution - Restarting Solr
Stop Solr: ./solr-4.10.4/bin/solr stop -all
Start Solr: ./solr-4.10.4/bin/solr start -p 8983
If you are trying to add documents to Solr 9, add them using the lines below...
import pysolr
solr = pysolr.Solr('http://localhost:8983/solr/core/', always_commit=True)
docs = {"id":3,"title": "Solr"} #This is what Solr LIKES. Solr will through an error if your do nested dictionary like this {"id":3,"title": "Solr", "_links": {"self": {"href": "/api/entries/7"}}}
solr.add(doc,commit=True)
The error I was getting in my case when I did nested dictionary is SolrError: Solr responded with an error (HTTP 400): [Reason: Error:[doc=3] Unknown operation for the an atomic update: self]

Django - populate table on startup with known values

I have a Dog Model that have a "dog_type" field. i want the dog_type to be chosen from a list of pre-defined dog types, i DO NOT want to use a textfield with choices but a ForeignKey to a "DogType" Model. How could I populate the DogType Model with types on server startup? is this a good practice or a hack?
thanks.
code:
class Dog(Model):
name = CharField(...)
dog_type = ForeignKey(DogType)
class DogType(Model):
type_name = CharField(...)
type_max_hight = IntegerField(...)
etc....
You'll probably want to write a data migration that will add your choices in database.
Advantages of using this approach is that the data will be loaded in all your databases (production, dev, etc.)
(If you're not using migrations yet, you should consider it, it's a clean and well-supported way to manage your database)
In your django project, just run python manage.py shell makemigrations myapp --empty. This will create an empty migration file under myapp/migrations.
You can then edit it:
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import models, migrations
DEFAULT_DOG_TYPES = (
('Labrador', 90),
('Berger Allemand', 66),
('Chihuaha', -2),
)
def add_dog_types(apps, schema_editor):
DogType = apps.get_model('myapp', 'DogType')
for name, max_height in DEFAULT_DOG_TYPES:
dog_type = DogType(name=name, max_height=max_height)
dog_type.save()
def remove_dog_types(apps, schema_editor):
# This is used to migrate backward
# You can remove your dog types, or just pass
# pass
for name, max_height in DEFAULT_DOG_TYPES:
dog_type = DogType.objects.get(name=name, max_height=max_height)
dog_type.delete()
class Migration(migrations.Migration):
dependencies = [
# if you're already using migrations, this line will be different
('myapp', '0001_initial'),
]
operations = [
migrations.RunPython(add_dog_types, remove_dog_types),
]
After that, all you need to do is to run python manage.py syncdb.
not so far after, i found this it's called "fixtures"...
basically what you need to do is to place a "fixture" file in a format of your choice (JSON\YAML...) "myapp/fixtures/" for example, a JSON file would look like this:
[
{
"model": "myapp.person",
"pk": 1,
"fields": {
"first_name": "John",
"last_name": "Lennon"
}
},
{
"model": "myapp.person",
"pk": 2,
"fields": {
"first_name": "Paul",
"last_name": "McCartney"
}
}
]
then simply run from the command line:
python manage.py loaddata <filename> # file with no path!

AttributeError: 'module' object has no attribute 'ElasticSearchError' : Using Haystack Elasticsearch

Using Django & Haystack with ElasticSearch.
After installing haystack and ES, and Rebuilding Index
./manage.py rebuild_index
WARNING: This will irreparably remove EVERYTHING from your search index in connection 'default'.
Your choices after this are to restore from backups or rebuild via the rebuild_index command.
Are you sure you wish to continue? [y/N] y
Removing all documents from your index because you said so.
All documents removed.
Indexing 1039 <django.utils.functional.__proxy__ object at 0x10ca3ded0>.
AttributeError: 'module' object has no attribute 'ElasticSearchError'
Updating index has the same problem
/manage.py update_index
Indexing 1039 <django.utils.functional.__proxy__ object at 0x10ea49d90>.
AttributeError: 'module' object has no attribute 'ElasticSearchError'
Clear index works fine though ( probably because there is no index )
./manage.py clear_index
WARNING: This will irreparably remove EVERYTHING from your search index in connection 'default'.
Your choices after this are to restore from backups or rebuild via the `rebuild_index` command.
Are you sure you wish to continue? [y/N] y
Removing all documents from your index because you said so.
All documents removed.
Versions
django-haystack==2.0.0-beta
pyelasticsearch==0.5
elasticsearch==0.20.6
localhost:9200 says :
{
"ok" : true,
"status" : 200,
"name" : "Jigsaw",
"version" : {
"number" : "0.20.6",
"snapshot_build" : false
},
"tagline" : "You Know, for Search"
}
Haystack settings :
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
'URL': 'http://127.0.0.1:9200/',
'INDEX_NAME': 'haystack',
},
}
search_indexes.py :
import datetime
import haystack
from haystack import indexes
from app.models import City
class CityIndex(indexes.SearchIndex, indexes.Indexable):
text = indexes.CharField(document=True, use_template=True)
name = indexes.CharField(model_attr='name')
state = indexes.CharField(model_attr='state')
country = indexes.CharField(model_attr='country')
lat = indexes.FloatField(model_attr='latitude')
lon = indexes.FloatField(model_attr='longitude')
alt = indexes.FloatField(model_attr='altitude')
pop = indexes.IntegerField(model_attr='population')
def get_model(self):
return City
Any help - why I am getting error ?
Solved it !
After debugging the process using pdb
./manage.py rebuild_index
At line 222 - in /haystack/backend/elasticsearch_backend.py
Changed
except (requests.RequestException, pyelasticsearch.ElasticSearchError), e:
To
# except (requests.RequestException, pyelasticsearch.ElasticSearchError), e:
except Exception as inst:
import pdb; pdb.set_trace()
I found out the core error was this
'ElasticSearch' object has no attribute 'from_python'.
To which I found solution here - https://github.com/toastdriven/django-haystack/issues/514#issuecomment-4058230
The version of pyelasticsearch I was using was from http://github.com/rhec/pyelasticsearch,
So I installed pyelasticsearch from a fork - http://github.com/toastdriven/pyelasticsearch using :
pip install --upgrade git+https://github.com/toastdriven/pyelasticsearch.git#3bfe1a90eab6c2dfb0989047212f4bc9fb814803#egg=pyelasticsearch
and That fixed it & Index was build !

Django fixtures. JSONDecodeError

I've a Django project and I want to add test data to database. When I make syncdb like this
python ~/django/foo/manage.py syncdb
After tables are installed I've got an error
Problem installing fixture '~/django/foo/shop/fixtures/initial_data.json':
Traceback (most recent call last):
raise JSONDecodeError("No JSON object could be decoded", s, idx)
JSONDecodeError: No JSON object could be decoded: line 1 column 0 (char 0)
My model is here:
# -*- coding: utf-8 -*-
from django.db import models
class Image(models.Model):
file = models.ImageField(upload_to = "img/")
title = models.CharField(
max_length=128,
blank = True
)
slug = models.SlugField(max_length=128, blank = True)
def __unicode__(self):
return unicode(self.title)
My fixture is this:
[
{
"pk": 2,
"model": "shop.image",
"fields": {
"slug": "",
"file": "img/8_m.jpg",
"title": "1"
}
}
]
Where is the problem?
Wild guess... maybe your fixture file is saved as a unicode file??? Try to open it in the simplest text editor you can, or run
hexdump ~/django/foo/shop/fixtures/initial_data.json
and make sure the first character in the dump is 5b not fe or something.