I was using Apache Solr for quite some time and only recently started running into some severe issues with it. I'm using it with haystack and a django project. When I do it from manage.py shell i'm getting the below:
>>> from haystack.query import SearchQuerySet
>>> emps = SearchQuerySet().filter(django_ct='web.employer').filter(name__icontains='Mi')[:10]
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/haystack/query.py", line 241, in __getitem__
self._fill_cache(start, bound)
File "/usr/local/lib/python2.7/dist-packages/haystack/query.py", line 140, in _fill_cache
results = self.query.get_results(**kwargs)
File "/usr/local/lib/python2.7/dist-packages/haystack/backends/__init__.py", line 469, in get_results
self.run(**kwargs)
File "/usr/local/lib/python2.7/dist-packages/haystack/backends/solr_backend.py", line 501, in run
results = self.backend.search(final_query, **search_kwargs)
File "/usr/local/lib/python2.7/dist-packages/haystack/backends/__init__.py", line 47, in wrapper
return func(obj, query_string, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/haystack/backends/solr_backend.py", line 202, in search
raw_results = self.conn.search(query_string, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/pysolr.py", line 578, in search
response = self._select(params)
File "/usr/local/lib/python2.7/dist-packages/pysolr.py", line 308, in _select
return self._send_request('get', path)
File "/usr/local/lib/python2.7/dist-packages/pysolr.py", line 293, in _send_request
error_message = self._extract_error(resp)
File "/usr/local/lib/python2.7/dist-packages/pysolr.py", line 372, in _extract_error
reason, full_html = self._scrape_response(resp.headers, resp.content)
File "/usr/local/lib/python2.7/dist-packages/pysolr.py", line 404, in _scrape_response
p_nodes = body_node.cssselect('p')
AttributeError: 'NoneType' object has no attribute 'cssselect'
I tried reinstalling haystack, lxml, cssselect, pysolr and still i'm getting these errors. Is there anything else I can try for this? Thanks for any help!
I also tried reading few other SO questions including this:
XML error object has no attribute 'cssselect'
Seems like the issue is with pysolr. You might find some help here.
I had the same issue persist even after bringing up pysolr and lxml to latest version.
Turned out it was because I was not using haystack generated schema which has a few additional fields compared to the default solr one.
You can confirm if this is the case by looking at your solr logs.
It is an issue with pysolr. It hasn't been fixed till 3.3.0.
The only alternative would be to override the pysolr code and make adjustments for when Solr returns a reponse status!=200.
You can check if the response has body attribute or not and make adjustments according to that.
Related
I have been trying to get the Elasticsearch to work in a Django application. It has been a problem because of the mess of compatibility considerations this apparently involves. I follow the recommendations, but still get an error when I actually perform a search.
Here is what I have
Django==2.1.7
Django-Haystack==2.5.1
Elasticsearch(django)==1.7.0
Elasticsearch(Linux app)==5.0.1
There is also DjangoCMS==3.7 and aldryn-search=1.0.1, but I am not sure how relevant those are.
Here is the error I get when I submit a search query via the basic text form.
GET /videos/modelresult/_search?_source=true [status:400 request:0.001s]
Failed to query Elasticsearch using '(video)': TransportError(400, 'parsing_exception')
Traceback (most recent call last):
File "/home/user-name/miniconda3/envs/project-web/lib/python3.7/site-packages/haystack/backends/elasticsearch_backend.py", line 524, in search
_source=True)
File "/home/user-name/miniconda3/envs/project-web/lib/python3.7/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
return func(*args, params=params, **kwargs)
File "/home/user-name/miniconda3/envs/project-web/lib/python3.7/site-packages/elasticsearch/client/__init__.py", line 527, in search
doc_type, '_search'), params=params, body=body)
File "/home/user-name/miniconda3/envs/project-web/lib/python3.7/site-packages/elasticsearch/transport.py", line 307, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/home/user-name/miniconda3/envs/project-web/lib/python3.7/site-packages/elasticsearch/connection/http_urllib3.py", line 93, in perform_request
self._raise_error(response.status, raw_data)
File "/home/user-name/miniconda3/envs/project-web/lib/python3.7/site-packages/elasticsearch/connection/base.py", line 105, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.RequestError: TransportError(400, 'parsing_exception')
Could someone tell me if this is an issue with compatibility or is there something else going on? How can I fix it?
The combination that appears to have worked for my setup is as follows. I believe the key was to drastically downgrade the Elasticsearch.
Elasticsearch=1.7.6 (with Java 8)
Django==2.1.7
Django-Haystack==2.8.1
elasticsearch==1.7.0
The two items below may or may not be relevant. I have not changed them.
DjangoCMS==3.7.0
aldryn-search==1.0.1
I am using celery for my django project.
java_path = "/your/Java/jdk/home/java.exe
os.environ['JAVAHOME'] = java_path
st = POSTagger('/your/postagger/models/path/english-bidirectional-distsim.tagger','/your/postagger/jar/file/path/stanford-postagger.jar')
tag = st.tag([key])
here the key is a list of feature words.
I got following errors, when using celery to execute:
raised unexpected:
LookupError('\n\n===========================================================================\nNLTK
`was unable to find the java file!\nUse software specific configuration paramaters or set the JAVAHOME environment`
variable.\n===========================================================================',)
Traceback (most recent call last):
File "/Users/Envs/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
R = retval = fun(*args, **kwargs)
File "/Users/Envs/lib/python2.7/site-packages/celery/app/trace.py", line 438, in __protected_call__
return self.run(*args, **kwargs)
File "/Users/Envs/src/evolvelearning/tasks.py", line 772, in RunProgram
tag = st.tag([key])
File "/Users/Envs/lib/python2.7/site-packages/nltk/tag/stanford.py", line 59, in tag
return self.tag_sents([tokens])[0]
File "/Users/Envs/lib/python2.7/site-packages/nltk/tag/stanford.py", line 64, in tag_sents
config_java(options=self.java_options, verbose=False)
File "/Users/Envs/lib/python2.7/site-packages/nltk/internals.py", line 82, in config_java
_java_bin = find_binary('java', bin, env_vars=['JAVAHOME', 'JAVA_HOME'], verbose=verbose, binary_names=['java.exe'])
File "/Users/Envs/lib/python2.7/site-packages/nltk/internals.py", line 544, in find_binary
binary_names, url, verbose))
File "/Users/Envs/lib/python2.7/site-packages/nltk/internals.py", line 538, in find_binary_iter
url, verbose):
File "/Users/Envs/lib/python2.7/site-packages/nltk/internals.py", line 517, in find_file_iter
raise LookupError('\n\n%s\n%s\n%s' % (div, msg, div))
LookupError:
===========================================================================
NLTK was unable to find the java file!
Use software specific configuration paramaters or set the JAVAHOME environment variable.
===========================================================================
I have set java_path and javahome and I would like to know why still thess errors occur?
my environment is MAC.
The problem is the java_path, should be:
java_path = "/your/Java/jdk/home
I'm trying to get into the new N1QL Queries for Couchbase in Python.
I got my database set up in Couchbase 4.0.0.
My initial try was to retreive all documents like this:
from couchbase.bucket import Bucket
bucket = Bucket('couchbase://localhost/dafault')
rv = bucket.n1ql_query('CREATE PRIMARY INDEX ON default').execute()
for row in bucket.n1ql_query('SELECT * FROM default'):
print row
But this produces a OperationNotSupportedError:
Traceback (most recent call last):
File "/Applications/PyCharm.app/Contents/helpers/pydev/pydevd.py", line 2357, in <module>
globals = debugger.run(setup['file'], None, None, is_module)
File "/Applications/PyCharm.app/Contents/helpers/pydev/pydevd.py", line 1777, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "/Users/my_user/python_tests/test_n1ql.py", line 9, in <module>
rv = bucket.n1ql_query('CREATE PRIMARY INDEX ON default').execute()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/couchbase/n1ql.py", line 215, in execute
for _ in self:
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/couchbase/n1ql.py", line 235, in __iter__
self._start()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/couchbase/n1ql.py", line 180, in _start
self._mres = self._parent._n1ql_query(self._params.encoded)
couchbase.exceptions.NotSupportedError: <RC=0x13[Operation not supported], Couldn't schedule n1ql query, C Source=(src/n1ql.c,82)>
Here the version numbers of everything I use:
Couchbase Server: 4.0.0
couchbase python library: 2.0.2
cbc: 2.5.1
python: 2.7.8
gcc: 4.2.1
Anyone an idea what might have went wrong here? I could not find any solution to this problem up to now.
There was another ticket for node.js where the same issue happened. There was a proposal to enable n1ql for the specific bucket first. Is this also needed in python?
It would seem you didn't configure any cluster nodes with the Query or Index services. As such, the error returned is one that indicates no nodes are available.
I also got similar error while trying to create primary index.
Create a primary index...
Traceback (most recent call last):
File "post-upgrade-test.py", line 45, in <module>
mgr.n1ql_index_create_primary(ignore_exists=True)
File "/usr/local/lib/python2.7/dist-packages/couchbase/bucketmanager.py", line 428, in n1ql_index_create_primary
'', defer=defer, primary=True, ignore_exists=ignore_exists)
File "/usr/local/lib/python2.7/dist-packages/couchbase/bucketmanager.py", line 412, in n1ql_index_create
return IxmgmtRequest(self._cb, 'create', info, **options).execute()
File "/usr/local/lib/python2.7/dist-packages/couchbase/_ixmgmt.py", line 160, in execute
return [x for x in self]
File "/usr/local/lib/python2.7/dist-packages/couchbase/_ixmgmt.py", line 144, in __iter__
self._start()
File "/usr/local/lib/python2.7/dist-packages/couchbase/_ixmgmt.py", line 132, in _start
self._cmd, index_to_rawjson(self._index), **self._options)
couchbase.exceptions.NotSupportedError: <RC=0x13[Operation not supported], Couldn't schedule ixmgmt operation, C Source=(src/ixmgmt.c,98)>
Adding query and index node to the cluster solved the issue.
I'm having strange error with installing fixture from dumped data. I am using psycopg2, and django1.1.1
silver:probsbox oleg$ python manage.py loaddata /Users/oleg/probs.json
Installing json fixture '/Users/oleg/probs' from '/Users/oleg/probs'.
Problem installing fixture '/Users/oleg/probs.json': Traceback (most recent call last):
File "/opt/local/lib/python2.5/site-packages/django/core/management/commands/loaddata.py", line 153, in handle
obj.save()
File "/opt/local/lib/python2.5/site-packages/django/core/serializers/base.py", line 163, in save
models.Model.save_base(self.object, raw=True)
File "/opt/local/lib/python2.5/site-packages/django/db/models/base.py", line 495, in save_base
result = manager._insert(values, return_id=update_pk)
File "/opt/local/lib/python2.5/site-packages/django/db/models/manager.py", line 177, in _insert
return insert_query(self.model, values, **kwargs)
File "/opt/local/lib/python2.5/site-packages/django/db/models/query.py", line 1087, in insert_query
return query.execute_sql(return_id)
File "/opt/local/lib/python2.5/site-packages/django/db/models/sql/subqueries.py", line 320, in execute_sql
cursor = super(InsertQuery, self).execute_sql(None)
File "/opt/local/lib/python2.5/site-packages/django/db/models/sql/query.py", line 2369, in execute_sql
cursor.execute(sql, params)
File "/opt/local/lib/python2.5/site-packages/django/db/backends/util.py", line 19, in execute
return self.cursor.execute(sql, params)
ProgrammingError: can't adapt
First I've checked similar issues on internet. This one seemed to be very related: http://code.djangoproject.com/ticket/5996, as my data has many non ASCII symbols
But actually I've checked my django installation and it's ok there
Could you advice what is wrong
====
Continued investigation after added print statement as suggested by the first answer. Log looks this way:
silver:probsbox oleg$ python manage.py loaddata /Users/oleg/probs.json
Installing json fixture '/Users/oleg/probs' from '/Users/oleg/probs'.
<DeserializedObject: Novice>
<DeserializedObject: Junior>
<DeserializedObject: Chess enthusiast>
<DeserializedObject: Experienced player >
<DeserializedObject: Smart player>
Problem installing fixture '/Users/oleg/probs.json': Traceback (most recent call last):
File "/opt/local/lib/python2.5/site-packages/django/core/management/commands/loaddata.py", line 153, in handle
print obj
File "/opt/local/lib/python2.5/site-packages/django/core/serializers/base.py", line 155, in __repr__
return "<DeserializedObject: %s>" % smart_str(self.object)
File "/opt/local/lib/python2.5/site-packages/django/utils/encoding.py", line 107, in smart_str
return str(s)
File "/opt/local/lib/python2.5/site-packages/django/db/models/base.py", line 335, in __str__
return force_unicode(self).encode('utf-8')
File "/opt/local/lib/python2.5/site-packages/django/utils/encoding.py", line 71, in force_unicode
s = unicode(s)
File "/Users/oleg/Sites/probsbox/registration/models.py", line 58, in __unicode__
return u"%s's profile" %(self.user.username)
File "/opt/local/lib/python2.5/site-packages/django/db/models/fields/related.py", line 257, in __get__
rel_obj = QuerySet(self.field.rel.to).get(**params)
File "/opt/local/lib/python2.5/site-packages/django/db/models/query.py", line 305, in get
% self.model._meta.object_name)
DoesNotExist: User matching query does not exist.
silver:probsbox oleg$
Error from very last comment
<DeserializedObject: qwert2000's profile>
Problem installing fixture '/Users/oleg/probs.json': Traceback (most recent call last):
File "/opt/local/lib/python2.5/site-packages/django/core/management/commands/loaddata.py", line 154, in handle
obj.save()
File "/opt/local/lib/python2.5/site-packages/django/core/serializers/base.py", line 163, in save
models.Model.save_base(self.object, raw=True)
File "/opt/local/lib/python2.5/site-packages/django/db/models/base.py", line 495, in save_base
result = manager._insert(values, return_id=update_pk)
File "/opt/local/lib/python2.5/site-packages/django/db/models/manager.py", line 177, in _insert
return insert_query(self.model, values, **kwargs)
File "/opt/local/lib/python2.5/site-packages/django/db/models/query.py", line 1087, in insert_query
return query.execute_sql(return_id)
File "/opt/local/lib/python2.5/site-packages/django/db/models/sql/subqueries.py", line 320, in execute_sql
cursor = super(InsertQuery, self).execute_sql(None)
File "/opt/local/lib/python2.5/site-packages/django/db/models/sql/query.py", line 2369, in execute_sql
cursor.execute(sql, params)
File "/opt/local/lib/python2.5/site-packages/django/db/backends/util.py", line 19, in execute
return self.cursor.execute(sql, params)
ProgrammingError: can't adapt
The can't adapt error is raised by psycopg2 when it receives an data type that it doesn't know how to translate into a value for a SQL statement. For example, if you accidentally pass a list, say, for a value that is supposed to be an integer, psycopg2 will raise this can't adapt error.
The faq.txt document that ships with the source distribution of psycopg2 explains it this way:
Why does !cursor.execute() raise the exception can't adapt?
Psycopg converts Python objects in a SQL string representation by looking
at the object class. The exception is raised when you are trying to pass
as query parameter an object for which there is no adapter registered for
its class. See :ref:adapting-new-types for informations.
Probably your best first-pass at finding the offending value is to run loaddata in fully verbose mode: python manage.py loaddata --verbosity=2 /Users/oleg/probs.json
Well, I was hoping loaddata verbosity would work and I wouldn't have to confess that I've never found an elegant way of debugging adaptation errors with django's loaddata. In the past, I've resorted to inserting print statements in django's loaddata function so that I can see the values being deserialized when the error occurs. I've edited django/core/management/loaddata.py. Look of obj.save() in the handle() function. I hope this confession inspires someone to share a better solution :-)
Ok, I ended copying dump from my database, and restoring it locally without python...
I'm trying to set up OpenID authentication in Django, using django-authopenid.
The instructions are pretty good, but having followed them and made all the requisite changes in settings.py and added the required templates, my whole site is now showing a 500 error, having previously worked fine. The Apache logs show:
Exception occurred processing WSGI script '/usr/local/www/wsgi-scripts/myapp.wsgi'.
Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/django/core/handlers/wsgi.py", line 241, in __call__
response = self.get_response(request)
File "/usr/local/lib/python2.6/dist-packages/django/core/handlers/base.py", line 73, in get_response
response = middleware_method(request)
File "/usr/local/lib/python2.6/dist-packages/django_authopenid-1.0.1-py2.6.egg/django_authopenid/middleware.py", line 36, in process_request
request.associated_openids = [rel.openid_url for rel in rels]
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 93, in _result_iter
self._fill_cache()
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 660, in _fill_cache
self._result_cache.append(self._iter.next())
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 207, in iterator
for row in self.query.results_iter():
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/query.py", line 287, in results_iter
for rows in self.execute_sql(MULTI):
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/query.py", line 2345, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python2.6/dist-packages/django/db/backends/util.py", line 19, in execute
return self.cursor.execute(sql, params)
ProgrammingError: relation "django_authopenid_userassociation" does not exist
Looks like a SQL error (I'm not a django expert)?
It's possible I've put my templates in the wrong place, the instructions aren't very clear. I just added two new directories, registration and openauthid, in the main templates folder.
Bit baffled - can anyone help? Thanks!
It looks to me like you've not yet set up the required tables. Try running:
python manage.py syncdb
from the project directory.