Spellcheck using Solr and Haystack not showing any results - django

Our Solr build is functioning as
http://192.168.1.106:8983/solr/spell?q=query&spellcheck=true&spellcheck.build=true
does successfully return spelling suggestions based off our index based dictionary
However the django-haystack variable {{suggestion}} or even python command SearchQuerySet().spelling_suggestion("query") return "None".
We use the standard view and url provided by haystack.
The install apps are
Python 2.7.2, Django 1.3.2,
Haystack 2.0, Apache Solr 3.6.1 (running on standard Jetty), PySolr 2.1.
Here is some of the code we are using:
In settings.py
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.solr_backend.SolrEngine',
'URL': 'http://192.168.1.106:8983/solr',
'INCLUDE_SPELLING': True,
},
}
In /PATH/TO/SOLR/example/solr/conf/solrconfig.xml:
<searchComponent name="spellcheck" class="solr.SpellCheckComponent">
<str name="queryAnalyzerFieldType">textSpell</str>
<lst name="spellchecker">
<str name="name">default</str>
<str name="field">text</str>
<str name="spellcheckIndexDir">spellchecker</str>
</lst>
</searchComponent>
<requestHandler name="/spell" class="solr.SearchHandler" startup="lazy">
<lst name="defaults">
<str name="df">text</str>
<str name="spellcheck.onlyMorePopular">false</str>
<str name="spellcheck.extendedResults">false</str>
<str name="spellcheck.count">10</str>
</lst>
<arr name="last-components">
<str>spellcheck</str>
</arr>
</requestHandler>
So the question is: where is the issue in the code to cause the installed app 'haystack' to not communicate with the results for spelling suggestions Solr is finding? Or in otherwords, why does haystack show no spelling suggestions while Solr provides some?

Do you have the INCLUDE_SPELLING setting defined as True in the CONNECTIONS in your Django settings file? http://django-haystack.readthedocs.org/en/latest/searchqueryset_api.html#spelling-suggestion
One thing that may be helpful is to see exactly what Haystack is sending to Solr. You can add a print statement into the SolrBackend class in Haystack's backends/solr_backend.py in the search function so you can see the URL being used. That would at least show you if Haystack is doing the search suggestion as ordered.
You may also want to check for Haystack updates direct from the Github repo. The development there is pretty active.

I was struggling with this error, and spelling_suggestion wasn't showing nothing, until I add to request handler "/select" the component spellchecker. So the default connection is as said above and the '/select' handler goes like this:
<requestHandler name="/select" class="solr.SearchHandler">
<lst name="defaults">
<str name="echoParams">explicit</str>
<int name="rows">10</int>
<str name="df">text</str>
</lst>
<arr name="last-components">
<str>spellcheck</str>
</arr>
Adittionally, I also add as attribute of my RestaurantIndex at search_indexes.py, something like:
#Suggestions - so obvious
suggestions = indexes.CharField()
def prepare(self, obj):
prepared_data = super(RestaurantIndex, self).prepare(obj)
prepared_data['suggestions'] = prepared_data['text']
return prepared_data
In this case 'text' will be your own field that will be suggestions.
After this, in python shell you could execute this and results would came up:
sqs = SearchQuerySet().auto_query('Restauront')
spelling = sqs.spelling_suggestion()
My suggestion to this is restaurant.
Cheers!
PS: If you need extra config, just say it.

Related

Tags with an underscore cause failure with BeautifulSoup selector

XML File:
<?xml version="1.0" encoding="UTF-8"?>
<sites>
<site>
<name>Default</name>
<url_namespace>default</url_namespace>
</site>
</sites>
Soup info:
soup = BeautifulSoup(xml)
soup.select('url_namespace')
Error:
ValueError: Unsupported or invalid CSS selector: "url_namespace"
How does one select an xml tag, or and id which contains an underscore?
I'd suggest lxml just because this could be done with a simple XPath, but just for the fun of showing how to select an invalid CSS selector... well, you actually don't. There are a couple of things that can be done, one of which is to replace the offensive tag with perhaps a div tag with a specific class, so you can select it.
However, one really hackish way of doing this really quickly is to just change the name property of each element you find.
from bs4 import BeautifulSoup as bsoup
data = """
<?xml version='1.0' encoding='UTF-8'?>
<sites>
<site>
<name>Default</name>
<url_namespace>default1</url_namespace>
<url_namespace>default2</url_namespace>
<url_namespace>default3</url_namespace>
<url_namespace>default4</url_namespace>
</site>
</sites>
"""
soup = bsoup(data)
elements = soup.find_all("url_namespace")
for element in elements:
element.name = "urlnamespace"
print soup
The above changes the soup to the following:
<html><body><sites>
<site>
<name>Default</name>
<urlnamespace>default1</urlnamespace>
<urlnamespace>default2</urlnamespace>
<urlnamespace>default3</urlnamespace>
<urlnamespace>default4</urlnamespace>
</site>
</sites>
</body></html>
Adding the following codeblock to the above code...
targets = soup.select("urlnamespace")
for target in targets:
print target.get_text()
... gives you the following result:
default1
default2
default3
default4
Not really the prettiest way, but it works. Out of sheer curiosity, though, why the need to select the tag this way? find_all works on the tag, as you can see above.
Anyway, let us know if this works.

Adding new view to Dexterity type causes "page not found" viewing items

I'm working through the recent Professional Plone 4 Development book, on a Plone 4.1.2 install.
I have successfully defined the content types via Dexterity and am now trying to create a custom view for one of the types. The schema & view are defined as such:
from zope import schema
from plone.directives import form
from five import grok
from ctcc.contenttypes import CTCCTypesMessageFactory as _
class ITrial(form.Schema):
"""A clinical trial."""
title = schema.TextLine(
title = _(u'label_title', default=u'Title'),
required = True,
)
description = schema.Text(
title=_(u'label_description', default=u'Description'),
description = _(u'help_description', default=u'A short summary of the content'),
required = False,
missing_value = u'',
)
class View(grok.View):
grok.context(ITrial)
grok.require('zope2.View')
grok.name('view')
Here is the relevant section from the type's FTI:
view
False
<alias from="(Default)" to="(selected layout)"/>
<alias from="edit" to="##edit"/>
<alias from="sharing" to="##sharing"/>
<alias from="view" to="##view"/>
<action title="View" action_id="view" category="object" condition_expr=""
url_expr="string:${folder_url}/" visible="True">
<permission value="View"/>
</action>
And the template itself, located in ctcc.contenttypes/trial_templates/view.pt, which should simply display the title & description:
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"
xmlns:tal="http://xml.zope.org/namespaces/tal"
xmlns:metal="http://xml.zope.org/namespaces/metal"
xmlns:i18n="http://xml.zope.org/namespaces/i18n"
lang="en"
metal:use-macro="context/main_template/macros/master"
i18n:domain="ctcc.contenttypes">
<body>
<metal:content-core fill-slot="content-core">
<metal:content-core define-macro="content-core">
<div tal:replace="structure context/text/output" />
</metal:content-core>
</metal:content-core>
</body>
</html>
Accessing any instances of the type with all this in place causes a "page not found" error. Something doesn't seem to be tying up the new view to the expected path, but as this is my first week with Plone I've no idea where to begin to track this down. I'm seeing no errors running the site in foreground mode either.
Any help whatsoever would be greatly appreciated.
did you included the dependency in setup.py?
install_requires=[
'setuptools',
'plone.app.dexterity',
...
],
did you initialized Grok in your configure.zcml?
<configure
xmlns="http://namespaces.zope.org/zope"
...
xmlns:grok="http://namespaces.zope.org/grok">
<includeDependencies package="." />
<grok:grok package="." />
...
</configure>
did you included Dexterity's GenericSetup profile in your metadata.xml?
<metadata>
<version>1</version>
<dependencies>
<dependency>profile-plone.app.dexterity:default</dependency>
</dependencies>
</metadata>
The problem was with this line in the template:
<div tal:replace="structure context/text/output" />
I had stripped back an example template to what I thought was the bare minimum. Thanks to David Glick's suggestion, I removed NotFound from the ignored exceptions list in error_log and saw the following:
Module Products.PageTemplates.Expressions, line 225, in evaluateText
Module zope.tales.tales, line 696, in evaluate
- URL: /opt/plone41/zeocluster/src/ctcc.contenttypes/ctcc/contenttypes/trial_templates/view.pt
- Line 13, Column 8
- Expression: <PathExpr standard:u'context/text/output'>
[...]
Module OFS.Traversable, line 299, in unrestrictedTraverse
- __traceback_info__: ([], 'text')
NotFound: text
Now that I can see what's causing the problem and have started reading deeper into TALs, I can see why it's failing: ignorance on my behalf, as suspected.
Thanks, everyone!

Solrj Questions for faceting and json resoponse

I am trying to use solrj for my application, my code is given below,
query.add("q", "simplify360");
query.add("facet", "true");
query.add("facet.range", "createdOnGMTDate");
query.add("facet.range.start", "2010-08-01T00:00:00Z+330MINUTES");
query.add("facet.range.end", "2011-05-31T00:00:00Z+330MINUTES");
query.add("facet.range.gap", "+1DAY");
//query.add("wt","json");
//query.add("wt.mime-type","application/json");
System.err.println(query.toString());
The code executes fine and when i execute the url on solr server, i get the following result for faceting,
<lst name="facet_counts"><lst name="facet_queries"/>
<lst name="facet_fields"/>
<lst name="facet_dates"/>
<lst name="facet_ranges">
<lst name="createdOnGMTDate">
<lst name="counts">
<int name="2010-01-01T00:00:00Z">0</int>
<int name="2010-01-02T00:00:00Z">0</int>
<int name="2010-01-03T00:00:00Z">0</int>
<int name="2010-01-04T00:00:00Z">0</int>
<int name="2010-01-05T00:00:00Z">0</int>
<int name="2010-01-06T00:00:00Z">0</int
</lst>
<str name="gap">+1DAY</str>
<date name="start">2010-01-01T00:00:00Z</date>
<date name="end">2011-05-31T00:00:00Z</date>
</lst>
</lst>
</lst>
</response>
1) How can i retrieve these values in java,
2) Also if there is anyway i can convert the json response to the json java object
Regards,
Rohit
Using SOLRJ API, you can use following code:
QueryResponse rsp = server.query(query); // you issue the query and get resuls
Map<String, Integer> fq = rsp.getFacetQuery(); // this returns facetQuery part of response
Hope this helps ...

Haystack more_like_this returns all

I am using Django, haystack, solr, to do searching. Ive am able to search and now I would like to find similar items using more_like_this. When I try to use the more_like_this functionality I get back all of the objects that are of that model type instead of just the ones that closely match it. Here is some code to show you how I am using it:
def resource_view(request, slug):
resource = Resource.objects.get(slug=slug)
versions = Version.objects.get_for_object(resource)
related = SearchQuerySet().more_like_this(resource)
add_comment_form = AddCommentForm()
return render_to_response('resources/resource.html',
{'resource': resource,
'versions': versions,
'related': related,
'add_comment_form': add_comment_form},
context_instance=RequestContext(request))
Apparently I need to enable mlt in the solrconfig.xml file. Anyone know how to do this, or an article/tutorial that is helpful?
stale question, but here's the answer anyway:
As John already pointed out, you need to add the more like this handler (MLT) to your solr config. This should do, put it somewhere in your solrconfig.xml, and reload SOLR (Tomcat):
<requestHandler name="/mlt" class="solr.MoreLikeThisHandler">
<lst name="defaults">
<str name="mlt.mintf">1</str>
<str name="mlt.mindf">1</str>
<str name="mlt.minwl">3</str>
<str name="mlt.maxwl">15</str>
<str name="mlt.maxqt">20</str>
<str name="mlt.match.include">false</str>
</lst>
</requestHandler>

Download specific version of a file from SharePoint using web services

Can someone provide a sample for downloading a specific version of a file from SharePoint using web services? Normally I would get the file with the Copy Web Service (.../_vti_bin/copy.asmx). But I don't really know how to specify a version.
Regards
Anton Kalcik
You can use the Versions web service to get which versions exist for a File
The GetVersions method will give you an xml like this:
<results xmlns="http://schemas.microsoft.com/sharepoint/soap/">
<list id="{26E516B0-8241-4B97-984D-000000000000}" />
<versioning enabled="1" />
<settings url="http://Server_Name/Site_Name/_layouts/
1033/LstSetng.aspx?
List={26E516B0-8241-4B97-984D-000000000000}" />
<result version="#4" url="http://Server_Name/Site_Name/
Shared Documents/File_Name.doc"
created="6/7/2003 5:55 PM" createdBy="DOMAIN\User" size="19968"
comments="" />
<result version="1" url="http://Server_Name/Site_Name/
_vti_history/1/Shared Documents/File_Name.doc"
created="6/7/2003 5:49 PM" createdBy="DOMAIN\User" size="19968"
comments="" />
.
.
.
</results>
You can then just use a HTTP GET request for the content of the "url" attribute of the "result" node for the right version
Here's the exact thing you want to do, with code:
Article