indexing all documents in doc folder in to solr FileListEntityProcessor - solrj

http://wiki.apache.org/solr/ExtractingRequestHandler does not provide much information how to configure this handler in an webapplication which has its own context and want to use solr as server features as embebdedd solr .
Can you please provide some information on how to upload the documents to solr and search for some content from those documents?
I have configured DIH as in solrConf.xml
<requestHandler name="/dataimport"
class="org.apache.solr.handler.dataimport.DataImportHandler">
<lst name="defaults">
<str name="config">tika-data-config.xml</str>
</lst>
</requestHandler>
and tika-data-config.xml looks like
<dataConfig>
<dataSource type="BinFileDataSource" name="bin" />
<document>
<entity name="sd"
processor="FileListEntityProcessor"
newerThan="'NOW-30DAYS'"
filenName=".*\.(DOC)|(PDF)|(pdf)|(doc)|(docx)|(ppt)"
baseDir="G:/workspace/FacetedSearch/src/solr/docs"
recursive="true"
rootEntity="false"
>
<field column="fileAbsolutePath" name="path" />
<field column="fileSize" name="size" />
<field column="fileLastModified" name="lastmodified" />
<field column="fileAbsolutePath" name="text" />
<!-- <field column="fileName" name="text" /> -->
<field column="baseDir" name="text" />
<!-- <entity name="tika-test" processor="TikaEntityProcessor"
url="${sd.fileAbsolutePath}" format="text" dataSource="bin">
-->
<entity name="tika-test"
dataSource="bin"
processor="TikaEntityProcessor"
url="G:/workspace/FacetedSearch/src/solr/docs"
format="text" >
<field column="Author" name="author" meta="true"/>
<field column="Content-Type" name="title" meta="true"/>
<field column="title" name="title" meta="true"/>
<field column="text" name="text"/>
</entity>
</entity>
</document>
</dataConfig>
the dir G:/workspace/FacetedSearch/src/solr/docs contains many pdf and html files
some of them are tutorial.pdf......index.pdf
after this configuration when i build solrQuery object as
CoreContainer.Initializer initializer = new CoreContainer.Initializer();
CoreContainer coreContainer = initializer.initialize();
EmbeddedSolrServer solrServer = new EmbeddedSolrServer(coreContainer, "");
SolrQuery solrQuery = new SolrQuery();
solrQuery.addField("literal.id");
solrQuery.setQuery("index.pdf");
QueryResponse queryResponse = null ;
try{
queryResponse = (QueryResponse) solrServer.query(solrQuery);
}catch(Exception e){
System.out.println("exception occured while processing the solrQuery "+
e.getMessage() +"stack trace " + e + solrQuery.toString());
}
out.println(queryResponse);
i do not get any result (here queryResponse is null).
I have the schema.xml distributed by solr 3.5 and added some fields as
<field name="path" type="text_general" indexed="true" stored="true" />
<field name="lastmodified" type="date" indexed="true" stored="true" />
I have question like are the documents in "G:/workspace/FacetedSearch/src/solr/docs"
will be indexed by solr on solr startup?
If these are indexed how can i get the result?
Can any one please let me know where i am doing wrong?
Please let me know if any more information needed from me in getting my answeres.

Related

How to change the colour of a field in view odoo?

I want to change a field colour based on its value.its a selection field.
field name--status
success--green
failed--red
manual--black
I tried following codes but failed to get desired output:
1.
<field name="status" readonly="1" attrs="{'invisible':[('status','!=','manual')]}"/>
<field name="status" style="color:green;" readonly="1" attrs="{'invisible':[('status','!=','success')]}"/>
<field name="status" style="color:red;" readonly="1" attrs="{'invisible':[('status','!=','failed')]}"/>
When I executed tthis code,view shows string name but no value
2.
<field name="status" readonly="1" options='{"fg_color": "red:status == "failed"; green:status == "success"; black:status == "manual"}'/>
I got an error while running this code :expected } got(name)
3.
<field name="status" readonly="1" colors=="red:status == 'failed'; green:status == 'success'; black:status == 'manual'"/>
No colours appeared when I executed this code
<field name="date" attrs="{'invisible': [('state','in', ('start','progress','close'))]}"/>
<field name="date" attrs="{'invisible': [('state','in', ('draft','progress','close'))]}" style="background-color:palegreen;" />
<field name="date" attrs="{'invisible': [('state','in', ('draft','start','close'))]}" style="background-color:red;" />
<field name="date" attrs="{'invisible': [('state','in', ('draft','progress','start'))]}" style="background-color:yellow;" />
You change field name and status according your requirements. It's work fine for me.

Solr DIH regextransformer - processes only one CSV line

Hi I have the following CSV file
132 1536130302256087040
133 1536130302256087041
134 1536130302256087042
the fields are seperated by a tab.
Now I have the Dataimporthandler (DIH) for the solr, and I try to import the CSV into solr, but I only get the first line into solr. Thats the result, but the other lines from the CSV are missing:
"response": {
"numFound": 1,
"start": 0,
"maxScore": 1,
"docs": [ {
"string": "1536130302256087040",
"id": "132",
"_version_": 1536202153221161000
} ] }
Here is my data-config.xml
<dataConfig>
<dataSource type="FileDataSource" encoding="UTF-8" name="fds"/>
<document>
<entity name="f"
processor="FileListEntityProcessor"
fileName="myfile.csv"
baseDir="/var/www/solr-5.4.0/server/csv/files"
recursive="false"
rootEntity="true"
dataSource="null" >
<entity
onError="continue"
name="jc"
processor="LineEntityProcessor"
url="${f.fileAbsolutePath}"
dataSource="fds"
rootEntity="true"
header="false"
separator="\t"
transformer="RegexTransformer" >
<field column="id" name="id" sourceColName="rawLine" regex="^(.*)\t"/>
<field column="string" name="string" sourceColName="rawLine" regex="\t(.*)$"/>
</entity>
</entity>
</document>
</dataConfig>
Here is my schema.xml
<field name="id" type="text_general" indexed="true" stored="true" multiValued="false" required="true"/>
<field name="string" type="text_general" indexed="true" stored="true" multiValued="false"/>
<field name="_version_" type="long" indexed="true" stored="true"/>
<uniqueKey>id</uniqueKey>
What I'm doing wrong?
You have rootEntity=true for both levels of entities. So, you will only get one document for the outer entity. Try setting the outer level rootEntity to false.
Also, you can just send tab-separated files to the Solr with CSV processor, no DIH required.

Not able to add columns to Document Library from Visual Studio template

I am trying to create new document library from SharePoint project template from Visual Studio 2015.
While doing so I am able to create Document Library Template and Instance. Also if I deploy solution to one of my site it creates list as well. Strange thing is if I add a column to the template and then deploy it creates the list but does not create new custom column.
Below is the schema file for the template.
<?xml version="1.0" encoding="utf-8"?>
<List xmlns:ows="Microsoft SharePoint" Title="SharedDocuments" Direction="$Resources:Direction;" Url="SharedDocuments" BaseType="1" xmlns="http://schemas.microsoft.com/sharepoint/" EnableContentTypes="TRUE">
<MetaData>
<ContentTypes>
<ContentType ID="0x0101000dba2955561e4de0a115152637e9f8e2" Name="ListFieldsContentType">
<FieldRefs>
<FieldRef ID="{fa564e0f-0c70-4ab9-b863-0177e6ddd247}" Name="Title" />
<FieldRef ID="{4b1bf6c6-4f39-45ac-acd5-16fe7a214e5e}" Name="TemplateUrl" />
<FieldRef ID="{cd1ecb9f-dd4e-4f29-ab9e-e9ff40048d64}" Name="xd_ProgID" />
<FieldRef ID="{fbf29b2d-cae5-49aa-8e0a-29955b540122}" Name="xd_Signature" />
<FieldRef ID="{e52012a0-51eb-4c0c-8dfb-9b8a0ebedcb6}" Name="Combine" />
<FieldRef ID="{5d36727b-bcb2-47d2-a231-1f0bc63b7439}" Name="RepairDocument" />
<FieldRef ID="{8f838bdc-4b95-4b26-b86f-3025d2df964f}" Name="TestColumn" />
</FieldRefs>
</ContentType>
<ContentTypeRef ID="0x0101">
<Folder TargetName="Forms/Document" />
</ContentTypeRef>
<ContentTypeRef ID="0x0120" />
</ContentTypes>
<Fields>
<Field ID="{fa564e0f-0c70-4ab9-b863-0177e6ddd247}" Type="Text" Name="Title" ShowInNewForm="FALSE" ShowInFileDlg="FALSE" DisplayName="$Resources:core,Title;" Sealed="TRUE" SourceID="http://schemas.microsoft.com/sharepoint/v3" StaticName="Title"></Field>
<Field ID="{4b1bf6c6-4f39-45ac-acd5-16fe7a214e5e}" Type="Text" Name="TemplateUrl" DisplaceOnUpgrade="TRUE" DisplayName="$Resources:core,Template_Link;" XName="TemplateUrl" Filterable="TRUE" Sortable="TRUE" Hidden="TRUE" FromBaseType="TRUE" PITarget="mso-infoPathSolution" PIAttribute="href" SourceID="http://schemas.microsoft.com/sharepoint/v3" StaticName="TemplateUrl"></Field>
<Field ID="{cd1ecb9f-dd4e-4f29-ab9e-e9ff40048d64}" Type="Text" Name="xd_ProgID" DisplaceOnUpgrade="TRUE" DisplayName="$Resources:core,Html_File_Link;" XName="ProgID" Filterable="TRUE" Sortable="TRUE" Hidden="TRUE" FromBaseType="TRUE" SourceID="http://schemas.microsoft.com/sharepoint/v3" StaticName="xd_ProgID"></Field>
<Field ID="{fbf29b2d-cae5-49aa-8e0a-29955b540122}" Type="Boolean" Group="_Hidden" Name="xd_Signature" DisplaceOnUpgrade="TRUE" SourceID="http://schemas.microsoft.com/sharepoint/v3" StaticName="xd_Signature" DisplayName="$Resources:core,Xml_signed;" XName="{FBF29B2D-CAE5-49aa-8E0A-29955B540122}" Filterable="TRUE" Sortable="TRUE" Sealed="TRUE" Hidden="TRUE" FromBaseType="TRUE" ReadOnly="TRUE"></Field>
<Field ID="{e52012a0-51eb-4c0c-8dfb-9b8a0ebedcb6}" ReadOnly="TRUE" Type="Computed" Name="Combine" DisplaceOnUpgrade="TRUE" DisplayName="$Resources:core,Merge;" Filterable="FALSE" Sortable="FALSE" Hidden="TRUE" SourceID="http://schemas.microsoft.com/sharepoint/v3" StaticName="Combine">
<FieldRefs>
<FieldRef Name="FSObjType" Key="Primary" />
<FieldRef Name="EncodedAbsUrl" />
<FieldRef Name="TemplateUrl" />
</FieldRefs>
</Field>
<Field ID="{5d36727b-bcb2-47d2-a231-1f0bc63b7439}" ReadOnly="TRUE" Type="Computed" Name="RepairDocument" DisplaceOnUpgrade="TRUE" DisplayName="$Resources:core,Relink;" Filterable="FALSE" Sortable="FALSE" Hidden="TRUE" SourceID="http://schemas.microsoft.com/sharepoint/v3" StaticName="RepairDocument">
<FieldRefs>
<FieldRef Name="FSObjType" Key="Primary" />
<FieldRef Name="ID" />
</FieldRefs>
</Field>
<Field Name="TestColumn" ID="{8f838bdc-4b95-4b26-b86f-3025d2df964f}" DisplayName="TestColumn" Type="Text" />
</Fields>
<Views>
--Removed--
</Views>
<Forms>
<Form Type="DisplayForm" SetupPath="pages\form.aspx" Url="Forms/DispForm.aspx" WebPartZoneID="Main" />
<Form Type="EditForm" SetupPath="pages\form.aspx" Url="Forms/EditForm.aspx" WebPartZoneID="Main" />
<Form Type="NewForm" Url="Forms/Upload.aspx" WebPartZoneID="Main" />
--Removed--
</Forms>
</MetaData>
</List>
When you upload a document in a Document Library there's an option for you to choose the Content Type, choose your custom Content Type and you'll see the fields you defined.

Quickfix MarketDataIncrementalRefresh: Tag not defined for this message type

I use Quickfix c++.
I receive FIX message from the server:
8=FIX.4.2|9=312|35=X|34=17|49=CX|52=20150413-13:33:39.691|56=CACIB_MD2|262=1|9883=IDB|268=2|279=2|269=0|55=535791|270=15|271=1|272=20150413|273=13:28:07.653|59=0|126=20150413-21:56:46.201|290=1|9139=cacib_uat2|279=0|269=0|55=535791|270=17|271=1|272=20150413|273=13:33:39.660|59=0|126=20150413-21:56:46.201|290=1|9139=cacib_uat2|10=015|
and Quickfix automatically sends back to the server reject message (tag 55 not defined for this message type):
8=FIX.4.2|9=121|35=3|34=17|49=CACIB_MD2|52=20150413-13:33:39.755|56=CX|45=17|58=Tag not defined for this message type|371=55|372=X|373=2|10=044|
But this tag (55) is defined in the dictionary file:
<message name='MarketDataIncrementalRefresh' msgcat='app' msgtype='X'>
<field name='MDReqID' required='N' />
<field name='CXMarketType' required='N' />
<group name='NoMDEntries' required='N'>
<field name='Symbol' required='N' />
</group>
</message>
...
<field number='55' name='Symbol' type='STRING' />
...
<field number='262' name='MDReqID' type='STRING' />
...
<field number='268' name='NoMDEntries' type='INT' />
...
<field number='9883' name='CXMarketType' type='STRING' />
...
And FIX options are:
UseDataDictionary=Y
DataDictionary=./conf/fix42_dictionary.xml
ValidateFieldsOutOfOrder=N
ValidateUserDefinedFields=N
I don't understand why Quickfix decides that tag 55 (symbol) is not defined, when it is part of the group.
Can you please help?
Problem is you can't directly add groups to messages, it should be inside a component first
<message name='MarketDataIncrementalRefresh' msgcat='app' msgtype='X'>
<field name='MDReqID' required='N' />
<field name='CXMarketType' required='N' />
<component name='NoMDEntriesComp' required='N' />
</message>
<component name="NoMDEntriesComp">
<group name='NoMDEntries' required='N'>
<field name='Symbol' required='N' />
</group>
</component>
NoMDEntries type should be NUMINGROUP instead of INT
<field number='268' name='NoMDEntries' type='NUMINGROUP' />
I faced same issue.It is QuickFix incorrect reporting of the error. It is necessary to define the user defined fields in the FIX4?.xml. Define the fields in the section and add them to the appropriate message as well.In your case 9883 is look like custom tag you have declared it but not defined in 35=X message properly. So if you use this custom tag in 35=X message inside NoMDEntries can solve your problem.
<message name='MarketDataIncrementalRefresh' msgcat='app' msgtype='X'>
<field name='MDReqID' required='N' />
<group name='NoMDEntries' required='N'>
<field name='Symbol' required='N' />
<field name='CXMarketType' required='N' />
</group>
</message>

how to get only the children nodelist using getElementByTagName?

I have this XML File from which I want to extract my message children fields
<message name="IOI" >
<field number="23" name="IOIID" type="STRING" required="Y" />
<field number="28" name="IOITransType" type="CHAR" required="Y" >
<value enum="C" description="CANCEL" />
<value enum="N" description="NEW" />
<value enum="R" description="REPLACE" />
</field>
<field number="26" name="IOIRefID" type="STRING" required="N" />
<component name="SecAltIDGrp" required="N" >
<field number="454" name="NoSecurityAltID" type="NUMINGROUP" required="N" >
<field number="455" name="SecurityAltID" type="STRING" required="N" />
<field number="456" name="SecurityAltIDSource" type="STRING" required="N" />
</field>
</component>
so I used getElementsByTagName() but it gave me all children with their children, so how could I only get my message fields without my fields's fields ??
DOMNodeList* fieldsNodeList = MessageElement->getElementsByTagName(xmlStrVar);
Xerces C++ provides partial XPath implementation. And here's an SO question with more details about it. With that said, depending on the version of Xerces C++ you're using, you should be able to run simple XPath expressions like /message/field.
Another option is to stay with the DOM API and just iterate through the child nodes using getChildNodes() available on the DOMNode looking for field elements.
Last but not the least, if you liked the XPath idea but your Xerces C++ version is not up to it, consider stepping out of your "comfort zone" and look at Xalan C++ or libxml2. Both should have full XPath 1.0 support.