MapReduce for Loading TSV - mapreduce

I am trying to load data into Cloud Bigtable using mapreduce job such that the values of the first column are set to hbase row key.Here is my code and a sample TSV file.
1011 v1 v2 v3 v4
1012 c1 c2 c3 c4
1013 k1 k2 k3 k4
1014 s1 s2 s3 s4
1015 r1 r3 r2 r4
1016 p1 p2 p7 p9
Here is my code sample :-
public static class TokenizerMapper extends
Mapper<Text, Text, Text,Text> {
#Override
public void map(Text key, Text value, Context context) throws IOException,
InterruptedException {
String fields[] = null;
CSVParser csvParser = new CSVParser('\t');
fields = csvParser.parseLine(value.toString());
LOG.info(fields[0]);
context.write(new Text(fields[0]), value);
}
}
public static class MyTableReducer extends
TableReducer<Text, Text, Text> {
#Override
public void reduce(Text key, Iterable<Text> values, Context context)
throws IOException, InterruptedException {
String[] fields = null;
CSVParser csvParser = new CSVParser('\t');
try {
for(Text value: values) {
fields = csvParser.parseLine(value.toString());
for (int i = 1; i < fields.length; ++i) {
Put put = new Put(Bytes.toBytes(fields[0]));
put.addColumn(COLUMN_FAMILY, Bytes.toBytes(cols[i]), Bytes.toBytes(fields[i]));
context.write(key, put);
}
}
} catch (Exception ex) {
context.getCounter("HBaseKVMapper", "PARSE_ERRORS").increment(1);
return;
}
}
}
public static void main(String[] args) throws Exception {
Configuration conf = HBaseConfiguration.create();
conf.set("mapreduce.input.keyvaluelinerecordreader.key.value.separator", "\t");
Job job = Job.getInstance(conf, "BigTableLoader");
job.setInputFormatClass(KeyValueTextInputFormat.class);
KeyValueTextInputFormat.addInputPath(job,new Path(args[0]));
TableName tableName = TableName.valueOf(args[1]);
job.setJarByClass(BigtableLoader.class);
job.setMapperClass(TokenizerMapper.class);
job.setMapOutputValueClass(Text.class);
job.setMapOutputKeyClass(Text.class);
TableMapReduceUtil.initTableReducerJob(tableName.getNameAsString(), MyTableReducer.class, job);
System.exit(job.waitForCompletion(true) ? 0 : 1);
}
}
The problem I am getting is that instead of the first column, the second column has been set as the key in the table, and also the first column is ignored completely i.e. it doesn't exist in the table.
Is there that something I am missing??

Related

How do i perform operations on a structures field?

I im currently having an issue with when performing an operation with my structure.
On form load, I am loading a comma delimited file with 6 indexes then using a 2d array i display the structured list's field to a grid view cell.
In the last for loop, perform the operation using my total field?
{
public partial class Form1 : Form
{
public struct Costomers
{
public string firstN;
public string lastN;
public string address;
public string item;
public double price;
public int quantity;
public double total;
}
public Form1()
{
InitializeComponent();
}
List<Costomers> consumers = new List<Costomers>();
private void Form1_Load(object sender, EventArgs e)
{
var inputfile = File.ReadAllLines("customers.txt");
for (int i = 0; i < inputfile.Length; i++)
{
var SplitArr = inputfile[i].Split(',');
// next declare a new instace of costomers
Costomers ConsumerInfo = new Costomers();
// the ConsumerInfo object now contains all of the fields i declared in the structer.
// next i want to access the Objects fields and assign them
ConsumerInfo.firstN = SplitArr[0];
ConsumerInfo.lastN = SplitArr[1];
ConsumerInfo.address = SplitArr[2];
ConsumerInfo.item = SplitArr[3];
ConsumerInfo.price = Convert.ToDouble(SplitArr[4]);
ConsumerInfo.quantity = Convert.ToInt32(SplitArr[5]);
//ConsumerInfo.total = consumers[i].price * consumers[i].quantity;
consumers.Add(ConsumerInfo);
}
// Next i need to disply each field in the grid view
// i will resue my code from section F to do this.
for (int i = 0; i < consumers.Count; i++)
{
dgInfo.Rows.Add();
dgInfo[0, i].Value = consumers[i].firstN;
dgInfo[1, i].Value = consumers[i].lastN;
dgInfo[2, i].Value = consumers[i].address;
dgInfo[3, i].Value = consumers[i].item;
dgInfo[4, i].Value = consumers[i].price;
dgInfo[5, i].Value = consumers[i].quantity;
}
// next in a seperat for loop i need too mutiply the price by the quantity
for (int i = 0; i < consumers.Count; i++)
{
Costomers totalPrice = new Costomers();
totalPrice.total = consumers[i].price * consumers[i].quantity;
dgInfo[6,i].Value = totalPrice.total;
}
}
}
}```
Second, because the file i load does not contain a index for total; how to i display the total in its own column?
Text file as shown below:
The, Batman, 123 Gotham Drive, bat belt, 193.82,17
The, Joker, 12432 Joker Way, Bat Spray, 19.99, 1022
Cat, Women, 8787 Meow St., Kibbles, 9.99, 4700
The, Penguin, 17 Waddel Ave., pointy cigarettes, 24.99, 51700
sidekick, Robin, 123 Gotham Drive, junior bat belt, 67.80, 10000
Adam, West, 1782 Hollywood Dr., hasbeen kit, 10305018.18, 1

LEVENSHTEIN_DISTANCE doesn't ignore order by comparing of lists

P.P.S. Ok, I founded hier Javers Comparing Lists following comment
There is no concept of move in JaVers list comparison algorithms. After a move there will be two changes reported: ValueAdded and ValueRemoved, just like you have mentioned.
But how then I can recognize, that the list actually has not been changed?
P.S. Even if I get #Entities and #Id to ZasFish, ZasCatchZone and ZasCatchArea I still get
Diff:
1. NewObject{globalId:'my.javers.comparator.ZasFish/2'}
2. ObjectRemoved{globalId:'my.javers.comparator.ZasFish/1'}
I'm trying to compare lists of custom objects. I set LEVENSHTEIN_DISTANCE and created custom comparators. The only difference between objects is the order of values in the lists. I would expect "no changes", but I got ListChange. The result and example is below. What am I doing wrong?
Many thanks and regards,
Andrej
Diff:
1. ListChange{globalId:'my.javers.comparator.ZasFish/', property:'zones', containerChanges:[(2).'my.javers.comparator.ZasCatchZone#7a9273a8'>>'my.javers.comparator.ZasCatchZone#26a7b76d', (1).'my.javers.comparator.ZasCatchZone#4abdb505'>>'my.javers.comparator.ZasCatchZone#7ce6a65d', (0).'my.javers.comparator.ZasCatchZone#1500955a'>>'my.javers.comparator.ZasCatchZone#e874448']}
2. ListChange{globalId:'my.javers.comparator.ZasFish/', property:'areas', containerChanges:[(2).'my.javers.comparator.ZasCatchArea#7113b13f'>>'my.javers.comparator.ZasCatchArea#45820e51', (1).'my.javers.comparator.ZasCatchArea#42d8062c'>>'my.javers.comparator.ZasCatchArea#6043cd28', (0).'my.javers.comparator.ZasCatchArea#cb51256'>>'my.javers.comparator.ZasCatchArea#59906517']}
package my.javers.comparator;
public class ZasCatchArea {
String catchArea;
public String getCatchArea() {
return catchArea;
}
public void setCatchArea(String catchArea) {
this.catchArea = catchArea;
}
}
public class ZasCatchZone {
String catchZone;
public String getCatchZone() {
return catchZone;
}
public void setCatchZone(String catchZone) {
this.catchZone = catchZone;
}
}
public class ZasFish {
String fischName;
List<ZasCatchZone> zones = new ArrayList<ZasCatchZone>();
List<ZasCatchArea> areas = new ArrayList<ZasCatchArea>();
public String getFischName() {
return fischName;
}
public void setFischName(String fischName) {
this.fischName = fischName;
}
public List<ZasCatchZone> getZones() {
return zones;
}
public void setZones(List<ZasCatchZone> zones) {
this.zones = zones;
}
public List<ZasCatchArea> getAreas() {
return areas;
}
public void setAreas(List<ZasCatchArea> areas) {
this.areas = areas;
}
}
public class ZasCatchAreaComparator implements
CustomPropertyComparator<ZasCatchArea, ValueChange> {
public ValueChange compare(ZasCatchArea left, ZasCatchArea right,
GlobalId affectedCdoId, Property propertyName) {
if (left.getCatchArea().equals(right.getCatchArea()))
return null;
return new ValueChange(affectedCdoId, propertyName.getName(), left, right);
}
}
public class ZasCatchZoneComparator implements
CustomPropertyComparator<ZasCatchZone, ValueChange> {
public ValueChange compare(ZasCatchZone left, ZasCatchZone right,
GlobalId affectedCdoId, Property propertyName) {
if (left.getCatchZone().equals(right.getCatchZone()))
return null;
return new ValueChange(affectedCdoId, propertyName.getName(), left, right);
}
}
public class MyComparator {
public static void main(String[] args) {
Javers javers = JaversBuilder.javers()
.registerCustomComparator(new ZasCatchAreaComparator(), ZasCatchArea.class)
.registerCustomComparator(new ZasCatchZoneComparator(), ZasCatchZone.class)
.withListCompareAlgorithm(ListCompareAlgorithm.LEVENSHTEIN_DISTANCE).build();
ZasFish fisch1 = new ZasFish();
ZasFish fisch2 = new ZasFish();
ZasCatchZone z1 = new ZasCatchZone();
z1.setCatchZone("zone1");
ZasCatchZone z2 = new ZasCatchZone();
z2.setCatchZone("zone2");
ZasCatchZone z3 = new ZasCatchZone();
z3.setCatchZone("zone3");
fisch1.getZones().add(z1);
fisch1.getZones().add(z2);
fisch1.getZones().add(z3);
ZasCatchArea a1 = new ZasCatchArea();
a1.setCatchArea("area1");
ZasCatchArea a2 = new ZasCatchArea();
a2.setCatchArea("area2");
ZasCatchArea a3 = new ZasCatchArea();
a3.setCatchArea("area3");
fisch1.getAreas().add(a1);
fisch1.getAreas().add(a2);
fisch1.getAreas().add(a3);
ZasCatchZone z4 = new ZasCatchZone();
z4.setCatchZone("zone3");
ZasCatchZone z5 = new ZasCatchZone();
z5.setCatchZone("zone2");
ZasCatchZone z6 = new ZasCatchZone();
z6.setCatchZone("zone1");
fisch2.getZones().add(z4);
fisch2.getZones().add(z5);
fisch2.getZones().add(z6);
ZasCatchArea a4 = new ZasCatchArea();
a4.setCatchArea("area3");
ZasCatchArea a5 = new ZasCatchArea();
a5.setCatchArea("area1");
ZasCatchArea a6 = new ZasCatchArea();
a6.setCatchArea("area2");
fisch2.getAreas().add(a4);
fisch2.getAreas().add(a5);
fisch2.getAreas().add(a6);
Diff diff = javers.compare(fisch1, fisch2);
System.out.println(diff);
}
}
I think I founded a solution for my issue. If I register values and value objects like this
final Javers javers = JaversBuilder.javers()
.registerCustomComparator(new ZasCatchAreaComparator(), ZasCatchArea.class)
.registerCustomComparator(new ZasCatchZoneComparator(), ZasCatchZone.class)
.registerValue(ZasCatchArea.class).registerValue(ZasCatchZone.class).registerValueObject(ZasFish.class)
.withListCompareAlgorithm(ListCompareAlgorithm.LEVENSHTEIN_DISTANCE).build();
Then I get as diff
1. ListChange{globalId:'my.javers.comparator.ZasFish/', property:'zones', containerChanges:[(2).'my.javers.comparator.ZasCatchZone#19bef4e5'>>'my.javers.comparator.ZasCatchZone#3f1abed', (1).'my.javers.comparator.ZasCatchZone#e37f307f'>>'my.javers.comparator.ZasCatchZone#2ad1e8a5', (0).'my.javers.comparator.ZasCatchZone#2ccd3209'>>'my.javers.comparator.ZasCatchZone#c0fd1f30']}
2. ListChange{globalId:'my.javers.comparator.ZasFish/', property:'areas', containerChanges:[(2).'my.javers.comparator.ZasCatchArea#48115f4e'>>'my.javers.comparator.ZasCatchArea#a1efa37', (1).'my.javers.comparator.ZasCatchArea#c08d9768'>>'my.javers.comparator.ZasCatchArea#d65a5a2', (0).'my.javers.comparator.ZasCatchArea#bb03583'>>'my.javers.comparator.ZasCatchArea#1ebaaab0']}
And as I don't have any ValueChange in this case I can ignore ListChange -> my Lists are identical.

Error "Expected invocation on the mock once, but was 0 times" when unit testing with moq

I have the following class I want to test:
public interface ISqlServiceByModule
{
DataSet GetPagedAggregateData(int clientId, int moduleId, int billTypeId, PagedTable result);
}
public class IncidentModuleService : IIncidentModuleService
{
private readonly ISqlServiceByModule sqlServiceByModule;
public IncidentModuleService(ISqlServiceByModule sqlServiceByModule)
{
if (sqlServiceByModule == null) throw new ArgumentNullException("sqlServiceByModule");
// Inject the ISqlServiceByModule dependency into the constructor
this.sqlServiceByModule = sqlServiceByModule;
}
public PagedTable GetData(int clientId, int moduleId, int billTypeId, Dictionary<string, string> queryStringParameters)
{
PagedTable result = new PagedTable(queryStringParameters);
DataSet dataSet = this.sqlServiceByModule.GetPagedAggregateData(clientId, moduleId, billTypeId, result);
// Map the DatSet to a PagedTable
if (dataSet == null || dataSet.Tables.Count == 0)
{
result.SetPagesFromTotalItems(0);
}
else
{
result.SetPagesFromTotalItems(Convert.ToInt16(dataSet.Tables[1].Rows[0][0]));
result.Listings = dataSet.Tables[0];
}
return result;
}
}
Specifically, I want to test the GetData method. My unit test looks like this:
[TestClass]
public class IncidentModuleServiceUnitTest
{
private DataSet incidentsData;
[TestInitialize]
public void SetUp()
{
this.incidentsData = new DataSet();
}
[TestMethod]
public void GetDataTestGetPagedAggregateDataIsCalled()
{
//-- Arrange
int billTypeId = 1;
int clientId = 1;
int moduleId = 1;
Dictionary<string, string> queryStringParameters = new Dictionary<string,string>();
PagedTable tempResult = new PagedTable(queryStringParameters);
DataSet dataSet = new DataSet();
dataSet.Tables.Add(new DataTable());
var mockSqlService = new Mock<ISqlServiceByModule>();
mockSqlService.Setup(r => r.GetPagedAggregateData(clientId, moduleId, billTypeId, tempResult)).Returns(this.incidentsData);
IncidentModuleService target = new IncidentModuleService(mockSqlService.Object);
//-- Act
var actual = target.GetData(clientId, moduleId, billTypeId, queryStringParameters);
//-- Assert
Assert.IsNull(actual.Listings);
mockSqlService.Verify(r => r.GetPagedAggregateData(clientId, moduleId, billTypeId, tempResult), Times.Once);
}
}
The error I am getting happens on the last line:
mockSqlService.Verify(r => r.GetPagedAggregateData(clientId, moduleId, billTypeId, tempResult), Times.Once);
And the exact error message is this:
{"\r\nExpected invocation on the mock once, but was 0 times: r =>
r.GetPagedAggregateData(.clientId, .moduleId, .billTypeId, .tempResult
Configured setups:\r\nr => r.GetPagedAggregateData(.clientId,
.moduleId, .billTypeId, .tempResult),
Times.Never
Performed invocations:\r\nISqlServiceByModule.GetPagedAggregateData(1,
1, 1, PagedTable)"}
Any idea why this is happening? It looks to me like the method in question is being called, but Moq doesn't like the parameters for some reason, even though they are the exact same ones in all three invocations, as far as I can tell.
PagedTable is a reference type not a value type. Therefore the parameters in Setup don't match what was called even though they look like they should be the same. You could use It.IsAny<PagedTable>() instead of tempResult.
See this answer for an example of how to check that the PagedTable parameter was the correct one.

MapReduce Hadoop - mapside joining the 2 data sets using customInputFormat

I am learning hadoop mapreduce framework ,i am trying to join 2 data sets have first record(Text) in the line as the Key , i tried to search in stackoverflow previous posts but nothing worked out.Here i am trying to customize the InputFormat and trying to join with the ID which is first record in the each line of data set.
My input file1:
cdd8dde3-0349-4f0d-b97a-7ae84b687f9c,Esther,Garner,4071 Haven Lane,Okemos,MI
81a43486-07e1-4b92-b92b-03d0caa87b5f,Timothy,Duncan,753 Stadium Drive,Taunton,MA
File2 :
cdd8dde3-0349-4f0d-b97a-7ae84b687f9c,517-706-9565,EstherJGarner#teleworm.us,Waskepter38,noL2ieghie,MasterCard,5305687295670850
81a43486-07e1-4b92-b92b-03d0caa87b5f,508-307-3433,TimothyDDuncan#einrot.com,Conerse,Gif4Edeiba,MasterCard,5265896533330445
**Driver class:**
conf.setInputFormat(CompositeInputFormat.class);
String strJoinStmt = CompositeInputFormat.compose("inner",
KeyValueLongInputFormat.class, dirEmployeesData, dirSalaryData);
conf.set("mapred.join.expr", strJoinStmt);
conf.setNumReduceTasks(0);
dirOutput.getFileSystem(conf).delete(dirOutput);
TextOutputFormat.setOutputPath(conf, dirOutput);
conf.setOutputKeyClass(Text.class);
conf.setOutputValueClass(Text.class);
conf.setOutputFormat(TextOutputFormat.class);
**Custom RecordReader class:**
public class KeyValueLongLineRecordReader implements
RecordReader<Text, Text> {
private final LineRecordReader lineRecordReader;
private byte separator = (byte) ',';
private LongWritable dummyKey;
private Text innerValue;
public Class getKeyClass() {
return Text.class;
}
public Text createKey() {
return new Text("");
}
public Text createValue() {
return new Text();
}
public KeyValueLongLineRecordReader(Configuration job, FileSplit split)
throws IOException {
lineRecordReader = new LineRecordReader(job, split);
dummyKey = lineRecordReader.createKey();
innerValue = lineRecordReader.createValue();
String sepStr = job.get("key.value.separator.in.input.line", ",");
this.separator = (byte) sepStr.charAt(0);
}
public static int findSeparator(byte[] utf, int start, int length, byte sep) {
for (int i = start; i < (start + length); i++) {
if (utf[i] == sep) {
return i;
}
}
return -1;
}
/** Read key/value pair in a line. */
public synchronized boolean next(Text key, Text value)
throws IOException {
Text tKey = key;
Text tValue = value;
byte[] line = null;
int lineLen = -1;
if (!lineRecordReader.next(dummyKey, innerValue)) {
return false;
}
else
line = innerValue.getBytes();
lineLen = innerValue.getLength();
if (line == null)
return false;
int pos = findSeparator(line, 0, lineLen, this.separator);
if (pos == -1) {
tKey.set(new String(line, 0, lineLen));
tValue.set("");
} else {
int keyLen = pos;
byte[] keyBytes = new byte[keyLen];
System.arraycopy(line, 0, keyBytes, 0, keyLen);
int valLen = lineLen - keyLen - 1;
byte[] valBytes = new byte[valLen];
System.arraycopy(line, pos + 1, valBytes, 0, valLen);
tKey.set(new String(keyBytes));
tValue.set(valBytes);
}
return true;
}
}
**InputFormat class:**
public class KeyValueLongInputFormat extends
FileInputFormat<Text, Text> implements JobConfigurable {
private CompressionCodecFactory compressionCodecs = null;
#Override
public void configure(JobConf conf) {
compressionCodecs = new CompressionCodecFactory(conf);
}
protected boolean isSplitable(FileSystem fs, Path file) {
return compressionCodecs.getCodec(file) == null;
}
#Override
public RecordReader<Text, Text> getRecordReader(
InputSplit genericSplit, JobConf job, Reporter reporter)
throws IOException {
reporter.setStatus(genericSplit.toString());
return new KeyValueLongLineRecordReader(job, (FileSplit) genericSplit);
}
}
**Finally Mapper class:**
enter code here
public class MapperMapSideJoinLargeDatasets extends MapReduceBase implements
Mapper<Text, TupleWritable, Text, Text> {
Text txtKey = new Text("");
Text txtValue = new Text("");
#Override
public void map(Text key, TupleWritable value,
OutputCollector<Text, Text> output, Reporter reporter)
throws IOException {
if (value.toString().length() > 0) {
txtKey.set(key.toString());
String arrEmpAttributes[] = value.get(0).toString().split(",");
String arrDeptAttributes[] = value.get(1).toString().split(",");
txtValue.set(arrEmpAttributes[1].toString() + "\t"
+ arrEmpAttributes[2].toString() + "\t"
+ arrDeptAttributes[0].toString());
output.collect(txtKey, txtValue);
}
}
In the logs Map input records is 0.No output can be seen on hdfs .Someone please help me to understand this issue. Thanks
The problem lies in the Driver class where in InputFormat mentioned as KeyValueLongInputFormat.class in the strJoinStmt property which actually works for LongWritable key and Text value. Instead KeyValueTextInputFormat.class can be used when both the key and value are of Text type.As the input is comma separated file, also specify a custom delimiter as comma by setting a property in the job's configuration object as follows in the Driver class.conf.set("key.value.separator.in.input.line",",");
For Complete details please check the below example:
https://github.com/sudha-pn/CompositeInputFormat

Is there any way to delete the index of only one row in solr using solrj

I have made a java apllication which can index my last row (which is what I wabt)
But now I ask Is there any wa yo deete the index of this role! Can you give me directions how to do that or maybe simple code to change my code?
;
public class indexSolr {
private Connection conn = null;
private static HttpSolrServer server;
private Collection docs = new ArrayList();
private int _totalSql = 0;
private long _start = System.currentTimeMillis();
public static void main(String[] args) throws SolrServerException, IOException, SQLException
{ String url = "http://localhost:8983/solr/db";
indexSolr idxer = new indexSolr(url);
idxer.doSqlDocuments();
idxer.endIndexing();
}
private void doSqlDocuments() throws SQLException {
try {
Class.forName("org.postgresql.Driver");
conn = DriverManager.getConnection(
"jdbc:postgresql://localhost:5432/biz_cat",
"postgres", "pos");
java.sql.Statement st = null;
st = conn.createStatement();
ResultSet rs = st.executeQuery("select * from pl_biz order by id DESC LIMIT 1");
while (rs.next()) {
SolrInputDocument doc = new SolrInputDocument();
Integer id = rs.getInt("id");
String name = rs.getString("name");
String midname = rs.getString("midname");
String lastname = rs.getString("lastname");
String frlsname = rs.getString("frlsname");
String biz_subject = rs.getString("biz_subject");
String company_type = rs.getString("company_type");
String obshtina = rs.getString("obshtina");
String main_office_town = rs.getString("main_office_town");
String address = rs.getString("address");
String role = rs.getString("role");
String country = rs.getString("country");
String nace_code = rs.getString("nace_code");
String nace_text = rs.getString("nace_text");
String zip_code = rs.getString("zip_code");
String phone = rs.getString("phone");
String fax = rs.getString("fax");
String email = rs.getString("email");
String web = rs.getString("web");
String location = rs.getString("location");
String geohash = rs.getString("geohash");
Integer popularity = rs.getInt("popularity");
doc.addField("id", id);
doc.addField("name", name);
doc.addField("midname", midname);
doc.addField("lastname", lastname);
doc.addField("frlsname", frlsname);
doc.addField("biz_subject", biz_subject);
doc.addField("company_type", company_type);
doc.addField("obshtina", obshtina);
doc.addField("main_office_town", main_office_town);
doc.addField("address", address);
doc.addField("role", role);
doc.addField("country", country);
doc.addField("nace_code", nace_code);
doc.addField("nace_text", nace_text);
doc.addField("zip_code", zip_code);
doc.addField("phone", phone);
doc.addField("fax", fax);
doc.addField("email", email);
doc.addField("web", web);
doc.addField("location", location);
doc.addField("geohash", geohash);
doc.addField("popularity", popularity);
docs.add(doc);
++_totalSql;
if (docs.size() > 1) {
// Commit within 5 minutes.
UpdateResponse resp = server.add(docs);
System.out.println (resp);
if (resp.getStatus() != 0) {
log("Some horrible error has occurred, status is: " +
resp.getStatus());
}
docs.clear();
}
}
}
catch (Exception ex)
{
ex.printStackTrace();
}
finally {
if (conn != null) {
conn.close();
}
}
}
private void endIndexing() throws IOException, SolrServerException {
if (docs.size() > 0) { // Are there any documents left over?
server.add(docs, 300000); // Commit within 5 minutes
}
try
{
server.commit();
}
catch (Exception ex)
{
ex.printStackTrace();
}
long endTime = System.currentTimeMillis();
log("Total Time Taken: " + (endTime - _start) +
" milliseconds to index " + _totalSql +
" SQL rows" );
}
private indexSolr(String url) throws IOException, SolrServerException {
// Create a multi-threaded communications channel to the Solr server.
try {
server = new HttpSolrServer(url);
server.setSoTimeout(1000); // socket read timeout
server.setConnectionTimeout(1000);
server.setMaxRetries(1);
}
catch (Exception ex)
{
ex.printStackTrace();
}
}
}
That's all you need to do to delete the index of a row with id for example 30682 from solr
SolrServer solrServer;
String url = "http://localhost:8983/solr/db";
solrServer = new HttpSolrServer(url);
solrServer.deleteByQuery("id:30682");
solrServer.commit();
System.out.println("index deleted");
I Hope it helps someone