RollbackException: caused by close() without success() on PlaceboTransaction - spring-data-neo4j

Since upgrading my Neo4j from 1.9.8 to 2.1.2. and Spring-Data from 3.2.0 to 3.2.5, I'm seeing a "RollbackException" when I attempt to call TopLevelTransaction.close(..) from within a Transaction. This does not occur if the thread doesn't already have a Transaction associated with it.
neo4j.kernel.TopLevelTransaction.close(TopLevelTransaction.java:134)
~[neo4j-kernel-2.1.2.jar:2.1.2]
at org.neo4j.kernel.TopLevelTransaction.finish(TopLevelTransaction.java:111)
.... Caused by: javax.transaction.RollbackException: Failed to commit, transaction rolled back
at org.neo4j.kernel.impl.transaction.TxManager.rollbackCommit(TxManager.java:629)
at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:390)
at org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:123)
at org.neo4j.kernel.TopLevelTransaction.close(TopLevelTransaction.java:124)
Before the exception is thrown, the transaction was marked as "rollback only". This occurred when I called CreateObjectiveTxTask.createObjectiveRuleRelationships(..), which results in a call to TypeRepresentationStrategyFactory.chooseStrategy(..).
private static Strategy chooseStrategy(GraphDatabase graphDatabaseService) {
try (Transaction tx = graphDatabaseService.beginTx()) {
if (AbstractIndexBasedTypeRepresentationStrategy.isStrategyAlreadyInUse(graphDatabaseService)) return Strategy.Indexed;
if (SubReferenceNodeTypeRepresentationStrategy.isStrategyAlreadyInUse(graphDatabaseService)) return Strategy.SubRef;
if (LabelBasedNodeTypeRepresentationStrategy.isStrategyAlreadyInUse(graphDatabaseService)) return Strategy.Labeled;
tx.success();
return Strategy.Labeled;
}
}
This method created a PlaceboTransaction, detected that the GraphDatabase instance is already using Index strategy, so closed the transaction without trying to call "success()" on it.
Is this likely to have caused the RollbackException?
What might cause SDN to close PlaceboTransaction before calling "success()" on it, and how can I stop SDN from doing this?

Perhaps there is a bug in TyperRepresentationStrategyFactory.chooseStrategy(..)
The most recent Git commit (b40ea309) for this code moved the tx.success() to where it only effects a Transaction if a strategy is not already in use.
private static Strategy chooseStrategy(GraphDatabase graphDatabaseService) {
Transaction tx = graphDatabaseService.beginTx();
try {
if (isAlreadyIndexed(graphDatabaseService)) return Strategy.Indexed;
if (isAlreadySubRef(graphDatabaseService)) return Strategy.SubRef;
if (isAlreadyLabeled(graphDatabaseService)) return Strategy.Labeled;
} finally {
tx.success();tx.finish();
}
Became
try (Transaction tx = graphDatabaseService.beginTx()) {
if (AbstractIndexBasedTypeRepresentationStrategy.isStrategyAlreadyInUse(graphDatabaseService)) return Strategy.Indexed;
if (SubReferenceNodeTypeRepresentationStrategy.isStrategyAlreadyInUse(graphDatabaseService)) return Strategy.SubRef;
if (LabelBasedNodeTypeRepresentationStrategy.isStrategyAlreadyInUse(graphDatabaseService)) return Strategy.Labeled;
tx.success();
return Strategy.Indexed;
}
When I revert this change, I no longer see the RollbackException.

Related

Flutter/Dart - Suggestions for how to Troubleshoot a Future?

Though it was working previously, for some reason my code has now stopped working. Though it fetches the required json data, the build doesn't render. The error on my app page which is supposed to display a page view was;
type String is not a subtype of 'Map<String,dynamic>'
But after some tweaks, now the error is;
invalid arguments(s)
I think I may have narrowed it down to the Future;
FutureBuilder<List<SpeakContent>> futureStage() {
return new FutureBuilder<List<SpeakContent>>(
future: downloadJSON(),
builder: (context, snapshot) {
if (snapshot.hasData) {
print("Snapshot has data.");
List<SpeakContent> speakcrafts = snapshot.data;
return new CustomPageView(speakcrafts);
} else if (snapshot.hasError) {
return Text('${snapshot.error}');
}
return new CircularProgressIndicator();
},
);
}
Future<List<SpeakContent>> downloadJSON() async {
final jsonEndpoint =
"http://example.com/getContent.php?";
final response = await get(jsonEndpoint);
if (response.statusCode == 200) {
List speakcrafts = json.decode(response.body);
debugPrint(speakcrafts.toString());
return speakcrafts
.map((speakcraft) => new SpeakContent.fromJson(speakcraft))
.toList();
} else
throw Exception('We were not able to successfully download the json data.');
}
Although it doesn't throw an error, I've noticed that it doesn't print my test statement after the "if (snapshot.hasData)" line.
Shouldn't I see "Snapshot has data." appear in my Android Studio console?
Based on what you provided, this
type String is not a subtype of 'Map<String,dynamic>'
must be this:
return Text('${snapshot.error}');
which means that your downloadJSON() threw an exception. In that case, print('Snapshot has data.'); never executes and the next case that I quoted above is executed.
Please put a breakpoint in the body of downloadJSON(), run it line by line and see what's thrown where.
ALSO, you are making an irrelevant but big mistake here. Do not call downloadJSON() like this. This function is executed at every re-render, which can be many times. You are initiating a JSON download at every redraw of your widget. This may stack up your backend bills... I explain it in this talk in more detail: https://youtu.be/vPHxckiSmKY?t=4771
After performing the troubleshooting tips as suggested by Gazihan Alankus and scrimau, I've found the culprit which was a single null entry in the MYSQL Database. Scary... gotta find out how to prevent that problem in the future.

Dataflow job run failing when templateLocation argument is set

Dataflow job is failing with below exception when I pass parameters staging,temp & output GCS bucket locations.
Java code:
final String[] used = Arrays.copyOf(args, args.length + 1);
used[used.length - 1] = "--project=OVERWRITTEN"; final T options =
PipelineOptionsFactory.fromArgs(used).withValidation().as(clazz);
options.setProject(PROJECT_ID);
options.setStagingLocation("gs://abc/staging/");
options.setTempLocation("gs://abc/temp");
options.setRunner(DataflowRunner.class);
options.setGcpTempLocation("gs://abc");
The error:
INFO: Staging pipeline description to gs://ups-heat-dev- tmp/mniazstaging_ingest_validation/staging/
May 10, 2018 11:56:35 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage
INFO: Uploading <42088 bytes, hash E7urYrjAOjwy6_5H-UoUxA> to gs://ups-heat-dev-tmp/mniazstaging_ingest_validation/staging/pipeline-E7urYrjAOjwy6_5H-UoUxA.pb
Dataflow SDK version: 2.4.0
May 10, 2018 11:56:38 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Printed job specification to gs://ups-heat-dev-tmp/mniazstaging_ingest_validation/templates/DataValidationPipeline
May 10, 2018 11:56:40 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Template successfully created.
Exception in thread "main" java.lang.NullPointerException
at org.apache.beam.runners.dataflow.DataflowPipelineJob.getJobWithRetries(DataflowPipelineJob.java:501)
at org.apache.beam.runners.dataflow.DataflowPipelineJob.getStateWithRetries(DataflowPipelineJob.java:477)
at org.apache.beam.runners.dataflow.DataflowPipelineJob.waitUntilFinish(DataflowPipelineJob.java:312)
at org.apache.beam.runners.dataflow.DataflowPipelineJob.waitUntilFinish(DataflowPipelineJob.java:248)
at org.apache.beam.runners.dataflow.DataflowPipelineJob.waitUntilFinish(DataflowPipelineJob.java:202)
at org.apache.beam.runners.dataflow.DataflowPipelineJob.waitUntilFinish(DataflowPipelineJob.java:195)
at com.example.DataValidationPipeline.main(DataValidationPipeline.java:66)
I was also facing the same issue, the error was throwing at p.run().waitForFinish();. Then I have tried following code
PipelineResult result = p.run();
System.out.println(result.getState().hasReplacementJob());
result.waitUntilFinish();
This was throwing the following exception
java.lang.UnsupportedOperationException: The result of template creation should not be used.
at org.apache.beam.runners.dataflow.util.DataflowTemplateJob.getState (DataflowTemplateJob.java:67)
Then to fix the issue I used the following code
PipelineResult result = pipeline.run();
try {
result.getState();
result.waitUntilFinish();
} catch (UnsupportedOperationException e) {
// do nothing
} catch (Exception e) {
e.printStackTrace();
}
I was running into the problem with java.lang.UnsupportedOperationException: The result of template creation should not be used. today aswell and I tried to fixed it by checking if the job was of type DataflowTemplateJob first:
val (sc, args) = ContextAndArgs(cmdlineArgs)
// ...
val result = sc.run()
if (!result.isInstanceOf[DataflowTemplateJob]) result.waitUntilFinish()
I think this should work for bare java jobs, but if you use Scio, then the result will be some anonymous type, so in the end I had to do the try catch version aswell.
try {
val result = sc.run().waitUntilFinish()
} catch {
case _: UnsupportedOperationException => // this happens during template creation
}
as displayed in the official Flex Template sample
There is a comment saying:
// For a Dataflow Flex Template, do NOT waitUntilFinish().
The same applies if you call any of those methods of the Runner if you pass the --templateRunner argument
if you change the pipeline to pipeline.run(); it is not going to fail.
The Issue is still flagged as opened by apache beam
https://github.com/apache/beam/issues/20106

Baqend onUpdate Handler

Will doing partialupdate() cause code in a data class' onUpdate Handler to run?
I have this setup in the data class:
exports.onUpdate = function(db, obj) {
DB.log.info(obj.ShiftID);
db.Shifts.load(obj.ShiftID)
.then((Shift) => {
DB.log.info(Shift);
if (Shift.User == db.User.me) {
Shift.User = null;
Shift.status = 0;
return Shift.update();
}
})
};
(yes, role 2 for node has permissions to query and update the Shifts data class)
But I am getting zero logs when I make a partialupdate(). Do I need to do a real update query...load the object, modify the data, update()?
Also it seems that this code causes the partialupdate() to not run at all, but when I delete the handler, it starts working again.
Yes, that is currently an unimplemented feature since a partial update can't execute an onUpdate handler since there is no object which can be passed to the update handler.
On the other hand, a partial update can't be executed directly since that will result in a security issue (since your onUpdate handler can contain validation code etc.)
So we currently reject any partial update on a class which has an onUpdate handler because there doesn't exist a way how we can actually validate the partial update against your onUpdate code.
We have planned that you can define an extra onPartial handler where you can take some extra steps before the partialUpdate is executed. But that handler will only get the partial update and not the object itself.
I'm pretty sure that partialupdate() will not cause the onUpdate Handler to run.
When I put the log line in and edit the records using website data manager it does log as expected. Not a big deal, I can just rewrite the query to be a full update.
BUT having any code in there does break partialupdate() which is not good.
Here is the code I'm using that works as long as there is nothing in the onUpdateHandler:
requestShift(shiftID) {
db.ready().then((db) => {
db.Applicants.find()
.where({
"shiftID": { "$in": [shiftID] },
})
.singleResult((applicants) => {
return applicants.partialUpdate()
.add("applicants", db.User.me.id)
.add("photos", this.props.UserData.photo)
.execute()
})
Alert.alert(
'Confirmation',
'Shift has been requested.',
)
this.props.navigation.dispatch(goToFindShifts)
})
}

Testing camel-sql route with in-memory database not fetching results

I have written the code using camel-sql which is working fine. Now I have to write test cases for the same. I have used in-memory database H2. I have initialized the database and assigned the datasource to sqlComponent.
// Setup code
#Override
protected JndiRegistry createRegistry() throws Exception {
JndiRegistry jndi = super.createRegistry();
// this is the database we create with some initial data for our unit test
database = new EmbeddedDatabaseBuilder()
.setType(EmbeddedDatabaseType.H2).addScript("createTableAndInsert.sql").build();
jndi.bind("myDataSource", database);
return jndi;
}
// Testcase code
#SuppressWarnings("unchecked")
#Test
public void testRoute() throws Exception {
Exchange receivedExchange = template.send("direct:myRoute", ExchangePattern.InOut ,exchange -> {
exchange.getIn().setHeader("ID", new Integer(1));
});
camelContext.start();
MyClass updatedEntity = (MyClass)jdbcTemplate.queryForObject("select * from MY_TABLE where id=?", new Long[] { 1l } ,
new RouteTest.CustomerRowMapper() );
// Here I can get the updatedEntity from jdbcTemplate
assertNotNull(receivedExchange);
assertNotNull(updatedEntity);
}
// Main code
from("direct:myRoute")
.routeId("pollDbRoute")
.transacted()
.to("sql:select * from MY_TABLE msg where msg.id = :#"+ID+"?dataSource=#myDataSource&outputType=SelectOne&outputClass=com.entity.MyClass")
.log(LoggingLevel.INFO,"Polled message from DB");
The problem is, as soon as the test case starts, it is saying
No bean could be found in the registry for: myDataSource of type: javax.sql.DataSource
I looked into camel-SQL component test cases and doing the same thing but the code is not able to find dataSource. Please help. Thanks in advance.
After spending a lot of time on this issue, I identified that H2 database was using JDBCUtils to fetch records and It was throwing ClassNotFoundException. I was getting it nowhere in Camel exception hierarchy because this exception was being suppressed and all I was getting a generic exception message. Here is the exception:
ClassNotFoundException: com.vividsolutions.jts.geom.Geometry
After searching for the issue I found out that It requires one more dependency. So I added it and it resolved the issue.
Issue URL: https://github.com/elastic/elasticsearch/issues/9891
Dependency: https://mvnrepository.com/artifact/com.vividsolutions/jts-core/1.14.0

How can I get optimistic concurrency with JPA annotation

I am using JPA 3, with annotation (no mapping file) and with provider org.hibernate.ejb.HibernatePersistence
I need to have optimistic concurrency.
1)I tried to rely on the tag called , it did not work.
2)
So I decided to do it with java code. I have a mergeServiceRequest method and an object with type Request as follows: I start a transaction, lock the request object,
then try to get a Request object newRequest from database, compare its timestamp with the current one request. If they do not match, I throw an exception; if they match, then I update the current request enter code herewith current time and save it to database.
I need to lock the object manually, because by starting a transaction from session, it does not put a lock on the row in database. I wrote some java code which shows that a transaction does not lock the record in database automatically.
Problem with this approach is the query
Request newRequest=entityManager.createQuery("select r from Request r where serviceRequestId = " + request.getServiceRequestId());
always return same object as request. "request" is in the session entityManger, and the query always return what is cached in the session. I tried all the five query.setHint lines and I still get same result: no database query is performed, the result is from session cache directly.
#Transactional
public void mergeServiceRequest(Request request) {
System.out.println("ServiceRequestDao.java line 209");
EntityTransaction transaction = entityManager.getTransaction();
transaction.begin();
entityManager.lock(request, LockModeType.WRITE); // use to lock the database row
Query query = entityManager.createQuery("select r from Request r where serviceRequestId = " + request.getServiceRequestId());
//query.setHint("javax.persistence.cache.retrieveMode", "BYPASS");
//query.setHint("org.hibernate.cacheMode", CacheMode.REFRESH);
//query.setHint("javax.persistence.cache.retrieveMode", CacheMode.REFRESH);
//query.setHint("javax.persistence.retrieveMode", CacheMode.REFRESH);
//query.setHint(QueryHints.CACHE_USAGE, CacheUsage.DoNotCheckCache);
Request newRequest=(Request)query.getSingleResult();
if (! newRequest.getLastUpdatedOn().equals(request.getLastUpdatedOn())) {
throw new StaleObjectStateException(request.getClass().toString(), request.getServiceRequestId());
}
request.setLastUpdatedOn(new Date(System.currentTimeMillis()));
entityManager.persist(request);
entityManager.flush();
transaction.commit();
}
3)So I also tried to use another session get query the newRequest, if I do that, the newRequest will be different from request. But for some reason, if I do that, then the lock on request object is never released, even after the transaction commit. Code looks like below
#Transactional
public void mergeServiceRequest(Request request) {
System.out.println("ServiceRequestDao.java line 209");
EntityTransaction transaction = entityManager.getTransaction();
transaction.begin();
entityManager.lock(request, LockModeType.WRITE); // use to lock the database row
Request newRequest=findRequest(request.getServiceRequestId()); // get it from another session
if (! newRequest.getLastUpdatedOn().equals(request.getLastUpdatedOn())) {
throw new StaleObjectStateException(request.getClass().toString(), request.getServiceRequestId());
}
request.setLastUpdatedOn(new Date(System.currentTimeMillis()));
entityManager.persist(request);
entityManager.flush();
transaction.commit();
//lock on the database record is not released after this, and even after entityManager is closed
}
Could anyone help me on this?
Thanks.
Daniel