I have a query in a Spring data JpaRepository like so:
#Lock(value = LockModeType.PESSIMISTIC_WRITE)
#QueryHints({#QueryHint(name = "javax.persistence.lock.timeout", value = "70000")})
Collection<AnyCLass> findBy ...
However, in my test, if I run the transaction ( that uses this query as the first query) in two concurrent threads I get an SQL Lock timeout (SQL Error: 50200, SQLState: HYT00) after one second, which is the default for the H2 in memory.
If the transaction is faster then one second everything works as expected.
Possible workaround for this Issue is a simple retry on the enclosing method:
#Retryable(backoff = #Backoff(value = 500,random = true))
void getSometingLocked(){
findById(id)
}
Related
I have written the code using camel-sql which is working fine. Now I have to write test cases for the same. I have used in-memory database H2. I have initialized the database and assigned the datasource to sqlComponent.
// Setup code
#Override
protected JndiRegistry createRegistry() throws Exception {
JndiRegistry jndi = super.createRegistry();
// this is the database we create with some initial data for our unit test
database = new EmbeddedDatabaseBuilder()
.setType(EmbeddedDatabaseType.H2).addScript("createTableAndInsert.sql").build();
jndi.bind("myDataSource", database);
return jndi;
}
// Testcase code
#SuppressWarnings("unchecked")
#Test
public void testRoute() throws Exception {
Exchange receivedExchange = template.send("direct:myRoute", ExchangePattern.InOut ,exchange -> {
exchange.getIn().setHeader("ID", new Integer(1));
});
camelContext.start();
MyClass updatedEntity = (MyClass)jdbcTemplate.queryForObject("select * from MY_TABLE where id=?", new Long[] { 1l } ,
new RouteTest.CustomerRowMapper() );
// Here I can get the updatedEntity from jdbcTemplate
assertNotNull(receivedExchange);
assertNotNull(updatedEntity);
}
// Main code
from("direct:myRoute")
.routeId("pollDbRoute")
.transacted()
.to("sql:select * from MY_TABLE msg where msg.id = :#"+ID+"?dataSource=#myDataSource&outputType=SelectOne&outputClass=com.entity.MyClass")
.log(LoggingLevel.INFO,"Polled message from DB");
The problem is, as soon as the test case starts, it is saying
No bean could be found in the registry for: myDataSource of type: javax.sql.DataSource
I looked into camel-SQL component test cases and doing the same thing but the code is not able to find dataSource. Please help. Thanks in advance.
After spending a lot of time on this issue, I identified that H2 database was using JDBCUtils to fetch records and It was throwing ClassNotFoundException. I was getting it nowhere in Camel exception hierarchy because this exception was being suppressed and all I was getting a generic exception message. Here is the exception:
ClassNotFoundException: com.vividsolutions.jts.geom.Geometry
After searching for the issue I found out that It requires one more dependency. So I added it and it resolved the issue.
Issue URL: https://github.com/elastic/elasticsearch/issues/9891
Dependency: https://mvnrepository.com/artifact/com.vividsolutions/jts-core/1.14.0
I've been researching A LOT for past 2 weeks and can't pinpoint the exact reason of my Meteor app returning results too slow.
Currently I have only a single collection in my Mongo database with around 2,00,000 documents. And to search I am using Meteor subscriptions on the basis of a given keyword. Here is my query:
db.collection.find({$or:[
{title:{$regex:".*java.*", $options:"i"}},
{company:{$regex:".*java.*", $options:"i"}}
]})
When I run above query in mongo shell, the results are returned instantly. But when I use it in Meteor client, the results take almost 40 seconds to return from server. Here is my meteor client code:
Template.testing.onCreated(function () {
var instance = this;
// initialize the reactive variables
instance.loaded = new ReactiveVar(0);
instance.limit = new ReactiveVar(20);
instance.autorun(function () {
// get the limit
var limit = instance.limit.get();
var keyword = Router.current().params.query.k;
var searchByLocation = Router.current().params.query.l;
var startDate = Session.get("startDate");
var endDate = Session.get("endDate");
// subscribe to the posts publication
var subscription = instance.subscribe('sub_testing', limit,keyword,searchByLocation,startDate,endDate);
// if subscription is ready, set limit to newLimit
$('#searchbutton').val('Searching');
if (subscription.ready()) {
$('#searchbutton').val('Search');
instance.loaded.set(limit);
} else {
console.log("> Subscription is not ready yet. \n\n");
}
});
instance.testing = function() {
return Collection.find({}, {sort:{id:-1},limit: instance.loaded.get()});
}
And here is my meteor server code:
Meteor.publish('sub_testing', function(limit,keyword,searchByLocation,startDate,endDate) {
Meteor._sleepForMs(200);
var pat = ".*" + keyword + ".*";
var pat2 = ".*" + searchByLocation + ".*";
return Jobstesting.find({$or:[{title:{$regex: pat, $options:"i"}}, { company:{$regex:pat,$options:"i"}},{ description:{$regex:pat,$options:"i"}},{location:{$regex:pat2,$options:"i"}},{country:{$regex:pat2,$options:"i"}}],$and:[{date_posted: { $gte : endDate, $lt: startDate }},{sort:{date_posted:-1},limit: limit,skip: limit});
});
One point I'd also like to mention here that I use "Load More" pagination and by default the limit parameter gets 20 records. On each "Load More" click, I increment the limit parameter by 20 so on first click it is 20, on second click 40 and so on...
Any help where I'm going wrong would be appreciated.
But when I use it in Meteor client, the results take almost 40 seconds to return from server.
You may be misunderstanding how Meteor is accessing your data.
Queries run on the client are processed on the client.
Meteor.publish - Makes data available on the server
Meteor.subscribe - Downloads that data from the server to the client.
Collection.find - Looks through the data on the client.
If you think the Meteor side is slow, you should time it server side (print time before/after) and file a bug.
If you're implementing a pager, you might try a meteor method instead, or
a pager package.
I have controller with method that blocks the Play server thread due to very slow Database query. I need to implement controller method in a way that it don't block the thread.
I have read documentation: http://www.playframework.org/documentation/1.2.4/asynchronous
There's absolutely no examples anywhere on how to do this. The only thing that I found close is this https://github.com/playframework/play/blob/master/samples-and-tests/chat/app/controllers/LongPolling.java
It simply wraps result in await();
When I try to do that it doesn't work.
routes:
GET /blog Controller.blog
Controller (this is not an actual slow query but everything else is identical):
public static void blog() {
String queryStr = "SELECT b FROM Blog b ORDER BY createTime DESC";
JPAQuery q = Blog.find(queryStr);
List<Blog> bList = q.fetch(100);
List<BlogDTO> list = new ArrayList<BlogDTO>(bList.size());
for (Blog b : bList) {
BlogDTO obj = new BlogDTO(b);
list.add(obj);
}
renderJSON(list);
}
I tried List<Blog> bList = await(q.fetch(100)); but that doesn't work.
I have not worked with Future and promises before.
Can anyone give me any pointers on how to approach this?
For me the best way to do this is to use a Job that returns a List object. Then in your controller you can await for the job termination :
public static void blog() {
List<BlogDTO> list = await(new BlogPostJob().now());
renderJSON(list);
}
and you put your jpa code in your job
Because JDBC uses blocking IO, any slow database query will always block a Thread.
The only way seems to be using Job for that purpose.
I am using JPA 3, with annotation (no mapping file) and with provider org.hibernate.ejb.HibernatePersistence
I need to have optimistic concurrency.
1)I tried to rely on the tag called , it did not work.
2)
So I decided to do it with java code. I have a mergeServiceRequest method and an object with type Request as follows: I start a transaction, lock the request object,
then try to get a Request object newRequest from database, compare its timestamp with the current one request. If they do not match, I throw an exception; if they match, then I update the current request enter code herewith current time and save it to database.
I need to lock the object manually, because by starting a transaction from session, it does not put a lock on the row in database. I wrote some java code which shows that a transaction does not lock the record in database automatically.
Problem with this approach is the query
Request newRequest=entityManager.createQuery("select r from Request r where serviceRequestId = " + request.getServiceRequestId());
always return same object as request. "request" is in the session entityManger, and the query always return what is cached in the session. I tried all the five query.setHint lines and I still get same result: no database query is performed, the result is from session cache directly.
#Transactional
public void mergeServiceRequest(Request request) {
System.out.println("ServiceRequestDao.java line 209");
EntityTransaction transaction = entityManager.getTransaction();
transaction.begin();
entityManager.lock(request, LockModeType.WRITE); // use to lock the database row
Query query = entityManager.createQuery("select r from Request r where serviceRequestId = " + request.getServiceRequestId());
//query.setHint("javax.persistence.cache.retrieveMode", "BYPASS");
//query.setHint("org.hibernate.cacheMode", CacheMode.REFRESH);
//query.setHint("javax.persistence.cache.retrieveMode", CacheMode.REFRESH);
//query.setHint("javax.persistence.retrieveMode", CacheMode.REFRESH);
//query.setHint(QueryHints.CACHE_USAGE, CacheUsage.DoNotCheckCache);
Request newRequest=(Request)query.getSingleResult();
if (! newRequest.getLastUpdatedOn().equals(request.getLastUpdatedOn())) {
throw new StaleObjectStateException(request.getClass().toString(), request.getServiceRequestId());
}
request.setLastUpdatedOn(new Date(System.currentTimeMillis()));
entityManager.persist(request);
entityManager.flush();
transaction.commit();
}
3)So I also tried to use another session get query the newRequest, if I do that, the newRequest will be different from request. But for some reason, if I do that, then the lock on request object is never released, even after the transaction commit. Code looks like below
#Transactional
public void mergeServiceRequest(Request request) {
System.out.println("ServiceRequestDao.java line 209");
EntityTransaction transaction = entityManager.getTransaction();
transaction.begin();
entityManager.lock(request, LockModeType.WRITE); // use to lock the database row
Request newRequest=findRequest(request.getServiceRequestId()); // get it from another session
if (! newRequest.getLastUpdatedOn().equals(request.getLastUpdatedOn())) {
throw new StaleObjectStateException(request.getClass().toString(), request.getServiceRequestId());
}
request.setLastUpdatedOn(new Date(System.currentTimeMillis()));
entityManager.persist(request);
entityManager.flush();
transaction.commit();
//lock on the database record is not released after this, and even after entityManager is closed
}
Could anyone help me on this?
Thanks.
Daniel
I'm researching a way to build an n-tierd sync solution. From the WebSharingAppDemo-CEProviderEndToEnd sample it seems almost feasable however for some reason, the app will only sync if the client has a live SQL db connection. Can some one explain what I'm missing and how to sync without exposing SQL to the internet?
The problem I'm experiencing is that when I provide a Relational sync provider that has an open SQL connection from the client, then it works fine but when I provide a Relational sync provider that has a closed but configured connection string, as in the example, I get an error from the WCF stating that the server did not receive the batch file. So what am I doing wrong?
SqlConnectionStringBuilder builder = new SqlConnectionStringBuilder();
builder.DataSource = hostName;
builder.IntegratedSecurity = true;
builder.InitialCatalog = "mydbname";
builder.ConnectTimeout = 1;
provider.Connection = new SqlConnection(builder.ToString());
// provider.Connection.Open(); **** un-commenting this causes the code to work**
//create anew scope description and add the appropriate tables to this scope
DbSyncScopeDescription scopeDesc = new DbSyncScopeDescription(SyncUtils.ScopeName);
//class to be used to provision the scope defined above
SqlSyncScopeProvisioning serverConfig = new SqlSyncScopeProvisioning();
....
The error I get occurs in this part of the WCF code:
public SyncSessionStatistics ApplyChanges(ConflictResolutionPolicy resolutionPolicy, ChangeBatch sourceChanges, object changeData)
{
Log("ProcessChangeBatch: {0}", this.peerProvider.Connection.ConnectionString);
DbSyncContext dataRetriever = changeData as DbSyncContext;
if (dataRetriever != null && dataRetriever.IsDataBatched)
{
string remotePeerId = dataRetriever.MadeWithKnowledge.ReplicaId.ToString();
//Data is batched. The client should have uploaded this file to us prior to calling ApplyChanges.
//So look for it.
//The Id would be the DbSyncContext.BatchFileName which is just the batch file name without the complete path
string localBatchFileName = null;
if (!this.batchIdToFileMapper.TryGetValue(dataRetriever.BatchFileName, out localBatchFileName))
{
//Service has not received this file. Throw exception
throw new FaultException<WebSyncFaultException>(new WebSyncFaultException("No batch file uploaded for id " + dataRetriever.BatchFileName, null));
}
dataRetriever.BatchFileName = localBatchFileName;
}
Any ideas?
For the Batch file not available issue, remove the IsOneWay=true setting from IRelationalSyncContract.UploadBatchFile. When the Batch file size is big, ApplyChanges will be called even before fully completing the previous UploadBatchfile.
// Replace
[OperationContract(IsOneWay = true)]
// with
[OperationContract]
void UploadBatchFile(string batchFileid, byte[] batchFile, string remotePeer1
I suppose it's simply a stupid example. It exposes "some" technique but assumes you have to arrange it in proper order by yourself.
http://msdn.microsoft.com/en-us/library/cc807255.aspx