I have a cypher query that is supposed to return nodes and edges so that I can render a representation of my graph in a web app. I'm running it with the query method in Neo4jOperations.
start n=node({id}) match n-[support:SUPPORTED_BY|INTERPRETS*0..5]->(argument:ArgumentNode)
return argument, support
Earlier, I was using spring data neo4j 3.3.1 with an embedded database, and this query did a fine job of returning relationship proxies with start nodes and end nodes. I've upgraded to spring data neo4j 4.0.0 and switched to using a remote server, and now it returns woefully empty LinkedHashMaps.
This is the json response from the server:
{"commit":"http://localhost:7474/db/data/transaction/7/commit","results":[{"columns":["argument","support"],
"data":[
{"row":[{"buildVersion":-1},[]]},
{"row":[{"buildVersion":-1},[{}]]}
]}],"transaction":{"expires":"Mon, 12 Oct 2015 06:49:12 +0000"},"errors":[]}
I obtained this json by putting a breakpoint in DefaultRequest.java and executing EntityUtils.toString(response.getEntity()). The query is supposed to return two nodes which are related via an edge of type INTERPRETS. In the response you see [{}], which is where data about the edge should be.
How do I get a response with the data I need?
Disclaimer: this is not a definitive answer, just what I've pieced together so far.
You can use the queryForObjects method in Neo4jOperations, and make sure that your query returns a path. Example:
neo4jOperations.queryForObjects(ArgumentNode.class, "start n=node({id}) match path=n-[support:SUPPORTED_BY|INTERPRETS*0..5]->(argument:ArgumentNode) return path", params);
The POJOs that come back should be hooked together properly based on their relationship annotations. Now you can poke through them and manually build a set of edges that you can serialize. Not ideal, but workable.
Docs suggesting that you return a path:
From http://docs.spring.io/spring-data/data-neo4j/docs/4.0.0.RELEASE/reference/html/#_cypher_queries:
For the query methods that retrieve mapped objects, the recommended
query format is to return a path, which should ensure that known types
get mapped correctly and joined together with relationships as
appropriate.
Explanation of why queryForObjects helps:
Under the hood, there is a distinction between different types of queries. They have GraphModelQuery, RowModelQuery, and GraphRowModelQuery, each of which pass a different permutation of resultDataContents: ["row", "graph"] to the server. If you want data sufficient to reconstruct the graph, you need to make sure "graph" is in the list.
You can find this code inside ExecuteQueriesDelegate:
if (type != null && session.metaData().classInfo(type.getSimpleName()) != null) {
Query qry = new GraphModelQuery(cypher, parameters);
...
} else {
RowModelQuery qry = new RowModelQuery(cypher, parameters);
...
}
Using queryForObjects allows you to provide a type, and kicks things over into GraphModelQuery mode.
Related
At the moment, when an article is added to the store, my view is not updated when I use store.query(), filtering server side, in my route but it's updated when I use store.findAll() with filtering client side.
With findAll, filtering client side
//route.js
model() {
return this.get('store').findAll('article');
}
//controller.js
articleSorted: computed.filterBy('model', 'isPublished', true),
and with query filtering server side
//route.js
model() {
return this.get('store').query('article', { q: 'isPublished' }),
}
The fact is that findAll is reloading and query is not.
I've found this but did not understand
https://github.com/emberjs/ember.js/issues/15256
thanks for the question. I'll try to answer it the best I can but it would seem like some more documentation should be added to the Ember Guides to explain this situation 🤔
Essentially this.store.findAll() and this.store.query() do two very different things. findAll() is designed to be a representation of all of the entities (articles in your case) so it makes sense that the result will automatically update as the store finds more articles it should care about. It does this because it doesn't return an array of articles, it returns a DS.RecordArray that will automatically update.
query() on the other hand is designed to ask the backend every time what it expects the result to be, and you are usually passing a number of parameters to the query() call that the backend is using to find or filter results. It would be impossible for the frontend to know exactly how the backend interprets these query parameters so it is not possible for it to "auto-update" when a new article is added that would satisfy the same query.
Does that make sense? Would you like me to go into any more detail?
When using store.query to fetch data from the server, the view can still be auto-updated with new client-created store data before it's saved to the server, by using a "live" record array for it.
While data from store.query isn't live, data from store.peekAll is, so you can query first but then leverage store.peekAll for display. You can query before setting your model to the peeked data, or keep your query as the model but use some other property of peeked data for display. The important part is to ensure the query is resolved before peeking at the store.
Example based on the current code in your question:
// route.js
beforeModel() {
// using return ensures this hook waits for the promise to resolve before moving on
return this.store.query('article', { q: 'isPublished' });
}
model() {
// Queried server data should now be available to peek at locally,
// so we can set the model to a live version of it. Don't append filterBy here,
// otherwise it will no longer be live.
return this.store.peekAll('article');
}
// controller.js
// seemingly redundant filter since using query, but needed if there are other records
// in the store that shouldn't be displayed, and is recomputed when
// a record is added or removed from the store-based model
articleSorted: filterBy('model', 'isPublished', true) // filterBy imported from '#ember/object/computed'
// template.hbs
{{#each articleSorted as |article|}}
{{!-- this displayed list should update as new records are added to the store --}}
{{article}}
{{/each}}
Note that after a new record is saved to the server, the query can be updated via its update method or via a route refresh. This will re-run the query and get the updated results from the server. If the query is the model, that would look like model.update(). If it was saved to someOtherProperty, then someOtherProperty.update(). In either case, route.refresh() could be used instead to re-run all route hooks.
Some specific comments/examples that I think are helpful:
https://github.com/emberjs/ember.js/issues/15256#issuecomment-302894768
https://github.com/emberjs/ember.js/issues/15256#issuecomment-302906077
https://github.com/pouchdb-community/ember-pouch/issues/232#issuecomment-428927114
I am currently using SDN 4 and trys to do the following query:
#Query("MATCH (n:TNode:{0}) RETURN n")
Collection<TNode> getNodes(String type);
where each node has a common label "TNode" and an individual label type. However, it always returns syntax error. I'm sure the query is correct because it returns nodes using Neo4j web client.
Does the error occurs because SDN can not find nodes by label?
This is a limitation of Cypher, not SDN. Labels (or relationship types) as parameters are not supported. See this and related feature requests.
You can work around this using where clause and labels(n) function:
MATCH (n:TNode)
WHERE {0} in labels(n)
RETURN n
This comes with a caveat - it will go through all nodes matched by the MATCH clause. In your situation having a :TNode label might solve the issue, but generally having simple MATCH (n) would go through all nodes in the database, which will be very slow.
Other option would be to build the query manually and use org.springframework.data.neo4j.template.Neo4jOperations#queryForObjects to run the query:
String query = "MATCH (n:TNode:" + type + ") RETURN n"; // ugly, but works, beware of query injections etc..
Collection<TNode> nodes = neo4jOperations.queryForObjects(TNode.class, query, params);
From the API docs dynamo db does support pagination for scan and query operations. The catch here is to set the ExclusiveStartIndex of current request to the value of the LastEvaluatedIndex of previous request to get next set (logical page) of results.
I'm trying to implement the same but I'm using DynamoDBMapper, which seems to have lot more advantages like tight coupling with data models. So if I wanted to do the above, I'm assuming I would do something like below:
// Mapping of hashkey of the last item in previous query operation
Map<String, AttributeValue> lastHashKey = ..
DynamoDBQueryExpression expression = new DynamoDBQueryExpression();
...
expression.setExclusiveStartKey();
List<Table> nextPageResults = mapper.query(Table.class, expression);
I hope my above understanding is correct on paginating using DynamoDBMapper.
Secondly, how would I know that I've reached the end of results. From the docs if I use the following api:
QueryResult result = dynamoDBClient.query((QueryRequest) request);
boolean isEndOfResults = StringUtils.isEmpty(result.getLastEvaluatedKey());
Coming back to using DynamoDBMapper, how can I know if I've reached end of results in this case.
You have a couple different options with the DynamoDBMapper, depending on which way you want go.
query - returns a PaginatedQueryList
queryPage - returns a QueryResultPage
scan - returns a PaginatedScanList
scanPage - returns a ScanResultPage
The part here is understanding the difference between the methods, and what functionality their returned objects encapsulate.
I'll go over PaginatedScanList and ScanResultPage, but these methods/objects basically mirror each other.
The PaginatedScanList says the following, emphasis mine:
Implementation of the List interface that represents the results from a scan in AWS DynamoDB. Paginated results are loaded on demand when the user executes an operation that requires them. Some operations, such as size(), must fetch the entire list, but results are lazily fetched page by page when possible.
This says that results are loaded as you iterate through the list. When you get through the first page, the second page is automatically fetched with out you having to explicitly make another request. Lazy loading the results is the default method, but it can be overridden if you call the overloaded methods and supply a DynamoDBMapperConfig with a different DynamoDBMapperConfig.PaginationLoadingStrategy.
This is different from the ScanResultPage. You are given a page of results, and it is up to you to deal with the pagination yourself.
Here is quick code sample showing an example usage of both methods that I ran with a table of 5 items using DynamoDBLocal:
final DynamoDBMapper mapper = new DynamoDBMapper(client);
// Using 'PaginatedScanList'
final DynamoDBScanExpression paginatedScanListExpression = new DynamoDBScanExpression()
.withLimit(limit);
final PaginatedScanList<MyClass> paginatedList = mapper.scan(MyClass.class, paginatedScanListExpression);
paginatedList.forEach(System.out::println);
System.out.println();
// using 'ScanResultPage'
final DynamoDBScanExpression scanPageExpression = new DynamoDBScanExpression()
.withLimit(limit);
do {
ScanResultPage<MyClass> scanPage = mapper.scanPage(MyClass.class, scanPageExpression);
scanPage.getResults().forEach(System.out::println);
System.out.println("LastEvaluatedKey=" + scanPage.getLastEvaluatedKey());
scanPageExpression.setExclusiveStartKey(scanPage.getLastEvaluatedKey());
} while (scanPageExpression.getExclusiveStartKey() != null);
And the output:
MyClass{hash=2}
MyClass{hash=1}
MyClass{hash=3}
MyClass{hash=0}
MyClass{hash=4}
MyClass{hash=2}
MyClass{hash=1}
LastEvaluatedKey={hash={N: 1,}}
MyClass{hash=3}
MyClass{hash=0}
LastEvaluatedKey={hash={N: 0,}}
MyClass{hash=4}
LastEvaluatedKey=null
I am developing a data mining application with the Weka API, Java and MySQL DB connectivity. I want to feed data from the database to the algorithm. I used http://weka.wikispaces.com/Use+Weka+in+your+Java+code#Instances-Database.
Since I use Hibernate and the hibernate.cfg.xml file has the database connection information, can't I just write a normal method in the DAO class to retrieve data and then pass that to the algorithm?
The Weka API is, unfortunately, in some points quite constrained. As such, you will need to have Instances objects. IIRC this its not an interface that you could implement otherwise, but an actual object you have to create.
Therefore, you will likely need to query all your database and produce Instances out of it. Not using hibernate but raw database accesses will save you from doing things twice, thus needing twice as much memory.
I've recently done this with Hibernate, but there is no way that a hibernate class can simply be put into WEKA. I've done it this way:
generate a table in the database that has the model information available as you need it (I've done this since i would have needed to do very complex, time consuming queries for every row. This way, I do the heavy work once and just read it from a simple table)
create you POJO, DAO and what not
then just set-up your WEKA model
Sample Code (WEKA 3.7)
ArrayList<Attribute> atts = new ArrayList<Attribute>();
atts.add(new Attribute("attribute1"));
atts.add(new Attribute("attribute1"));
atts.add(new Attribute("id", (ArrayList<String>) null));
data = new Instances("yourData", atts, 0);
DAOModel dao = getYourDaoModelHereFromHibernateHoweverYouWantIt();
for (Model m : dao.findAll()) {
vals = new double[data.numAttributes()];
vals[0] = m.getAttribute1();
vals[1] = m.getAttribute2();
vals[2] = data.attribute(2).addStringValue(m.getId());
data.add(new DenseInstance(1.0, vals));
}
data now has the proper format and the algorithms can work with it (you could also save it to an .arff file if you want to work with the GUI)
I have a method that builds and runs a Criteria query. The query does what I want it to, specifically it filters (and sorts) records based on user input.
Also, the query size is restricted to the number of records on the screen. This is important because the data table can be potentially very large.
However, if filters are applied, I want to count the number of records that would be returned if the query was not limited. So this means running two queries: one to fetch the records and then one to count the records that are in the overall set. It looks like this:
public List<Log> runQuery(TableQueryParameters tqp) {
// get the builder, query, and root
CriteriaBuilder builder = em.getCriteriaBuilder();
CriteriaQuery<Log> query = builder.createQuery(Log.class);
Root<Log> root = query.from(Log.class);
// build the requested filters
Predicate filter = null;
for (TableQueryParameters.FilterTerm ft : tqp.getFilterTerms()) {
// this section runs trough the user input and constructs the
// predicate
}
if (filter != null) query.where(filter);
// attach the requested ordering
List<Order> orders = new ArrayList<Order>();
for (TableQueryParameters.SortTerm st : tqp.getActiveSortTerms()) {
// this section constructs the Order objects
}
if (!orders.isEmpty()) query.orderBy(orders);
// run the query
TypedQuery<Log> typedQuery = em.createQuery(query);
typedQuery.setFirstResult((int) tqp.getStartRecord());
typedQuery.setMaxResults(tqp.getPageSize());
List<Log> list = typedQuery.getResultList();
// if we need the result size, fetch it now
if (tqp.isNeedResultSize()) {
CriteriaQuery<Long> countQuery = builder.createQuery(Long.class);
countQuery.select(builder.count(countQuery.from(Log.class)));
if (filter != null) countQuery.where(filter);
tqp.setResultSize(em.createQuery(countQuery).getSingleResult().intValue());
}
return list;
}
As a result, I call createQuery twice on the same CriteriaBuilder and I share the Predicate object (filter) between both of them. When I run the second query, I sometimes get the following message:
Exception [EclipseLink-6089] (Eclipse Persistence Services -
2.2.0.v20110202-r8913):
org.eclipse.persistence.exceptions.QueryException Exception
Description: The expression has not been initialized correctly. Only
a single ExpressionBuilder should be used for a query. For parallel
expressions, the query class must be provided to the ExpressionBuilder
constructor, and the query's ExpressionBuilder must always be on the
left side of the expression. Expression: [ Base
com.myqwip.database.Log] Query: ReportQuery(referenceClass=Log ) at
org.eclipse.persistence.exceptions.QueryException.noExpressionBuilderFound(QueryException.java:874)
at
org.eclipse.persistence.expressions.ExpressionBuilder.getDescriptor(ExpressionBuilder.java:195)
at
org.eclipse.persistence.internal.expressions.DataExpression.getMapping(DataExpression.java:214)
Can someone tell me why this error shows up intermittently, and what I should do to fix this?
Short answer to the question : Yes you can, but only sequentially.
In the method above, you start creating the first query, then start creating the second, the execute the second, then execute the first.
I had the exact same problem. I don't know why it's intermittent tough.
I other words, you start creating your first query, and before having finished it, you start creating and executing another.
Hibernate doesn't complain but eclipselink doesn't like it.
If you just start by the query count, execute it, and then create and execute the other query (what you've done by splitting it in 2 methods), eclipselink won't complain.
see https://issues.jboss.org/browse/SEAMSECURITY-91
It looks like this posting isn't going to draw much more response, so I will answer this in how I resolved it.
Ultimately I ended up breaking my runQuery() method into two methods: runQuery() that fetches the records and runQueryCount() that fetches the count of records without sort parameters. Each method has its own call to em.getCriteriaBuilder(). I have no idea what effect that has on the EntityManager, but the problem has not appeared since.
Also, the DAO object that has these methods used to be #ApplicationScoped. It now has no declared scope, so it is now constructed on demand from the various #RequestScoped and #ConversationScoped beans that use it. I don't know if this has any effect on the problem but since it has not appeared since I will use this as my code pattern from now on. Suggestions welcome.