I am using https://aws-quickstart.github.io/quickstart-neo4j/ to deploy Neo4j on AWS.
My server connect to the Leader cluster IP.
But when I perform query , sometimes it takes me very long time for an simple query. It can be up to 70second while another time with the same query only take about 1 seconds.
My query is just like :
MATCH (user:USER {alias:"userAlias"}) RETURN user
I am using .Net Neo4jClient.
I already set an Unique Constraints.
Any one has facing this issue ? Please let me know how to solve this ?
Related
I am using DMS to move data (5.200.000 records) between an RDS SQL Server Express database and an RDS MySQL database. The number of records transferred is perfect : every record count per table matches exactly.
However when running the data validation step with the AWS tool, I get a lot of mismatched records that I believe is an issue in the DMS validation process at the driver level (charset configuration probably or character encoding).
Example:
RECORD_DIFF [{'Title': 'سلام ژوند څنګه دی (1) 😀🥳'}, {'Title': 'سلام ژوند څنګه دی (1) ??'},]
If I do the select query directly in the MySQL database with MySQL Workbench I get exactly the same emojis as I the DMS validation tool gets from the SQL Server.
Besides this I am also getting a lot of RECORD_DIFF [{'Content': '<VALUE_PRINT_NOT_SUPPORTED>'}, {'Content': '<VALUE_PRINT_NOT_SUPPORTED>'},] which I am not sure what they mean querying both database I see the values perfectly for those rows.
Should I consider using another validation tool or there is a known way to address the problems above?
Any help will be greatly appreciated.
We have enabled advanced throttling for WSO2AM 2.6.0. Once this was enabled and the execution plans were appropriately created, we are noticing that over 35M select count queries per hour are executing against MB_METADATA table.
Also, MB_METADATA and MB_CONTENT table are constantly growing and the row count never goes down.
I have disabled all statistics as well as tracing. we have 4 WSO servers, each one running independently with the gateway, key manager, and traffic manager on the same box. The DB is oracle.
we are seeing this query run 35 million times / hr:
SELECT COUNT(MESSAGE_ID) AS count
FROM MB_METADATA
WHERE QUEUE_ID=:1
AND MESSAGE_ID BETWEEN :2 AND :3
AND DLC_QUEUE_ID=-1
I would expect the table sizes to be manageable and this query not be run at this high of a rate.
Any suggestions on what might be going on? may be a configuration that I need to disable?
Sharing the MB database is not correct. Each traffic manager node should have its own MB database, and it can be the default H2 one.
Quoted from docs:
Do not share the WSO2_MB_STORE_DB database among the nodes in an Active-Active set-up
or Traffic Manager HA scenario, because each node should have its own local WSO2_MB_STORE_DB
database to act as separate Traffic Managers.
The latter mentioned DBs can be either H2 DBs or any RDBMS such as MySQL.
If the database gets corrupted then you need to replace the database with a fresh database
that is available in the product distribution.
Ref: https://docs.wso2.com/display/AM260/Installing+and+Configuring+the+Databases
We have been using Couchbase for about two years but we finally decided to switch to Amazon DynamoDB service for many reasons.
Now I started the migration of data to dynamodb. First everything was alright and going as expected, but after some time the response time from dynamo is getting higher and higher and the migration process is getting slower by time.
I tried to change my strategies but with no luck.
What can I do to increase the response time?
Basically I am scanning an SQL table getting 100 items per query then asking Couchbase to retrieve the data I want about these 100 items. At first I was getting high response times (as shown in the image bellow).
The following info might help:
I am running the migration code on an ec2 micro server running Ubuntu 14.04 with node v 4.4.1.
After looking at the graphs I started measuring the time for each dynamodb request (so I don't know what is the average was at first), the average response time is 800 ms for about 150,000 requests (get & put only, no batch commands or queries)
I am storing the items in two tables one with integer hash key and the other is with integer hash and sort keys
The second table is a huge one (having about 4.4 millions of items and counting)
Thanks to #Vor for mentioning the "Hot partition keys" I have made further readings and in reference to the Guidelines for Working with Tables I have followed what they are recommending and also I distributed my requests into batch requests made the difference.
Thank you all for your help.
We have sysdig running on our WSO2 API gateway machine and we notice that it fires a large number of SQL queries to the database for a minute, than waits a minute and repeats.
The query looks like this:
Every minute it goes wild, waits for a minute and goes wild again with a request of the following format:
SELECT REG_PATH, REG_USER_ID, REG_LOGGED_TIME, REG_ACTION, REG_ACTION_DATA
FROM REG_LOG
WHERE REG_LOGGED_TIME>'2016-02-29 09:57:54'
AND REG_LOGGED_TIME<'2016-03-02 11:43:59.959' AND REG_TENANT_ID=-1234
There is no load on the server. What is causing this? What can we do to avoid this?
screen shot sysdig api gateway process
This particular query is the result of the registry indexing task that runs in the background. The REG_LOG table is being queried periodically to retrieve the latest registry actions. The indexing task cannot be stopped. However, one can configure the frequency of the indexing task through the following parameter that is in the registry.xml. See [1] for more information.
indexingFrequencyInSeconds
If this table is filled up, one can clean the data using a simple SQL query. However, when deleting the records, one must be careful not to delete all the data. The latest records of each resource path should be left in the REG_LOG table since reindexing of data requires at least one reference of each resource path.
Also, if required, before clearing up the REG_LOG table, you can take a dump of the data in case you do not want to loose old records. Hope this answer provides information you require.
[1] - https://docs.wso2.com/display/Governance510/Configuration+for+Indexing
We've got a pretty typical django app running on postgresql 9.0. We've recently discovered some db queries that have run for over 4 hours, due to inefficient searches in the admin interface. While we plan to fix these queries, as a safeguard we'd like to artificially constrain database query time to 15 seconds--but only in the context of a web request; batch jobs and celery tasks should not be bounded by this constraint.
How can we do that? Or is it a terrible idea?
The best way to do this would be to set up a role/user that is only used to run the web requests, then set the statement_timeout on that role.
ALTER ROLE role_name SET statement_timeout = 15000
All other roles will use the global setting of statement_timeout (which is disabled in a stock install).
You will need to handle this manually. That is checking for the 15 second rule and killing the queries that violate it.
Query pg_stat_activity and find the violators and issue calls to pg_terminate_backend(procpid) to kill the offenders.
Something like this in a loop:
SELECT pg_terminate_backend(pg_stat_activity.procpid)
FROM pg_stat_activity
WHERE pg_stat_activity.datname = 'TARGET_DB'
AND usename = 'WEBUSERNAME'
AND (now()-query_start) > '00:00:15';
As far as the timing goes, you could pass all of your queries through a class which, on instantiation, spawns two threads: one for the query, and one for a timer. If the timer reaches 15 seconds, then kill the thread with the query.
As far as figuring out if the query is instantiated from a web request, I don't know enough about Django to be able to help you. Simplistically, I would say, in your class that handles your database calls, an optional parameter to the constructor could be something like context, which could be http in the event of a web request and "" for anything else.