I was faced with a very strange behavior of Data Service Server (v. 3.5.0).
I prepared quite simple service with some resources for MS SQL Server RDBMS data source. When I call one of resources with simple select query I get answer after 6 seconds (six seconds).
The same select in MS SQL Server Management Studio returns data after 15-100 ms.
The same select in WSO2 DSS Database Explorer returns data after max 15 ms.
The same select in Netbeans returns data after max 100 ms.
I tried JSTL and Microsoft drivers. The result is the same.
Everywhere except DSS service I get answer in max. 100 milliseconds. Result of these queries is very small - 6-8 rows with about 10 columns.
Which is the reason for such behavior?
Could somebody help me?
The reason is very, very strange.
I tried a simple query like:
SELECT a1,a2 .... FROM someView WHERE a1=:parameter
When parameter is of type STRING (I think, like parameter for PreparedStatement) query returns results in about 4000 milliseconds (???).
If parameter is of type QUERY_STRING query returns results in 10 milliseconds (!!!!). Result is very small - about 10 rows with 5 columns.
Time was measured by net.sf.log4jdbc.DriverSpy, but the same difference I get with pure Microsoft Jdbc and Jtds drivers.
Why such a big difference? 400 times faster?
One question remained: What is to blame:
WSO2 DSS software?
JDBC driver?
SQL Server?
Related
I have a dataset for 3 days I can not update it on Power BI service, knowing that on DESKTOP it is normally updated in 30 minutes. the dataset is powered from a SQL server database, via a data gateway. the data gateway is well updated. the incremental update is activated and the dataset retrieves only the data of the last 3 days to each update.
Here is the generated error message :
Data source error: The XML for Analysis request timed out before it was completed. Timeout value: 17998 sec.
Cluster URI: WABI-WEST-EUROPE-B-PRIMARY-redirect.analysis.windows.net
Activity ID: 680ec1d7-edea-4d7c-b87e-859ad2dee192
Application ID: fcde3b0f-874b-9321-6ee4-e506b0782dac
Time: 2020-12-24 19:03:30Z
What is the solution to this problem please.
Thank you
What license are you using?
Without premium capacity, the max dataset size can be 1GB. Maybe your dataset size has crossed this mark? If you are using the shared capacity, then you can check the workspace utilized storage size by clicking on ellipses at top right corner. Then click on storage to see how much is utilized for that workspace.
Also note that in shared capacity there is a 10GB uncompressed dataset limit at the gateway (but this should not be an issue as you have only 3 day data in refesh)
Also check whether your power query query is foldable (on the final step you should be able to see the 'show native query' option). If not then incremental refresh does not work and results in querying the entire data.
Also, note that 17998 sec means 5 hours. What is your internet speed?
I am running a functional test of a 3rd party application on an Azure SQL Data Warehouse database set at DWU 1000. In reviewing the current activity via:
sys.dm_pdw_exec_requests
I see:
prepare statements taking 30+ seconds,
NULL statements taking up to 25 seconds,
compilation of statements takes up to 60 seconds,
explain statements taking 60+ seconds, and
select count(1) from empty tables take 60+ seconds.
How does one identify the bottleneck involved?
The test has been running for a few hours and the Azure portal shows little DWU consumed on average, so I doubt that modifying the DWU will make any difference.
The third-party application has workload management feature, so I've specified a limit of 30 connections to the ADW database (understanding that only 32 sessions are active on the database itself.)
There are approximately ~1,800 tables and ~350 views in the database across 29 schemas (per information_schema.tables).
I am in a functional testing mode, so many of the tables involved in the queries have not yet been loaded, but statistics have been created on every column on every table in the scope of the test.
One userID is being used in the test. It is in smallrc.
have a look at your tables - in the query? Make sure all columns in joins, group by, and order by have up-to-date statistics.
https://learn.microsoft.com/en-us/azure/sql-data-warehouse/sql-data-warehouse-tables-statistics
We have been using Couchbase for about two years but we finally decided to switch to Amazon DynamoDB service for many reasons.
Now I started the migration of data to dynamodb. First everything was alright and going as expected, but after some time the response time from dynamo is getting higher and higher and the migration process is getting slower by time.
I tried to change my strategies but with no luck.
What can I do to increase the response time?
Basically I am scanning an SQL table getting 100 items per query then asking Couchbase to retrieve the data I want about these 100 items. At first I was getting high response times (as shown in the image bellow).
The following info might help:
I am running the migration code on an ec2 micro server running Ubuntu 14.04 with node v 4.4.1.
After looking at the graphs I started measuring the time for each dynamodb request (so I don't know what is the average was at first), the average response time is 800 ms for about 150,000 requests (get & put only, no batch commands or queries)
I am storing the items in two tables one with integer hash key and the other is with integer hash and sort keys
The second table is a huge one (having about 4.4 millions of items and counting)
Thanks to #Vor for mentioning the "Hot partition keys" I have made further readings and in reference to the Guidelines for Working with Tables I have followed what they are recommending and also I distributed my requests into batch requests made the difference.
Thank you all for your help.
My function running query, its taking sometime depending on requested data. While query running, I want to show real query loading cfprogressbar & changing status/title while cfprogressbar doing progress. im still searching on google till now no luck all examples showing static time.
I was thinking if i can get real cfquery loading time & i'll pass that value to cfprogressbar. Please advise
coldfusion 11
windows 2012
cfprogressbar
cfquery
Unfortunately you can't show an accurate progress bar. The ColdFusion engine does not run your query, the database server runs your query and for that reason the ColdFusion engine does not know where your db server is at while running the query. You can show a 'spinner' if you want to let your user know that something is going on.
This is regarding Microsoft sync framework where we are syncing Ipad data with Sql server database.
It is working fine.
But here, I want to limit my records to be synched to only 20 records at a time. Right now, all the records are getting synched.
I there a out-of box feature by sync framework which will enable us to do so.
If not, how can I write a custom code to achieve this.
you should be able to set batching. but it's in terms of size, not number of rows. you can simply get your row size and multiply by the number of rows you have in mind.
lookup SetDownloadBatchSize and SetBatchSpoolDirectory in the documentation.
e.g. config.SetDownloadBatchSize = some value in Kb