Using mongo Bi connector via Atlas as source database
Using mysql for pre-aggregation
Currently, it's not possible to use pre-aggregations if the source data is stored in MongoDB. Please see this GitHub issue for details: https://github.com/cube-js/cube.js/issues/2858
If you'd like to use pre-aggregations for performance, syncing data from MongoDB into another data store is the only viable option.
Related
I am trying to use pre aggregations over CLOUD SQL on Google Cloud Platform but the database is denying access and giving error Statement violates GTID consistency.
Any help is appreciated.
Cube.js done pre-aggregation by CREATE TABLE ... SELECT, but you are using MySQL on top of Google SQL with --enforce-gtid-consistency (has limitations).
Since only transactionally safe statements can be logged, there is a limitation to use CREATE TABLE ... SELECT (and some another SQL), because this statement is actually logged as two separate events.
There are two ways how to solve this issue:
1. Use pre-aggregations to an external database. (recommended way).
https://cube.dev/docs/pre-aggregations/#read-only-data-source-pre-aggregations
2. Use not documented flag loadPreAggregationWithoutMetaLock
Attention: This flag is an experimental and can be removed or changed in the feature..
Take a look at the source code
You can pass it directly in the driver constructor. This will produce two SQL statements to pass the limitation:
CREATE TABLE
INSERT INTO
Thanks
We have google cloud MySQL Instance and we are getting error like:
InnoDB: Cannot add field CE_Lead__c in table client_1067.#sql-ib1179556-1243586840 because after adding it, the row size is 8148 which is greater than maximum allowed size (8126) for a record on index leaf page.
The workaround to fix this issue is to set internal_tmp_disk_storage_engine=MYISAM
I can't see this flag in Database flags section, also as per the google documents this variable is not exist in google cloud, however I can see it exist in MySQL 5.7
Database flag not exist in google cloud supported flag:
https://cloud.google.com/sql/docs/mysql/flags#list-flags-mysql
Database flag exist in MySQL 5.7
https://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html#sysvar_internal_tmp_disk_storage_engine
Thanks
SKS
Some flags are not supported in Cloud SQL MySQL due to the managed nature of the service (stability/streamline reasons, etc.) You could try other workarounds like decreasing the memory usage mentioned further below on this post, if you decide you need to use flags that are not supported on Cloud SQL you can choose to create your own self managed MySQL instance in order to bypass the Cloud SQL limitations. (for example using Compute Engine)
We are working on the ETL. How to read data from the POSTGRESQL data base using streams in DATA ANALYTICS SERVER and manipulate some operations using the streams and insert the manipulated data into another POSTGRESQL data base on a scheduled time. Please share the procedures to follow.
Actually, you don't need to publish data from your PostgreSQL server. Using WSO2 Data Analytics Server (DAS) you can pull data from your database and do the analysis. Finally, you can push results back to the PostgreSQL server. In DAS, we have a special connector called "CarbonJDBC" and using that connector you can easily do this.
The current version of the "CarbonJDBC" connector supports following database management systems.
MySQL
H2
MS SQL
DB2
PostgreSQL
Oracle
You can use following query to pull data from your PostgreSQL database and populate a spark table. Once spark table is populated with data, you can start you data analysis tasks.
create temporary table <temp_table> using CarbonJDBC options (dataSource "<datasource name>", tableName "<table name>");
select * from <temp_table>;
insert into / overwrite table <temp_table> <some select statement>;
For more information regarding "CarbonJDBC" connector please refer following blog post [1].
[1]. https://pythagoreanscript.wordpress.com/2015/08/11/using-the-carbon-spark-jdbc-connector-for-wso2-das-part-1/
I'm not too familiar with ArcGIS, but is it possible to export geo-spatial data from SQL Server in a format that can easily be imported by ArcGIS? Is there a proprietary format for ArcGIS?
I'm not interested (I don't think) in purchasing ArcServer, I just want my data exposed (via web services) a client who does use Arc.
Thanks
If you are using ArcMap 10.0 version, then use the Query Layer feature to add SQL Server Spatial table data into ArcMap 10.
Use "File->Add Data-> Add Query Layer.." interface.
Using this You can connect to a SQL Server spatial tables and add them into ArcMap. Once you added into ArcMap, You can convert these data into Shape file format (a generally used GIS data files for ESRI ArcGIS tools). There are certain limitation with this method.
Another way is use ogr2ogr tool
see the link for more details
http://alastaira.wordpress.com/2011/09/29/exporting-spatial-data-from-sql-server-to-esri-shapefile/
I have C++ code, and from it I need to access the DB and make a query in table (with name NECE_TABLE, which has 2 columns - IntID and Status).
Here I need to get "status" column value from DB table (NECE_TABLE) using the IntID from C++ code.
Any help will be greatly helpful. Thanks in advance
Your question is very vague, but in summary you need to:
Use an appropriate client library supported by your database to connect to that database using some user credentials with appropriate permissions for SELECTing from your table
Execute a SQL select to fetch the data you want
There's some confusion as to which database you're using.
If you're using Oracle, you can use the OCCI client library to connect to the database and execute SQL statements. See section 2 of the linked document, where it describes connecting to a database and executing SQL queries.
Take a look at this link - it's a simple tutorial on how to get started with MySQL and C++. You say you are using vanilla SQL in your tags, but the two should be compatible as long as you stick to the more basic queries.