I know AWS RDS-postgresql supports postgis extension.
Does AWS Redshift support postgis extension?
As per this answer was written (June 2016), AWS Redshift itself still does not support PostGIS extensions.
The official documentation have not much changed. In an AWS blog post, it said implicitly:
You can use the dblink extension to connect to Amazon Redshift and leverage PostgreSQL functionality. .... There are likely many other uses for the dblink extension with Amazon Redshift, such as PostGIS or LDAP support in PostgreSQL (Amazon EC2 only), ....
From the above AWS blog post, we can combine Amazon Redshift and RDS/self-hosted PostgreSQL database to make a PostGIS queries by using dblink.
If your data is in latitude-longitude pairs (EPSG-4326/WGS84) format, you can convert your data to single-dimension data using GeoHash or S2 Geometry Library hash. Personally, I prefer S2 Geometry Library because it has more precision levels and functionalities. After that, you can make some query over that hash column. And of course, you can combine the result to PostGIS database.
According to AWS docs,
Amazon Redshift is based on PostgreSQL 8.0.2. Amazon Redshift and PostgreSQL have a number of very important differences that you must be aware of as you design and develop your data warehouse applications.
(http://docs.aws.amazon.com/redshift/latest/dg/c_redshift-and-postgres-sql.html)
It is not supporting the basic types that PostGIS depends upon
(https://forums.aws.amazon.com/message.jspa?messageID=425664#)
Therefore, the answer is no.
Related
We are migrating our existing jobs from Hadoop to GCP.
I encountered two hdfs functions createSnapshot and snapshotDiff in our existing hadoop code. Do we have thier equivalent in GCP?
Table format projects Delta Lake and Apache Iceberg are now available in the latest version of Cloud Dataproc (version 1.5 Preview).
With these table formats, you can now use Dataproc for workloads that need:
ACID transaction
Data versioning (a.k.a. time travel)
Schema enforcement
Schema evolution and more
Data versioning provides a snapshot of your data in history. You can look up data history and roll back to the data at a certain time or version in history.
Please refer to the documentation. I hope it helps.
I'm facing the following situation:
I have to do a logical replication of my Cloud SQL PostgreSQL instance to an external PostgreSQL database and vice-versa. More specifically, the CloudSQL and the external instance will contain some master and slave tables that must be replicated between each database. However, PostgreSQL logical and external replication are currently not supported by the GCP.
Since it is not supported, I've searched for an alternative, and I noticed that Debezium + Kafka can be used with CloudSQL MySQL to capture database changes and do logical replication. I asked the same question on the Debezium google group (link below)
https://groups.google.com/forum/#!topic/debezium/yS61un46x8k
And they've answered to me:
"Debezium requires a specific logical decoding plug-in (either ProtoBufs or wal2json) installed within the source Postgres, I'm not sure whether you have that flexibility with your cloud SQL provider (but e.g. on Amazon RDS, wal2json is installed by default)."
Can anybody answer me if one of these plugins are available or can be installed within Cloud SQL PostgreSQL? If not, are there any alternative approach for logical replication while it is not supported by the platform? Thanks in advance.
Cloud SQL Postgres does not support logical replication right now. You also have no way to use Debezium + Kafka because of missing logical replication.
I have to migrate from Cloud SQL Postgres to on-premises Postgres.
Took them a long time, but is supported now
https://cloud.google.com/sql/docs/postgres/replication/configure-logical-replication
We are currently using AWS RDS as our databases. In tables, we defined some insert or update triggers on tables. I would like to know if Bigquery also support triggers?
thanks
BigQuery is a data warehouse product, similar to AWS Redshift and AWS Athena and there is no trigger support.
If you used AWS RDS so far, you need to check Google CloudSQL.
Google Cloud SQL is an easy-to-use service that delivers fully managed
SQL databases in the cloud. Google Cloud SQL provides either MySQL or
PostgreSQL databases.
If you have a heavy load, then check out Google Cloud Spanner it's even better for full scalable relational db.
Cloud Spanner is the only enterprise-grade, globally-distributed, and
strongly consistent database service built for the cloud specifically
to combine the benefits of relational database structure with
non-relational horizontal scale.
Big Query doesn't have the feature as stated by the colleague above.
However it has an event api based on it's audit logs. You can inspect it and trigger events with cloud functions as per:
https://cloud.google.com/blog/topics/developers-practitioners/how-trigger-cloud-run-actions-bigquery-events
Regards
I tried looking online but I couldn't find anything. We are trying to see if Amazon Aurora supports Updatable Views. We are not using Amazon Aurora yet but the plan is to use it in future.
Thanks,
Sridhar.
Because Amazon Aurora implements the Postgres and Mysql engines directly, you can expect all the features of the engine and version that you use on Aurora to work the same way. Updatable views are available in the current database engine versions for both of these engines in Amazon Aurora, and require the same configuration regarding rules and grants as you'd find on a non-Aurora environment.
Postgres engine versions: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostgreSQL.Updates.20180305.html
Mysql engine versions: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Updates.html
I'd like to determine what version of Amazon Athena I'm connected to by running a query. Is this possible? If so, what is the query?
Searching Google, SO, and AWS docs have not found an answer.
Amazon Redshift launches as a cluster, with virtual machines being used for that specific cluster. The cluster must be specifically updated between versions because it is continuously running and is accessible by only one AWS account. Think of it as software running on your own virtual machines.
From Amazon Redshift Clusters:
Amazon Redshift provides a setting, Allow Version Upgrade, to specify whether to automatically upgrade the Amazon Redshift engine in your cluster if a new version of the engine becomes available.
Amazon Athena, however, is a fully-managed service. There is no cluster to be created -- you simply provide your query and it uses the metastore to know where to find data. Think of it just like Amazon S3 -- many servers provide access to multiple AWS customers simultaneously.
From Amazon Athena – Interactive SQL Queries for Data in Amazon S3:
Behind the scenes, Athena parallelizes your query, spreads it out across hundreds or thousands of cores, and delivers results in seconds.
As a fully-managed service, there is only ever one version of Amazon Athena, which is the version that is currently available.