I have try to create view in athena using pyspark like below.
spark.sql("CREATE OR REPLACE VIEW work.test_view123 AS SELECT emp_name,emp_no FROM emp.emp_table")
But I am not able to see view in work db.
I think you need to use the SDK to get a reference to the Athena API and use it to execute a query with the create view statement.
Related
I have been working with AWS Athena for a while and need to do create a backup and version control of the views. I'm trying to build an automation for the backup to run daily and get all the views.
I tried to find a way to copy all the views created in Athena using boto3, but I couldn't find a way to do that. With Dbeaver I can see and export the views SQL script but from what I've seen only one at a time which not serve the goal.
I'm open for any way.
I try to find answer to my question in boto3 documentation and Dbeaver documentation. read thread on stack over flow and some google search did not took me so far.
Views and Tables are stored in the AWS Glue Data Catalog.
You can Query the AWS Glue Data Catalog - Amazon Athena to obtain information about tables, partitions, columns, etc.
However, if you want to obtain the DDL that was used to create the views, you will probably need to use SHOW CREATE TABLE [db_name.]table_name:
Analyzes an existing table named table_name to generate the query that created it.
Have you tried using get_query_results in boto3? get_query_results
After creating a view in Athena (Presto), I would like to add a comment for each column and also a comment for the view itself. Looking at the AWS official documentation, I can't find any reference for this. Any ideas?
I want to select a saved query as datasource in my charts but Superset only displays views and tables as data sources.
What is required to select a query as a datasource?
In SQL Lab, after you execute the query, you have an option to visualize the query, and then you are able to create a chart using the query, save it, and use it in a Dashboard once saved.
Not very intuitive, I had to look for it too ;-)
IDK if it is still relevant but you can query the dataset. This can be very useful when you're doing nested queries. You can do this with the help of jinja templating. make sure you have enabled them in the config file
select * from {{dataset(233)}}
I've created a simple view in Redshift that is select * from source_a with no schema binding. Now I have a new table, source_b, that I would like the view to point to instead that has no new columns or datatypes.
How can I repoint the view without dropping permissions?
You run the CREATE again but include the OR REPLACE syntax.
CREATE OR REPLACE VIEW my_view AS …
WITH NO SCHEMA BINDING ;
You can retrieve the existing view definition with pg_get_viewdef.
SELECT pg_get_viewdef('my_view', true);
First off take a look at the create view documentation page - https://docs.aws.amazon.com/redshift/latest/dg/r_CREATE_VIEW.html
You have made a late-binding view which checks permissions on the tables when executed so you need to make sure you have the correct permissions on that new table - "source_b". These late-binding views are a the lookup / reference to the tables rather than a predefined link in the database. You also cannot grant column lever access to a late binding view. Otherwise permissions work the same as regular views.
To change the view without having to change the permissions on it you will want to use "create or replace view ..." (see link above) to update the definition. Just remember that the permissions of the new table will matter.
Is there a bug in the information_schema.views implementation in AWS Athena?
I'm getting 0 rows returned when running the query SELECT * FROM information_schema.views in AWS Athena even though the database I'm running on has tables and views in it.
Wanted to check if anyone else is facing the same issue.
I'm trying to fetch the view definition script for ALL views in the AWS Athena database as a single result set instead of using the SHOW CREATE VIEW statement.
You should be using
SHOW TABLES IN example_database; to get information about tables in Athena database.
And the loop through use describe query to fetch details.
Hope this will help!