I am trying to execute a sql statement against Athena using sqlworkbench. I have executed several queries and know I have a connection if that is the first question. What would be the solution to renaming a database in Athena, or maybe Athena through the jdbc?
alter schema geoosm rename to geo_osm
An error occurred when executing the SQL command: alter schema
geoosm rename to geo_osm
[Simba]AthenaJDBC An error has been thrown from the AWS
Athena client. line 1:24: mismatched input 'rename' expecting 'SET'
[Execution ID not available] [SQL State=HY000, DB Errorcode=100071] 1
statement failed.
Execution time: 0.27s
My SQL syntax comes in Athena from Presto documentation which from my understanding is the syntax used by Athena.
8.1. ALTER SCHEMA Synopsis
ALTER SCHEMA name RENAME TO new_name
Sorry but there is no way to rename a database in AWS Athena. Fortunately, table data and table definition are two completely different things in Athena.
You can just create a new database with the right name, generate all DDL's for your table and execute them using the new database.
The "new" tables in the new database will still pointing to the same location so nothing to worry about.
Related
HIVE_CANNOT_OPEN_SPLIT: Error opening Hive split s3://exp-mahesh-sandbox/Demo/Year=2017/Month=1/Day=3/part-00015-d0e1263a-616e-435f-b4f4-9154afb3f07d.c000.snappy.parquet (offset=0, length=12795): Schema mismatch, metastore schema for row column statistical has 17 fields but parquet schema has 9 fields
I have used AWS Glue crawler to get the schema of the Parquet files. Initially I am having few files in the partition Day=1 and Day=2, run crawler and able to query it using Athena. After adding few more files in the partition Day=3, where the schema of file with "statistical"(type:struct) column has some missing fields, Athena throws the above mentioned error.
Is there any way to solve this issue. I am expecting null value in the missing fields.
I have tried UPDATE THE TABLE DEFINITION IN THE DATA CATALOG option in the crawler, but it gives the same result.
Crawler Settings
You're getting that error because at least one of your Parquet files has a schema that is either different from the other files that compose the table or from the table's definition itself; it appears to be your "Day=3" partition.
This is a limitation in Athena, that requires that the files that are the data source for a table have the same schema, i.e. all the files' columns need to match Athena's table definition, even struct members.
This error happens despite the Glue crawler running successfully; the table definition is indeed updated by the crawler, but when you execute a query that touches a file with a different schema (e.g. missing a column) you get a HIVE_CANNOT_OPEN_SPLIT error.
I have a .sql file filled with Athena queries.
Is there a way I can tell Athena to run the sql queries saved in s3://my-bucket/path/to/queries.sql?
In MySQL can do something like this (based in SO answer), but curious if possible in Athena
mysql> source \home\user\Desktop\test.sql;
Is there a way I can tell Athena to run the sql queries saved in s3://my-bucket/path/to/queries.sql?
I think there is no direct way to tell Athena to run query stored in S3.
In MySQL can do something like this (based in SO answer), but curious if possible in Athena.
If you want to do it at all, then yes, you should be able to run the query using AWS CLI.
Your steps should be look like this.
Get the query from S3 using CLI and store in temp variable
Pass the query stored in a temp variable to Athena Query CLI
Hope this will help.
I am trying to execute a sql statement against Athena using sqlworkbench. What would be the solution to switching a databases in Athena, or more generally Athena through the jdbc?
use AwsDataCatalog.geoosm
An error occurred when executing the SQL command: use
AwsDataCatalog.geoosm [Simba]AthenaJDBC An error has been
thrown from the AWS Athena client. line 1:19: mismatched input '.'
expecting [Execution ID not available] [SQL State=HY000, DB
Errorcode=100071] 1 statement failed.
Execution time: 0.18s
My SQL syntax in Athena comes from Presto documentation which from my understanding is the syntax used by Athena.
8.39. USE Synopsis
USE catalog.schema USE schema
The use statement which is supported in presto is not supported in Athena at this time.
But for cross database queries lower-case awsdatacatalog.geoosm, actually works.
I am trying to change a column name in an AWS Athena table.
From old_name to new_name.
Normal DDL commands does not affect the table (They cannot be executed).
Is It possible to change a column name without deleting and re-creating the table from scratch ?
I was mistaken, Athena uses HIVE DDL syntax so the correct command is :
ALTER TABLE %%table-name%% CHANGE %%old-column-name%% %%new-column-name%%<string>;
I based my answer on a hive related question.
You can find more about supported and unsupported DDLs here
I created a database and some tables with Data on AWS Athena and would like to rename the database without deleting and re-creating the tables and database. Is there a way to do this? I tried the standard SQL alter database but it doesn't seem to work.
thanks!
I'm afraid there is no way to do this according to this official forum thread. You would need to remove the database and re-create it. However, since Athena does not store any data by itself, deleting a table or a database won't impact your data stored on S3. Therefore, if you kept all the scripts that create external tables, re-creating a database should be fairly quick thing to do.
Athena doesn't support renaming database. You need to recreate database with a new name.
You can use Presto which is an open source version of Athena and Presto supports more DDL queries.