What is a No page sink provider error in AWS Athena?
Full Error
GENERIC_INTERNAL_ERROR: No page sink provider for catalog
'awsdatacatalog'. You may need to manually clean the data at location
's3://bucket/tables/97577168-8d0d-4a9d-914d-74a6a4ed4b99' before
retrying. Athena will not delete data in your
account..'
SQL Query:
select * from database.table
This error in AWS Athena seems transient but not sure what is even wrong, we are able to query that database for other tables output location is the same. Later we can even query this database.table successfully.
Related
I'm trying to create a crawler a Redshift table. Although the crawler runs successfully, table is not created in the catalog, logs mentions "Finished writing to Catalog".
IAM role has full access permissions for Redshift,Glue,S3
I have 2 buckets in S3, where the second one is created with the settings of the first copied.
Both buckets are transformed into tables in Glue (in the same database, the first table via a crawler, the second manually created).
Both tables can be queried in Athena.
If I try to visualize the tables in QuickSight, the first table can visualized in graphs/etc.
If I try to visualize (or query in QuickSight) the second table. It gives me the error:
An error has been thrown from the AWS Athena client. Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID:
How can I avoid this?
I suspect it has to do with the security settings of the second bucket (because if it would have to do with IAM roles or QuickSight, I would expect the first table also not to visualize). But I copied these settings exactly from the first table, which are the default settings (no public access, no policies, etc.).
I tried to add the same S3 bucket via a manifest file, which gives me the same error.
Anyone that had the same issue and knows where to look?
Any help would be great.
Chris
I see I had to change the allowed buckets in the IAM policy for QuickSightAccessForS3StorageManagementAnalyticsReadOnly (and Glue in my case)
I am trying to migrate redshift to BQ, and and few stating steps goes fine at the last step I am getting some permission denied error to table get.
Error code 7 : Access Denied: Table XXXXXXXXXX:MA_test.zip_code: The user redshift#bigquery-data-connectors.iam.gserviceaccount.com does not have bigquery.tables.get permission for table XXXXXXXXXX:MA_test.zip_code.
I also allowed the popup that popups when creating the Redshift data transfer service.
We have access to other AWS redshift cluster. We want to unload one of the tables query results into our S3 bucket. Can we know what are all the permissions required to unload those results into our S3 bucket.
Tried to search in google but didn't find any related doc.
If you can access a Redshift, any table/view and could do SELECT on it, then you should be able to unload that select as well. No special permissions are required as far as Redshift is concerned.
Though you need to have valid S3 IAM role or S3 Access/Secret Key to unload the data.
Other way to look into unload is, its more of selecting bulk data and redirecting it to a physical store S3 for other purposes.
Here is official unload documentation page.
https://docs.aws.amazon.com/redshift/latest/dg/t_Unloading_tables.html
With S3 credentials
unload ('select * from venue') to 's3://mybucket/tickit/venue_' access_key_id <access-key-id>' secret_access_key '<secret-access-key>'
With IAM Role,
unload ('select * from venue') to 's3://mybucket/tickit/unload/venue_' iam_role 'arn:aws:iam::0123456789012:role/MyRedshiftRole';
I have 2 AWS accounts: Account1, and Account2. I have some data stored in S3 in Account1, and I registered that data into an Athena table in Account1. Now, I would like to access the same Athena table from Account2. I realize that I could create an Athena table in Account2 to query data in Account1, but ideally I would like to keep all the tables under Account1.
Since May 2021 it is now possible to register a data catalog from a different account in Amazon Athena, see the User Guide.
Athena Query Engine v2 is required though and there are some other limitations.
As of today, it seems possible only by deploying in Account2 (the account you want to query from) a Lambda with the proper cross-account permissions to access the data catalog in Account1.
See this other answer and the associated post on AWS blog.