I was trying to edit schema from AWS Glue console.
I am getting following error while trying to save my changes.
{"service":"AWSGlue","statusCode":400,"errorCode":"AccessDeniedException","requestId":"644bfcb2-75a8-456b-b17a-e22e829345d2","errorMessage":"Insufficient Lake Formation permission(s): Required Alter on recidivism_clustering_model_output_csv","type":"AwsServiceError"}
I have provided necessary access to the s3 buckets and tables in glue using Lakeformation.
Can anyone tell me what the best way to edit schema via glue?
And wny am i getting that error?
I had to manually add my user as a database owner to the database resource. Not sure why this sometimes happens. My role is a datalake administrator but for some reason that did not work if I added my role as a Database admin then I could alter the database.
There is probably something else going on but I could not find it.
Related
I am getting this message when I try to create a crawler on AWS Glue:
{"service":"AWSGlue","statusCode":400,"errorCode":"AccessDeniedException","requestId":"RequestIDNumber","errorMessage":"Account <AccountID> is denied access.","type":"AwsServiceError"}
already attached all those policies below to the IAM
All my Policies here
Already setup permissions to the AWS lake formation for the role too
Already created a custom policy kms to it too
And I am stucked, I cannot create a crawler!
I am in the root account, actually there`s no other account just the root, It's a super new account I created in AWS so I don't know what to do to be able to create this simple crawler
My ideia its to use it with dynamodb as data source
The message says there is something wrong with my account permission not with a role
Someone has an idea?
Thank you so much
Not sure the reason, but loads of people have been having this issue. You can submit a ticket through AWS account support. I actually submitted two, including one through the unpaid support version and one through my paid account. They answered both tickets.
Basically, just tell them you need access to Glue and they should alter whatever it is to give you access. Sorry it's not a better answer, but I found no other useful information anywhere.
I propose to try using a separate user and attaching the policy you created to it.
I am trying to create a crawler in AWS Glue, but it gives error: {"service":"AWSGlue","statusCode":400,"errorCode":"AccessDeniedException","requestId":"<requestId>","errorMessage":"Account <accountId> is denied access.","type":"AwsServiceError"}.
This is what I've done so far:
Create a database in AWS Glue
Add tables in the database using a crawler
Name the crawler
Choose Amazon S3 as the data store and specified a path to a csv file inside a bucket in my account
Choose an existing IAM role I've created before
Choose a database I've created before
Press finish.
When I press finish, the above error is occurred.
I have grant AdministratorAccess both to IAM user and role used to create the crawler, so I assume there is no lack of permission issues. The bucket used is not encrypted and located in the same region as the AWS Glue.
I also have tried to create another database and specified a path to a different csv file but it is not solved the problem.
Any help would be very appreciated. Thanks.
I have contacted the owner (the root user) of this account and the owner asked for help to AWS Premium Support. The AWS Premium Support told us that all the required permissions to create AWS Glue Crawler are already provided and there is no SCPs attached to the account. After waiting around 7-working-day, finally I can create AWS Glue Crawler without any errors.
Unfortunately, I don't have any further information on how the AWS Premium Support solve the issue. For those of you who encounter similar errors like me, just try to contact the owner of the account, because most likely the issue is out of your control. Hope this helps in the future. Thanks.
I am getting the following error
"Amazon.Runtime.AmazonServiceException: Unable to get IAM security credentials from EC2 Instance Metadata Service"
when trying to run a request to create a table for DynamoDb
Its my first time using this, and very new to AWS, I have managed to publish an AWS Lambda function and a webHook calls it and all working fine, and in doing this I created an AWS profile so that i can publish the project
However, I'm now trying to create a table within DynamoDB, which is causing the above error :(
Its probably something simple, like I have forgotten a config value somewhere, or haven't created the user for Dynamo or something... or maybe need to activate Dynamo on my AWS account
Any help will be very much appreciated
Thank you
Is it possible to directly access AWS Glue Data Catalog of Account B via the Athena interface of Account A?
I was just trying to resolve this same issue in my own setup, but then stumbled across this bummer (the last bullet under Cross-Account Access Limitations on this page):
Cross-account access to the Data Catalog is not supported when using an AWS Glue crawler, Amazon Athena, or Amazon Redshift.
So it sounds like even with the cross-account access that is possible today, they won't naturally replicate through those services (including the asked about Athena).
That said, I was able to set up cross-account access to the AWS Glue Data Catalog in a way that allowed me to use Account A to pull all relevant info about Data Catalog objects from Account B. I can update my answer to incorporate how far I got, if you want, but a hacky method that might solve this question would be to set up the cross-account access that is possible today then run a recurring Lambda function that replicates over all the relevant metadata in the Data Catalog from Account B to Account A so users in Account A can view that within Account A's AWS Glue Data Catalog. I'm not sure whether Athena specifically would work in that setup, as I know it requires PutObject access when it queries data in S3 (which could be solved via the appropriate S3 bucket policies, but that'd be another cross-account permissions thing to manage).
Let me know whether you'd like to see those details on what cross-account stuff I was able to get working.
AWS has started supporting this using Lambda, please follow below link
https://aws.amazon.com/blogs/big-data/cross-account-aws-glue-data-catalog-access-with-amazon-athena/
Since May 2021 it is now possible to register a data catalog from a different account in Amazon Athena, see the User Guide.
Athena Query Engine v2 is required though and there are some other limitations.
My IAM users can't see the Athena tables I've created a long time ago using the root account.
Their group has the following permissions:
AmazonS3FullAccess
AmazonAthenaFullAccess
They only see the sampledb databases, which is unfortunate, because they need the one we actually use. The documentation is not clear on how to make the databases accessible to everyone. How do I achieve that?
Your permissions are correct.
Athena's context is not currently shared across regions. Ensure that the users are viewing Athena from the same region as the root account. When they login to AWS, they may be initially placed in another region.
You need Glue permissions, Glue is the service in charge of manage Databases and Tables in AWS