I am new to AWS infra and currently doing some POC/Feasibility for new work.
So I have created a S3 bucket in Ireland server, train and publish Sagemaker endpoint in Ireland server and its giving result in Jupyter notebook there. Now I want to use that endpoint in my browser javascript library to show some graphics. When I try to test my endpoint in Postman then its giving region specific error
{
"message": "Credential should be scoped to a valid region, not 'us-east-1'.
Credential should be scoped to correct service: 'sagemaker'. "
}
My AWS account is not yet enterprise managed so I am using as 'root user', Whenever I go to my profile>Security_Credential page and generate any security credential then it always create for 'us-east-1' region, As Sagemaker is region specific service, I am not able to find the way to create region specific security key for root user, can someone please help
You should create an IAM role first that defines what should be permitted (mainly calling the invoke-endpoint API call for SageMaker runtime). Then you should create an IAM user, add the above role to that user, and then generate credentials that you can use in your Postman to call the service. Here you can find some details about the IAM role for SageMaker that you can use in this process: https://docs.aws.amazon.com/sagemaker/latest/dg/using-identity-based-policies.html
A popular option to achieve external access to a SageMaker endpoint, is to create an API Gateway that calls a Lambda Function that is then calling the invoke-endpoint API. This chain is giving you various options such as different authentication options for the users and API keys as part of API-GW, processing the user input and inference output using API-GW and Lambda code, and giving the permission to call the SageMaker endpoint to the Lambda function. This chain removes the need for the credentials creation, update and distribution.
Related
Is there a way to retrieve ARN of a resource through AWS SDK for Go? I created a couple of tables in DynamoDB and I want to retrieve the ARNs.
The ARN format:
arn:aws:service:region:account-id:resource-type:resource-id
How to retrieve the account-id and region via SDK for Go?
There is no generic way to get region from AWS SDK. By generic, here we consider simple code that returns a correct AWS region for your service deployed to ANY environment.
AWS assumes the opposite process. As a client, you are expected to know where your AWS resources deployed, and you have to inject region into an app that connects to AWS.
Think about your code running on your local machine in Europe, accessing AWS DynamoDB deployed in us-east-2 region, or code that needs to copy data from DB in region1 to DB in region2. In both these cases, the application cannot get the correct region without a hint.
In many cases, though, the environment where your code is deployed can provide that hint.
A few examples:
For local environment, you can configure default region for AWS SDK - https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html#cli-configure-quickstart-region. Your service picks up that region if you create client using config.LoadDefaultConfig
Another example is running your service on EC2. EC2 provides AWS metadata that includes current region and account. Current region can be requested using client.GetMetadata. GetInstanceIdentityDocument API returns both Region and Account ID.
If you control how your service is deployed, you can try to get current Region and Account ID from environment, otherwise common practice is setting ENV variables with Region and Account ID when you deploy your code.
I want to call api gateway from our own backend which is a ECS cluster, and I want to use IAM authorization, is there any way to not manually sign the request using Access Key and Secret Key?
For example when Lambda call KMS to decrypt environment variables, no need to configure the AWS SDK. Wondering if there's similar thing for API gateway.
It is definitely possible, even more - it's a security best practice. You can assign IAM roles to all computing services of AWS: Lambda, EC2, ECS, Beanstalk etc. On ECS you can assign IAM roles to your tasks.
It gives a great benefit, which is well described in official docs:
Benefits of Using IAM Roles for Tasks
Credential Isolation: A container can only retrieve credentials for the IAM role that is defined in the task definition to which it belongs; a container never has access to credentials that are intended for another container that belongs to another task.
Authorization: Unauthorized containers cannot access IAM role credentials defined for other tasks.
Auditability: Access and event logging is available through CloudTrail to ensure retrospective auditing. Task credentials have a context of taskArn that is attached to the session, so CloudTrail logs show which task is using which role.
This link will help you: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html
This page shows how to send an email using SES. The example works by reading the credentials from ~/.aws/credentials, which are the root (yet "shared"??) credentials.
The documentation advises in various places against using the root credentials.
Acquiring temporary credentials
using roles is mentioned as an option, yet assume_role() is not defined for SES client objects.
How do I send an email through SES with temporary SES-specific credentials?
Update
The context for my question is an application running on an EC2 instance.
There are a few pieces to this.
First you need an IAM policy. You can use one of the built-in policies, such as AmazonSESFullAccess or you can create your own. The holder of a particular policy will be able to access the resources and actions defined in the policy. You can create this policy manually, or work through the AWS console and it will walk you through it. IAM --> Policies --> Create Policy
Secondly, you will need a role. Also, easily done in the console. IAM --> Roles --> Create role. Trusted entity is AWS service. Highlight EC2. In the next screen, select the policy you want to associate with this role. This is the policy you created above. If your EC2 already has a role, then you can add the IAM policy to this role. Assigning an IAM policy to a role, is what they refer to as a trust policy.
Now any code that runs on your EC2 instance will be able to send messages to your SES service. The EC2 assumes the role assigned to it. And the SES policy is defined for that role. This will allow EC2 to get temporary credentials (behind the scenes).
The back story is as follows. Any API call to an AWS service needs to have a key and secret. When you make API calls from your local computer, you may use your personal key and secret (or even root ones). When you need to make API calls from another service, you do not have that key and secret. It would not be secure or practical to store the credentials on an EC2. Or even worse, in an S3 bucket. That is why AWS came up with the Role concept. Roles can request temporary credentials from an internal service called Simple Token Service (STS). A role is attached to an EC2 instance for example. And if the right policy is attached to that role, the EC2 instance can request to get temporary credentials to make an API call to another service. All of this happens behind the scenes.
Two options...
You could create IAM User credentials with the appropriate permissions and put them in the ~./aws/credentials file. Then your application will find them and use them to connect with Amazon SES.
Or, your application could use a set of IAM User credentials to call assume_role() (which is an IAM command). This will return a set of temporary credentials that could be used with Amazon SES. However, if you are going to provide a set of credentials that will be used to call assume_role(), then you may as well just use those credentials directly with Amazon SES.
An IAM User can be used for people OR applications.
I am trying to setup spinnaker locally to manage AWS EC2 instances. The current documentaion depicts the steps which need to have spinnaker instance to be running on EC2. They are creating one role and attaching it to spinnaker instance. As I am running spinnaker in my local environment, I am finding a way which will allow my local spinnaker instance to access the AWS resources. Will it be possible to have one such policy/role ? May be using AWS-STS ( Security Toke Service ), but i dont know how to use that creds with spinnaker instance
You can do this directly by creating an IAM user with required policies to access AWS Resources and use the Programmatic access Credentials in your local machine to use AWS CLI, API or SDKs.
For an existing IAM User, the step are as follows.
IAM User -> Security Credentials -> Create Access Keys
Note: If you cannot trust your local environment, then you can use AWS STS service (For this you need to implement a separate service, where you can pass user credentials and request for a temporal token from AWS STS)
You can create the IAM role for your local machine to assume, like
this example, or stricter,
spinnaker will handle the STS assume role given its configured properly
as for the temporary credential, if what you mean is MFA compatibility, I am myself still figuring out the way to do it. I think one workaround is to create a wrapper script that call sts:assumeRole, ask the user to provide the MFA token, then set AWS_ACCESS_KEY, AWS_SECRET_KEY, and AWS_SESSION_TOKEN which will be honored by clouddriver, but then deployment to multiple AWS accounts will be a problem
One of the things that drives me nuts is that AWS has loads of docs about the format of an ARN, but doesn't have any kind of generator to make you confident that the ARN is correct.
In IAM, I'm trying to set up a policy to allow access to an API Gateway and I've read the following docs about it:
http://docs.aws.amazon.com/apigateway/latest/developerguide/permissions.html#api-gateway-control-access-using-iam-policies
http://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html#arn-syntax-apigateway
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/AWSHowTo.iam.policies.arn.html
But I can't get any ARN to validate, even just a wide open API Gateway ARN. See screenshot:
What am I doing wrong here?
From the documentation: To create an IAM policy using the Policy Generator in the IAM console, select Manage Amazon API Gateway as AWS Service to set permissions statements for apigateway and select Amazon API Gateway as AWS Service to set permission statements for execute-api.
If you are creating a policy to manage creating/editing your API, then you will need to select Manage - Amazon API Gateway and then use * to give permission for all resources. If you want to give permissions for specific resources, then use this format: (note that the service name is apigateway)
arn:aws:apigateway:region::resource-path-specifier.
If you are creating a policy to manage invoking your API, then you will need to select Amazon API Gateway and then use * to give permission for all resources. If you want to give permissions for specific resources, then use this format: (note that the service name is execute-api)
arn:aws:execute-api:region:account-id:api-id/stage-name/HTTP-VERB/resource-path-specifier.