Is there a way to retrieve ARN of a resource through AWS SDK for Go? I created a couple of tables in DynamoDB and I want to retrieve the ARNs.
The ARN format:
arn:aws:service:region:account-id:resource-type:resource-id
How to retrieve the account-id and region via SDK for Go?
There is no generic way to get region from AWS SDK. By generic, here we consider simple code that returns a correct AWS region for your service deployed to ANY environment.
AWS assumes the opposite process. As a client, you are expected to know where your AWS resources deployed, and you have to inject region into an app that connects to AWS.
Think about your code running on your local machine in Europe, accessing AWS DynamoDB deployed in us-east-2 region, or code that needs to copy data from DB in region1 to DB in region2. In both these cases, the application cannot get the correct region without a hint.
In many cases, though, the environment where your code is deployed can provide that hint.
A few examples:
For local environment, you can configure default region for AWS SDK - https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html#cli-configure-quickstart-region. Your service picks up that region if you create client using config.LoadDefaultConfig
Another example is running your service on EC2. EC2 provides AWS metadata that includes current region and account. Current region can be requested using client.GetMetadata. GetInstanceIdentityDocument API returns both Region and Account ID.
If you control how your service is deployed, you can try to get current Region and Account ID from environment, otherwise common practice is setting ENV variables with Region and Account ID when you deploy your code.
Related
I'm trying to create an AWS EC2 instance using Terraform. It says it creates the instance and if I try run apply again it says it's already there. It isn't being created in my AWS account though and I can't figure out what's going on. I've regenerated the access id and secret from my AWS account and placed those directly in the .tf file and according to Terraform successfully created the instance - it's just not appearing in AWS.
Any ideas?
The most common reason for not being able to find resources in the AWS UI is due to looking at the wrong AWS region. Make note of the region in the top-right corner of the AWS console, and the region you are configuring Terraform to use.
I am new to AWS infra and currently doing some POC/Feasibility for new work.
So I have created a S3 bucket in Ireland server, train and publish Sagemaker endpoint in Ireland server and its giving result in Jupyter notebook there. Now I want to use that endpoint in my browser javascript library to show some graphics. When I try to test my endpoint in Postman then its giving region specific error
{
"message": "Credential should be scoped to a valid region, not 'us-east-1'.
Credential should be scoped to correct service: 'sagemaker'. "
}
My AWS account is not yet enterprise managed so I am using as 'root user', Whenever I go to my profile>Security_Credential page and generate any security credential then it always create for 'us-east-1' region, As Sagemaker is region specific service, I am not able to find the way to create region specific security key for root user, can someone please help
You should create an IAM role first that defines what should be permitted (mainly calling the invoke-endpoint API call for SageMaker runtime). Then you should create an IAM user, add the above role to that user, and then generate credentials that you can use in your Postman to call the service. Here you can find some details about the IAM role for SageMaker that you can use in this process: https://docs.aws.amazon.com/sagemaker/latest/dg/using-identity-based-policies.html
A popular option to achieve external access to a SageMaker endpoint, is to create an API Gateway that calls a Lambda Function that is then calling the invoke-endpoint API. This chain is giving you various options such as different authentication options for the users and API keys as part of API-GW, processing the user input and inference output using API-GW and Lambda code, and giving the permission to call the SageMaker endpoint to the Lambda function. This chain removes the need for the credentials creation, update and distribution.
I have a scaling group of several EC2 instances.
I have API keys which I would like to distribute to the instances using round-robin.
How can I code the instances to get the credentials once they go live?
Is there an AWS service for that?
It is not AWS credentials which could be solved by defining IAM Roles.
Thanks
Use "user data" option when you start your EC2 instance, You can run the bash script.
I recommend the following step.
1-put your cred or other shared information to S3 or dynamoDB.
2-write script to read and setting this data when your EC2 was starting.
The closest thing AWS has to this is called IAM Roles. A role includes a set of IAM permissions (like an IAM user). When you start a VM, you can set the role of the VM. The VM can then call the AWS API and get temporary credentials that give it access to the services that are defined in the IAM role.
See here for more details:http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html
This does not exactly meet your requirement for round-robin credentials distribution. But it might be a better option. IAM roles are as secure a method of distributing credentials to EC2 instances as you can get.
AWS now provides two services that could be used for that purpose:
The Secrets Manager would seem to be the most fitting, but does cost money from the start.
The Parameter Store is also an option and is free for up to 10k parameters.
I'm setting up a Kubernetes cluster on AWS and as part of the configuration for say the API Server, I provide the --cloud-provider=aws setting.
Once it starts up, however, I see in the logs that it complains about not having AWS credentials:
NoCredentialProviders: no valid providers in chain
After some searching, it seems that this issue was resolved for most people by using the "kube-up" script. However, for those who are not using the script to set up their cluster, how do we provide Kubernetes with AWS credentials?
It sounds like you don't have the appropriate IAM instance profile set on your master VM. The kube-up script for AWS creates a role and associated policy that is attached to the master VM when it is created. Having the IAM policy attached should give you the credentials necessary to make API calls into AWS.
When using the web UI for AWS Elastic Beanstalk Environment management, I see:
Code change deployments will complete faster if you associate an
instance profile with this environment.
(also see this forum post mentioning the same thing: http://www.infosys.tuwien.ac.at/staff/leitner/cs_study/forum/viewtopic.php?pid=186#p186)
What is an instance profile? Why does it matter? How does it work / what is it doing?
I found these articles:
http://docs.aws.amazon.com/IAM/latest/UserGuide/instance-profiles.html
http://docs.aws.amazon.com/cli/latest/reference/iam/create-instance-profile.html
but I still don't understand instance profiles.
Like Celine said in the comment above, instance profiles allow you to associate an IAM role to your instance. This IAM role must be provided with certain permissions to access your AWS resources. Your EC2 instance (launched by Elastic Beanstalk) can then perform certain extra tasks. For example if you launch a worker tier environment with Elastic Beanstalk the daemon needs to poll from SQS queue, publish metrics to cloudwatch from the EC2 instance. This means that the EC2 instance needs some credentials to poll from a queue. If you have an IAM role with appropriate policies associated with the EC2 instance you essentially permit your instance to call SQS using the credentials of that role.
You can do other interesting things like automatic log publication to your S3 bucket if you have an instance profile associated with your environment.
Having an instance profile allows you to control permissions you want to give to the instance and also frees you from storing long term credentials on all your EC2 instances.
From the documentation:
Instance profiles provide applications and services access to AWS
resources. For example, your application may require access to
DynamoDB. Every API request made to AWS services must be signed using
AWS security credentials. One way to grant applications access to AWS
resources is to distribute your credentials to each instance; however,
distributing long-term credentials to each instance is challenging to
manage and a potential security risk. Instead, you can create an IAM
role with the permissions that applications require when the
application makes calls to other AWS resources. When AWS Elastic
Beanstalk launches the Amazon EC2 instances, it uses the instance
profile associated with that role. All applications that run on the
instances can use the role credentials to sign requests. Because role
credentials are temporary and rotated automatically, you don't have to
worry about long-term security risks.
The message you are seeing on the console regarding console is recommending you to use an instance profile because that allows your EC2 instance to take a faster path to deploy your application version each time you update your environment with a new copy of your source code. The end result is the same but having an instance profile enables optimizations in deployment speed which are not possible without one.
You can read more about instance profiles with Elastic Beanstalk here.
Although you can create a custom role and associate it with a beanstalk environment by giving it appropriate permissions, you can for convenience just get a default role when you launch an environnment using the AWS console. You will have the option of choosing which role you want to associate with an environment in the create environment wizard.