Actually, I use kubernetes service accounts mostly with NodeJS, and this works fine, but I have this one service made in Go and I can't seem to make it work with service accounts (I know that the service account is correctly configured because I tested it with a pod).
I'm using this lib https://github.com/aws/aws-sdk-go
Up till now I tried this:
sess := session.Must(session.NewSession())
creds := stscreds.NewCredentials(sess, os.Getenv("AWS_ROLE_ARN"))
svc := s3.New(sess, &aws.Config{Credentials: creds})
And also this (just in case):
region := os.Getenv("AMAZON_REGION")
sess := session.Must(session.NewSession(&aws.Config{Region: ®ion}))
svc := s3.New(sess)
for the first case I got the following error:
AccessDenied: User: arn:aws:sts::xxxxxxxx:assumed-role/staging-worker-node/i-0xxxxxxxxx is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::xxxxxxxx:role/EKSServiceAccount-app
and for the second case, I got a generic permission error.
I read the docs and tried a few things more (that may not be relevant here), but I can't see to make it work, maybe because I don't have much experience with golang.
There are a few things you can try to get your Go service to work with service accounts on Kubernetes:
Verify that your Go service is properly configured to use the Kubernetes service account. This can be done by checking that the service account is correctly mounted as a volume in the pod definition and that the service is able to read the credentials from the volume.
Make sure that the AWS SDK for Go you are using (https://github.com/aws/aws-sdk-go) is configured to use the correct credentials. The SDK supports several methods for providing credentials, including environment variables, shared credentials file, and IAM roles.
You can try using the k8s.io/client-go library instead of the AWS SDK for Go, this will help you to use the Kubernetes service account to authenticate with the Kubernetes API and obtain the required credentials for the AWS SDK.
If you are using the Kubernetes service account to authenticate with an external service such as AWS, you may also need to configure an IAM role that allows the service account to access the necessary resources.
Double check that your Go service is correctly using the Kubernetes service account token and is passing it along as an authentication token to the AWS SDK.
You can also try to use the k8s.io/client-go library to get the secret and use it in your go code.
Related
How can I make the GO SDK fetch the access keys for AWS from the Instance Metadata Service (169.254.169.254) provided by AWS.
I checked the official AWS SDK for go documentation and there seems to be only ways of fetching the access keys from environment variables, but no credentials retriever from IMS.
How is this done in go?
I checked the official AWS SDK for go documentation and there seems to be only ways of fetching the access keys from environment variables, but no credentials retriever from IMS.
You just missed it. The Go SDK supports the instance metadata service as well as every other common credentials provider.
From https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html:
If you have configured your instance to use IAM roles, the SDK uses these credentials for your application automatically.
You don't have to do anything to configure this. It should just work. If you're having problems, make sure that you're not manually configuring some other credentials source.
Usually you don't have to do anything more than something like:
sess := session.Must(
session.NewSessionWithOptions(session.Options{
SharedConfigState: session.SharedConfigEnable,
}),
)
And with or without CLI configuration, metadata service, or environment variables, it should just work wherever you run it.
How do I enable API services specifically for a service account and not a user account?
Context: I'm using a Python script to locally test a cloud function (query BQ, convert results to json, drop in GCS bucket). I can do this fine with my own test account where I'm able to enable services, but not sure how I would do it (or how a client would go about doing it) for a client's service account. This is how I do it for my own service account:
Get service account credentials as json
Follow installations for gcloud cloud sdk
Issue: gcloud auth activate-service-account --key-file="/path/to/json-todd-credentials.json" --project="json-todd"
Enable API services like so: gcloud enable --account="json-todd#json-todd.iam.gserviceaccount.com" cloudfunctions.googleapis.com pubsub.googleapis.com etc.
I have the client's service account json and I can auth activate-service-account the service account but I can't enable because I don't have permissions - but how would the client enable APIs it specifically for a service account on GCP without having to install/initialise/auth the service account in the way above?
There is a misunderstanding I think. The API are enabled for a project, not for a service account (or a user account). Then, you have permissions to access to the API that you have activated.
If you have a service account, on a new project, without the API enable, there is no issue to grant this service account with the roles/serviceusage.serviceUsageAdmin. Like this, the service account will be able to activate the API on the project, possibly, ALL the API. BUT if the service account has only the permission to access to BigQuery (for example) and the service account activate the compute engine API, it won't be able to access to VM, even if, the API is enabled.
At the opposite, if the API are already enabled on the project, the service account doesn't need to have the roles/serviceusage.serviceUsageAdmin role granted, only the permission to use the activated API.
How to configure the credential to use AWS service from inside the EKS. I can not use AWS SDK for this specific purpose. I have mentioned a role with required permission in the yaml file but it does not seem like it is picking up the role.
ThankYou
Any help is appreciated.
Typically you'd want to apply some level of logic to allow the pods themselves to obtain IAM credentials from STS. AWS does not currently (its re:Invent now so you never know) provide a native-way to do this. The two community solutions we've implemented are:
kube2IAM: https://github.com/jtblin/kube2iam
kIAM: https://github.com/uswitch/kiam
Both work well in production/large environments in my experience. I prefer kIAM's security model, but both get the job done.
Essentially the work the same basic way ... intercepting (for lack of a better word) communications b/t the SDK libraries in the container and STS, matching identity of the pod with an internal role dictionary, and then obtaining STS credentials for that role and handing those creds back to the container. The SDK isn't inherently aware its in a container, its just doing what it does anywhere ... walking its access tree until it sees the need to obtain creds from STS and receiving those.
My use case is as follows:
I need to push some data into AWS SQS queue using JAVA SDK and by help of IAM role (not using credential provider implementation).
Is there any way to do that?
Thanks for help in advance.
It's been a while, but this is not currently the case, it is now possible to use assume role with the Java SDK with a user. You can configure credentials in your .aws/credentials file as follows:
[useraccount]
aws_access_key_id=<key>
aws_secret_access_key=<secret>
[somerole]
role_arn=<the ARN of the role you want to assume>
source_profile=useraccount
Then, when you launch, set an environment variable: AWS_PROFILE=somerole
The SDK will use the credentials defined in useraccount to call assumeRole with the role_arn you provided. You'll of course need to be sure that the user with those credentials has the permissions to assume that role.
Note that if you're not including the full Java SDK in your project (i.e. you're including just the libraries for the services you need), you also need to include the aws-java-sdk-sts library in your classpath for this to work.
It is also possible to do all of this programmatically using STSAssumeRoleSessionCredentialsProvider, but this would require you to directly configure all of the services so it might not be as convenient as the profile approach which should just work for all services.
You can use role based authentication only on EC2 Instances, ECS Containers and Lambda functions. It is not possible to use them locally or on on premise servers.
DefaultAWSCredentialsProviderChain will automatically pick the EC2 Instance Role if it can't find the credentials via any of other methods. You can also create a custom AWSCredentialsProviderChain object with only injecting a instance of InstanceProfileCredentialsProvider to it like here
AWSCredentialsProviderChain myCustomChain = new AWSCredentialsProviderChain(new InstanceProfileCredentialsProvider());
For more info: https://docs.aws.amazon.com/java-sdk/latest/developer-guide/java-dg-roles.html
I've been looking in to getting the AWS (web) console hooked up to an AD or ADFS setup for managing users. It was reasonable easy to get working with a SAML Identity Provider in IAM and some existing ADFS infrastructure.
The problem is that users that authenticate that way, as opposed to normal AWS user accounts, don't have any way to have associated access keys so far as I can tell. Access keys are a key concept for authenticating stuff such as the AWS CLI, which needs to be tied to individual user accounts.
What are the workarounds to allow a user authenticated via a SAML identity provider to still be able to easily use the aws CLI? The only thing I've come up with to far is some hacky crap that would proxy the aws cli command, request temporary 1-hour credentials from the aws STS service, put them in the aws credentials file, and forward the command to the normal AWS cli. But, that makes me want to throw up a little bit; plus, I have no idea if it would work if a command took over an hour to complete (large s3 uploads, etc..)
Suggestions? I would try the official Directory Service AD connector, but my understanding is users still just assume IAM roles and would ultimately have the same problem.
https://github.com/Versent/saml2aws was created to address this, and has a vibrant open source community behind it.
I've had success with aws-adfs for AWS CLI via ADFS
The repo owner is currently adding support for DUO MFA as well.
It works by authenticating the user to the same page you'd use for console access then scraping the roles available. You choose a role and then aws-adfs sets the default user to the credential set needed for sts access.
After the default user is set you can cli like normal: aws s3 ls
https://github.com/venth/aws-adfs