I have a maven plugin which uses application_default_credentials.json file to authenticate against google cloud services.
I am trying to figure out to get default credential using service account instead of using my account.
I tried setting GOOGLE_APPLICATION_CREDENTIALS environment variable to the path of service account's credential file. But, the plugin does not use this environment variable.
I know, it might be a problem with the plugin. But, I am wondering is there any way to set application-default-credentials.json using google service account.
format of application-default-credentials.json:
{
"client_id": "76....408.apps.googleusercontent.com",
"client_secret": "d-....D0Ty",
"refresh_token": "1/r............................emnY02",
"type": "authorized_user"
}
Service account's key format:
{
"type": "service_account",
"project_id": "ID",
"private_key_id": "9a4.................................bbaad80",
"private_key": "-----BEGIN PRIVATE KEY-----\nMI................................e\n-----END PRIVATE KEY-----\n",
"client_email": "name#project.iam.gserviceaccount.com",
"client_id": "10..................886",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/name%40project.iam.gserviceaccount.com"
}
I know, it might be a problem with the plugin. But, I am wondering is
there any way to set application-default-credentials.json using google
service account.
The answer is no. See below for details.
What you are calling application default credentials is actually OAuth Client Secrets. These credentials are used to authenticate (login) by a human to Google Accounts to generate OAuth tokens. You cannot use this type of credential file with GOOGLE_APPLICATION_CREDENTIALS.
Application Default Credentials (ADC) is not a credential, but a strategy to locate credentials.
A Service Account JSON file is used by a computer/machine to authenticate with Google Accounts and generate an OAuth Access Token (and optionally an OIDC Client ID token).
The two types of authentication result in similar types of tokens but cannot be interchanged without using different types of code and integration with the Google authentication systems.
The key point is that one requires a human to interact with Google, the other interacts silently with Google.
Related
I've created Service Account A and granted roles Service Account Admin and Service Account Key Admin. I did this work in the GCP Console.
Service Account A's function is to create other service accounts programmatically, using the GCP Java SDK. It successfully creates new service accounts, but when it goes to create a key for the newly created service account, I get the following response:
{
"code": 403,
"errors": [
{
"domain": "global",
"message": "Permission iam.serviceAccountKeys.create is required to perform this operation on service account projects/-/serviceAccounts/<new_service_account_name>#<project_id>.iam.gserviceaccount.com.",
"reason": "forbidden"
}
],
"message": "Permission iam.serviceAccountKeys.create is required to perform this operation on service account projects/-/serviceAccounts/<new_service_account_name>#<project_id>.iam.gserviceaccount.com.",
"status": "PERMISSION_DENIED"
}
I've tried waiting to see if perhaps I tried to create the key too soon after creating the service account, but waiting hours resulted in no change.
Service Account A can successfully create a key for itself, just not for other service accounts it creates.
How do I resolve?
You have one of three problems:
Service Account A actually does not have the IAM role Service Account Key Admin in the project. Use the CLI command gcloud projects get-iam-policy and double-check.
Your code is using the wrong identity. You believe that you are using the service account but instead, another identity is being loaded by ADC (Application Default Credentials), or you made a mistake in your code.
You assign the correct role but on the service account instead of the project. Use the CLI command gcloud iam service-accounts get-iam-policy. If you find the role listed in the output, you assigned the role in the wrong place. Use the CLI command gcloud projects add-iam-policy-binding instead.
Note: There is a fourth method to prevent you from creating service account keys. Constraints might be enabled:
Restricting service account usage
I want to authenticate my API Gateway requests with Cognito.
For that i created custom scopes. API Gateway checks those scopes and proxies these requests to my Elastic Beanstalk API. This works fine.
But another part of my Authorization are groups. Based on a assigned group some actions have restricted Access. I need to use groups because i want to be able to add or remove those groups during user-lifecycle. The group will be checked in my Elastic Beanstalk API.
Problem
The documentation states that Access Tokens contain the cognito:groups claim. But a setup like in the Image below does not include this claim in my token.
The following decoded jwt will be produced after a login via hosted-UI. As you can see the claim is missing. ID tokens (with openid scope) will include this group. I am also sure that i've tested Cognito earlier with Amplify JS-SDK which included the group. But there i was unable to include my custom scopes.
{
"sub": "xxxxxxxxxxxxxxxxxxxxxx",
"token_use": "access",
"scope": "api.example.com/item.read api.example.com/item.write",
"auth_time": 1615325374,
"iss": "https://cognito-idp.eu-central-1.amazonaws.com/eu-central-1_xxxxxxx",
"exp": 1615328974,
"iat": 1615325374,
"version": 2,
"jti": "f37219a5-c8b0-411b-bdb3-ab7d9201b491",
"client_id": "xxxxxxxxxxxxxxx",
"username": "xxxxxxxxxxxxxxxxxxxxxxxxxx"
}
Do I miss about a restriction or configuration issue? Why is the group missing inside my Access Tokens?
Thanks for your help!
I had the same issue. The cognito:groups value appeared after I added the openid scope:
and the access token is still supplied, as per earlier comments.
I m using a Google Cloud Scheduler to call an external application. Google Cloud Scheduler uses OIDC authentication and uses a service account. I could get only the service account's private key from Google Service Accounts UI Console page. How do I get the public of that user managed service account?
I found the public key of this service account by pasting the Bearer token here : https://jwt.io/
But , is this the only way to get it public key of a service account? Is there any other way to get this ? (like libraries, etc) ? Is there any way to get this from Google utils or gcloud or Google console?
In one of the sites it was mentioned that "The public key can be widely distributed, so every consumer of the token can verify its integrity." .So, where is this Google service account's public key distributed to ? is there a server/place where all Google service account public keys are stored?
Also, there is an option to embed the public key as part of the jwt token. If I get a bearer token from google cloud scheduler, how do I know if it has embedded public key or not? or is it distributed public key ?
Thanks in advance for any support!
Regards
P.S: I read through these but not very helpful:
1.Get developer keys for Google Cloud Service Accounts
2. https://www.pingidentity.com/fr/company/blog/posts/2019/the-hard-parts-of-jwt-security-nobody-talks-about.html
According to the official documentation:
Creating and managing service account keys
Google ensures that all public keys for all service accounts are
publicly accessible by anyone and available to verify signatures that
are created with the private key. The public key is publicly
accessible at the following URLs:
1.x.509 certificate: https://www.googleapis.com/service_accounts/v1/metadata/x509/[SA-NAME]#[PROJECT-ID].iam.gserviceaccount.com
2.JSON web key (JWK):
https://www.googleapis.com/service_accounts/v1/jwk/[SA-NAME]#[PROJECT-ID].iam.gserviceaccount.com
3.Raw endpoint:
https://www.googleapis.com/service_accounts/v1/metadata/raw/[SA-NAME]#[PROJECT-ID].iam.gserviceaccount.com
I used curl to access the URLs:
curl -i https://www.googleapis.com/service_accounts/v1/jwk/[SA-NAME]#[PROJECT-ID].iam.gserviceaccount.com
{
"keys": [
{
"e": "xxxx",
"kty": "xxx",
"alg": "xxxx",
"n": "xxxxxxxxxxxxxxxxxxxxxxxxxxx",
"use": "xxx",
"kid": "xxxxxxxxxxxxxxxx"
}
]
}
Or to access the raw endpoint:
curl -i https://www.googleapis.com/service_accounts/v1/metadata/raw/[SA-NAME]#[PROJECT-ID].iam.gserviceaccount.com
{
"keyvalues": [
{
"exponent": "xxx",
"keyid": "xxxxxxxxxxx",
"modulus": "xxxxxxxxxxx",
"algorithm": "xxx"
}
]
}
I'm trying to verify that an IAM user exists with an access key using the AWS SDK(js).
I am trying to build an access control module for an API. I can't deploy it on AWS. Instead of trying to build the whole thing, I want to handle user management using IAM but will need to build a custom module to check if current IAM user has access to resources.
I've checked the docs and looks like you can only get user by username. I thought maybe I can list the users and filter the user array by access key but obviously the list does not have access key info.
The AWS Identity and Access Management (IAM) service is designed specifically for granting access to AWS resources. It is not designed as an authentication system for applications.
A more appropriate product would be Amazon Cognito:
Amazon Cognito lets you easily add user sign-up and sign-in and manage permissions for your mobile and web apps. You can create your own user directory within Amazon Cognito. You can also choose to authenticate users through social identity providers such as Facebook, or Amazon; with SAML identity solutions; or by using your own identity system. In addition, Amazon Cognito enables you to save data locally on users' devices, allowing your applications to work even when the devices are offline. You can then synchronize data across users' devices so that their app experience remains consistent regardless of the device they use.
With Amazon Cognito, you can focus on creating great app experiences instead of worrying about building, securing, and scaling a solution to handle user management, authentication, and synchronization across devices.
You can check whether the user exist or not by IAM User's credentials using AWS CLI/SDK
Attach iam:GetUser and iam:SimulatePrincipalPolicy policies to your IAM user.
Here iam:GetUser will be used to check the user existence and iam:SimulatePrincipalPolicy will be used to check the resource access.
1. Check existence:
You can use getUser() function of AWS-IAM to verify whether user exist or not
const iam = new AWS.IAM({
// Iam User access and secret Key
})
iam.getUser({}, (err,data)=>{
if(err)
console.log("User not exist");
else
console.log("User exist ", data);
})
if user exists:=>
`{
"User": {
"Path": "/",
"UserName": "userName",
"UserId": "AIDAY357ZXJ7ADSEWNGWA3",
"Arn": "arn:aws:iam::60977878822:user/userName", // required this in simulator
"CreateDate": "2020-08-05T14:48:49Z"
}
}`
2. For resource access you can use simulatePrincipalPolicy() function of AWS-IAM.
let params = {
PolicySourceArn = "Paste IAM user arn", // arn:aws:iam::60977878822:user/userName
ActionNames = ["ec2:RunInstances"]
}
iam.simulatePrincipalPolicy(params, (err, data)=> {
if(err)
console.log("Error", err);
else
console.log("Data ", data);
})
output:=>
`{
ResponseMetadata: { RequestId: '3e7cbc9a-ed7b-472a-b054-a6f3f37bf8c4' },
EvaluationResults:
[ { EvalActionName: 'iam:SimulatePrincipalPolicy',
EvalResourceName: '*',
**EvalDecision: 'allowed',** // check this
MatchedStatements: [Array],
MissingContextValues: [],
ResourceSpecificResults: [] },
],
IsTruncated: false
}`
If the EvalDecision is 'allowed' means your IAM User have the access to create new instances.
so I am able to make a valid request to the video intelligence api with the sample video given in the quickstart. https://cloud.google.com/video-intelligence/docs/getting-started I have tried many different ways of authenticating to the api as well. The API token I am using was created from the Credentials page in the console. There are no options to tie it to the video api so I figured it should automatically work. The API has been enabled on my account.
export TOKEN="foobar"
curl -XPOST -s -k -H"Content-Type: application/json" "https://videointelligence.googleapis.com/v1beta1/videos:annotate?key=$TOKEN" --data '{"inputUri": "gs://custom-bucket/IMG_3591.mov", "features": ["LABEL_DETECTION"]}'
{
"error": {
"code": 403,
"message": "The caller does not have permission",
"status": "PERMISSION_DENIED"
}
}
curl -XPOST -s -k -H"Content-Type: application/json" "https://videointelligence.googleapis.com/v1beta1/videos:annotate?key=$TOKEN" --data '{"inputUri": "gs://cloud-ml-sandbox/video/chicago.mp4", "features": ["LABEL_DETECTION"]}'
{
"name": "us-east1.18013173402060296928"
}
Update:
I set the file as public and it worked. But I need to access this as private, so I gave the service account access to the file and tried to get the API key like suggested.
export TOKEN="$(gcloud auth print-access-token)"
curl -XPOST -s -k -H"Content-Type: application/json" "https://videointelligence.googleapis.com/v1beta1/videos:annotate?key=$TOKEN" --data '{"inputUri": "gs://custom-bucket/IMG_3591.mov", "features":["LABEL_DETECTION"]}'
{
"error": {
"code": 400,
"message": "API key not valid. Please pass a valid API key.",
"status": "INVALID_ARGUMENT",
"details": [
{
"#type": "type.googleapis.com/google.rpc.Help",
"links": [
{
"description": "Google developers console",
"url": "https://console.developers.google.com"
}
]
}
]
}
}
It seems like the token returned by this print-access-token function does not work. I do have an API key, but it does not have access to the bucket and I don't see a way to give an API key access.
Update 2:
So it looks like we were setting our token wrong. We were following this example https://cloud.google.com/video-intelligence/docs/analyze-labels#videointelligence-label-file-protocol which is where we got the apiKey=$TOKEN from. But it looks like we needed to set the Bearer Header. We did try this at first but we were having the first issue of not having access to the bucket. So thank you.
TL;DR - Video Intelligence service is unable to access the file on your Cloud storage bucket because of lack of permissions. Since the API uses the permissions of the service account token being passed, you will need to grant your service account permissions to read the file in the GCS bucket or the entire GCS bucket itself.
Long version
The access token you pass should correspond to an IAM service account key. The service account will belong to a project (where you need to enable the Video intelligence API access) and the service account should have permissions to access the GCS bucket you're trying to access.
Each such service account has an associated email id in the form SERVICE_ACCOUNT_NAME#PROJECT_NAME.iam.gserviceaccount.com.
In Cloud console, you can go to the Cloud storage bucket/file and grant Reader permissions for the IAM service account email address. There is no need to make this bucket public.
If you use gsutil, you can run the following equivalent command:
gsutil acl ch -u SERVICE_ACCOUNT_NAME#PROJECT_NAME.iam.gserviceaccount.com:READ gs://custom-bucket/IMG_3591.mov`
I confirmed this myself with an IAM service account that I created in my project and used this to invoke the video intelligence API. The file was not made public, but granted Reader permissions only to the service account.
I used gcloud to activate the service account and fetch the access token, although you can do this manually as well using the google OAuth APIs:
gcloud auth activate-service-account SERVICE_ACCOUNT_KEY.json
export TOKEN="$(gcloud auth print-access-token)"
The steps for creating the IAM service account using gcloud are in the same page.
I can repro this issue. I believe the problem is that you don't have proper permission setup for your video file in your gs bucket. To test out this hypothesis try sharing it publicly (checkbox next to the blob in Google Storage) and then run the request again.