Google OAuth 2.0 for Server to Server Applications - google-cloud-platform

I wanted to access GCP storage bucket from outside. So I used following steps which google has provided.
Created the service account
Generated the jwt token using the private key that provided for service account.
When I called the above API to get access token by providing jwt token it gives following error.
{
"error": "invalid_scope",
"error_description": "Empty or missing scope not allowed."
}
Thanks in advance!
https://developers.google.com/identity/protocols/OAuth2ServiceAccount
This is the Java code I used to generate the JWT
long now = System.currentTimeMillis();
try {
GoogleCredential credential = GoogleCredential.fromStream(new FileInputStream("service.json"));
PrivateKey privateKey = credential.getServiceAccountPrivateKey();
String privateKeyId = credential.getServiceAccountPrivateKeyId();
Algorithm algorithm = Algorithm.RSA256(null, (RSAPrivateKey) privateKey);
String signedJwt = JWT.create()
.withKeyId(privateKeyId)
.withIssuer("***********#************-******.iam.gserviceaccount.com")
.withSubject("***********#************-******.iam.gserviceaccount.com")
.withAudience("https://www.googleapis.com/oauth2/v4/token")
.withIssuedAt(new Date(now))
.withExpiresAt(new Date(now + 3600 * 1000L))
.sign(algorithm);
System.out.println(signedJwt);
} catch(Exception e) {
System.out.println(e);
}

I figure out the issue. It was with the payload that I passed to generate the JWT token.
Below I attched the python code which I used to genarate jwt token.
I got the reference from https://www.jhanley.com/google-cloud-creating-oauth-access-tokens-for-rest-api-calls/ below python code
import jwt
import time
# Permissions to request for Access Token
scopes = "https://www.googleapis.com/auth/devstorage.read_write"
# private key id
pkey_id = ""
# private key
pkey = ""
serviceid = ""
# Google Endpoint for creating OAuth 2.0 Access Tokens from Signed-JWT
auth_url = "https://www.googleapis.com/oauth2/v4/token"
# Set how long this token will be valid in seconds
expires_in = 3600 # Expires in 1 hour
issued = int(time.time())
expires = issued + expires_in # expires_in is in seconds
# JWT Payload
payload = {
"iss": serviceid, # Issuer claim
"sub": serviceid, # Issuer claim
"aud": auth_url, # Audience claim
"iat": issued, # Issued At claim
"exp": expires, # Expire time
"scope": scopes # Permissions
}
# JWT Headers
additional_headers = {
'kid': pkey_id,
"alg": "RS256",
"typ": "JWT" # Google uses SHA256withRSA
}
sig = jwt.encode(payload, pkey, algorithm="RS256", headers=additional_headers)
print(sig)

Related

Google Cloud - Accessing Storage API with JWT token: Illegal URI error

I am trying to make a JWT call to storage API using the example listed here with some changes as below -
def generate_jwt():
"""Generates a signed JSON Web Token using a Google API Service Account."""
now = int(time.time())
sa_email = os.environ["FUNCTION_IDENTITY"]
expiry_length = 3600
# build payload
payload = {
'iat': now,
# expires after 'expiry_length' seconds.
"exp": now + expiry_length,
# iss must match 'issuer' in the security configuration in your
# swagger spec (e.g. service account email). It can be any string.
'iss': sa_email,
# aud must be either your Endpoints service name, or match the value
# specified as the 'x-google-audience' in the OpenAPI document.
'aud': "https://storage.googleapis.com",
# sub and email should match the service account's email address
'sub': sa_email,
'email': sa_email
}
# sign with keyfile
sa_keyfile="cred.json"
signer = google.auth.crypt.RSASigner.from_service_account_file(sa_keyfile)
jwt = google.auth.jwt.encode(signer, payload)
return jwt
and calliing it here
def make_jwt_request(signed_jwt, url="https://storage.googleapis.com/storage/v1/b/BUCKET_NAME"):
"""Makes an authorized request to the endpoint"""
headers = {
'Authorization': 'Bearer {}'.format(signed_jwt.decode('utf-8')),
'content-type': 'application/json',
"Host": "www.googleapis.com",
}
response = requests.get(url, headers=headers)
print(response.status_code, response.content)
response.raise_for_status()
but getting error as Couldn't parse the specified URI. Illegal URI.
I dont understand why it is a illegal URI. I tried with https://googleapis.com/storage/b/BUCKETNMAE but still same error. could not find anything on SO or google docs about this. any idea what wrong am I doing here ?
Google Cloud Storage does not accept a Signed JWT for authorization. Once you create the Signed JWT you must exchange the JWT for an Access Token.
Refer to my answer here or my article for a complete example in Python.
def exchangeJwtForAccessToken(signed_jwt):
'''
This function takes a Signed JWT and exchanges it for a Google OAuth Access Token
'''
auth_url = "https://www.googleapis.com/oauth2/v4/token"
params = {
"grant_type": "urn:ietf:params:oauth:grant-type:jwt-bearer",
"assertion": signed_jwt
}
r = requests.post(auth_url, data=params)
if r.ok:
return(r.json()['access_token'], '')
return None, r.text

Generate JWT for service account using compute metadata from cloud function

I'm trying to generate a JWT for a given service account serviceA from a Google/Firebase Cloud function. Service account serviceB is running the function.
I got it working by using the account keys from JSON.
Given that the CF is running within Google Cloud, I want to leverage compute metadata to not having to store the private key with the functions.
I've been trying to access the metadata server for serviceA while serviceB is executing the CF. I deliberately don't want serviceA to run the CF.
The code
const request = require('request-promise');
const serviceAccountEmail = 'serviceA#<projectA>.iam.gserviceaccount.com';
const metadataServerTokenURL = `http://metadata/computeMetadata/v1/instance/service-accounts/${serviceAccountEmail}/identity?audience=<audience>`;
const tokenRequestOptions = {
uri: metadataServerTokenURL,
headers: {
'Metadata-Flavor': 'Google'
}
};
const token = await request(tokenRequestOptions);
The error
I'm currently getting a 404 not found error for the email provided
I guess it's
a) not possible what I'm trying to do, or
b) I'm missing some IAM permissions for serviceA
You can do this with the metadata server because they can only generate ID Token for the service account loaded with your instance (in this case the serviceB).
You can use another API for this: Service Account Credentials API, especially the generateIdToken method
In your case, you can do something like this (in python here)
import google.auth
from google.auth.transport.requests import AuthorizedSession
import json
# IAP audience is the ClientID of IAP-App-Engine-app in
# the API->credentials page
# Cloud Function and Cloud Run need the base URL of the service
audience = 'YOUR AUDIENCE'
# #1 Get the default credential to generate the access token
credentials, project_id = google.auth.default(
scopes='https://www.googleapis.com/auth/cloud-platform')
# #2 To use the current service account email
service_account_email = credentials.service_account_email
# Don't work with user account, so define manually the email
# service_account_email = 'MY SERVICE ACCOUNT EMAIL'
# #3 prepare the call the the service account credentials API
sa_credentials_url = f'https://iamcredentials.googleapis.com/' \
f'v1/projects/-/serviceAccounts/' \
f'{service_account_email}:generateIdToken'
headers = {'Content-Type': 'application/json'}
# Create an AuthorizedSession that includes
# automatically the access_token based on your credentials
authed_session = AuthorizedSession(credentials)
# Define the audience in the request body
# add the parameter "'includeEmail':true" for IAP access
body = json.dumps({'audience': audience})
# Make the call
token_response = authed_session.request('POST',sa_credentials_url,
data=body, headers=headers)
jwt = token_response.json()
id_token = jwt['token']
I wrote an article on this, this week
I adopted #guillaume blaquiere's solution to Typescript:
import { GaxiosOptions, Headers } from 'gaxios';
import { GoogleAuth } from 'google-auth-library';
interface TokenRequestResponse {
token: string
}
const service_account_email = 'MY SERVICE ACCOUNT EMAIL'
const audience: string = 'MY AUDIENCE';
const sa_credentials_url = `https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/${service_account_email}:generateIdToken`
const headers: Headers = { 'Content-Type': 'application/json' }
const body = {
'audience': audience
};
const options: GaxiosOptions = {
method: 'POST',
url: sa_credentials_url,
body: JSON.stringify(body),
headers: headers,
};
const auth = new GoogleAuth({
scopes: 'https://www.googleapis.com/auth/cloud-platform'
});
const client = await auth.getClient();
const tokenResponse = await client.request(options);
const tokenRequestResponse = tokenResponse.data as TokenRequestResponse;
const token = tokenRequestResponse.token;

Sign google cloud storage blob using access token

Goal: Generate Signed-URL Using OAuth2.0 Access Token
The examples and source codes I find for signing Google Cloud Storage blobs all require service account credentials file (the private key to be specific). For instance:
https://cloud.google.com/storage/docs/access-control/signing-urls-with-helpers#storage-signed-url-get-object-python
However, since I follow the authorization flow discussed here, I only have OAuth2.0 access token (and I do NOT have the credentials file and private key of a service account with access to GCS bucket/object). Hence, I was wondering how I can sign blobs using OAuth2.0 access tokens.
The Code Used:
I use the following to sign blob:
# First, get access token:
service_account = "<email address of a service account>"
access_token = build(
serviceName='iamcredentials',
version='v1',
http=http
).projects().serviceAccounts().generateAccessToken(
name="projects/{}/serviceAccounts/{}".format(
"-",
service_account),
body=body
).execute()["accessToken"]
credentials = AccessTokenCredentials(access_token, "MyAgent/1.0", None)
# Second, use the access token to sign a blob
url = "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/{}:signBlob".format(service_account)
encoded = base64.b64encode(blob)
sign_blob_request_body = {"payload": encoded}
response = requests.post(url,
data=json.dumps(sign_blob_request_body),
headers={
'Content-Type': 'application/json',
'Authorization': 'Bearer {}'.format(credentials.access_token)})
signature = response.json()["signedBlob"]
# Third, use the signature to create signed URL:
encoded_signature = base64.b64encode(signature)
signed_url = "https://storage.googleapis.com/<BUCKET>/<OBJECT>?" \
"GoogleAccessId={}&" \
"Expires={}&" \
"Signature={}".format(service_account,
expiration,
encoded_signature)
The Error Message Received:
<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>
The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.
</Message>
<StringToSign>GET 1561832204 /<BUCKET>/<OBJECT></StringToSign>
</Error>
In case you do NOT want to use API secret key, follow procedure described in this sample that is using iamcredentials.signBlob() API signing URL 'remotely' for a service account with no need to distribute API secret key.
Signature string (that has to be signed) has this format:
signature_string = ('{verb}\n'
'{content_md5}\n'
'{content_type}\n'
'{expiration}\n'
'{resource}')

Google Cloud Storage JSON API with JWT Token

I'm trying to use the JSON API for Google Cloud Storage to retrieve a file from Google Cloud Storage. I am not allowed to use the SDKs. Is it possible to create a JWT from a ServiceAccount.json file and use the JWT to access files from Google Cloud Storage? I have a script in node.js that generates a JWT from the service account, but i'm not sure if the audience is right
const jwt = require('jsonwebtoken');
const serviceAccount = require('./serviceAccount.json');
const issuedAt = Math.floor(Date.now() / 1000);
const TOKEN_DURATION_IN_SECONDS = 3600;
let params = {
'iss': serviceAccount.client_email,
'sub': serviceAccount.client_email,
'aud': serviceAccount.project_id,
'iat': issuedAt,
'exp': issuedAt + TOKEN_DURATION_IN_SECONDS,
};
let options = {
algorithm: 'RS256',
header: {
'kid': serviceAccount.private_key_id,
'typ': 'JWT',
'alg': 'RS256',
},
};
let token = jwt.sign(params, serviceAccount.private_key, options);
console.log(token);
I then use that JWT to call the Google Cloud Storage JSON API:
https://www.googleapis.com/storage/v1/b/test
Using the header: Authorization Bearer {token}
That simply resulted in a Invalid Credentials response.
A few questions:
I'm not sure what the 'aud' should be when creating the JWT. I've seen examples where it's a url and also where it's the projectId. Neither work for me.
One of the JSON API examples said the Authorization token should be an oauth token. Can I use a JWT instead or do I need to make a call using the JWT to get an access token?
Is my bucket path correct? Is the base folder for the bucket path your projectId? Should my path be /{projectId}/test. I've tried both and neither work.
Recap
This is an IoT project and I need embedded devices to download files from Google Cloud Storage. I need to create a web portal to upload files to (using Firebase Functions) and pass to the device either a bucket path or a private/signed URL that. The bottom line being I need to access a Google Cloud Storage bucket using a service account key. If there is an embedded SDK - great, but I couldn't find one for C. My only thought was to use the JSON API. If there is a way I can sign a URL which can only be accessed using a service account - that works too.
Thanks!
Yes, you can create your own Signed JWT from a service account Json (or P12) file and exchange the JWT for an Access Token that you then use as Authorization: Bearer TOKEN
I have written a number of articles on how to use Json and P12 credentials.
Google Cloud – Creating OAuth Access Tokens for REST API Calls
For your questions:
I'm not sure what the 'aud' should be when creating the JWT. I've seen
examples where it's a url and also where it's the projectId. Neither
work for me.
Set aud to "https://www.googleapis.com/oauth2/v4/token"
One of the JSON API examples said the Authorization token should be an
oauth token. Can I use a JWT instead or do I need to make a call using
the JWT to get an access token?
Some APIs accept signed JWTs, others expect an OAuth Access Token. It is just easier to always obtain the OAuth Access Token. In my example code below, I show you how.
Is my bucket path correct? Is the base folder for the bucket path your
projectId? Should my path be /{projectId}/test. I've tried both and
neither work.
Your url shold look like this (Python string building example)
url = "https://www.googleapis.com/storage/v1/b?project=" + project
Below I show you how to call two services (GCE and GCS). Most Google APIs will follow similar styles for building the REST API urls.
From the code in your question, you are missing the last step in the OAuth process. You need to exchange your Signed JWT for an Access Token.
def exchangeJwtForAccessToken(signed_jwt):
'''
This function takes a Signed JWT and exchanges it for a Google OAuth Access Token
'''
auth_url = "https://www.googleapis.com/oauth2/v4/token"
params = {
"grant_type": "urn:ietf:params:oauth:grant-type:jwt-bearer",
"assertion": signed_jwt
}
r = requests.post(auth_url, data=params)
if r.ok:
return(r.json()['access_token'], '')
return None, r.text
Here is a complete Python 3.x example that will list GCE instances. Below this code are changes to display GCS Buckets.
'''
This program lists lists the Google Compute Engine Instances in one zone
'''
import time
import json
import jwt
import requests
import httplib2
# Project ID for this request.
project = 'development-123456'
# The name of the zone for this request.
zone = 'us-west1-a'
# Service Account Credentials, Json format
json_filename = 'service-account.json'
# Permissions to request for Access Token
scopes = "https://www.googleapis.com/auth/cloud-platform"
# Set how long this token will be valid in seconds
expires_in = 3600 # Expires in 1 hour
def load_json_credentials(filename):
''' Load the Google Service Account Credentials from Json file '''
with open(filename, 'r') as f:
data = f.read()
return json.loads(data)
def load_private_key(json_cred):
''' Return the private key from the json credentials '''
return json_cred['private_key']
def create_signed_jwt(pkey, pkey_id, email, scope):
'''
Create a Signed JWT from a service account Json credentials file
This Signed JWT will later be exchanged for an Access Token
'''
# Google Endpoint for creating OAuth 2.0 Access Tokens from Signed-JWT
auth_url = "https://www.googleapis.com/oauth2/v4/token"
issued = int(time.time())
expires = issued + expires_in # expires_in is in seconds
# Note: this token expires and cannot be refreshed. The token must be recreated
# JWT Headers
additional_headers = {
'kid': pkey_id,
"alg": "RS256",
"typ": "JWT" # Google uses SHA256withRSA
}
# JWT Payload
payload = {
"iss": email, # Issuer claim
"sub": email, # Issuer claim
"aud": auth_url, # Audience claim
"iat": issued, # Issued At claim
"exp": expires, # Expire time
"scope": scope # Permissions
}
# Encode the headers and payload and sign creating a Signed JWT (JWS)
sig = jwt.encode(payload, pkey, algorithm="RS256", headers=additional_headers)
return sig
def exchangeJwtForAccessToken(signed_jwt):
'''
This function takes a Signed JWT and exchanges it for a Google OAuth Access Token
'''
auth_url = "https://www.googleapis.com/oauth2/v4/token"
params = {
"grant_type": "urn:ietf:params:oauth:grant-type:jwt-bearer",
"assertion": signed_jwt
}
r = requests.post(auth_url, data=params)
if r.ok:
return(r.json()['access_token'], '')
return None, r.text
def gce_list_instances(accessToken):
'''
This functions lists the Google Compute Engine Instances in one zone
'''
# Endpoint that we will call
url = "https://www.googleapis.com/compute/v1/projects/" + project + "/zones/" + zone + "/instances"
# One of the headers is "Authorization: Bearer $TOKEN"
headers = {
"Host": "www.googleapis.com",
"Authorization": "Bearer " + accessToken,
"Content-Type": "application/json"
}
h = httplib2.Http()
resp, content = h.request(uri=url, method="GET", headers=headers)
status = int(resp.status)
if status < 200 or status >= 300:
print('Error: HTTP Request failed')
return
j = json.loads(content.decode('utf-8').replace('\n', ''))
print('Compute instances in zone', zone)
print('------------------------------------------------------------')
for item in j['items']:
print(item['name'])
if __name__ == '__main__':
cred = load_json_credentials(json_filename)
private_key = load_private_key(cred)
s_jwt = create_signed_jwt(
private_key,
cred['private_key_id'],
cred['client_email'],
scopes)
token, err = exchangeJwtForAccessToken(s_jwt)
if token is None:
print('Error:', err)
exit(1)
gce_list_instances(token)
To display GCS Buckets instead, modify the code:
# Create the HTTP url for the Google Storage REST API
url = "https://www.googleapis.com/storage/v1/b?project=" + project
resp, content = h.request(uri=url, method="GET", headers=headers)
s = content.decode('utf-8').replace('\n', '')
j = json.loads(s)
print('')
print('Buckets')
print('----------------------------------------')
for item in j['items']:
print(item['name'])
I found this [Service account authorization without OAuth].(https://developers.google.com/identity/protocols/oauth2/service-account#jwt-auth
You can avoid having to make a network request to Google's authorization server before making an API call.
Available APIs are listed in https://github.com/googleapis/googleapis.
It looks like Google Cloud Storage api is not yet published as per the comments in the repository.
Were you able to use the cloud storage API with JWT?

What are the valid grant_type values for IdentityServer4 with a Client using Hybrid grant type?

Im using version version 1.0.0 of the IdentityServer4 package.
"IdentityServer4": "1.0.0"
I've created a Client
new Client
{
ClientId = "MobleAPP",
ClientName = "Moble App",
ClientUri= "http://localhost:52997/api/",
AllowedGrantTypes = GrantTypes.HybridAndClientCredentials,
ClientSecrets =
{
new Secret("SecretForMobleAPP".Sha256())
},
AllowedScopes =
{
IdentityServerConstants.StandardScopes.OpenId,
IdentityServerConstants.StandardScopes.Profile,
"api"
},
AllowOfflineAccess = true
}
And the scope/ApiResources
public static IEnumerable<ApiResource> GetApiResources()
{
return new List<ApiResource>
{
new ApiResource("api", "My API")
};
}
With the following user/TestUser
public static List<TestUser> GetUsers()
{
return new List<TestUser>
{
new TestUser
{
SubjectId = "2",
Username = "bob",
Password = "password",
Claims = new []
{
new Claim(JwtClaimTypes.Name, "Bob Smith")
}
}
};
}
I'm trying to test the IdentityServer that I have setup from Postman and determine the possible values for the grant_type key value pair.
I can successfully connect when I set the grant_type to client_credentials and wasn't sure if there were other options for the grant_type value.
Working Postman configuration with grant_type set to client_credentials
Short answer
client_credentials is the only grant_type value you can use directly against the token endpoint when using both hybrid and client credentials grant types.
Longer answer
The client credentials grant type is the only one allowing you to hit the token endpoint directly, which is what you did in your Postman example. In that case the authentication is done against the client itself - i.e. the application you registered.
When you use the hybrid grant type, the authentication will be done against the end-user - the user using your application. In that case, you cannot hit the endpoint token directly but you'll have to issue an authorization request to IdentityServer.
When you do so, you won't use the grant_type parameter but the response_type parameter, to instruct IdentityServer what you expect back.
The possible values for response_type when you use the hybrid grant type can be found in IdentityServer constants - they are the last 3 items in the dictionary:
code id_token, which will return an authorization code and an identity token
code token, returning an authorization code and an access token
code id_token token, giving you back an authorization code, an identity token and an access token
After you get the authorization code, you'll be able to exchange it for an access token and possibily a refresh token by hitting the token endpoint.