I am trying to implement user authentication via JWTs in Google Cloud API Gateway.
I have configured the security requirement object and a security definitions object in the API config as per the documentation
securityDefinitions:
google_id_token:
authorizationUrl: ""
flow: "implicit"
type: "oauth2"
x-google-issuer: "https://accounts.google.com"
x-google-jwks_uri: "https://www.googleapis.com/oauth2/v3/certs"
security:
- google_id_token: []
And the backend service is a Cloud Run service
x-google-backend:
address: https://my-apis-fskhw40mta-uk.a.run.app
However when I call the API with my user bearer token, I receive a 403 error and this error in the stackdriver logs
"jwt_authn_access_denied{Audiences_in_Jwt_are_not_allowed}"
The Python client code to call the API is
id_token = subprocess.run(['gcloud', 'auth', 'print-identity-token'], capture_output=True, text=True, shell=True).stdout
http = urllib3.PoolManager()
encoded_args = urlencode({'arg1': "val1"})
r = http.request(
'GET',
API_URL + "/run-api?" + encoded_args,
headers={"Authorization": f"Bearer {id_token}"}
)
What is the correct way to include an audience when using a User account (not service account)
Update 1
I have found one way to do it, however I'm not sure it is correct. If I add 32555940559.apps.googleusercontent.com
to the securityDefinitions so it looks like this
securityDefinitions:
google_id_token:
authorizationUrl: ""
flow: "implicit"
type: "oauth2"
x-google-issuer: "https://accounts.google.com"
x-google-jwks_uri: "https://www.googleapis.com/oauth2/v3/certs"
x-google-audiences: "https://oauth2.googleapis.com/token, 32555940559.apps.googleusercontent.com"
It will allow unauthenticated access to Cloud Run, however I still can not call Cloud Run with authentication enabled. Cloud Run returns 403 error due to the API gateway service account not having permmissions - It has Cloud Run Invoker
Is there anything special I need to do to enable API Gateway to call cloud run other than granting Cloud Run Invoker
Adding 32555940559.apps.googleusercontent.com is not recommended, since this is the default. Ideally the audience should be unique for every service, which is why we normally use the service's own URL for this purpose. This prevents the tokens being reused, e.g. by a malicious or insecure server, to authenticate to other services which expect a different audience.
You can specify the audience you want to use when you create your identity token. For example: gcloud auth print-identity-token --audiences "https://service-acldswax.fx.gateway.dev"
You can specify the same audience in x-google-audiences to make this work. Alternatively, the service name prefixed with "https://" will be accepted by default. This can be specified as "host" in the API specification file and would normally be something like "api.example.com".
Note that anyone can generate a valid identity token, which will be accepted by the gateway. The gateway is performing /authentication/, but not /authorization/. You can either do authorization in the app, or for a private API you may wish to use a different oauth2 client.
When this is set up correctly you should be able to connect to the API gateway, but you will probably want your Cloud Run service to be locked down, to prevent the gateway from being bypassed. As you mentioned, the permission required to do this is included in the "Cloud Run Invoker" role, this needs to be granted to the gateway's service account on the Cloud Run service one of its containing resources (e.g. project, folder, organization).
I would suggest running the following commands to confirm/check the settings again :
Verify URL and API config in the gateway: gcloud api-gateway gateways describe $GATEWAY --location $REGION
Verify gateway config service account and backend URL (in base64 encoded document.contents): gcloud api-gateway api-configs describe --api $API $API_CONFIG --view FULL
Verify permission on Cloud Run service : gcloud run services --region $REGION get-iam-policy $SERVICE
Related
It doesn't seem possible to create an API Gateway config for a gateway i've created using:
gcloud api-gateway apis create test-api --project=acme-prd
Then the following command fails
gcloud api-gateway api-configs create 01 \
--api=test-api --openapi-spec=./acme-web-gateway-v2.yaml \
--project=acme-prd --backend-auth-service-account=svc-owner#acme-prd.iam.gserviceaccount.com
With the error:
ERROR: (gcloud.api-gateway.api-configs.create) FAILED_PRECONDITION: API Gateway Management Service Agent does not have permission to create Service Configs for Service "test-api-3qz6mxqfw7klr.apigateway.acme-prd.cloud.goog", or the Service does not exist.
Noting the service account svc-owner#acme-prd.iam.gserviceaccount.com has Owner privileges on the project.
Is there something I am missing? This is preventing a Terraform deployment. I've used gcloud commands to demonstrate the issue.
Also of note, this does not work in the GCP UI either. :(
Permissions granted to the account being used:
Cheers
KH
To resolve this, you will need to ensure that the Service Agent account has the necessary permissions for the specified service. Check API Gateway Service Account and verify if it has “Service Account User '' role associated with it.The apigateway.apis.create should have owner/editor permissions.
Check the Google Cloud Console or by using command gcloud services list to see if the Gateway API, Service Management API, Service Control API are enabled because these api are prerequisites.You will need to enable it if it is not already enabled.you enable by using below commands:
gcloud services enable apigateway.googleapis.com
gcloud services enable servicemanagement.googleapis.com
gcloud services enable servicecontrol.googleapis.com
Attaching documents for creating an api, Gateway API access , Troubleshooting for your reference.
Edit-1:
I have tried to create an API Gateway config for a gateway using below steps and successfully created an api config
Create an api gateway using below command
gcloud api-gateway apis create test-api
Creating an API config using the below command.
gcloud api-gateway api-configs create 01 --api=test-api --openapi-spec=openapi2-functions.yaml --project=project-id
Output is
waiting for API Config [01] to be created for API [test-api]...done.
I have taken openapi2-functions. Yaml file for this doc. Can you check if your yaml files has any mistakes.
The image below has the api config that i have created.
I have followed this guide, can try to create an api gateway using this and let me know if you have an issues.
I recently switched to google cloud functions gen 2 and am having an issue with authentication via API Gateway. I have the new cloud function being called by my gateway with the function itself not allowing unauthorized users. The gateway has a service account attached to it with the cloud functions invoker role (and owner role since I'm troubleshooting).
The error message in the cloud function log is
The request was not authenticated. Either allow unauthenticated invocations or set the proper Authorization header. Read more at https://cloud.google.com/run/docs/securing/authenticating Additional troubleshooting documentation can be found at: https://cloud.google.com/run/docs/troubleshooting#unauthorized-client
I have a test function with the same code that uses cloud functions gen 1 and does not allow unauthenticated users. This function works great when you hit my api gateway. I also have a cloud function gen 2 that does allow unauthenticated users and that also works as expected when hitting the gateway, it's just my gen 2 that doesn't allow unauthenticated users.
For my API gateway yaml file, this is what the file looks like (with sensitive information XXX'd out),
swagger: "2.0"
info:
title: XXXXXXXX api gateway for carson blade analytics app
description: Sample API on API Gateway with a Google Cloud Functions backend
version: 1.0.0
schemes:
- https
produces:
- application/json
paths:
/convert_csv:
get:
summary: Converts an XLSX file to a CSV file
operationId: convert
x-google-backend:
address: https://convert-csv-XXXXX-ue.a.run.app
responses:
"200":
description: A successful response
schema:
type: string
When I hit the unauthenticated allowed one and look at it's headers, I see
{
"aud": "https://convert-csv-XXXXX-ue.a.run.app/",
"azp": "XXXX",
"email": "XXX#XXX.iam.gserviceaccount.com",
"email_verified": true,
"exp": 1660101164,
"iat": 1660097564,
"iss": "https://accounts.google.com",
"sub": "XXXXX"
}
with the email being the email that is associated with the service account that has the correct cloud function invoker role and the aud being the same address as the address in the cloud api yaml. So it's replacing the authentication header that I'm sending in with its service account one as expected. I've tried reading all the documentation and have tried a bunch of different yaml configurations, but I just can't figure this out.
To summarize, With each one being attached to a cloud api gateway and hitting the endpoint from postman, gen 2 with allow unauthenticated turned off -> doesn't work. gen 2 with allow unauthenticated turned on -> works. gen 1 with allow unauthenticated turned off -> works
Any help would be extremely appreciated, thanks.
Cloud Functions 2nd gen is a wrapper on top of Cloud Run. Many times, the 2 products are entangled.
You must have the Cloud Functions Invoker role AND the Cloud Run Invoker roles (your issue)
You can filter the logs by selecting Cloud Run service and not Cloud Functions
It's absolutely not clear, and from the beginning in the Alpha version. Despite my remarks to the PM and engineering team, the product has been publicly released and now there are a lot of mistakes and confusions because of this lack of clear separation. Too bad :(
I am calling a cloud function from within my GCP project.
I receive 403 (Permission Denied) when the function is configured with Allow internal traffic only, see
https://cloud.google.com/functions/docs/networking/network-settings#ingress_settings
When removing the ingress control there is no issue, the function responds with status 200.
The function does not allow un-authenticated access, IAM policies are configured.
Following the example from https://cloud.google.com/functions/docs/securing/authenticating#function-to-function:
# main.py
import requests
# TODO<developer>: set these values
# REGION = None
# PROJECT_ID = None
RECEIVING_FUNCTION = 'hello-get'
# Constants for setting up metadata server request
# See https://cloud.google.com/compute/docs/instances/verifying-instance-identity#request_signature
function_url = f'https://{REGION}-{PROJECT_ID}.cloudfunctions.net/{RECEIVING_FUNCTION}'
metadata_server_url = \
'http://metadata/computeMetadata/v1/instance/service-accounts/default/identity?audience='
token_full_url = metadata_server_url + function_url
token_headers = {'Metadata-Flavor': 'Google'}
def hello_trigger(request):
token_response = requests.get(token_full_url, headers=token_headers)
jwt = token_response.text
function_headers = {'Authorization': f'bearer {jwt}'}
function_response = requests.get(function_url, headers=function_headers)
function_response.raise_for_status()
return function_response.text
def hello_get(req):
return 'Hello there...'
Deploying the function and the triggering function with desired ingress settings:
gcloud functions deploy hello-get --trigger-http --entry-point hello_get --runtime python37 --ingress-settings internal-only
gcloud functions deploy hello-trigger --trigger-http --entry-point hello_trigger --runtime python37 --ingress-settings all --allow-unauthenticated
Calling hello-trigger returns 403.
Changing ingress of hello-get solves the issue:
gcloud functions deploy hello-get --trigger-http --entry-point hello_get --runtime python37 --ingress-settings all
Now calling hello-trigger returns 200.
The service account used for Cloud Functions is given the Functions Invoker Role for this setup.
When you set the ingress traffic to internal-only, only the traffic coming from your VPC or from the VPC SC (Service Control) is accepted.
Here, in your trigger function, you don't come from YOUR vpc, but from another one (a serverless VPC, managed by Google, the land where the Cloud Functions are deployed). Therefore, the ingress setting isn't respected and you get a 403.
So, for this you have 2 solutions:
Use only IAM service to filter who can invoke or not your function, and let "public" your function with an ingress=all. (Solution proposed by John in his 2nd comment). It's already a high level of security.
However, sometime, for regulatory reason (or for old fashion security team design) network control is preferred.
If you want to pass through your VPC, you need to
Create a serverless VPC connector in the same region as your trigger function
Deploy your trigger function with this serverless VPC connector
Set the egress traffic to all --egress-settings=all
Like this, all the outgoing traffic of your trigger function will pass through the serverless VPC connector, thus, the traffic is routed in your VPC before trying to reach your "ingress-internal" cloud functions. And it will be accepted.
If your function use ingress=all settings, anyone can reach it from internet.
However, if you don't make the function publicly accessible, I mean, authorized to unauthenticated user, only the valid requests (authenticated AND authorized with the role cloudfunctions.invoker) will be processed by your Cloud Functions
In fact, there is a common layer to any Google service name GFE: Google Front End. This layer is in charge of many things (expose your service in HTTPS, manage your certificates, discard DDoS attack OSI layer 4,...) whom the check of the authentication header and the authorization check against the IAM service.
Therefore, in case of DDoS attack on the layer 4, GFE filter by default these attacks. In case of attack of layer 7, only the authorized request (valid) are allowed and you will pay only for them. The filter performed by GFE is free.
I am trying to get a session token for the given IAM in postman but not able to receive a token.
If I use boto3.client('sts'), I am able to get the token.
Use Case: I am trying to Invoke VPC Rest Endpoint from EC2 instance where ServiceNow mid-server instance is running. Since we have ServiceNow mid-server agent running on EC2 instance, I want to use IAM Role attached to EC2 to authenticate other VPC endpoints that are deployed in the same AWS account.
I have permission policy attached to IAM Role to allow Assume Role policy. If there any other approach, please suggest.
here HTML HTML response in postman. Postman redirecting to IAM Docs
client = boto3.client('sts')
response = client.assume_role(
RoleArn='arn:aws:iam::**************:role/ServiceNow-midserver-Role',
RoleSessionName='Session1',
DurationSeconds=3600
)
print(response)
anything wrong with postman request body or endpoint.
Authentication on postman is none.
To call AssumeRole from Postman (or curl etc.) as opposed to using a supported AWS SDK, you should follow the AssumeRole API documentation. You will also need to authenticate using AWS credentials.
Specifically, the request is an HTTP GET and parameters are passed as query strings, for example:
GET https://sts.amazonaws.com/
?Version=2011-06-15
&Action=AssumeRole
&RoleSessionName=stackoverflow-64706420
&RoleArn=arn:aws:iam::123456781234:role/myrole
&DurationSeconds=3600
Here's what this looks like in Postman:
And you will need to add AWS credentials so that your API request is signed correctly, for example:
Click 'Send' and the response will look something like this:
<AssumeRoleResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
<AssumeRoleResult>
<AssumedRoleUser>
<Arn>arn:aws:sts::123456781234:assumed-role/123456781234/stackoverflow-64706420</Arn>
<AssumedRoleId>ARO123EXAMPLE123:stackoverflow-64706420</AssumedRoleId>
</AssumedRoleUser>
<Credentials>
<AccessKeyId>ASIAIOSFODNN7EXAMPLE</AccessKeyId>
<SecretAccessKey>wJalrXUtnFEMI/K7MDENG/bPxRfiCYzEXAMPLEKEY</SecretAccessKey>
<SessionToken>
AQoDYXdzEPT//////////wEXAMPLEtc764bNrC9SAPBSM22wDOk4x4HIZ8j4FZTwdQW
LWsKWHGBuFqwAeMicRXmxfpSPfIeoIYRqTflfKD8YUuwthAx7mSEI/qkPpKPi/kMcGd
QrmGdeehM4IC1NtBmUpp2wUE8phUZampKsburEDy0KPkyQDYwT7WZ0wq5VSXDvp75YU
9HFvlRd8Tx6q6fE8YQcHNVXAkiY9q6d+xo0rKwT38xVqr7ZD0u0iPPkUL64lIZbqBAz
+scqKmlzm8FDrypNC9Yjc8fPOLn9FX9KSYvKTr4rvx3iSIlTJabIQwj2ICCR/oLxBA==
</SessionToken>
<Expiration>2020-12-09T13:34:41Z</Expiration>
</Credentials>
<PackedPolicySize>6</PackedPolicySize>
</AssumeRoleResult>
<ResponseMetadata>
<RequestId>c6104cbe-af31-11e0-8154-cbc7ccf896c7</RequestId>
</ResponseMetadata>
</AssumeRoleResponse>
You need to use credentials for an IAM user or an IAM role to call AssumeRole. boto3 must be getting credentials from the standard locations it look for (like ~/.aws/config) [ref:https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html]. May be you could try providing the AWS creds in Authorization tab in Postman selecting type as AWS Signature and then call assumeRole.
I am trying to invoke Google cloud function which is Http triggered by cloud scheduler.
But whenever I try to run cloud scheduler it always says permission denied error
httpRequest: {
status: 403
}
insertId: "14igacagbanzk3b"
jsonPayload: {
#type: "type.googleapis.com/google.cloud.scheduler.logging.AttemptFinished"
jobName: "projects/***********/locations/europe-west1/jobs/twilio-cloud-scheduler"
status: "PERMISSION_DENIED"
targetType: "HTTP"
url: "https://europe-west1-********.cloudfunctions.net/function-2"
}
logName: "projects/*******/logs/cloudscheduler.googleapis.com%2Fexecutions"
receiveTimestamp: "2020-09-20T15:11:13.240092790Z"
resource: {
labels: {
job_id: "***********"
location: "europe-west1"
project_id: "**********"
}
type: "cloud_scheduler_job"
}
severity: "ERROR"
timestamp: "2020-09-20T15:11:13.240092790Z"
}
Solutions I tried -
Tried putting Google cloud function in the same region as the App engine as suggested by some users.
Gave access to Google provided cloud scheduler sa service-****#gcp-sa-cloudscheduler.iamaccount.gserviceaccount.com owner role and Cloud Functions Admin role
My cloud function has ingress setting of Allow all traffic.
My cloud scheduler only works when I run below command
gcloud functions add-iam-policy-binding cloud-function --member="allUsers" --role="roles/cloudfunctions.invoker"
On Cloud Scheduler page, you have to add a service account to use to call the private Cloud Function. In the Cloud Scheduler set up, you have to
Click on SHOW MORE on the bottom
Select Add OIDC token in the Auth Header section
Add a service account email in the service account email for the Scheduler
Fill in the Audience with the same base URL as the Cloud Functions (the URL provided when you deployed it)
The service account email for the Scheduler must be granted with the role cloudfunctions.invoker
In my case the problem was related to restricted ingress setting for the cloud function. I set it to 'allow internal traffic only', but that allows only traffic from services using VPC, whereas Cloud Scheduler doesn't as per doc explanation:
Internal-only HTTP functions can only be invoked by HTTP requests that are created within a VPC network, such as those from Kubernetes Engine, Compute Engine, or the App Engine Flexible Environment. This means that events created by or routed through Pub/Sub, Eventarc, Cloud Scheduler, Cloud Tasks and Workflows cannot trigger these functions.
So the proper way to do it is:
set the ingress to 'all traffic'
remove the permission for allUsers with role Cloud Function Invoker
add the permission for created service account with role Cloud Function Invoker
or just set that permission globally for the service account in IAM console(you could do that when creating service account as well)
If you tried all of the above (which should be the first things to look at, such as Add OIDC token, giving your service account role Cloud Function Invoker and/or Cloud Run Invoker (for 2nd gen functions) etc.), please also check the following:
For me the only thing that fixed this, was adding the following google internal service account to IAM:
service-YOUR_PROJECT_NUMBER#gcp-sa-cloudscheduler.iam.gserviceaccount.com
And give this internal service account the following role:
Cloud Scheduler Service Agent
See also:
https://cloud.google.com/scheduler/docs/http-target-auth
And especially for this case:
https://cloud.google.com/scheduler/docs/http-target-auth#add