In my GCP project, I have a python API running in a docker container (using connexion). I want to expose the API (with an API key) using API Gateway.
When I deploy the docker container with --ingress internal, I get Access is forbidden. on API calls over the Gateway. So the API gateway cannot access the Google Run container.
When I use --ingress all, all works as expected, but then my internal API is accessible from the web, which is not what I want.
I created a service account for this:
gcloud iam service-accounts create $SERVICE_ACCOUNT_ID \
# --description="the api gateway user" \
# --display-name="api gateway user"
... gave the account run.invoker permissions:
gcloud projects add-iam-policy-binding $PROJECT_ID \
--role=roles/run.invoker --member \
serviceAccount:$SERVICE_ACCOUNT_EMAIL
... and used the service account to create the API Config:
gcloud api-gateway api-configs create $CONFIG_ID \
--api=$API_ID --openapi-spec=$API_DEFINITION \
--project=$PROJECT_ID --backend-auth-service-account=$SERVICE_ACCOUNT_EMAIL
But I can't access the docker API from API Gateway. What am I missing here? How can I secure my API, so API Gateway can connect internally.
Update1:
Also applied the role to my run service:
gcloud run services add-iam-policy-binding $SERVICE_ID \
--region $REGION --member="serviceAccount:$SERVICE_ACCOUNT_EMAIL" \
--role="roles/run.invoker"
Update2:
Some extra info as requested by John Hanley:
My gateway yml looks like this:
swagger: '2.0'
info:
title: "title"
description: "description"
version: "0.1"
schemes:
- https
x-google-backend:
address: <CLOUD_RUN_SERVICE_URL>
paths:
/api:
post:
operationId: api
consumes:
- application/json
produces:
- application/json
security:
- api_key: []
parameters:
- in: body
name: request
description: request
required: true
schema:
$ref: '#/definitions/Request'
responses:
200:
description: "success"
400:
description: "bad data"
503:
description: "internal error"
definitions:
Request:
properties:
parameter1:
type: string
parameter1:
type: string
required:
- parameter1
securityDefinitions:
api_key:
type: "apiKey"
name: "key"
in: "query"
gcloud api-gateway api-configs describe api-config --api api-api
createTime: '2021-06-12T15:02:27.382098034Z'
displayName: api-config
gatewayServiceAccount: projects/-/serviceAccounts/apigatewayuser#projectid.iam.gserviceaccount.com
name: projects/722514052893/locations/global/apis/api-api/configs/api-config
serviceConfigId: api-config-3hytlxf4gfvzj
state: ACTIVE
updateTime: '2021-06-12T15:05:09.778404414Z'
gcloud api-gateway gateways describe api-gateway --location europe-west1
apiConfig: projects/722514052893/locations/global/apis/api-api/configs/api-config
createTime: '2021-06-12T15:06:03.383002459Z'
defaultHostname: api-gateway-97x27n6l.ew.gateway.dev
displayName: api-gateway
name: projects/projectid/locations/europe-west1/gateways/api-gateway
state: ACTIVE
updateTime: '2021-06-12T15:07:37.590520122Z'
gcloud run services describe api --region europe-west1
✔ Service api in region europe-west1
URL: https://api-o3rf5h4boa-ew.a.run.app
Ingress: internal
Traffic:
100% LATEST (currently api-00010-lig)
Last updated on 2021-06-12T17:42:49.913232Z by myemail#gmail.com:
Revision api-00010-lig
Image: gcr.io/projectid/api
Port: 8080
Memory: 512Mi
CPU: 1000m
Concurrency: 80
Max Instances: 100
Timeout: 300s
Tried debugging directly on Cloud Run:
gcloud iam service-accounts keys create $KEY_FILE --iam-account=$SERVICE_ACCOUNT_EMAIL
gcloud auth activate-service-account $SERVICE_ACCOUNT_EMAIL --key-file $KEY_FILE
BEARER=$(gcloud auth print-identity-token $SERVICE_ACCOUNT_EMAIL)
curl --header "Content-Type: application/json" \
--header "Authorization: bearer $BEARER" \
--request POST \
--data '{"parameter1":"somedata"}' \
$SERVICE_URL/api
The result is still a Forbidden:
<!DOCTYPE html>
<html lang=en>
<meta charset=utf-8>
<meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
<title>Error 403 (Forbidden)!!1</title>
<style>
*{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}#media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}#media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}#media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}
</style>
<a href=//www.google.com/><span id=logo aria-label=Google></span></a>
<p><b>403.</b> <ins>That’s an error.</ins>
<p>Access is forbidden. <ins>That’s all we know.</ins>
So the problem lies in the Cloud Run application not being accessible by the service account. I'm not sure why this does not work, since the run.invoker role was added to the Run service.
Ingress internal means "Accept only the requests coming from the project's VPC or VPC SC perimeter".
When you use API Gateway, you aren't in your VPC, it's serverless, it's in Google Cloud managed VPC. Therefore, your query are forbidden.
And because API Gateway can't be plugged to a VPC Connector (for now) and thus can't route the request to your VPC, you can't use this ingress=internal mode.
Thus, the solution is to set an ingress to all, which is not a concern is you authorize only the legit accounts to access it.
For that, check in Cloud Run service is there is allUsers granted with the roles/run.invoker in your project.
If yes, remove it
Then, create a service account and grant it the roles/run.invoker on the Cloud Run service.
Follow this documentation
Step 4: update the x-google-backend in your OpenAPI spec file to add the correct authentication audience when you call your Cloud Run (it's the base service URL)
Step 5: create a gateway with a backend service account; set the service account that you created previously
At the end, only the account authenticated and authorized will be able to reach your Cloud Run service
All the unauthorized access are filtered by Google Front End and discarded before reaching your service. Therefore, your service isn't invoked for nothing and therefore your pay nothing!
Only API Gateway (and the potential other accounts that you let on the Cloud Run service) can invoke to the Cloud Run service.
So, OK, your URL is public, reachable from the wild internet, but protected with Google Front End and IAM.
Related
I created a cluster.yaml file which contains the below information:
---
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
name: eks-litmus-demo
region: ${AWS_REGION}
version: "1.21"
managedNodeGroups:
- instanceType: m5.large
amiFamily: AmazonLinux2
name: eks-litmus-demo-ng
desiredCapacity: 2
minSize: 2
maxSize: 4
EOF
When i run $ eksctl create cluster -f cluster.yaml to create the cluster through my terminal, I get the below error:
Error: checking AWS STS access – cannot get role ARN for current session: MissingEndpoint: 'Endpoint' configuration is required for this service
How can I resolve this? Please help!!!
Note: I have the global and regional endpoints under STS set to "valid in all AWS regions".
In my case, it was a typo in the region. I had us-east1 as the value. When it is corrected to us-east-1, the error disappeared. So it is worth checking if there are typos in any of the fields.
mention --profile if you use any aws profile other than default
eksctl create cluster -f cluster.yaml --profile <profile-name>
My SSO session token had expired:
aws sts get-caller-identity --profile default
The SSO session associated with this profile has expired or is otherwise invalid. To refresh this SSO session run aws sso login with the corresponding profile.
Then I needed to refresh my SSO session token:
aws sso login
Attempting to automatically open the SSO authorization page in your default browser.
If the browser does not open or you wish to use a different device to authorize this request, open the following URL:
https://device.sso.us-east-2.amazonaws.com/
Then enter the code:
XXXX-XXXX
Successfully logged into Start URL: https://XXXX.awsapps.com/start
Error: checking AWS STS access – cannot get role ARN for current session:
According to this, I think its not able to get the role (in your case, cluster creator's role) which is responsible to create the cluster.
Create an IAM user with appropriate role. Attach necessary policies to that role to create the EKS cluster.
Then you can use aws configure command to add the AWS Access Key ID, AWS Secret Access Key, and Default region name.
[Make sure that the user has the appropriate access to create and access the eks cluster in your aws account. You can use aws cli to verify if you have the appropriate access]
It is important to configure the default profile for AWS CLI correctly on the command line using
set AWS_ACCESS_KEY_ID <your_access_key>
set AWS_SECRET_ACCESS_KEY <your_secret_key>
I am trying to implement user authentication via JWTs in Google Cloud API Gateway.
I have configured the security requirement object and a security definitions object in the API config as per the documentation
securityDefinitions:
google_id_token:
authorizationUrl: ""
flow: "implicit"
type: "oauth2"
x-google-issuer: "https://accounts.google.com"
x-google-jwks_uri: "https://www.googleapis.com/oauth2/v3/certs"
security:
- google_id_token: []
And the backend service is a Cloud Run service
x-google-backend:
address: https://my-apis-fskhw40mta-uk.a.run.app
However when I call the API with my user bearer token, I receive a 403 error and this error in the stackdriver logs
"jwt_authn_access_denied{Audiences_in_Jwt_are_not_allowed}"
The Python client code to call the API is
id_token = subprocess.run(['gcloud', 'auth', 'print-identity-token'], capture_output=True, text=True, shell=True).stdout
http = urllib3.PoolManager()
encoded_args = urlencode({'arg1': "val1"})
r = http.request(
'GET',
API_URL + "/run-api?" + encoded_args,
headers={"Authorization": f"Bearer {id_token}"}
)
What is the correct way to include an audience when using a User account (not service account)
Update 1
I have found one way to do it, however I'm not sure it is correct. If I add 32555940559.apps.googleusercontent.com
to the securityDefinitions so it looks like this
securityDefinitions:
google_id_token:
authorizationUrl: ""
flow: "implicit"
type: "oauth2"
x-google-issuer: "https://accounts.google.com"
x-google-jwks_uri: "https://www.googleapis.com/oauth2/v3/certs"
x-google-audiences: "https://oauth2.googleapis.com/token, 32555940559.apps.googleusercontent.com"
It will allow unauthenticated access to Cloud Run, however I still can not call Cloud Run with authentication enabled. Cloud Run returns 403 error due to the API gateway service account not having permmissions - It has Cloud Run Invoker
Is there anything special I need to do to enable API Gateway to call cloud run other than granting Cloud Run Invoker
Adding 32555940559.apps.googleusercontent.com is not recommended, since this is the default. Ideally the audience should be unique for every service, which is why we normally use the service's own URL for this purpose. This prevents the tokens being reused, e.g. by a malicious or insecure server, to authenticate to other services which expect a different audience.
You can specify the audience you want to use when you create your identity token. For example: gcloud auth print-identity-token --audiences "https://service-acldswax.fx.gateway.dev"
You can specify the same audience in x-google-audiences to make this work. Alternatively, the service name prefixed with "https://" will be accepted by default. This can be specified as "host" in the API specification file and would normally be something like "api.example.com".
Note that anyone can generate a valid identity token, which will be accepted by the gateway. The gateway is performing /authentication/, but not /authorization/. You can either do authorization in the app, or for a private API you may wish to use a different oauth2 client.
When this is set up correctly you should be able to connect to the API gateway, but you will probably want your Cloud Run service to be locked down, to prevent the gateway from being bypassed. As you mentioned, the permission required to do this is included in the "Cloud Run Invoker" role, this needs to be granted to the gateway's service account on the Cloud Run service one of its containing resources (e.g. project, folder, organization).
I would suggest running the following commands to confirm/check the settings again :
Verify URL and API config in the gateway: gcloud api-gateway gateways describe $GATEWAY --location $REGION
Verify gateway config service account and backend URL (in base64 encoded document.contents): gcloud api-gateway api-configs describe --api $API $API_CONFIG --view FULL
Verify permission on Cloud Run service : gcloud run services --region $REGION get-iam-policy $SERVICE
API Gateway in GCP requires an Open API definition. It looks like this definition must be very precise (includes all HTTP methods and all endpoints). Ex:
paths:
/v1/hello:
get:
summary: Hi Service
operationId: hello-v1
x-google-backend:
address: <CLOUD_RUN_URL>
responses:
'200':
description: OK
Is there something in the spec to define only partial definitions? End goal would be to use API Gateway as a kind of reverse proxy.
Example:
anything behind /path1 goes to Cloud Run URL 1,
anything behind /path2 goes to Cloud Run URL 2.
You are able to create this figure
and by adding the relavant path-rules you can accomplish what you described
gcloud compute url-maps add-path-matcher api-gateway-url-map \
--path-matcher-name=my-pm2 \
--default-service=my-host-default-backend \
--path-rules="/video=video-service,/video/*=video-service" \
--new-hosts my-hosts.com
See the full documentation page for: Getting started with HTTP(S) Load Balancing for API Gateway
You can set up a wildcard and path_translation
/v1/hello/{another_path=**}:
get:
summary: Hi Service
operationId: hello-v1
x-google-backend:
address: <CLOUD_RUN_URL>
path_translation: APPEND_PATH_TO_ADDRESS
responses:
'200':
description: OK
That means that the path <API_GATEWAY_URL>/v1/hello/world/123 will be forwarded to <CLOUD_RUN_URL>/v1/hello/world/123
I am trying to invoke Google cloud function which is Http triggered by cloud scheduler.
But whenever I try to run cloud scheduler it always says permission denied error
httpRequest: {
status: 403
}
insertId: "14igacagbanzk3b"
jsonPayload: {
#type: "type.googleapis.com/google.cloud.scheduler.logging.AttemptFinished"
jobName: "projects/***********/locations/europe-west1/jobs/twilio-cloud-scheduler"
status: "PERMISSION_DENIED"
targetType: "HTTP"
url: "https://europe-west1-********.cloudfunctions.net/function-2"
}
logName: "projects/*******/logs/cloudscheduler.googleapis.com%2Fexecutions"
receiveTimestamp: "2020-09-20T15:11:13.240092790Z"
resource: {
labels: {
job_id: "***********"
location: "europe-west1"
project_id: "**********"
}
type: "cloud_scheduler_job"
}
severity: "ERROR"
timestamp: "2020-09-20T15:11:13.240092790Z"
}
Solutions I tried -
Tried putting Google cloud function in the same region as the App engine as suggested by some users.
Gave access to Google provided cloud scheduler sa service-****#gcp-sa-cloudscheduler.iamaccount.gserviceaccount.com owner role and Cloud Functions Admin role
My cloud function has ingress setting of Allow all traffic.
My cloud scheduler only works when I run below command
gcloud functions add-iam-policy-binding cloud-function --member="allUsers" --role="roles/cloudfunctions.invoker"
On Cloud Scheduler page, you have to add a service account to use to call the private Cloud Function. In the Cloud Scheduler set up, you have to
Click on SHOW MORE on the bottom
Select Add OIDC token in the Auth Header section
Add a service account email in the service account email for the Scheduler
Fill in the Audience with the same base URL as the Cloud Functions (the URL provided when you deployed it)
The service account email for the Scheduler must be granted with the role cloudfunctions.invoker
In my case the problem was related to restricted ingress setting for the cloud function. I set it to 'allow internal traffic only', but that allows only traffic from services using VPC, whereas Cloud Scheduler doesn't as per doc explanation:
Internal-only HTTP functions can only be invoked by HTTP requests that are created within a VPC network, such as those from Kubernetes Engine, Compute Engine, or the App Engine Flexible Environment. This means that events created by or routed through Pub/Sub, Eventarc, Cloud Scheduler, Cloud Tasks and Workflows cannot trigger these functions.
So the proper way to do it is:
set the ingress to 'all traffic'
remove the permission for allUsers with role Cloud Function Invoker
add the permission for created service account with role Cloud Function Invoker
or just set that permission globally for the service account in IAM console(you could do that when creating service account as well)
If you tried all of the above (which should be the first things to look at, such as Add OIDC token, giving your service account role Cloud Function Invoker and/or Cloud Run Invoker (for 2nd gen functions) etc.), please also check the following:
For me the only thing that fixed this, was adding the following google internal service account to IAM:
service-YOUR_PROJECT_NUMBER#gcp-sa-cloudscheduler.iam.gserviceaccount.com
And give this internal service account the following role:
Cloud Scheduler Service Agent
See also:
https://cloud.google.com/scheduler/docs/http-target-auth
And especially for this case:
https://cloud.google.com/scheduler/docs/http-target-auth#add
I am following Google's tutorial for setting up an Endpoint for my cloud function.
When I try to access the endpoint from my browser using URL service_name.a.run.app/function1 I get
Error: Forbidden
Your client does not have permission to get URL /function1GET from this server
As part of the mentioned tutorial and answer from a Google product manager , I'm securing my function by granting ESP permission to invoke my function.
gcloud beta functions add-iam-policy-binding function1 --member "serviceAccount:id-compute#developer.gserviceaccount.com" --role "roles/cloudfunctions.invoker" --project "project_id"
My openapi-functions.yaml
swagger: '2.0'
info:
title: Cloud Endpoints + GCF
description: Sample API on Cloud Endpoints with a Google Cloud Functions backend
version: 1.0.0
host: HOST
x-google-endpoints:
- name: "HOST"
allowCors: "true
schemes:
- https
produces:
- application/json
paths:
/function1:
get:
operationId: function1
x-google-backend:
address: https://REGION-FUNCTIONS_PROJECT_ID.cloudfunctions.net/function1GET
responses:
'200':
description: A successful response
schema:
type: string
Note that I added
- name: "HOST"
allowCors: "true'
to my .yaml file because I need to access the endpoint from a static site hosted on Firebase.
I have followed the tutorial you have mentioned, and indeed I came across the exact same error.
Nothing regarding permissions and roles seemed wrong.
After digging a bit what solved the issue was removing the “GET” at the end of the address.
So the openapi-functions.yaml would be like this:
swagger: '2.0'
info:
title: Cloud Endpoints + GCF
description: Sample API on Cloud Endpoints with a Google Cloud Functions backend
version: 1.0.0
host: [HOST]
schemes:
- https
produces:
- application/json
paths:
/function-1:
get:
summary: Greet a user
operationId: function-1
x-google-backend:
address: https://[REGION]-[PROJECT_ID].cloudfunctions.net/function-1
responses:
'200':
description: A successful response
schema:
type: string
Then make sure you are following all the steps mentioned in the tutorial correctly (except the above part).
In case you get a Permissions Denied error when running any of the steps, try running it again as sudo.
I have also tried adding the same as you:
host: [HOST]
x-google-endpoints:
- name: [HOST]
allowCors: "true"
And all is working well.
Pay extra attention to the CONFIG_ID that changes with each new deployment
Example:
2019-12-03r0
then it goes like:
2019-12-03r1
In case the deployment step fails (it shows some successful messages but it might fail in the end), then make sure you delete the existing endpoint service to avoid issues:
gcloud endpoints services delete [SERVICE_ID]
Also you can use the following to give cloudfunctions.invoker role to all users (Just for testing)
gcloud functions add-iam-policy-binding function-1 \
--member="allUsers" \
--role="roles/cloudfunctions.invoker"