I am trying my hands on configuring Endpoints on Cloud Functions by following the article.
Have performed the following steps:
1) Create a Google Cloud Platform (GCP) project, and deployed the following Cloud Function.
export const TestPost = (async (request: any, response: any) => {
response.send('Record created.');
});
using following command
gcloud functions deploy TestPost --runtime nodejs10 --trigger-http --region=asia-east2
Function is working fine till here.
2) Deploy the ESP container to Cloud Run using following command
gcloud config set run/region us-central1
gcloud beta run deploy CLOUD_RUN_SERVICE_NAME \
--image="gcr.io/endpoints-release/endpoints-runtime-serverless:1.30.0" \
--allow-unauthenticated \
--project=ESP_PROJECT_ID
ESP container is successfully deployed as well.
3) Create an OpenAPI document that describes the API, and configure the routes to the Cloud Functions.
swagger: '2.0'
info:
title: Cloud Endpoints + GCF
description: Sample API on Cloud Endpoints with a Google Cloud Functions backend
version: 1.0.0
host: HOST
schemes:
- https
produces:
- application/json
paths:
/Test:
get:
summary: Do something
operationId: Test
x-google-backend:
address: https://REGION-FUNCTIONS_PROJECT_ID.cloudfunctions.net/Test
responses:
'200':
description: A successful response
schema:
type: string
4) Deploy the OpenAPI document using following command
gcloud endpoints services deploy swagger.yaml
5) Configure ESP so it can find the configuration for the Endpoints service.
gcloud beta run configurations update \
--service CLOUD_RUN_SERVICE_NAME \
--set-env-vars ENDPOINTS_SERVICE_NAME=YOUR_SERVICE_NAME \
--project ESP_PROJECT_ID
gcloud alpha functions add-iam-policy-binding FUNCTION_NAME \
--member "serviceAccount:ESP_PROJECT_NUMBER-compute#developer.gserviceaccount.com" \
--role "roles/iam.cloudfunctions.invoker" \
--project FUNCTIONS_PROJECT_ID
This is done successfully
6) Sending requests to the API
Works absolutely fine.
Now I wanted to Implement authentication so I made following changes to OpenAPI document
swagger: '2.0'
info:
title: Cloud Endpoints + GCF
description: Sample API on Cloud Endpoints with a Google Cloud Functions backend
version: 1.0.0
host: HOST
schemes:
- https
produces:
- application/json
security:
- client-App-1: [read, write]
paths:
/Test:
get:
summary: Do something
operationId: Test
x-google-backend:
address: https://REGION-FUNCTIONS_PROJECT_ID.cloudfunctions.net/Test
responses:
'200':
description: A successful response
schema:
type: string
securityDefinitions:
client-App-1:
authorizationUrl: ""
flow: "implicit"
type: "oauth2"
scopes:
read: Grants read access
write: Grants write access
x-google-issuer: SERVICE_ACCOUNT#PROJECT.iam.gserviceaccount.com
x-google-jwks_uri: https://www.googleapis.com/robot/v1/metadata/x509/SERVICE_ACCOUNT#PROJECT.iam.gserviceaccount.com
I created a service account using following command.
gcloud iam service-accounts create SERVICE_ACCOUNT_NAME --display-name DISPLAY_NAME
Granted Token Creator role to service account using following
gcloud projects add-iam-policy-binding PROJECT_ID --member serviceAccount:SERVICE_ACCOUNT_EMAIL --role roles/iam.serviceAccountTokenCreator
Redeploy the OpenAPI document
gcloud endpoints services deploy swagger.yaml
Now when I test the API I get following error
{
"code": 16,
"message": "JWT validation failed: BAD_FORMAT",
"details": [
{
"#type": "type.googleapis.com/google.rpc.DebugInfo",
"stackEntries": [],
"detail": "auth"
}
] }
I am passing the access token generated via gcloud into the request using BearerToken
cmd for generating access token is gcloud auth application-default print-access-token
Can some one point out what the issue here. Thanks...
Edit#1:
I am using Postman to connect to my API's
After using the following command I am getting a different error.
Command:
gcloud auth print-identity-token SERVICE_ACCOUNT_EMAIL
Error:
{
"code": 16,
"message": "JWT validation failed: Issuer not allowed",
"details": [
{
"#type": "type.googleapis.com/google.rpc.DebugInfo",
"stackEntries": [],
"detail": "auth"
}
]
}
For ESP, you should use jwt-token, or identity token. not access token. Please check this out.
Finally I manager to solve the issue.
Two things were wrong.
1) The format of the raw JWT token, it should be as following
{
"iss": SERVICE_ACCOUNT_EMAIL,
"iat": 1560497345,
"aud": ANYTHING_WHICH_IS_SAME_AS_IN_OPENAPI_YAML_FILE,
"exp": 1560500945,
"sub": SERVICE_ACCOUNT_EMAIL
}
and then we need to generate a signed JWT token using following command
gcloud beta iam service-accounts sign-jwt --iam-account SERVICE_ACCOUNT_EMAIL raw-jwt.json signed-jwt.json
2) The security definition in the YAML file should be like following
securityDefinitions:
client-App-1:
authorizationUrl: ""
flow: "implicit"
type: "oauth2"
scopes:
read: Grants read access
write: Grants write access
x-google-issuer: SERVICE_ACCOUNT_EMAIL
x-google-jwks_uri: "https://www.googleapis.com/robot/v1/metadata/x509/SERVICE_ACCOUNT_EMAIL
x-google-audiences: ANYTHING_BUT_SAME_AS_IN_RAW_JWT_TOKEN
Related
I am using API Gateway with firebase JWT authorisation (so that users can use google sign in) that is forwarding the requests to cloud run services and one cloud function service.
Here is how my API Gateway config looks like:
swagger: '2.0'
info:
version: '1.0.0'
title: 'BFF'
description: Backend For Frontend
schemes:
- https
security:
- firebase: []
securityDefinitions:
firebase:
authorizationUrl: ""
flow: "implicit"
type: "oauth2"
x-google-issuer: "https://securetoken.google.com/${PROJECT}"
x-google-jwks_uri: "https://www.googleapis.com/service_accounts/v1/metadata/x509/securetoken#system.gserviceaccount.com"
x-google-audiences: ${PROJECT}
paths:
/test/auth:
post:
operationId: testAuth
summary: Test auth
produces:
- application/json
x-google-backend:
address: https://${REGION}-${PROJECT}.cloudfunctions.net/auth-test
responses:
'200':
description: 'Response returns user related data from JWT'
/record/new:
post:
operationId: crateRecord
summary: Create new record
x-google-backend:
address: ${RUN_SERVICE_URL}:${RUN_SERVICE_PORT}/new
produces:
- application/json
parameters:
- in: body
name: data
description: Data for new record
schema:
$ref: '#/definitions/Record'
responses:
'200':
description: New record data
schema:
$ref: '#/definitions/Record'
'400':
description: Invalid input data
The problem is that API Gateway for some reason can't invoke cloud run service but it can invoke cloud functions:
┍ Client is passing authorization token in header
|
| ┍ Auth is successful and request is forwarded to cloud run
| |
| | ┍ 401 unauthorized to invoke cloud run
| | |
↓ ↓ ↓
Client -----------> API Gateway -----X-----> Cloud run service
API Gateway service account has these relevant roles: roles/cloudfunctions.invoker, roles/run.invoker and roles/iam.serviceAccountUser
Run service also has IAM binding for gateway service account with the role roles/run.invoker
When I use /test/auth route I can see that firebase auth is working as expected and I can trigger cloud functions without any problem and in response cloud function returns data from x-apigateway-api-userinfo as expected. But when i make request with same authorization token to run service route /record/new I get:
www-authenticate: Bearer error="invalid_token" error_description="The access token could not be verified"
401 Unauthorized
Your client does not have permission to the requested URL /new.
I am running out of ideas on what might be the problem, any advice would be helpful.
With Cloud Functions, the identity token created contains automatically the correct audience. It's not the case when you invoke Cloud Run, you have to explicitly mention the Cloud Run audience
/record/new:
post:
operationId: crateRecord
summary: Create new record
x-google-backend:
address: ${RUN_SERVICE_URL}:${RUN_SERVICE_PORT}/new
jwt_audience: ${RUN_SERVICE_URL}
Have a try, it should work now.
I was trying GCP API Gateway using firebase authentication. I can see my request has been processed from the logs and completed with response code 200. However, I am not getting the response back to my client. I am getting the response when I call the function directly. Am I missing something ?
API Config
swagger: "2.0"
info:
title: API Endpoints
description: API Endpoints
version: 1.0.1
schemes:
- https
produces:
- application/json
securityDefinitions:
firebase:
authorizationUrl: ""
flow: "implicit"
type: "oauth2"
x-google-issuer: "https://securetoken.google.com/my-project"
x-google-jwks_uri: "https://www.googleapis.com/service_accounts/v1/metadata/x509/securetoken#system.gserviceaccount.com"
x-google-audiences: "my-project"
paths:
/hello:
get:
summary: Test link
operationId: hello
x-google-backend:
address: https://us-central1-my-project.cloudfunctions.net/hello
security:
- firebase: []
responses:
"200":
description: A successful response
schema:
type: string
"403":
description: Failed to authenticate
Function
* Responds to any HTTP request.
*
* #param {!express:Request} req HTTP request context.
* #param {!express:Response} res HTTP response context.
*/
exports.helloWorld = (req, res) => {
let message = req.query.message || req.body.message || 'Hello World!';
res.status(200).send(message);
};
Logs
Additional Query
Initially, I had my functions private and was getting 403. It gave me 200 once I added allUsers with Cloud Functions Invoker to the permissions to the function I am trying to invoke. So I am a bit confused here. Part of the reason I am using API gateway with firebase auth is to protect it against unauthorised calls. And for firebase auth to work, I had to add allUsers, making it public. My understanding was that the API gateway alone would be public while all the backend services that it invokes would be private. In this case, the function can be directly invoked by anyone, rendering API Gateway useless. How can I setup the backend to private and only respond to authenticated calls through API gateway ?
Additional Logs
{
insertId: "8c13b49c-2752-4216-8188-d445f4724ef14850908905639612439#a1"
jsonPayload: {
api_key_state: "NOT CHECKED"
api_method: "1.myapi.GenerateRTCToken"
api_name: "1.myapi"
api_version: "1.0.1"
client_ip: "999.999.999.999"
http_method: "GET"
http_response_code: 200
location: "us-central1"
log_message: "1.myapi.GenerateRTCToken is called"
producer_project_id: "myproject"
request_latency_in_ms: 161
request_size_in_bytes: 4020
response_size_in_bytes: 579
service_agent: "ESPv2/2.17.0"
service_config_id: "myapi-config"
timestamp: 1606313914.9804168
url: "/generateRTCToken"
}
logName: "projects/myproject/logs/myapi%2Fendpoints_log"
receiveTimestamp: "2020-11-25T14:18:36.521292489Z"
resource: {
labels: {
location: "us-central1"
method: "1.myapi.GenerateRTCToken"
project_id: "myproject"
service: "myapi"
version: "1.0.1"
}
type: "api"
}
severity: "INFO"
timestamp: "2020-11-25T14:18:34.980416865Z"
}
To call the Google Cloud Function from the Api-Gateway without making it public you can grant the service account used by the API-Gateway the role CloudFunctions Invoker
On Creating an API config (4) : you set a service Account, take note of this service account.
Once you have identified the service account you can grant it the Cloud Funtions Invoker role to the service account. For these you can follow these steps:
Go to IAM&Admin
Look for the service account, and click on the pencil next to it( If you don't see the service account click on Add and type the service account email)
For role Select Cloud Functions Invoker
Click on Save
This will grant the service account the permission to call the functions without having them public.
I have an similar issue with API Gateway throwing the same response validating a token against keycloak.
The issue was with the JWT, it was too long, we remove unused roles from the user and work perfectly.
Hope it helps.
Take a look at this document where it is explained in detail how to use Firebase to authenticate in API Gateway.
In general there are two conditions that we need to keep on mind in order to configure these services:
When your client application sends an HTTP request, the authorization header in the request must contain the following JWT claims:
iss (issuer)
sub (subject)
aud (audience)
iat (issued at)
exp (expiration time)
To support Firebase authentication, add the following to the security definition in your API config:
securityDefinitions:
firebase:
authorizationUrl: ""
flow: "implicit"
type: "oauth2"
# Replace YOUR-PROJECT-ID with your project ID
x-google-issuer: "https://securetoken.google.com/YOUR-PROJECT-ID"
x-google-jwks_uri: "https://www.googleapis.com/service_accounts/v1/metadata/x509/securetoken#system.gserviceaccount.com"
x-google-audiences: "YOUR-PROJECT-ID"
Hi to all I created and used openAPI by yaml and I created endpoint that maps 2 cloud functions which use path templating to route the call no error by google sdk cli.
Now I call by POST https://myendpointname-3p5hncu3ha-ew.a.run.app/v1/setdndforrefcli/12588/dnd?key=[apikey] because it's mapped by below open api and reply me "Path does not match any requirement URI template.".
I don't know why path template in endpoint not work I added path_translation: APPEND_PATH_TO_ADDRESS to avoid google to use CONSTANT_ADDRESS default which append id in query string with brutal [name of cloud function]?GETid=12588 and overwrite query parameters with same name.
Somebody can tell me how can I debug the endpoint or the error in openAPI (that have green check ok icon in endpoint)?
# [START swagger]
swagger: '2.0'
info:
description: "Get data "
title: "Cloud Endpoint + GCF"
version: "1.0.0"
host: myendpointname-3p5hncu3ha-ew.a.run.app
# [END swagger]
basePath: "/v1"
#consumes:
# - application/json
#produces:
# - application/json
schemes:
- https
paths:
/setdndforrefcli/{id}/dnd:
post:
summary:
operationId: setdndforrefcli
parameters:
- name: id # is the id parameter in the path
in: path # is the parameter where is in query for rest or path for restful
required: true
type: integer
format: int64
minimum: 1
security:
- api_key: []
x-google-backend:
address: https://REGION-PROJECT-ID.cloudfunctions.net/mycloudfunction
path_translation: APPEND_PATH_TO_ADDRESS
protocol: h2
responses:
'200':
description: A successful response
schema:
type: string
# [START securityDef]
securityDefinitions:
# This section configures basic authentication with an API key.
api_key:
type: "apiKey"
name: "key"
in: "query"
# [END securityDef]
I had the same error, but after did some test I was able to use successfully the path templating (/endpoint/{id}). I resolved this issue as follows:
1 .- gcloud endpoints services deploy openapi-functions.yaml \
--project project
Here you will get a new Service Configuration that you will to use in the next steps.
2.-
chmod +x gcloud_build_image
./gcloud_build_image -s SERVICE \
-c NEWSERVICECONFIGURATION -p project
Its very important change the service configuration with every new deployment of the managed service.
3.- gcloud run deploy SERVICE \
--image="gcr.io/PROJECT/endpoints-runtime-serverless:SERVICE-NEW_SERVICE_CONFIGURATION" \
--allow-unauthenticated \
--platform managed \
--project=PROJECT
Trying to use Google Cloud platform with a GKE deployed backend.
I have a swagger file for the endpoints that works fine when not using security.
I added the api key definition in the swagger file:
paths:
/create:
post:
...
security:
- api_key: []
securityDefinitions:
api_key:
type: "apiKey"
name: "key"
in: "query"
and now if I try to post on I get the expected
{
"code": 16,
"message": "Method doesn't allow unregistered callers (callers without established identity). Please use API Key or other form of API consumer identity to call this API.",
"details": [
{
"#type": "type.googleapis.com/google.rpc.DebugInfo",
"stackEntries": [],
"detail": "service_control"
}
]
}
Good, now I created an API key in the credential sections of GCP
I update the post request to include ?key=API_KEY and get the following error:
{
"code": 13,
"message": "\b#The caller does not have permission",
"details": [
{
"#type": "type.googleapis.com/google.rpc.DebugInfo",
"stackEntries": [],
"detail": "service_control"
}
]
}
I can't find any info about this error, does it mean that my API key has no right for this endpoint? If so how can I fix this?
Confirm that you have the required services enabled
gcloud services enable servicemanagement.googleapis.com
gcloud services enable servicecontrol.googleapis.com
gcloud services enable endpoints.googleapis.com
Also enable your Endpoint service gcloud services enable ENDPOINTS_SERVICE_NAME
I'm now developing REST API with Cloud endpoints and App engine.
I will like to implement api key authentication but it does not work.
Looks good without query params of 'key=${API KEY}'.
# curl -X POST https://hogehoge.com/test -d '{"key":"value"}'
{
"code": 16,
"message": "Method doesn't allow unregistered callers (callers without established identity). Please use API Key or other form of API consumer identity to call this API.",
"details": [
{
"#type": "type.googleapis.com/google.rpc.DebugInfo",
"stackEntries": [],
"detail": "service_control"
}
]
}
But any key can be granted to access to the backend.
# curl -X POST https://hogehoge.com/test?key=aaa -d '{"key":"value"}'
POST is sended.
Of course, API key generated via API management will work.
# curl -X POST https://hogehoge.com/test?key=${realkey} -d '{"key":"value"}'
POST is sended.
Cloud endpoint file definition is
swagger: "2.0"
info:
title: "xxxxxxxxx"
description: "xxxxxxxxx"
version: "1.0.0"
host: "hogehoge.com"
schemes:
- "https"
security: []
paths:
"/test":
post:
description: "test"
operationId: "test"
security:
- api_key: []
parameters:
- name: body
in: body
required: true
schema:
$ref: '#/definitions/testRequest'
responses:
201:
description: "Success"
schema:
$ref: '#/definitions/testResponse'
definitions:
testRequest:
type: object
required:
- data
properties:
data:
type: object
required:
- key
properties:
token:
type: string
example: value
maxLength: 20
testResponse:
type: string
securityDefinitions:
api_key:
type: "apiKey"
name: "key"
in: "query"
What I expect is only key generated via API management will be granted to access.
Let me know how to solve this issue.
Thanks.
It seems that the Service Control API might not be enabled on your project.
In order to check that, you can run
gcloud services list --enabled
If servicecontrol.googleapis.com is not listed in the result of the previous command, you should run
gcloud services enable servicecontrol.googleapis.com
Furthermore, you could check that you have all the required services for Endpoints enabled. You can see how to do this in the documentation