I am using gcp cloud endpoint with a cloud run backend Service. My Problem is that the backend is configured with a default timeout of 15 seconds. Thats why I would like to set openAPI "x-google-backend" deadline parameter to increase timeout for the endpoint: (https://cloud.google.com/endpoints/docs/openapi/openapi-extensions)
Currently I am using the following grpc service configuration for my endpoint.
https://cloud.google.com/endpoints/docs/grpc/grpc-service-config
openAPI extension is not supported for this kind of configuration. Now I am looking for a way to combine
the grpc configuration with openAPI. I have read that it is possible to publish several configuration files for one endpoint.
OK, this kind of configuration works well.
type: google.api.Service
config_version: 3
name: ${cloud_run_hostname_endpoint}
title: ${endpoint_title}
apis:
- name: my_endpoint_name
usage:
rules:
# No APIs can be called without an API Key
- selector: "*"
allow_unregistered_calls: false
backend:
rules:
- selector: "*"
address: grpcs://${cloud_run_hostname_backend}
deadline: 300.0
deadline parameter is accepted.
Related
I've searched a lot of the "Not Found" errors on StackOverflow, along with other APIGW HTTP posts, and they don't match this exact scenario.
Tldr: Custom domain with APIGW V2 HTTP works when the "Path" is blank in the API mappings, but returns 404 "Not Found" when I add the exact path.
Details
I'm using a custom domain (dev.myapp.com) configured in API Gateway. The DNS is setup correctly as you will see shortly.
Here is the API mapping I have setup:
The issue is:
When I use the optional "Path", and invoke dev.myapp.com/oauth/callback/app the API Response returns 404 "Not Found"
I have logs enabled for the HttpApi and the logs don't show any events, so I'm guessing it is not matching to the mapping
When I remove the optional "Path", then invoking dev.myapp.com/oauth/callback/app returns a successful 200
The logs show for the API that is mapped in this case
Been trying to solve this for a day and a half now. Tried many things like re-deploying, etcc.. I need to have multiple API mappings for this custom domain so using a single mapping with the Path field blank is not an option.
Other info:
Custom domain config's "Endpoint type" is "Regional"
The API is hooked up to invoke a lambda, which works properly when the "Path" is removed in the API mappings
I'm using Serverless Framework to deploy the V2 API. Here is part of the serverless yml:
app: myApp
service: ${self:app}-callback
provider:
name: aws
runtime: nodejs16.x
region: us-east-1
logs:
httpApi: true # enable httpApi logs
functions:
oauth-callback:
handler: src/infra/oauth-app-callback-service/index.handler
name: ${self:service}-oauth-callback-${sls:stage}
events:
- httpApi:
path: /oauth/callback/app
method: GET
I've tried re-deploying, smaller "path" such as only oauth, turning off "default endpoint", but none have worked. I've looked at this solution but its a different issue which was with load balancer path rules which is not applicable here
What I expect is when using APIGW V2 HTTP with a custom domain and an API mappings with a "Path", the mapped API should be found and not return "Not Found"
After much searching, I went to AWS's community and found this post.
Go to your API Gateway and select Route
Click Create
Leave the Any method as is and type in the path $default without a leading slash.
Click on Attach integration and select your web service endpoint
This worked and I no longer get the 404. Only issue is I have to do this manually and can't do this via Serverless Framework. I'll post a comment here if I figure it out.
Edit:
To set this automatically up with serverless framework, you need to use the $default path, which can only be done as such:
functions:
oauth-callback:
handler: src/infra/oauth-app-callback-service/index.handler
name: ${self:service}-oauth-callback-${sls:stage}
events:
- httpApi: '*'
Now you're custom domain API mapping will work without having to deploy and then set the $default route for each API.
I am having a problem deploying my application in AWS with CloudFormation using the Serverless Framework.
I am using the "Single API Gateway Project" strategy for the deployment. I have my Backend divided into services, each with its directory inside the repo and its serverless.yml file.
To have a single API Gateway for each one of them, I first deploy a root service that creates said API Gateway for me and outputs the ApiGatewayRestApiId and ApiGatewayRestApiRootResourceId as I could see in the following document of the same Serverless Framework:
https://www.serverless.com/framework/docs/providers/aws/events/apigateway#easiest-and-cicd-friendly-example-of-using-shared-api-gateway-and-api-resources
My root service that creates the API Gateway is something like:
...
resources:
Outputs:
ApiGatewayRestApiId:
Value:
Ref: ApiGatewayRestApi
Export:
Name: ${self:provider.stage}-ApiGatewayRestApiId
ApiGatewayRestApiRootResourceId:
Value:
Fn::GetAtt:
- ApiGatewayRestApi
- RootResourceId
Export:
Name: ${self:provider.stage}-ApiGatewayRestApiRootResourceId
...
Then, from the rest of the microservices, I use those values by importing them as follows:
service: name
custom:
APP_ENV: ${env:SERVERLESS_APP}
providers:
apiGateway:
restApiId: !ImportValue ${env:${self:custom.APP_ENV}_API_STAGE}-ApiGatewayRestApiId
restApiRootResourceId: !ImportValue ${env:${self:custom.APP_ENV}_API_STAGE}-ApiGatewayRestApiRootResourceId
...
I never had any problems deploying until today, when I tried to deploy only the root service. The error I am having is the following:
An error occurred: root-beta - Template error: RootResourceId attribute of API Gateway RestAPI d8zc1j912b doesn't exist.
I checked everywhere but I can't find the reason why I get this error.
Operating System: linux
Node Version: 13.10.0
Framework Version: 1.60.0
PluginVersion: 3.8.4
SDK Version: 2.3.2
Components Core Version: 1.1.2
Components CLI Version: 1.6.0
Based on a comment I just read on Github, this is caused by an API that consists of more than 200 resources in total. CloudFormation (which is used to deploy serverless resources) only retrieves up to 8 pages of 25 resources to locate the root resource id, so if it happens to be on page 9+, it'll throw that error.
I'm not quite sure what the order of these resources is. In my own tests, sometimes it works, and other times it fails.
Original text from AWS (copied here in case the Github issue disappears):
... CloudFormation is making an API call with action “apiateway:GetResources” in order to get the root resource ID. This corresponds to the following API call [1]. This is a paginated response, meaning that it only returns resources 25 at a time (by default). Furthermore, to prevent throttling and time out issues, this is only checked a maximum of 8 pages deep. This means, that this root resource ID must be in the first 200 responses in order to get the root resource ID. Otherwise, it will fail to get this value.
...
The service team is aware of this issue and are working on a way to fix this. However, there is no ETA for this to be available.
Source: https://github.com/serverless/serverless/issues/9036#issuecomment-1047240189
I'm trying to deploy the basic serverless express api example - https://github.com/serverless/examples/tree/v2/aws-node-express-api with a custom domain. It uses HTTP Api instead of the older REST API gateway.
The domain has been set up successfully, certificate in place etc. but when I run serverless deploy it throws an error:
Error: Unable to setup base domain mappings for mycustom.domain.name
Adding SLS_DEBUG revealed:
Service Information
-------------------
service: my-service-name
stage: staging
region: us-east-1
stack: my-service-name-staging
resources: 11
api keys:
None
endpoints:
ANY - https://123323.execute-api.us-east-1.amazonaws.com
functions:
api: my-service-name-staging-api
layers:
None
Serverless: [AWS cloudformation 400 1.296s 0 retries] describeStackResource({
LogicalResourceId: 'ApiGatewayRestApi',
StackName: 'my-service-name-staging'
})
Serverless: [AWS cloudformation 200 1.592s 0 retries] describeStacks({})
Serverless Domain Manager: Error: mycustom.domain.name: Error: Failed to find a stack my-service-name-staging
Serverless Domain Manager: Error: mycustom.domain.name: Error: Failed to find CloudFormation resources for mycustom.domain.name
Clearly the stack exists, but serverless domain manager can't find it. Can I not use custom domain via serverless-domain-manager with HTTP api?? The documentation only shows a manual way to do this.. Any help would be greatly appreciated, thanks!
It's hard to answer 100% without seeing your configuration, but are you sure you've specified apiType: http in your customDomain config section? If not, it defaults to rest. Setting it to http should resolve your problem.
Since your complete serverless.yml is not here, I guess you are using the default one. I had the same issue, and here is my solution.
This is probably the customDomain conf in your .yml file:
customDomain:
domainName: sub.domain.com
certificateName: '*.domain.com'
basePath: ${sls:stage}
stage: ${sls:stage}
createRoute53Record: true
The only thing you need to do is add these two attributes to convert your domain from edge to regional:
customDomain:
domainName: sub.domain.com
certificateName: '*.domain.com'
basePath: ${sls:stage}
stage: ${sls:stage}
createRoute53Record: true
endpointType: regional <-- New Attribute (default is edge)
apiType: http <-- New Attribute (default is rest)
Remember, if you already have run the sls create_domain command (with the previous serverless.yml configuration), you might face this error:
Error:
Error: Unable to setup base domain mappings for 'sub.domain.com':
Unable to create base path mapping for 'sub.domain.com':
Only REGIONAL domain names can be managed through the API Gateway V2 API. For EDGE domain names, please use the API Gateway V1 API. Also note that only REST APIs can be attached to EDGE domain names.
The reason for this error is that you have created an edge domain with the previous configuration, and now, the deployment wants to use it as a regional domain. To solve this error, just delete the domain with sls delete_domain and then rerun the sls create_domain command. I will automatically create the new records for you.
I am following Google's tutorial for setting up an Endpoint for my cloud function.
When I try to access the endpoint from my browser using URL service_name.a.run.app/function1 I get
Error: Forbidden
Your client does not have permission to get URL /function1GET from this server
As part of the mentioned tutorial and answer from a Google product manager , I'm securing my function by granting ESP permission to invoke my function.
gcloud beta functions add-iam-policy-binding function1 --member "serviceAccount:id-compute#developer.gserviceaccount.com" --role "roles/cloudfunctions.invoker" --project "project_id"
My openapi-functions.yaml
swagger: '2.0'
info:
title: Cloud Endpoints + GCF
description: Sample API on Cloud Endpoints with a Google Cloud Functions backend
version: 1.0.0
host: HOST
x-google-endpoints:
- name: "HOST"
allowCors: "true
schemes:
- https
produces:
- application/json
paths:
/function1:
get:
operationId: function1
x-google-backend:
address: https://REGION-FUNCTIONS_PROJECT_ID.cloudfunctions.net/function1GET
responses:
'200':
description: A successful response
schema:
type: string
Note that I added
- name: "HOST"
allowCors: "true'
to my .yaml file because I need to access the endpoint from a static site hosted on Firebase.
I have followed the tutorial you have mentioned, and indeed I came across the exact same error.
Nothing regarding permissions and roles seemed wrong.
After digging a bit what solved the issue was removing the “GET” at the end of the address.
So the openapi-functions.yaml would be like this:
swagger: '2.0'
info:
title: Cloud Endpoints + GCF
description: Sample API on Cloud Endpoints with a Google Cloud Functions backend
version: 1.0.0
host: [HOST]
schemes:
- https
produces:
- application/json
paths:
/function-1:
get:
summary: Greet a user
operationId: function-1
x-google-backend:
address: https://[REGION]-[PROJECT_ID].cloudfunctions.net/function-1
responses:
'200':
description: A successful response
schema:
type: string
Then make sure you are following all the steps mentioned in the tutorial correctly (except the above part).
In case you get a Permissions Denied error when running any of the steps, try running it again as sudo.
I have also tried adding the same as you:
host: [HOST]
x-google-endpoints:
- name: [HOST]
allowCors: "true"
And all is working well.
Pay extra attention to the CONFIG_ID that changes with each new deployment
Example:
2019-12-03r0
then it goes like:
2019-12-03r1
In case the deployment step fails (it shows some successful messages but it might fail in the end), then make sure you delete the existing endpoint service to avoid issues:
gcloud endpoints services delete [SERVICE_ID]
Also you can use the following to give cloudfunctions.invoker role to all users (Just for testing)
gcloud functions add-iam-policy-binding function-1 \
--member="allUsers" \
--role="roles/cloudfunctions.invoker"
So i've got a scenario that i want to use an endpoint and map the provided requests directly into Kinesis stream.
I was able to do that manually in the aws console.
But is there a way to do change the integration to aws service using serverless or serverless plugin?
I tried to find a way to deploy an endpoint that communicates directly with an aws service, without lambdas, and could not find it.
It's been a while but recently i noticed that there's a plugin now that helps setup this exact configuration, https://github.com/horike37/serverless-apigateway-service-proxy
custom:
apiGatewayServiceProxies:
- kinesis:
path: /kinesis
method: post
streamName: { Ref: 'YourStream' }
cors:
origin: '*'
headers:
- Content-Type
- X-Amz-Date
- Authorization
- X-Api-Key
- X-Amz-Security-Token
- X-Amz-User-Agent
allowCredentials: false
Hope this helps others that are still having this problem