FIle upload in GCP API Gateway - google-cloud-platform

I'm trying to create the following api config for GCP API Gateway (ommitted real backend URL):
swagger: '2.0'
info:
title: upload
description: upload
version: 1.0.0
schemes:
- https
produces:
- application/json
security:
- api_key: [ ]
paths:
/upload:
post:
summary: uploads a file.
consumes:
- multipart/form-data
operationId: uploadFile
parameters:
- in: formData
name: file
description: The file to upload.
required: true
type: file
responses:
'200':
description: upload successful
x-google-backend:
address: https://XXXXX.XXXXX
path_translation: APPEND_PATH_TO_ADDRESS
securityDefinitions:
api_key:
type: "apiKey"
name: "key"
in: "query"
Running gcloud (replaced real variables with placeholders)
gcloud api-gateway api-configs create uploadconfig --api=[API] --openapi-spec=openapi.yaml --project=[MYPROJECT] --backend-auth-service-account=[ACCOUNT]
This results in this error message:
ERROR: (gcloud.api-gateway.api-configs.create) INVALID_ARGUMENT: Cannot convert to service config.
'location: "unknown location"
kind: ERROR
message: "http: repeated message field 'google.protobuf.Struct.fields' referred to by message 'UploadFileRequest' cannot be mapped as an HTTP parameter."
location: "unknown location"
kind: ERROR
message: "http: cyclic message field 'google.protobuf.Struct.FieldsEntry.value' referred to by message 'UploadFileRequest' in method 'method 1.xxxxxxx.UploadFile' cannot be mapped as an HTTP parameter."
I verified the gcloud command with a different api configurations and backends.
The config itself seems fine, i.e. it validates with Swagger editor, still gcloud won't accept it.
How do I define a file upload via API Gateway?

As per file upload Endpoints does not accept the type: file for file upload parameters, type: string should be used instead.And I tried config with changed parameters and it validates with Swagger editor results are here.

With SwaggerHub and OpenAPI 2 spec, this is how I got this file upload to work. Let me know if you have any questions. This is written in YAML as opposed to JSON spec. It took some reading and some experimentation, but here's the Gcloud documentation: https://cloud.google.com/storage/docs/uploading-objects
And the SwaggerHub info as well: https://swagger.io/docs/specification/2-0/file-upload/
/upload/storage/v1/b/{bucket}/o?uploadType=media&name={objectName}:
post:
summary: Upload an object directly to a bucket
description: Upload an object directly to a bucket
consumes:
- multipart/form-data
produces:
- application/json
parameters:
- in: header
name: Authorization
type: string
required: true
- in: path
name: bucket
required: true
type: string
description: The name of the bucket to upload to.
- in: path
name: objectName
required: true
type: string
description: The name the object will receive in the bucket.
- in: header
name: Content-Type
type: string
required: true
description: The content type of the upload. Ex. text/plain
- in: formData
name: fileToUpload
type: file
description: The file to upload.
responses:
200:
description: OK
204:
description: Success - No Content
400:
description: Bad Request
401:
description: Insufficient Privileges
404:
description: Not Found
It DOES accept type: file in the formData request.

Related

I can't create a Gateway API Google Error: "Cannot convert to service config. 'location: "unknown location"

I'm trying to create an API gateway on Google Cloud Platform, after filling in all the fields and clicking on create gateway the following error message appears:
Cannot convert to service config. 'location: "unknown location" kind: ERROR message: "Unable to parse the content. while parsing a block mapping\n in 'reader', line 1, column 1:\n swagger: '2.0'\n ^\nexpected , but found BlockMappingStart\n in 'reader', line 2, column 5:\n info:\n ^\n\n at [Source: (StringReader); line: 1, column: 15]" '
I believe this is related to the configuration of the Yaml file that is required in the API Spec field as shown in the image below:
My yaml file is configured as follows:
swagger: '2.0'
info:
title: API Gateway for Cycle
description: "Send a deal object for the data to be treated"
version: "1.0.0"
host: teste.apigateway.project-teste-homolog.cloud.goog
schemes:
- "https"
produces:
- "application/json"
paths:
"/data-verification-homologation":
post:
x-google-backend:
address: URL.example
description: "Jailson esteve aqui =)"
operationId: "dataVerification"
parameters:
-
name: iataCode
in: query
required: true
type: string
responses:
200:
description: "Sucess"
schema:
type: string
400:
description: "Error"
I've already checked the following google documentation https://cloud.google.com/endpoints/docs/grpc/troubleshoot-config-deployment, but I couldn't solve the error.
Your indentation is incorrect.
swagger: "2.0"
info:
title: "API Gateway for Cycle"
description: ...
YAML requires very precise indentation.
See YAML Swagger (OpenAPI) example here: https://swagger.io/docs/specification/basic-structure/

GCP Cloud endpoint portal does not show xml definition defined in swagger

Having an issue with displaying xml structure defined in swagger file on cloud endpoint portal (Developer portal) for example it does not show the namespaces and example defined, but it works fine when uploaded on swagger editor
Following is example of xml definition declared
MsgResp:
type: object
properties:
Code:
type: string
example: RC_001_SUCCESS
Message:
type: string
example: Message sent
xml:
name: 'MessageResponse'
wrapped: true
namespace: http://MsgResponse
Edit:
Swagger file
# [START swagger]
swagger: '2.0'
info:
title: <Endpoint-name>
description: <Endpoint-name>
version: 1.0.0
# Connects to the cloud run running the ESP Beta 2 image
host: <Endpoint-address> # CloudRun/Esp url
security: []
schemes:
- https
paths:
"/status":
post:
description: "Test API for sending request from system 1 to IIP. "
operationId: "status-api"
# Defines which service it should connect to for backend processing, It can be Cloud function/ Cloud Run url
x-google-backend:
address: https://<Function1-address> # Backend Cloud function URL
deadline: 3600.0
# Defines Authentication mechanism to use, Following mentions to use API KEYS
security:
- api_key: []
# MIME Types expected as request and response
produces:
- "application/xml"
consumes:
- "application/xml"
parameters:
- in: body
name: schema
description: Input Schema for /status
schema:
$ref: '#/definitions/InSchema'
responses:
200:
description: OK
schema:
$ref: '#/definitions/MessageResponse'
404:
description: Not Found
500:
description: Internal Service Error
definitions:
InSchema:
type: object
xml:
name: 'Identifier'
prefix: 'msg'
wrapped: true
namespace: 'http://Identifier'
properties:
Number:
type: integer
LogIdentifier:
type: object
properties:
Code:
type: integer
Type:
type: string
xml:
name: 'LogicalIdentifier'
wrapped: true
namespace: http://LogicalIdentifier
prefix: sample
example: # <----------
Number: 38
LogIdentifier:
Code: 100
Type: CDC
MessageResponse:
type: object
properties:
Code:
type: string
example: SUCCESS
Message:
type: string
example: Message sent
# [START securityDef]
securityDefinitions:
api_key:
type: "apiKey"
name: "key"
in: "query"
# [END securityDef]
As seen on swagger editor
As seen on Cloud endpoint portal/ application portal
According to Cloud Endpoints on Cloud Run Official Documentation, I can only see the .json MimeType is used in the example.
The Cloud Endpoint service definition should be based on OpenAPI Specification v2.0, also known as Swagger 2, which describes the surface of your backend service and any authentication requirements.
So checking the OpenAPI Specification v2.0 in GitHub, I was not able to see xml specification in the MimeType Section. However in the Swagger Official Documentation, I can see that the xml media type is supported as well.
So I would like to ask you to check all the steps provided in the Cloud Endpoint on Cloud Run Official Documentation.
In the screenshot, I can see 404 NOT_FOUND error, this error is mentioned in the Troubleshooting section of Cloud Endpoints, so please have a look into it.

Google cloud API Gateway with authentication yaml config file deployment error

I am new in API Gateway and trying to enable security on my API. I follow some instruction which I found on the internet like this:
https://medium.com/swlh/manage-serverless-apis-with-api-gateway-in-gcp-b7f906efec1a
Here is my YAML file:
# openapi2-functions.yaml
swagger: '2.0'
info:
title: simple-test
description: Sample API on API Gateway with a Google Cloud Functions backend
version: 1.0.0
schemes:
- https
produces:
- application/json
paths:
/direcciones:
get:
summary: get direcciones
operationId: direcciones
x-google-backend:
address: http://publicIP/api/v1/app/catalogos/direcciones
security:
- api_key: []
responses:
'200':
description: A successful response
schema:
type: string
securityDefinitions:
api_key:
type: "apiKey"
name: "key"
in: "query"
While deploying this config file in API gateway config, I am getting the following error:
INVALID_ARGUMENT Cannot convert to service config. 'location: "evva.yaml: x-google-backend" kind: ERROR message: "Extension x-google-backend cannot be converted into proto type google.api.BackendRule. Details: Cannot find field: security in message google.api.BackendRule" location: "evva.yaml: x-google-backend" message: "Address field in extension x-google-backend is empty. In this case, the backend address must be provided to the proxy via a runtime flag." location: "evva.yaml: Operation \'get\' in path \'/direcciones\'" message: "Operation does not require an API key; callers may invoke the method without specifying an associated API-consuming project. To enable API key all the SecurityRequirement Objects (https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md#security-requirement-object) inside security definition must reference at least one SecurityDefinition of type : \'apiKey\'." ' com.google.apps.framework.request.BadRequestException: Cannot convert to service config. 'location: "evva.yaml: x-google-backend" kind: ERROR message: "Extension x-google-backend cannot be converted into proto type google.api.BackendRule. Details: Cannot find field: security in message google.api.BackendRule" location: "evva.yaml: x-google-backend" message: "Address field in extension x-google-backend is empty. In this case, the backend address must be provided to the proxy via a runtime flag." location: "evva.yaml: Operation \'get\' in path \'/direcciones\'" message: "Operation does not require an API key; callers may invoke the method without specifying an associated API-consuming project. To enable API key all the SecurityRequirement Objects (https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md#security-requirement-object) inside security definition must reference at least one SecurityDefinition of type : \'apiKey\'." '
I don't understand this error what change should i need to do in YAML file to make it acceptable while deploying.
The security entry mustn't be "in" the x-google-backend, but bellow the get:. Like this.
paths:
/direcciones:
get:
summary: get direcciones
operationId: direcciones
x-google-backend:
address: http://publicIP/api/v1/app/catalogos/direcciones
security:
- api_key: []

passing path parameter in google endpoints to backend not working

My setup contains google-endpoints with google-cloud-functions as my backend.
Google endpoints is defined with the following swagger v2 yaml:
swagger: "2.0"
info:
description: "yada..."
version: "0.0.1"
title: "yadada.."
termsOfService: "http://swagger.io/terms/"
contact:
name: "blah"
email: "email#mail.com"
url: "https://example.com"
host: "(generated service url by google when endpoints is deployed, i.e. 'api-gateway-xyz123123-ew.a.run.app')"
tags:
- name: "Documents"
description: "blah"
schemes:
- "https"
paths:
/api/documents:
post:
tags:
- "Documents"
summary: "Add a new document"
description: ""
security:
- firebase: []
operationId: "addDocument"
x-google-backend:
address: "(cloud functions http url)/documents"
consumes:
- "application/json"
produces:
- "application/json"
parameters:
- in: "body"
name: "body"
description: "Document supplied"
required: true
schema:
$ref: "#/definitions/Document"
responses:
201:
description: "The document was successfully created."
schema:
$ref: "#/definitions/Document"
400:
description: "Invalid input. See response for details"
schema:
items:
$ref: "#/definitions/Error"
/api/documents/{document_id}:
get:
tags:
- "Documents"
summary: "Get a document with the given ID"
description: ""
security:
- firebase: []
operationId: "getDocument"
x-google-backend:
address: "(cloud function http url)/documents/"
path_translation: APPEND_PATH_TO_ADDRESS
produces:
- "application/json"
parameters:
- in: "path"
name: "document_id"
description: "ID of the document to modify"
required: true
type: "string"
responses:
200:
description: "success."
schema:
type: "array"
items:
$ref: "#/definitions/Document"
404:
description: "Document not found"
schema:
items:
$ref: "#/definitions/Error"
securityDefinitions:
firebase:
authorizationUrl: ""
flow: "implicit"
type: "oauth2"
x-google-issuer: "https://securetoken.google.com/%%GOOGLE_PROJECT_ID%%"
x-google-jwks_uri: "https://www.googleapis.com/service_accounts/v1/metadata/x509/securetoken#system.gserviceaccount.com"
x-google-audiences: "%%GOOGLE_PROJECT_ID%%"
definitions:
(a lot of type definitions)
This works with the POST endpoint without any problems.
The problem is with the GET REST endpoint where the path variable is not passed correctly to the backend.
As in https://cloud.google.com/endpoints/docs/openapi/openapi-extensions I tried to add the x-google-backend parameter as in the swagger api above. (path_translation: APPEND_PATH_TO_ADDRESS).
However this does not work.
I get an Unauthorized Error (403) as the cloud function is not hit by the endpoints frontend.
Currently I use an ugly workaround without the path_translation parameter which translates the google endpoints path variable to a query parameter in the cloud function backend with the same name. I.e. in the backend the url /documents?document_id=xyz is called.
(What I try to achieve is to pass the call with the backend url /documents/{document_id})
Does anyone know how to configure path based parameters correctly so that they are passed correctly to the cloud function backend?
Thank you in advance.
Regards,
Sebastian
TL;DR:
I assume that your 403 error isn't the correct error. It should be a 404, but because the endpoint is unknown, I guess that 403 is answered.
Cloud Endpoint is frustrating about this behavior. With the path_translation: APPEND_PATH_TO_ADDRESS, you think that your final called address will be /documents/{document_id}, but NO. The full openAPI path is append to your backend address, in your case: /documents/api/documents/{document_id}
That's why the endpoint doesn't exist and you should have a 404 (and not a 403).
For more details, you can have a look to this page.
Note: I'm in relation with Google team on this topic, and it will take time before having an update on this behavior.

Proxying to multiple services using Google ESP

I have been reviewing Google Cloud Extensible Service Proxy, which promises to be a serverless NGINX instance, however I am unsure on how to proxy to multiple services running in app engine through it, and essentially use it as a gateway. It seems to demand a host in the swagger JSON configuration and an environment variable that points to the endpoint service name, and I don't see how it could proxy to multiple services given this constraint.
My understanding is that you could host the Endpoints ESP using Cloud Run. This would then give you a single URL from which to access it but would spin up enough instances of the ESP if needed. The Open API specification document you would then register with it would contain paths corresponding to each instance of a service you want to expose. For each path, you would then define an x-google-backend pointing to the service URL that each path would resolve against.
EDIT:
the yaml file will look somehow like this:
info:
title: Cloud Endpoints with API Keys
description: Sample API on Cloud Endpoints with multiple App Engine with IAP backend
version: 1.0.0
host: <ENDPOINT_URL>
schemes:
- https
produces:
- application/json
paths:
/hello-gae1:
get:
summary: Greet a user from App Engine
operationId: hello_gae
x-google-backend:
address: https://<PROJECT_ID>.appspot.com
parameters:
- in: query
name: name
required: false
type: string
responses:
'200':
description: A successful response
schema:
type: string
/hello-gae2:
get:
summary: Greet a user from App Engine
operationId: hello_gae
x-google-backend:
address: https://<SERVICE-dot-PROJECT_ID>.appspot.com
parameters:
- in: query
name: name
required: false
type: string
responses:
'200':
description: A successful response
schema:
type: string
/hello-gae3:
get:
summary: Greet a user from App Engine
operationId: hello_gae
x-google-backend:
address: https://<SERVICE-dot-PROJECT_ID>.appspot.com
parameters:
- in: query
name: name
required: false
type: string
responses:
'200':
description: A successful response
schema:
type: string
securityDefinitions:
# This section configures basic authentication with an API key.
api_key:
type: "apiKey"
name: "key"
in: "query"
References:
Getting Started with Endpoints for Cloud Run