I am currently trying to execute the AWS cloud watch APIs and get the logs from the cloud watch environment. I have successfully implemented this by using the SDK provided by AWS. Also in the documentations they mentioned that we can also use HTTP requests to get the logs from cloud watch. In the following documentation they have mentioned the params that we have to pass from the request.
https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_FilterLogEvents.html.
The problem is there are also some common params to be sent from the request. I need to clarify the way that we want to send them. As query params, path params or request headers?
The request parameters for the Cloud Watch Rest API should be sent in the JSON format inside the {}, similar to the way in the POST example given. Of these only logGroupName is required, while the other paramaters mentioned are optional:
**
endTime
filterPattern
interleaved
limit
logGroupName
logStreamNamePrefix
logStreamNames
nextToken
startTime
**
In the context of the entire HTTP request:
POST / HTTP/1.1
Host: logs.<region>.<domain>
X-Amz-Date: <DATE>
Authorization: AWS4-HMAC-SHA256 Credential=<Credential>, SignedHeaders=content-type;date;host;user-agent;x-amz-date;x-amz-target;x-amzn-requestid, Signature=<Signature>
User-Agent: <UserAgentString>
Accept: application/json
Content-Type: application/x-amz-json-1.1
Content-Length: <PayloadSizeBytes>
Connection: Keep-Alive
X-Amz-Target: Logs_20140328.FilterLogEvents
{
"endTime": number,
"filterPattern": "mystring",
"interleaved": boolean,
"limit": number,
"logGroupName": "string",
"logStreamNamePrefix": "string",
"logStreamNames": [ "string" ],
"nextToken": "string",
"startTime": number
}
The Common Parameters are sent as HTTP params as seen in the example above. They are needed here for signing your requests to AWS with the proper authentication.
(Authentication occurs automatically in the background when using the CLI)
This is an official walkthrough of how to construct a canonical,signed HTTP request for AWS APIs
For example:
Action=ListUsers&
Version=2010-05-08&
X-Amz-Algorithm=AWS4-HMAC-SHA256&
X-Amz-Credential=AKIDEXAMPLE%2F20150830%2Fus-east-1%2Fiam%2Faws4_request&
X-Amz-Date=20150830T123600Z&
X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-date
Related
I'm trying to test Amazon Translate Services using Postman. When I send my post request, I get the error:
Could not send request -- Error: Request timed out
In the "Authorization" tab, I selected "AWS Signature" and entered my AccessKey, SecretKey, and AWS Region.
In the "Headers" tab, I entered:
Content-Type: application/x-amz-json-1.1
X-Amz-Date: 20180820T184626Z
Z-Amz-Target: AWSShineFrontendService_20170701.TranslateText
In the "Body" tab, I entered:
{
"SourceLanguageCode": "en",
"TargetLanguageCode": "fr",
"Text": "this is a test"
}
I tried to increase my timeout in settings and that did not help.
Any help would be appreciated!
Just learning my way through AWS - I have an APIGateway REST API setup with Lambda proxy integration. The API has a model defined, and request validation setup on the body using this model.
Say the model is
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"propertyA": {
"type": "string"
},
"propertyB": {
"type": "string"
},
"propertyC": {
"type": "string"
},
"propertyD": {
"type": "string"
}
},
"required": ["propertyA", "propertyB", "propertyC", "propertyD"]
}
Now, if I test the API via APIGateway console, and purposely give an invalid input (omitting a required property propertyD):
{
"propertyA": "valueA",
"propertyB": "valueB",
"propertyC": "valueC"
}
the request fails with the error(400): Sun Jul 11 13:07:07 UTC 2021 : Request body does not match model schema for content type application/json: [object has missing required properties (["propertyD"])]
But when I invoke the same API(and stage) with the same invalid input from Postman, the validation seems to be not happening, and request is proxied to Lambda, which even returns a 200 OK as long as I comment out the parts of code that depend on propertyD.
What's the difference here? Should I be passing in any request header from the client side? I couldn't find anything from the AWS documentations
Answering my question-
Issue was with the headers used in the request - Postman defaulted the JSON as a Content-Type of text/plain, I had to switch to JSON using the dropdown in Body tab to make PostMan set the Content-Type to application/json
Following this post seems to have fixed the problem: https://itnext.io/how-to-validate-http-requests-before-they-reach-lambda-2fff68bfe93b, although it doesn't explain how
Apparently the magic lies with the config adding Content-Type Header under HTTP Request Headers section, even though the header is set correctly as application/json in PostMan.
I have a Google Cloud Storage bucket with the following CORS configuration:
[
{
"origin": ["http://localhost:8080"],
"responseHeader": [
"Content-Type",
"Access-Control-Allow-Origin",
"Origin"],
"method": ["GET", "HEAD", "DELETE", "POST", "PUT", "OPTIONS"],
"maxAgeSeconds": 3600
}
]
I am generating a signed URL with the following code:
let bucket = storage.bucket(bucketName);
let file = bucket.file(key);
const options = {
version: "v4",
action: "write",
expires: Date.now() + 15 * 60 * 1000, // 15 minutes
contentType: "application/zip"
};
let url = await file.getSignedUrl(options))[0];
For my requests I am using the following headers:
Origin: http://localhost:8080
Content-Type: application/zip
When I try using a PUT request to upload the data everything works fine and I get the Access-Control-Allow-Origin header containing my Origin. But when I do a OPTIONS request with the exact same headers it fails to return the Access-Control-Allow-Origin header. I have tried many alterations to my CORS config but none have worked like:
Changing the origins to *
The different changes described in Stackoverflow it's answer and comments.
The different changes described in GitHub Google Storage API Issues
I solved my own problem with some help from my colleagues, when I was testing the function in Postman the CORS header was not sent as a response to the OPTIONS request because the request was missing the Access-Control-Request-Method header. When I added this header it worked fine.
Notice that as stated on the public documentation you shouldn't specify OPTIONS in your CORS configuration and notice that Cloud Storage only supports DELETE, GET, HEAD, POST, PUT for the XML API and DELETE, GET, HEAD, PATCH, POST, PUT for the JSON API. So, I believe that what you are experiencing with the OPTIONS method should be expected behavior.
I need to build a function that takes 5 particular HTTP headers + the request params, aggregate, order, encode, and then hash them in order to validate/authenticate the overall request. However, I am unable to get the header 'Content-Length' to come through to the lambda.
I used Terraform to create the API Gateway (aws_api_gateway_domain_name) and then Serverless to create the endpoints:
functions:
alerts:
handler: src/event.handler
role: arn:aws:iam::${env:AWS_ACCOUNT_ID}:role/alerts_lambda
environment:
API_TRANS_KEY: ${env:API_TRANS_KEY}
REGION: ${self:custom.region}
SNS_ARN: arn:aws:sns:us-east-1:${env:AWS_ACCOUNT_ID}:Transactions
STAGE: ${self:custom.deploymentStage}
events:
- http:
path: /alerts/AccountEvent
method: post
cors: true
integration: lambda-proxy
however the headers I get are :
"headers": {
"Accept": "*/*",
"accept-encoding": "gzip, deflate",
"Cache-Control": "no-cache",
"CloudFront-Forwarded-Proto": "https",
"CloudFront-Is-Desktop-Viewer": "true",
"CloudFront-Is-Mobile-Viewer": "false",
"CloudFront-Is-SmartTV-Viewer": "false",
"CloudFront-Is-Tablet-Viewer": "false",
"CloudFront-Viewer-Country": "US",
"Content-Type": "application/x-www-form-urlencoded",
"Date": "20170504:141752UTC",
"Encryption-Type": "HMAC-SHA256",
"Host": "events.dev.myapi.com",
"Postman-Token": "84bd0cc3-f339-4b2a-8017-31ec9174c37e",
"User-Agent": "PostmanRuntime/7.11.0",
"User-ID": "galileo",
"Via": "1.1 50c3c79d5d7adbc8948ea11709b61d17.cloudfront.net (CloudFront)",
"X-Amz-Cf-Id": "1OE1aGP_3Q-CkXFuJbRwvkGAR2ZaHAPuozckZ6747EP64zZcmXjphw==",
"X-Amzn-Trace-Id": "Root=1-5d0bf01b-8afdb9628f42a9357dbb5c68",
"X-Forwarded-For": "73.72.58.46, 70.132.57.87",
"X-Forwarded-Port": "443",
"X-Forwarded-Proto": "https"
},
Do I need to use a mapping template at this point? Is CloudFront/Api Gateway stripping this header out for some reason (note, I don't set up a CloudFront distribution but API Gateway is creating one due to 'edge' type, but I could change it to 'regional' if that would solve this)?
Based on my tests, the Content-Length header can be passed through when making an API request. However, it won't appear in the event payload for a Lambda proxy integration.
Two alternatives that you could use to receive the Content-Length header:
Use an HTTP API with Lambda proxy integration instead of a REST API in API Gateway.
Use a Lambda non-proxy Integration for the REST API and configure it as follows:
In the Mapping Templates for Integration Request:
Set application/json as the Content-Type
Change the default 'Method Passthrough Template' template by adding the statement - "X-Content-Length" : $input.body.length()
In the Lambda function, the variable event ['X-Content-Length'] will contain the Content-Length.
In the Integration Request stage of the API Gateway for a given endpoint, you can map header values to the body. You could also use the Integration Request step to move the lost headers into the body first. You may also be able to directly re-map it to the headers passed to the Lambda function as well.
More information on header/body mapping can be found here:
https://docs.aws.amazon.com/apigateway/latest/developerguide/request-response-data-mappings.html
Is it possible to send request with: Content-Type: multipart/form-data to API Gateway?
In my case, I try to send form-data like below via Postman:
user[email]:extest829#ex.com
user[password]:password
user[password_confirmation]:password
user[username]:testUser
But It seems that API Gateway loses the content.
Everything works fine when I send it as: application/x-www-form-urlencoded or application/json.
Using mulipart/form-data is not fully supported by AWS API Gateway, especially when we try to send file via mulipart/form-data.
To send image along with other data from form, probably the best solution would be send it as JSON, where image is encoded with base64.
For example, when someone want to send:
Username (string)
Avatar (image)
Avatar should be encoded with base64. It gives an image as a text, like:
data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAyA...
The content of POST message would be:
{
"user_account": {
"avatar": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAyA...",
"username": "my name"
}
}
API Gateway
In API Gateway, under your API, open Models and create new model, for example UserAccount:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "UserAccountUpdate",
"type": "object",
"properties": {
"user": {
"type": "object",
"properties": {
"avatar": { "type": "string" },
"username": { "type": "string" }
}
}
}
}
In Method Request -> HTTP Request Headers set: Content-Type.
In Method Request -> Request Body set: application/json and as model use created UserAccount model.
In Integration Request -> Content Handling set as: Passthrough.
In Integration Request -> Body Mapping Templates choose: When no template matches the reuqest Content-Type header. (You can also use two other options here and set additional mapping templates).
Backend
Backend receives an image as a text encoded with base64. So probably before it uses futher, it needs to decode it from text to image.
Encoding/decoding images with base64 is quite popular, so you should find some appropriate tools / libs in your frameworks.
When you are sending request with Content-Type: multipart/form-data to API Gateway, you need to pass through your original Content-Type header to your integration endpoint.
aws apigateway update-integration \
--rest-api-id a1b2c3d4e5 \
--resource-id a1b2c3 \
--http-method POST \
--patch-operations op='replace',path='/requestParameters/integration.request.header.Content-Type',value='method.request.header.Content-Type'
Code example for managing binary data in AWS Gateway and Proxy+ Lambda. Or to upload and retrieve files from servers using AWS Gateway and Proxy Lambda+
Click here