Requirement:
Use AWS Lambda + API Gateway
Function language: Python
Convert a text from UTF-8 to Shift_JIS (such as: 武蔵野冷蔵名古屋)
Response with csv format (application/csv)
Issue:
I'm using response = codecs.encode(inputString, 'SHIFT_JIS', errors='ignore') with inputString something like 武蔵野冷蔵名古屋: encode input string with shift_jis
And I return response to user. But facing error Invalid lambda response received: Invalid API Gateway Response Keys: {'requestId', 'errorMessage', 'stackTrace', 'errorType'} in {'errorMessage': "Unable to marshal response: 'utf-8' codec can't decode byte 0x95 in position 0: invalid start byte".
Seem that API Gateway just support UTF-8 for text.
My concern is how to response non UTF-8 text when combine Lambda + API Gateway.
I'm checking the guide setup Binary data with API Gateway but not work now
I don't want to base64.encode it and decode later at client side.
Any help for this issue? Thanks!
Thank God!
I was guided to solve this issue.
Just base64 encode the string_after_encode_to_shift_jis.
return {
'statusCode': 200,
'headers': {
'Access-Control-Allow-Origin': '*',
'Content-Type': f'application/csv; charset=shift_jis'
},
'body': base64.b64encode(data.encode(enc)).decode('utf-8'),
'isBase64Encoded': True
}
Related
I have the following architecture:
(1) A front-end form, which has a user input of file:
var data = new FormData();
data.append('username', 'USER123');
data.append('file', selectedFile); //selectedFile is a file which I capture form a form (user-submitted)
await fetch('myURL/test', {
method: 'post',
headers: {
"Access-Control-Allow-Origin": "*"
},
body:data
}).then(res => res.json()).then(json => console.log(json))
console.log('Done')
(2) An AWS API Gateway (which has a POST route of 'myURL/test'),
(3) An AWS Lambda Integration with the following Python Code:
def lambda_handler(event, context): #Lambda functions works and returns a response of "Hello from Lambda" during a POST request
# TODO implement
print(event['body']) //prints some base64 string
return {
'statusCode': 200,
'body': json.dumps('Hello from Lambda!')
}
What I am trying to achieve: User submits form (with .zip file) from my front-end, (2) Lambda function receives the file, (3) Lambda function checks whether file is correct size, type, (4) Lambda function uploads file to S3 Bucket. The thing is I am unable to receive the file in my lambda function. Printing of the event['body'] prints out some weird string that I'm unsure on how to process.
Any tips?
Verify that the Incoming Request at the Gateway is actually passing the POST body content (the file) to the Lambda. Likely you'll need to setup a mapping template along these lines...
{ "content": "$input.body" }
Here's a pretty good article showing the end-to-end setup to upload a file:
https://medium.com/swlh/upload-binary-files-to-s3-using-aws-api-gateway-with-aws-lambda-2b4ba8c70b8e
I want to POST form data to api gateway which then sends it to lambda for processing. However, I get
{
"message": "Could not parse request body into json: Unexpected character (\'-\' (code 45))
in numeric value: expected digit (0-9) to follow minus sign, for valid numeric value\n
at [Source: (byte[])\"----------------------------086228525798973846089611\r\nContent-
Disposition: form-data; name=\"first_name\"\r\n\r\nfake\r\n---------------------------
-086228525798973846089611\r\nContent-Disposition: form-data;
name=\"last_name\"\r\n\r\nname\r\n---------------------------
-086228525798973846089611\r\nContent-Disposition: form-data;
name=\"email\"\r\n\r\nfakename1#something.com\r\n---------------------------
-086228525798973846089611\r\nContent-Disposition: form-data; name=\"mobile\"\r\n\r\n\r\n-
---------------------------086228525798973\"[truncated 752 bytes]; line: 1, column: 3]"
}
In the cloudwatch logs I can see lambda gives this error: Lambda invocation failed with status: 400. Can't see any errors in lambda logs just api gateway logs.
My api gateway POST method looks like this:
Edit:
Adding image of form I am trying to submit that the API isn't liking:
Any ideas how to POST through to lambda without api gateway transformations or other issue?
I'm uploading an image to s3, through a lambda, and everything works well, with no errors, but the response from API Gateway is 500 Internal Server Error.
I configured my api-gateway following this tutorial: Binary Support for API Integrations with Amazon API Gateway.
My lambda receives the base64Image, decode it and successfully upload to s3.
This is my lambda code:
def upload_image(event, context):
s3 = boto3.client('s3')
b64_image = event['base64Image']
image = base64.b64decode(b64_image)
try:
with io.BytesIO(image) as buffer_image:
buffer_image.seek(0)
s3.upload_fileobj(buffer_image, 'MY-BUCKET', 'image')
return {'status': True}
except ClientError as e:
return {'status': False, 'error': repr(e)}
This is what i'm receiving:
{
"message": "Internal server error"
}, with a 500 status code.
Obs: I'm not using lambda proxy integration.
You need to return a header in the response, e.g. in Python:
return {
"statusCode": 200,
'headers': { 'Content-Type': 'application/json' },
"body": json.dumps(body)
}
That example looks like it falls short on mapping the responses section in favor of a pass through. In which case changing your return to: return {'status': True, 'statusCode': 200} might work.
Generally speaking there are two paths when building a response with ApiGateway-Lambda. One is the lambda-proxy (where your lambda function defines the response), the other is where ApiGateway transforms your responses and generates the appropriate headers/status based on a mapping.
The path from the example is for the latter.
Personally I would change:
return {'status': True}
to return {'status': "Success"} And create a regex that looks for the word "Success" and "Error" respectively.
I have used this blog post successfully with this technique (it also describes at length the differences between the two approaches). Once you get one mapping working you could adjust it as is more appropriate for your implementation.
EDIT: hot tip these decorators are awesome and make python & lambda even cleaner/easier but mostly for the proxy setup
I setup an API using amazon api gateway and want to put data into Kinesis streams. Amazon API gateway has inbuilt support for it. But when I try to put JSON data it gives "Serialization exception".
var data = {"ua_platform":"iPhone","ua_browsercodename":"Mozilla","ua_browserlanguage":"en-us","ua_header":"Mozilla\/5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) AppleWebKit\/601.1.46 (KHTML, like Gecko) Version\/9.0 Mobile\/13B143 Safari\/601.1","ua_browsername":"Netscape","key":"livestream_hindi",,"datetime_ut":"1458711871","datetime_dt":"2016-03-23","value":"15","source":"0","browser":"Mobile Safari-9.0.","os":"iOS-9.1.","device_detail":"iPhone Apple iPhone"};
var json = JSON.stringify(data);
var params = {
'ContentType': 'application/json',
'Access-Control-Allow-Headers': 'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'
};
var body = {
"Data": json,
"StreamName": "XXXXXX",
"PartitionKey": "XXXX"
};
After this I make a put request
apigClient.functionPut(params, body, additionalParams)
.then(function(result) {
// This is where you would put a success callback
console.log("success");
})
.catch(function(result) {
// This is where you would put an error callback
console.log("catch");
});
The API gives 200 ok along with the serialization exception. It is stated in put request the "Data" key in body variable will accept only "blob" type. Now I also tried converting JSON data to BLOB but no luck at all.
I am not able to figure out what I am doing wrong. Please help.
There's a walkthrough of how to set up API Gateway in front of Kinesis in the official AWS docs at http://docs.aws.amazon.com/apigateway/latest/developerguide/integrating-api-with-aws-services-kinesis.html
You must use the $utils variable to convert to base64 on your integration request template, here's my integration request template:
#set($msgBody = $util.parseJson($input.body))
#set($msgId = $msgBody.messageId)
{
"Data": "$util.base64Encode($input.body)",
"PartitionKey": "$msgId",
"StreamName": "arena-hub-dev-ks"
}
See the line with the "Data" I'm using $util.base64Encode. In this case my API Gateway endpoint accepts text/plain.
I have a lambda function that returns a string that contains UTF-8 characters (german umlaute). However, The AWS API-Gateway response does not contain the utf-8 header. How can I add it so the client receives a readable response?
I was able to solve this problem like this in Python:
return {
'statusCode': 200,
'headers': {
"content-type":"application/json; charset=utf-8"},
'body': json.dumps(data)
}
I just needed to set the additional header "content-type".
But I'm not sure whether you need to have "HTTP-Proxy-Integration" enabled (like i did).