I created a python flask api which accepts image uploading. Something similar to the following:
#app.route("/image_upload", methods=['POST'])
def image_upload():
image = request.files['image']
image_data = image.read()
base64encoded = base64.b64encode(image_data).decode('utf-8')
return base64encoded
I deployed it to okteto, and if I upload image against the okteto endpoint, it works well.
I then add an AWS API gateway to proxy the requests to the okteto endpoint:
I then try to upload the same image through the API gateway endpoint, then the image received in the flask will be corrupt.
I noticed that the payload seems twice as big as without the API gateway. And the base64 string can NOT be rendered as an image anymore.
So how does the API Gateway handle the file uploading? I tried searching in both google and AWS documents, there are many saying how to use API Gateway to upload files to S3. But I don't want to upload to S3. I just want the API Gateway forward the payload to my Flask app.
The image can be as small as needed. So it's far less than the limit of the gateway.
Fixed by adding the binary media types to the gateway.
Detail explained (in Chinese though):
https://zhuanlan.zhihu.com/p/595401224
Related
Let's say I have a service to allow people to upload photos. I store these photos in S3, and to make it efficient, I use Cloudfront. In order to upload these photos to S3, I was recommended to use Lambdas with API Gateway. However, I'd also like to send this request to a custom HTTP endpoint as well, and return that in the response from the Gateway. So my ideal process is:
User submits upload photo
Photo gets sent to API Gateway
API Gateway calls Lambda to store photo in S3 and also forwards the request to custom backend API
Backend sends back some info
API Gateway sends back this info to client
Is this possible? From the integrations doc, it seems like I can only do Lambdas or HTTP custom endpoint. Not sure how to do both.
Your flow should be like this
Create a lambda endpoint to create signedUrl and return to frontend for s3 upload
Once you have the signed URL upload the pic to the signed URL from the frontend
Once the operation is successful either you can send a request to the lambda from the frontend or a event on the s3 bucket to do further processing
Im learning about AWS and Flutter to develop a mobile app (Android) like Instagram for a subject in the university.
My question is if there is a way to send an image from the app directly to a S3 Bucket and just store the URL related to that image in DynamoDB.
I’m using Flutter for the frontend (it doesn't use AWS SDK) and to develop the backend AWS Api Gateway (Api REST), Lambda and DynamoDB.
I read that the image can be passed in the request body if it is converted to base64 but I’m worried about if it is the optimal way to do it. I’m thinking that if I pass the image in that way the AWS free tier of Api Gateway and Lambda will be consumed because of the size of the request body that is passed and processed to be saved in a S3 Bucket considering that I will upload several images.
The dimensions of the image will be 1080x1350px so I think the size will be around 500KB and I'm planning to upload videos too.
I will be grateful for any kind of guidance because I am new to AWS.
You may use S3 presigned url as it will provide scalable solution which will overcome limits imposed by API Gateway. Because video file - particularly HD files, will introduce issues on limit w.r.t API Gateway later on if you pass file directly to API Gateway.
I'm working on the application that will receive files from users and then upload to Amazon S3. The application is accessed using API Gateway. The service API Gateway has limits for payload size for both WebSocket and REST APIs. Is there any way to access my service from the Internet through API Gateway?
API gateway is not intended to be data transfer gateway, but lightweight API definition layer.
The most suitable approach is to generate temporary pre-signed upload URL and redirect (30X) requests there. API Gateway should define an endpoint, calling lambda function which generates pre-signed S3 URL and redirect post request there (after user's authentication of course).
Please refer an example of app with API Gateway and pre-signed S3 URLs to upload files
API documentation for generating pre-signed S3 URLs in Python, AWS CLI and even Go-lang
We need to save product's images in AWS. There are 2 ways, it can be uploaded from frontend(website or mobile application) or from backend.
On frontend side we need to store AWS credentials, which can be an issue. So, we want to go with upload on AWS from backend. The flow will be: user select an image, and upload it to backend, and backend upload it to AWS.
Is this ok? What issues can appear?
There is nothing wrong with uploading it to your backend, presumably an ec2 instance, and then having the ec2 instance upload the file to s3 - thats a secure way of doing it and a method I often use.
However, you do not need to expose your aws credentials to the browser if you would prefer to do the upload directly from your browser to s3 - you would just need to add AWS Cognito to the equation.
Using cognito you can get temporary credentials that will allow you to do the upload without compromising security.
https://aws.amazon.com/sdk-for-browser/
You can also used a pre-signed S3 URL (see: Uploading Objects Using Pre-Signed URLs) which you generate in the backend and pass to the frontend app. Then the flow would be something like this:
Request a pre-signed URL from your backend service
The frontend app PUTs the file to the signed URL
Signing the URL on the backend would look something like this (Ruby):
s3 = Aws::S3::Resource.new(region: 'us-east-1')
url = s3.bucket('my-bucket').object('name-of-file').presigned_url(:put)
And on the frontend you could simply do something like this using fetch:
fetch(signedUrl, { method: 'PUT', body: file })
Is it possible for AWS Gateway API to respond with a file (zip file) from a HTTP endpoint integration? I heard somewhere AWS Gateway API doesn't support binary formats but wasn't sure if that was for input or input and output.
I have an existing HTTP endpoint and I want to add AWS Gateway API over it; it currently returns a file (zip) on the response.
You cannot respond with a Zip(any binary type) file using API Gateway so far. (As stated in AWS official forum)
As a work around, you can store your file on S3 and dispatch the link of the file using API Gateway.
Binary payloads are not yet natively supported as API Gateway currently encodes content as UTF-8. For the purposes of serving files, serving them via S3 may be an appropriate workaround. You could configure your API to return a link to S3 or to redirect to the public S3 URL.
Thanks,
Ryan