Let's say I have a service to allow people to upload photos. I store these photos in S3, and to make it efficient, I use Cloudfront. In order to upload these photos to S3, I was recommended to use Lambdas with API Gateway. However, I'd also like to send this request to a custom HTTP endpoint as well, and return that in the response from the Gateway. So my ideal process is:
User submits upload photo
Photo gets sent to API Gateway
API Gateway calls Lambda to store photo in S3 and also forwards the request to custom backend API
Backend sends back some info
API Gateway sends back this info to client
Is this possible? From the integrations doc, it seems like I can only do Lambdas or HTTP custom endpoint. Not sure how to do both.
Your flow should be like this
Create a lambda endpoint to create signedUrl and return to frontend for s3 upload
Once you have the signed URL upload the pic to the signed URL from the frontend
Once the operation is successful either you can send a request to the lambda from the frontend or a event on the s3 bucket to do further processing
Related
I have a react application uploaded on S3 and node application on Lambda which has endpoints available for only authenticated users. Authentication on React app is done by Cognito, I have hardcoded Google Sign in button. After authentication, user is redirected to S3 endpoint(/Code) which gets the code=... from URL, sends it using Axios to specific Lambda endpoint. From Lambda side when it receives code it gets token from /oauth2/token and sends it back to S3, which sets it as a cookie using universal-cookie package. After this S3 sends every request to Lambda using {credentials: true}
For Cognito authentication to work properly, it needs login Callback url to be HTTPS. I created a Cloudfront distribution to forward requests to S3. So I setup my callback url to be Cloudfront URL. The authentication is done correctly and I can see that cookie is set and is sent in every request:
But it's not forwarded to S3, I see a request that is sent to Lambda:
Which doesn't contain the cookie. I changed Cloudfront behaviour:
But it didn't help. I'm missing some knowledge in Cloudfront. Should I set some other option to forward Cookies to S3?
I have a use case where I have to upload a large file in the S3. I am planning to do so creating a REST API as an Amazon S3 proxy in API gateway.
I have some queries regarding the same
1. Is it possible to upload a file more than 10 MB? Is the size of the file restricted due to the payload size of API Gateway ?
Is there any other mechanism to fetch the bucket name other than fetching it from URL request param aa it is currently being done in most examples ?
I'm working on the application that will receive files from users and then upload to Amazon S3. The application is accessed using API Gateway. The service API Gateway has limits for payload size for both WebSocket and REST APIs. Is there any way to access my service from the Internet through API Gateway?
API gateway is not intended to be data transfer gateway, but lightweight API definition layer.
The most suitable approach is to generate temporary pre-signed upload URL and redirect (30X) requests there. API Gateway should define an endpoint, calling lambda function which generates pre-signed S3 URL and redirect post request there (after user's authentication of course).
Please refer an example of app with API Gateway and pre-signed S3 URLs to upload files
API documentation for generating pre-signed S3 URLs in Python, AWS CLI and even Go-lang
I'm trying to configure AWS API Gateway for a Lambda function which receive the event Webhook request from Sendgrid (https://sendgrid.com/docs/API_Reference/Webhooks/event.html).
The requests will be sent from outside of my internal system so I want to have some kind of authentication for it. But according to Sendgrid'd documents, only basic http authentication is supported. The URL will look like : http(s)://username:password#domain/foo.php
I have no idea how to setting up API Gateway so it can at least pass the username and password to the Lambda function.
I want to create an Api Gateway route which is connected via service proxy to s3.
Only authenticated and authorized users (from Cognito Userpool) which have a specific permission (which is stored in a DynamoDb table) should be able to upload the file.
Since I'm not using S3 as the service proxy and not lambda, is it ok to put the code checking against the Dynamodb in the Custom Authorizer lambda? (After the token has been verified and before sending the success callback).
The query is simple, based on the Cognito user unique Id, I check in a table that user if user is authorized to upload.
I wouldn't want the upload to be done via a lambda function since some files are big.
Thanks
You can also consider
Setup CloudFront infront of S3 bucket.
Using IAM authorization at API Gateway.
Write a Lambda endpoint which checks Cognito ID in request context against Dynamodb.
Return Signed URL from Lambda for authorized users to directly upload files to S3 from browser client.