The Cloudformation template "Serverless Image Handler" by AWS is nice, but the images do not seem to cache automatically in the browser.
Do you know where to set this so that the same URL is not downloaded again, at least during the same session? I'm using the same image on two different screens of a single page app and it always just redownloads. There should effectively be no second HTTP request, not even to check an ETag or something.
The setup uses a Cloudfront distribution, a Lambda function that does the resizing, API Gateway, and S3 as image storage.
Any ideas?
Found the answer, there is an open pull request to fix this on Github.
https://github.com/awslabs/serverless-image-handler/pull/158
Related
We have some videos in an S3 bucket. they've been transformed using AWS Elastic Transcoder to .m3u8 / .ts
We want the users to be able to stream these videos on both a web app and a mobile app.
Now, we want to secure this streaming, so our videos won't get pirated.
So, our proposed solution is as follows:
Prevent public access to the S3 bucket
create a cloudfront distribution with the bucket as the origin
Only enable access to this CDN using pre-signed URLs/cookies
For web app: use a pre-signed cookie (set by an endpoint at our backend that requires authentication), so that it works well with HLS (since the app needs to fetch a new segment every few seconds)
But now we don't know what to do with our mobile app. We can't use pre-signed cookies since there's no browser, and we can't use pre-signed URLs, since we'll need a signed URL for each segment we need to fetch. Any suggestions and solutions are welcome.
For our similar use-case:
We used CloudFront url and not S3 signed url. Because S3 signed URL is valid at object level and not folder level.
For paid videos, security and access was managed by Lambda#Edge on viewer requests.
Although we used OAuth and database inside that lambda, but surprisingly, we didn't face any bottlenecks on Lambda#Edge. For future plans we considered using Redis for seamless access validation inside Lambda#Edge.
I've deployed the AWS Cloud formation Serverless image handler to resize images in an S3 bucket which is working well. It is using Cloudfront, API Gateway, Lamdba, and a custom domain.
My original images (PNGs) are of a reasonable size (~5MB or ~5000px across) and my derivatives are of various sizes. Up until about fit-in/2048x2048 everything works well. Above that though I get a 502 "Internal server error".
I've tried maxing out memory and timeouts to no avail. Does anyone have any idea what's going on there? Is there some invisible limit?
{
"message": "Internal server error"
}
This issue is caused by the payload limit of 6mb for Lambda Functions. As a workaround it is possible to save the images directly to a output bucket and deliver them from this bucket to the customer.
The issue is in the future backlog since 07/2018 (see here or here)
The architecture may need a correction here. Whenever you deliver large files, you need to leave the responsibility to S3 / CloudFront. Lambda should take care of authenticating / authorization of the content.
API Gateway has a limitation on the lambda response limit.
https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html
Using AWS cloudfront with S3 to host an angular-based web client.
Is there any rewrite rule or settings allowing one of the following examples? It is so unclear from what AWS are trying to exaplain.
Using friendly route, for example:
domain.com?lang=en&fun=no => domain.com/en/no
Configuration folders to have a default file, for example:
domain.com\en => domain.com (but now the client knows it has a parameter lang=en)
Obviously both of the example can be done with an html file which routes to the desired url BUT it doesn't work well with some sort of analytics models such as google's.
I would suggest using 'AWS Lamda at the Edge' functionality to provide the custom rewriting you want:
Using CloudFront with Lambda#Edge
Lambda#Edge is an extension of AWS Lambda, a compute service that lets you execute functions that customize the content that CloudFront delivers. Lambda#Edge scales automatically, from a few requests per day
to thousands per second. Processing requests at AWS locations closer
to the viewer instead of on origin servers significantly reduces
latency and improves the user experience.
When you associate a CloudFront distribution with a Lambda#Edge
function, CloudFront intercepts requests and responses at CloudFront
edge locations. You can execute Lambda functions when the following
CloudFront events occur:
When CloudFront receives a request from a viewer (viewer request)
Before CloudFront forwards a request to the origin (origin request)
When CloudFront receives a response from the origin (origin response)
Before CloudFront returns the response to the viewer (viewer response)
and here is an aCloudGuru blog post with lots of good examples, including one specifically about url rewriting:
https://read.acloud.guru/supercharging-a-static-site-with-lambda-edge-da5a1314238b
In a multipage web app (say 12 pages), you will want to use an automated and worry-less strategy via AWS Lamda#Edge. It solves this completely.
First, create an AWS Lambda function and then attach your CloudFront as a trigger.
In the code section of this AWS Lamda page, add the snippet in the repository below.
https://github.com/CloudUnder/lambda-edge-nice-urls/blob/master/lambdaRewrite.js
Content delivery will still be as fast as you can blink your eyes.
PS: Note the options in the readme section of the repo above
Sorry for doing this kind of question.. but I´m a bit lost here....
I have an app which consist in an Angular4 as frontend and Java app as Backend.
But I´m planning to use AWS Lambda as I´m interested after seeing the videos in Amazon.
The issue is that I don´t know how to get the best from AWS.
My Java app has a very time consuming task to process some images (which takes several seconds).
But I'm not sure if I can deploy all my app in Lambda, or if the idea is to use a EC2 server and then the specific task for the image processing in the lambda. Can anyone please shed some light here?
Also, the frontend app can be deploy in a lambda, or again, lambda is just for specific task?
EDIT:
The application flow would be:
The user in the angular app upload an image, the image goes to the backend server in Java and it´s stored in (maybe) a AWS bucket.. Then the Java app with imagemagick process the image and the result is store in (maybe) another bucket.
So the question is when I need to use Lambda? just to convert the image or if the full backend (and maybe frontend) app would be there?<
I'm asking because I cannot find enough information about that...
First of all you can deploy your Angular frontend to Amazon S3. Also you can use AWS CloudFront to add custom domains and free SSL certificates from Amazon using Amazon Certificate Manager for your domain. For more details refer the article Deploying Angular/React Apps in AWS.
If you don't need to show tge image processing results immediately in frontend
For the image processing backend you can use AWS API Gateway and Lambda along with S3. For this recommended flow is you can use the API Backend to get an Signed URL or AWS STS in Lambda (Or Use Cognito Federated Identities) to get temporary access to Amazon S3 Bucket to Upload the image directly to S3 from Angular App. For more details on this refer the article Upload files Securely to AWS S3 Directly from Browser.
Note: AWS recently released a JavaScript Library called AWS Amplify to simplify the implementation of the above tasks.
After Uploading the image to S3 you can setup an event driven workflow by using Amazon S3 triggers to invoke an Lambda function to perform the image processing and save the process image back to S3 (If you need to store the result).
If you need to show the result immediately
Still use tge previous approach upto Upload to S3 from frontend and then invoke an API Gateway Lambda function passing the file path in S3 to process the image.
To understand the details in connecting both frontend and backend with AWS serverless technologies refer the article Full Stack Serverless Web Apps with AWS.
As a side note, you should be able to implement the required functionality with AWS Lambda without using AWS EC2.
I had plans to use the AWS API Gateway for three purposes. All of these endpoints are configured with custom domain names, with AWS issued SSL certificates and I have CNAME records configure to match the could front urls.
api.my-domain.com (REST api calls that return json data) (Working as expected)
images.my-domain.com (Proxy pass through of binary image data from S3) - Working as expected.
videos.my-domain.com (DOH!... )
Unfortunately dealing with videos I've run into a few issues. Smaller videos start to work but then generate an error. But.. that's not the main issue.
There is a 10MB max payload size on the response data from an API integration endpoint, so I must come up with another solution for the videos.
I don't want to host the images, or videos via cloundfront. And I want to use the same AWS issues wildcard certificate *.my-domain.com on all the endpoints. I wanted to use the API gateway for the image request because the images are small, and won't exceed the limit, and I can cache them at the api level.
a CNAME pointed to my video S3 bucket works, but can't use the same SSL certificate, and I wanted all traffic to originate vie the API gateway and not have request going directly to the bucket endpoint.
```
So.. what are my options?
It seems like my best option will be to transcode the MP4 videos to HLS, and host the S3 bucket via cloudfront. I hadn't really wanted to incur the charges of using cloudfont, but I don't see any better option, for the design I want.
The most recent videos will be viewed occasionally, not high demand, older videos will be viewed rarely so hosting them in cloudfront seems like a waste.
Typical setup for Video streaming in AWS is to Stream the Video stored in S3 through AWS CloudFront RTMP Distribution.
Going forward with CloudFront hosted content from my S3 bucket to see how it works.
The reason I picked CloudFront overall was the tight integration with other AWS services, and the ability to have complete control over the path/name of the assets in the S3 bucket, whereas with Vimeo you don't have very good control over the name of the assets.