I'm currently utilizing an S3 bucket's rest api to access images for my company's application. I've written lambda functions to help lighten the load on my application, but in order to utilize these functions I have to change the settings in my S3 bucket to act as a static website host. When I do this I can longer access the original rest api. I would like to utilize both the REST api and the static website url to make the transition from one to the other smooth, but I can't find any details about doing this. Is it even possible?
Rest API request example https://s3.amazonaws.com/bucketname.mycompany.com/spacebackground.jpg
Static Website Example
http://bucketname.mycompany.com.s3-website-us-west-1.amazonaws.com/spacebackground(1)Full.jpg
Once I change a bucket to static website host I get a "permanent redirect" http response.
Is it possible to utilize both?
You are missing the cloud service named CloudFront.
https://aws.amazon.com/cloudfront/
You can map both lambda and your static website to a single domain. For Example,
https://www.example.com/ --- will be a static site
https://www.example.com/api -- belongs to API.
Create a CloudFront Distribution, configure your origins and Map to your url pattern.
You can also specify which url need to be self signed or which one to keep public as well.
It will be a breeze.
Hope it helps.
Related
I have an input provided by a user, that would be used as the endpoint url for bucket operations for an S3 bucket.
Is there a way to differentiate if the url is a REST API endpoint or a website endpoint?
I did read: https://docs.aws.amazon.com/AmazonS3/latest/userguide/WebsiteEndpoints.html
which mentions "Supports only GET and HEAD requests on objects" for a website endpoint.
However, i have come across cases where the other operations worked even with a website endpoint.
I am using python boto3 for these APIs.
With only S3 you can't upload any data. It's a static website hosting, without any logic or ability to process data. You're just storing files that browser renders.
If you need logic I'd suggest adding some Lambda functions with REST endpoint. For your needs, you'll probably stay in free tier.
I need to secure my s3 bucket objects. In my web application I'm using aws-sdk to upload media to s3 bucket and get an http link back to access that object. This http link is public by default and I want to make it secure so that only authorized users can access the media. aws s3 allows to make the object private but it wont let anyone with the link access the object.
This link will be accessed from a mobile app where I dont want to use aws-sdk, Instead I want to execute some logic on aws side whenever someone tries to access the http link for the object.
What I would like to happen is, before the user gets access to s3 object, Some authorizer code would execute (like a jwt token authorizer) and depending on it user would be granted/denied access.
I'm currently looking into Amazon API Gateways, I believe they can be accessed as an http link and AWS Lambda could be used to secure them(where i would execute my jwt authorizer). Then these apis would have access to s3 internally.
If someone could point me in the right direction, If this is at all possible.
If I could use the same jwt token issued from my web-application to send along the request to Amazon API Gateway, that would be great.
I would make the bucket private, and place a CloudFront distribution in front of it. Using an Origin access identity to allow only CloudFront to directly access the S3 bucket.
Then to provide security I would use either CloudFront signed cookies, or Lambda#Edge with a custom JWT token validation.
The easiest solution to expose private objects in an S3 bucket is to create a pre-signed URL. Pre-signed URLs use the permissions from the service (which pre-signs the URL) to determine access and have only a limited duration in which they can be used. They can also be used to upload an object directly to S3 instead of having to proxy the upload through a lambda function.
For a download functionality and a smooth user experience, you can - for example - have a lambda function that generates a pre-signed URL and returns it as an HTTP 302 response, which should instruct the browser to automatically download the file from the new URL.
(Edit)
Following on what I've stated in the comments on this answer, if you're proxying the upload/download of the objects through services such as API Gateway or Lambda, you will be severely limited in the size of files that you are able to upload to S3. The payload size limit on an API Gateway is 10 MB and for requests to lambda your payload is capped at 6MB for synchronous invocations. If you want to upload something larger than 10 MB, you will need to use direct upload to S3 for which pre-signed URLs are the safest solution.
I know I am bit late here, but I wanted to give my opinion in case someone has the same problems.
Your mobile app should communicate with a server app (backend app) for authentication and authorization. let's say you are deploying your server app on AWS VPC. Now, it's simple to manage the files access by creating a policy which allow just your server app (IP, or VPC) to access the bucket. the authorization part will be managed on your application.
I wonder, for example using AWS, if it was possible to detect the session of the user and send him to CDN if no session, or to the actual servers if logged in. Any ideas? I'm not too practical of CloudFront or Load balancers. Feel free to give examples using other cloud providers or Saas.
The CloudFront CDN has an integration with AWS Lambda, which allows you to intercept, inspect and modify requests at various points in their lifecycle. This is called Lambda#Edge.
You can use an Lambda#Edge function to dynamically select an origin as the source for the data based on information like the headers. AWS has published a blog post that describes this process in more details.
We have some videos in an S3 bucket. they've been transformed using AWS Elastic Transcoder to .m3u8 / .ts
We want the users to be able to stream these videos on both a web app and a mobile app.
Now, we want to secure this streaming, so our videos won't get pirated.
So, our proposed solution is as follows:
Prevent public access to the S3 bucket
create a cloudfront distribution with the bucket as the origin
Only enable access to this CDN using pre-signed URLs/cookies
For web app: use a pre-signed cookie (set by an endpoint at our backend that requires authentication), so that it works well with HLS (since the app needs to fetch a new segment every few seconds)
But now we don't know what to do with our mobile app. We can't use pre-signed cookies since there's no browser, and we can't use pre-signed URLs, since we'll need a signed URL for each segment we need to fetch. Any suggestions and solutions are welcome.
For our similar use-case:
We used CloudFront url and not S3 signed url. Because S3 signed URL is valid at object level and not folder level.
For paid videos, security and access was managed by Lambda#Edge on viewer requests.
Although we used OAuth and database inside that lambda, but surprisingly, we didn't face any bottlenecks on Lambda#Edge. For future plans we considered using Redis for seamless access validation inside Lambda#Edge.
Title says it all: is it possible to host a static-generated site on S3 and restrict access to it based on Github OAuth?
Scenario: You generate a static site (using Hugo or whatever) and you upload it to S3 as a static site. Is there some combination of AWS services (API Gateway, Lambda, etc.) that can make it so that only members of my Github organization can access this site? Authenticating with Github's OAuth? Anyone not authenticated should be treated with a 404.
Has anyone done this or know how to do it? Is such a thing possible without running a backing server (EC2, Heroku, DigitalOcean Droplet, etc.) of some kind?
Looking for a complete answer that outlines steps on how to set this up (if possible) with the lowest cost/complexity.
Yes this is possible. I've followed this tutorial in the past to secure a static site on S3 behind an Auth0 authentication, using DynamoDB as the user account store, and API Gateway + AWS Lambda as the authentication service. That was an AngularJS SPA, but the general concept should be the same.
Github is listed as an Auth0 identity provider. I would recommend looking at this documentation for using Auth0 on your site. Once you get it working you would have to configure your static site generator to include the authentication JavaScript code on every page it generates.