I have a feature on my website where users can upload images. Users can see their own images but not others. The images are stored on Amazon S3 but uploaded and viewed on my website which is at a web hosting and not S3.
I have tried to show the pictures on my website through my private key when pictures are private at Amazon but failed.
Found this post: http://blog.learningtree.com/configuring-amazon-s3-to-serve-images which describes how to make the images/files more private even if they are set to public on S3. The site suggest to stop search engines with robots.txt file and only serves images to people who are coming from my domain to stop hot-linking.
Do you think this is enough if I make them public on S3 or should I think about something else?
You can also configure the images on S3 to be private, and then generate pre-signed URLs in your app. That way, you can include an expiry time within the link.
From Authenticating REST Requests in the S3 docs:
For example, if you want to enable a user to download your private data directly from S3, you can insert a pre-signed URL into a web page before giving it to your user.
People can then only use the generated URL for a certain time. If they come through your app, it will always generate a link for some time in the future (say, 15 minutes as an example). If people pass around the links to these images, these links auto-expire.
Most S3 SDKs have higher-level methods to pre-sign those URLs.
Relevant: How secure are presigned URLs in AWS S3? here on SO.
Related
I'm creating a platform whereby users upload and download data. The amount of data uploaded isn't trivial---this could be on the order of GB.
Users should be able to download a subset of this data via hyperlinks.
If I'm not mistaken, my AWS account will be charged for the egress of downloaded these files. If that's true, I'm concerned about two related scenarios:
Users who abuse this, and constantly click on the download hyperlinks (more than reasonable)
More concerning, robots which would click the download links every few seconds.
I had planned to make the downloads accessible to anyone who visits the website as a public resource. Naturally, if users logged in to the platform, I could easily restrict the amount of data downloaded over a period of time.
For public websites, how could I stop users from downloading too much? Could I use IP addresses maybe?
Any insight appreciated.
IP address can be easily changed. Thus, its a poor control, but probably better than nothing.
For robots, use capcha. This is an effective way of preventing automated scraping of your links.
In addition, you could considered providing access to your links through API gateway. The gateway has throttling limits which you can set (e.g. 10 invocations per minute). This way you can ensure that you will not go over some pre-defined.
On top of this you could use S3 pre-signed URLs. They have expiration time so you could adjust this time to be valid for short time. This also prevents users from sharing links as they would expire after a set time. In this scenario, he users would obtained the S3 pre-signed urls through a lambda function, which would be invoked from API gateway.
You basically need to decide whether your files are accessible to everyone in the world (like a normal website), or whether they should only be accessible to logged-in users.
As an example, let's say that you were running a photo-sharing website. Users want their photos to be private, but they want to be able to access their own photos and share selected photos with other specific users. In this case, all content should be kept as private by default. The flow would then be:
Users login to the application
When a user wants a link to one of their files, or if the application wants to use an <img> tag within an HTML page (eg to show photo thumbnails), the application can generate an Amazon S3 pre-signed URLs, which is a time-limited URL that grants temporary access to a private object
The user can follow that link, or the browser can use the link within the HTML page
When Amazon S3 receives the pre-signed URL, it verifies that it is correctly created and the expiry time has not been exceeded. If so, it provides access to the file.
When a user shares a photo with another user, your application can track this in a database. If a user requests to see a photo for which they have been granted access, the application can generate a pre-signed URL.
It basically means that your application is in control of which users can access which objects stored in Amazon S3.
Alternatively, if you choose to make all content in Amazon S3 publicly accessible, there is no capability to limit the downloads of the files.
I have an application that stores images in AWS S3. It is like a profile picture upload case. After uploading the profile picture, the image will be stored in AWS S3 and the S3 link will be stored in a database. The application will then show the profile picture using that link in the database.
Right now, as the bucket is private the profile picture is not visible in my application. How can I use this link to show the image without making the bucket public?
I don't think, I can use AWS's signed URL because this link can't be time-limited. The link need to be available all the time for showing the image on the application.
Is there any method to do so? Or is there any other industry-standard method for making this feature possible?
Regarding images, best way is the serve them via CDN (you can link it with S3). Their long, scaffolded URL should be enough (and make a dedicated S3 bucket, public). Check a photo from a friend's Facebook account, it will should even if not logged in. How to setup CDN https://learnetto.com/blog/cloudfront-s3
If you are really concerned about security, you can fetch the images in Base64 (https://stackoverflow.com/a/2429959/290036). Make your bucket private and allow access only to your internal services. That way you have better control, but lose all the benefits of a CDN.
Well I have a service which generate PDF files every day and I need to share a download link to this pdf files through email to send to our clients.
I am thinking to save these files to a s3 bucket, but we need to make sure these download links will works all the time, I went through Amazon S3 Pre-Signed URLs but asfaik presigned urls only valid for a maximum of 7 days but the links should be valid all the time, is there any other way to give access to s3 objects for longer term?
It's not possible to generate Amazon S3 pre-signed URLs with a validity of more than one week.
For longer term, you need to use CloudFront Signed URLs for which you can set an expiry date far in the future.
See Serving Private Content with Signed URLs and Signed Cookies and Using Signed URLs in the CloudFront documentation and Using CloudFront Signed URLs to Serve Private S3 Content on Medium.
You can create a custom download link that hits your backend service (use a jwt, or something else with expiry of your choice).
After validating the JWT on your server, generate a new S3 Pre-signed link(short expiry) and proceed with download.
Can I allow a 3rd party file upload to an S3 bucket without using IAM? I would like to avoid the hassle of sending them credentials for an AWS account, but still take advantage of the S3 UI. I have only found solutions for one or the other.
The pre-signed url option sounded great but appears to only work with their SDKs and I'm not about to tell my client to install python on their computer to upload a file.
The browser based upload requires me to make my own front end html form and run in on a server just to upload (lol).
Can I not simply create a pre-signed url which navigates the user to the S3 console and allows them to upload before expiration time? Of course, making the bucket public is not an option either. Why is this so complicated!
Management Console
The Amazon S3 management console will only display S3 buckets that are associated with the AWS account of the user. Also, it is not possible to limit the buckets displayed (it will display all buckets in the account, even if the user cannot access them).
Thus, you certainly don't want to give them access to your AWS management console.
Pre-Signed URL
Your user does not require the AWS SDK to use a pre-signed URL. Rather, you must run your own system that generates the pre-signed URL and makes it available to the user (eg through a web page or API call).
Web page
You can host a static upload page on Amazon S3, but it will not be able to authenticate the user. Since you only wish to provide access to specific people, you'll need some code running on the back-end to authenticate them.
Generate...
You ask: "Can I not simply create a pre-signed url which navigates the user to the S3 console and allows them to upload before expiration time?"
Yes and no. Yes, you can generate a pre-signed URL. However, it cannot be used with the S3 console (see above).
Why is this so complicated?
Because security is important.
So, what to do?
A few options:
Make a bucket publicly writable, but not publicly readable. Tell your customer how to upload. The downside is that anyone could upload to the bucket (if they know about it), so it is only security by obscurity. But, it might be a simple solution for you.
Generate a very long-lived pre-signed URL. You can create a URL that works for months or years. Provide this to them, and they can upload (eg via a static HTML page that you give them).
Generate some IAM User credentials for them, then have them use a utility like the AWS Command-Line Interface (CLI) or Cloudberry. Give them just enough credentials for upload access. This assumes you only have a few customers that need access.
Bottom line: Security is important. Yet, you wish to "avoid the hassle of sending them credentials", nor do you wish to run a system to perform the authentication checks. You can't have security without doing some work, and the cost of poor security will be much more than the cost of implementing good security.
you could deploy a lambda function to call "signed URL" then use that URL to upload the file. here is an example
https://aws.amazon.com/blogs/compute/uploading-to-amazon-s3-directly-from-a-web-or-mobile-application/
I am using Amazon S3 to host images for a public REST API and serve them. Currently, my bucket allows anyone to enter in the URL to the image, without any signature included in the params, and the image will be accessible. However, I'd like to require an expiring signature in each image request, so users will have to go through the API to fetch images. How do I do this? Is this a bucket policy?
You simply set all of the files to private. If you want to be able to give out pre-signed URLs, you'll need to generate them as you hand them out.
AWS’ official SDKs make this trivial.