I need to store different dimensions of an image on S3. I am new to AWS Services, so it is taking some time for me to figure out how I can accomplish this.
Say I have an image called abc.png. I want to get different versions of this image using query string parameters, i.e apc.png?s=medium for medium 400x400, and abc.png?s=large for 1200*1200. I do not want to do a preprocessing, or on the fly resizing.
Is there a way to do this on the S3 level only?
The S3 API doesn't allow selecting a different file through the query string. Put the size in the image path instead: /medium/apc.png or /apc.png/medium. Don't forget to set the content-type option to the appropriate MIME type for each S3 object, which should allow the browser to render the images correctly.
Try this service, they work with s3 as well I believe
http://cloudinary.com/
Related
I’m trying to build application with backend in java that allows users to create a text with images in it (something like a a personal blog). I’m planning to store these images to s3 bucket. When uploading image files to bucket i’m hashing the original name and store the hashed one in the bucket. Images are for display purpose only, no user will be able to download them. Frontend displays these images by getting a path to them from the server. So the question is, is there any need to store original name of the image file in the database? And what are the reasons, if any, of doing so?
I guess in general it is not needed because what is more important is how these resources are used or managed in the system.
Assuming your service is something like data access (similar to google drive), I don't think it's necessary to store it in DB, unless you want to make faster search queries.
I have some functionality that uploads Documents to an S3 Bucket.
The key names are programmatically generated via some proprietary logic for the layout/naming convention needed.
The results of my S3 upload command is the actual url itself. So, it's in the format of
REGION/BUCKET/KEY
I was planning on storing that full url into my DB so that users can access their uploads.
Given that REGION and BUCKET probably wouldn't change, does it make sense to just store the KEY - and then dynamically generate the full url when the client needs it?
Just want to know what the desired pattern here is and what others do. Thanks!
Storing the full URL is a bad idea. As you said in the question, the region and bucket are already known, so storing the full URL is a waste of disk space. Also, if in the future say, you want to migrate your assets to a different bucket may be in a different region, having full URLs stored in the DB just make things harder.
For my app I need users to be able to upload their profile pictures.
The way it works is they send their info (name, email...) and their pictures to a lambda function. The Lambda function stores the pictures in S3 and stores the info and the link to the picture in S3 in DynamoDB.
Users should be able to upload a new picture and use it as their profile picture. Would it be possible to upload a picture that would use the same link in S3 (meaning I would replace the old picture by the new one while keeping the link the same)?
This way I don't have to update any table in dynamoDB. The thing is that I need to use the link in other tables and this would avoid having to update every tables it is in.
To replace the file upload it again with the same key. e.g.
aws s3 cp ./hello1.text s3://document/hello.text
May receive old data until replication is completed. refer - https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html#BasicsKeys
I want to do the following: a user in a browser types some text and after he presses a 'Save' button, the text should be saved in a file (for example: content.txt) in a folder (for example: /username_text) on the root of an S3 bucket.
Also, I want the user to be able, when he visits the same page, load the content from S3 and continue working on the file. Then, if he/she is done, save the file to S3 again.
Probably important to mention, but I plan on using NodeJS for my back-end...
My question now is: What is the best way to set this storing-and-retrieving thing up? Do I create an API gateway + Lambda function to GET and POST files through that? Or do I for example use the aws-sdk in Node to directly push and pull files from S3? Or is there a better way to do this?
I looked at the following two guides:
Using AWS S3 Buckets in a NodeJS App – Codebase – Medium
Image Upload and Retrieval from S3 Using AWS API Gateway and Lambda
Welcome to StackOverflow!
I think you are worrying too much about the not-so-important stuff. S3 is nothing but a storage system. You could have decided to store the content of these files on DynamoDB, RDS, etc. What would you do if you stored its contents on these real databases? You'd fetch for data and display it to the user, wouldn't you?
This is what you need to do with S3! S3 is a smart choice on your scenario because your "file" can grow very big and S3 is a great place for storing files. However, apparently, you're not actually storing files (think of .pdf, .mp4, .mov, etc.), you're essentially only storing human-readable text.
So here's one approach on how to solve your problem:
FETCHING FILE CONTENT
User logs in
You fetch the user's personal information based on some token. You can store all the metadata in DynamoDB, where given a user_id, fetch all the "files" from this user. These "files" (metadata only) would be the bucket and key for the actual file on S3.
You use the getObject API from S3 to fetch the file based on your query and display the body of your file to your user in a RESTful way. Your response should look something like this:
{
"content": "some content"
}
SAVING FILE CONTENT
User logs in
The user writes anything in a form and submits it. In your Lambda function, you grab the content of this form and process it. This request should look something like this:
{
"file_id": "some-id",
"user_id": "some-id",
"content": "some-content"
}
If the file_id exists, update the content in S3. Otherwise, upload a new file in S3 and then create a new entry in DynamoDB. You'd then, of course, have to handle if the user submitting the changes actually owns the file, but if you're using UUIDs it shouldn't be too much of a problem, but still worth checking in case an ID is leaked somehow.
This way, you don't need to worry about uploading/downloading files as these are CPU intensive tasks, so you can keep your costs low as well as using very little RAM in your functions (128MB should be more than enough), after all, you're now only serving text. Not only this will simplify your way of designing it, but will also make things simpler both in API Gateway and in your code as you won't have to deal with binary types. The maximum you'll do is convert the buffer from S3 to a String when serving some content, but this should be completely fine.
EDIT
On your question regarding whether you should upload it from the browser or not, I suggest you take a look into this answer where I cover the pros/cons of doing it via API Gateway vs from the Browser.
I'm trying to make an ExternalQuestion HIT and was wondering how I could pass S3 image URLs to the hit and display them.
I considered passing the URL as a url parameter but that doesn't make sense since it's a url.
Is it possible to do something like this?
<ExternalQuestion xmlns="[the ExternalQuestion schema URL]">
<ExternalURL>http://tictactoe.amazon.com/gamesurvey.cgi?gameid=01523</ExternalURL>
<FrameHeight>400</FrameHeight>
<ImageURL1>[image_url]</ImageURL1>
<ImageURL2>[image_url]</ImageURL2>
....
</ExternalQuestion>
It looks like you're trying to do something like what the Requester User Interface does when creating a batch, which involves building an HTMLQuestion out of a template HTML page with some named variables and a CSV file containing the values for each variable.
The ExternalQuestion only contains the ExternalURL and FrameHeight parameters. If you want to display different images, those need to be on their own pages (i.e., your server needs to decide what image to display to a given worker OR you have to create multiple HITs, one for each page that contains a distinct URL).