store s3 images in testing machine - amazon-web-services

I am working with zf2, and and storing my user profile pictures in my public folder, like this
zf2-project/public/uploads/users/photo.png
so i will be configuring the above path like in config file:
ASSET_BASE_URL = "http://my-project/uploads/users"
and while retrieving, <img src = ASSET_BASE_URL. $photo; >
In my production machine, i am storing the images outside the project folder
my project is in opt/mybucket/uploads/users/photo.png
my boss said that, when we put the images in the above place, it will mount directly to amazon bucket.
so i will be accessing the file from the following url..
http://s3.amazonbucket.com/uploads/users/photo.png
ASSET_BASE_URL = "http://s3.amazonbucket.com/uploads/users/photo.png"
i.e <img src = ASSET_BASE_URL.$photo; >
for the above scenarios, its ok to do like that, local machine and production machine.
But how can i do that in testing machine, because i am not using amazon service here. My boss said that, find the way to do this.
Because the scenario here is,
storing images in /opt/virtual-bucket/users/
But my project folder is in /var/www/my-project/
How can i show the images here in img src..
/
Anyone got idea, can you help me on this,

Related

AWS S3 filename

I’m trying to build application with backend in java that allows users to create a text with images in it (something like a a personal blog). I’m planning to store these images to s3 bucket. When uploading image files to bucket i’m hashing the original name and store the hashed one in the bucket. Images are for display purpose only, no user will be able to download them. Frontend displays these images by getting a path to them from the server. So the question is, is there any need to store original name of the image file in the database? And what are the reasons, if any, of doing so?
I guess in general it is not needed because what is more important is how these resources are used or managed in the system.
Assuming your service is something like data access (similar to google drive), I don't think it's necessary to store it in DB, unless you want to make faster search queries.

Springboot server in Elastic Beanstalk creates files that I can't see

I have a Springboot server that is deployed to an Elastic Beanstalk environment in AWS. The basic functionality is this:
1. Upload a file to the server
2. The server processes file by doing some data manipulation.
3. Then the file that is created is sent to a user via email.
The strange thing is that, the functionality mentioned above is working. The output file is sent to my email inbox successfully. However, the file cannot be seen when SSHed into the instance. The entire directory that gets created for the data manipulation is just not there. I have looked everywhere.
To test this, I even created a simple function in my Springboot Controller like this:
#GetMapping("/")
public ResponseEntity<String> dummyMethod() {
// TODO : remove line below after testing
new File(directoryToCreate).mkdirs();
return new ResponseEntity<>("Successful health check. Status: 200 - OK", HttpStatus.OK);
}
If I use Postman to hit this endpoint, the directory CANNOT be seen via the terminal that I am SSHed into. The program is working so I know that the code is correct in that sense, but its like the files and directories are invisible to me.
Furthermore, if I were to run the server locally (using Windows OR Linux) and hit this endpoint, the directory is successfully created.
Update:
I found where the app lives in the environment at /var/app. But my folders and files are still not there, only the source code files, ect are there. The files that my server is supposed to be creating are still missing. I can even print out the absolute path to the file after creating it, but that file still doesn't exist. Here is an example:
Files.copy(source, dest);
logger.info("Successfully copied file to: {}", dest.getAbsolutePath());
will print...
Successfully copied file to: /tmp/TESTING/Test-Results/GVA_output_2021-12-13 12.32.58/results_map_GVA.csv
That path DOES NOT exist in my server, but I CAN send it to me via email from the server code after being processed. But if I SSH into the instance and go to that path, nothing is there.
If I use the command: find . -name "GVA*" (to search for the file I am looking for) then it prints this:
./var/lib/docker/overlay2/fbf04e23e39d61896a1c935748a63f2d3836487d9b166bae490764c30b8870ae/diff/tmp/TESTING/Test-Results/GVA_output_2021-12-09 18.15.59
./var/lib/docker/overlay2/fbf04e23e39d61896a1c935748a63f2d3836487d9b166bae490764c30b8870ae/diff/tmp/TESTING/Test-Results/GVA_output_2021-12-13 12.26.34
./var/lib/docker/overlay2/fbf04e23e39d61896a1c935748a63f2d3836487d9b166bae490764c30b8870ae/diff/tmp/TESTING/Test-Results/GVA_output_2021-12-13 12.32.58
./var/lib/docker/overlay2/fbf04e23e39d61896a1c935748a63f2d3836487d9b166bae490764c30b8870ae/merged/tmp/TESTING/Test-Results/GVA_output_2021-12-09 18.15.59
./var/lib/docker/overlay2/fbf04e23e39d61896a1c935748a63f2d3836487d9b166bae490764c30b8870ae/merged/tmp/TESTING/Test-Results/GVA_output_2021-12-13 12.26.34
./var/lib/docker/overlay2/fbf04e23e39d61896a1c935748a63f2d3836487d9b166bae490764c30b8870ae/merged/tmp/TESTING/Test-Results/GVA_output_2021-12-13 12.32.58
But this looks like it is keeping track of differences between versions of files since I see diff and merged in the file path. I just want to find where that file is actually residing.
If you need to store an uploaded file somewhere from a Spring BOOT app, look at using an Amazon S3 bucket as opposed to writing the file to a folder on the server. For example, assume you are working with a Photo app and the photos can be uploaded via the Spring BOOT app. Instead of placing this in a directory on the server, use the Amazon S3 Java API to store the file in an Amazon S3 bucket.
Here is an example of using a Spring BOOT app and handling uploaded files by placing them in a bucket.
Creating a dynamic web application that analyzes photos using the AWS SDK for Java
This example app also shows you how to use the SES API to send data (a report in this example) to a user via email.

Sitecore - How to load images on CD server from CM server?

I have two separated servers: one is CD server and one is CM Server. I upload images on CM server and publish them. On the web database, although I saw the the images under Media Library item
But they aren't displayed on CD server (e.g on website), it indicates that the images not found. Please help me to know how I can solve that problem or I do need some configuration for that.
Many thanks.
Sitecore media items can carry actual media file either as:
Blob in the database - everything works automatically OOB
Files on the file system - one needs to configure either WebDeploy, or DFS
Database resources are costly, you might not want to waste them on something that can be achieved by free tools.
Since WebDeploy by default locates modified files by comparing file hashes between source, and target, it will become slower after a while.
You might have uploaded image in media library as a file. As such, image is stored as a File on file system. To verify this, your image item in media library will have a path value set in 'File Path' field of your image item. Such files have to be moved to file system of CD server as well.
If you uploaded your images in bulk, you can store them as blob in DB by default rather than as a File in file system using following setting-
<setting name="Media.UploadAsFiles" value="false">

Django Media File deployment - change in file location

I am having a web server running in Digitalocean. We are in a situation where we need to upload our clients' images (media files). Now, as far as the understating goes, in the following case:
class Car(models.Model):
...
photo = models.ImageField(storage=fs)
The photo field will store the url of the file, right?
Now we might move to amazon (with S3 as the storage system) in near future. Will the model field value change?
How do you, then, suggest us in taking care of the change (as we will have two servers then, one EC2 for django & S3 for media & static files)?
You can upload your static and media files to S3 during deployment after collectstatic. In case of multiple servers, you only need to run collectstatic in one server, then you will be able to reach same content from S3 for multiple servers.
If you take a look https://www.caktusgroup.com/blog/2014/11/10/Using-Amazon-S3-to-store-your-Django-sites-static-and-media-files/ there is a detailed guide on how to store files on S3.

How to set Amazon S3 Bucket default image

I have a bucket on S3 with some public images in it. If I browse to the folder without specifying a file name, it serves me an image.
So using a link like this, I am still getting an image back.
https://s3.amazonaws.com/bucket_name/folder_name/
The image served is one of mine that I've obviously uploaded at some point in the past but I don't recall ever setting it as a folder default. Is there an option somewhere to do this?
Thanks.
I suspect that /bucket_name/folder_name/ is the object path. In S3, there really is not a true concept of folders. / in an object path, can be displayed as folders, but they are still just part of the key in the end.