Is that possible to shorten a file path and get the file with that shortened path?
Ex: In my rails public folder I have a file like
/users/000/001/991/image.png
I want to shorten it like /<dynamic user_name>/image.png. But when I use that shortened url I want to refer to the original file path. I googled but couldn't find a way. Appreciate if I get any reference.
Logic depends as per your need and current DB structure. some possible solution could be
-If your path is fix, just store image name. and while creating full path join +
-If path is no fixed and storing path in DB, you can create one column more column and store unique id(any integer or alpanumeric as per logic but unique value) and use that unique value to replace path
Related
I'm trying to understand why there needs to be two arguments here:
key which is basically the filename
path argument, the path of the file
Isn't it redundant to still pass the key?
Can't we just pass the bucket and the filename?
key in S3 could be a long string, like /my-prefix/YYYY/MM/DD/UUID.txt. It can, and usually will, contain things like slash characters. So it makes sense to have to specify the local filename argument separately, because you may not want to save the file in the same path that it is saved in S3, and you also may not want to save it using the same name that was used in S3.
You can download the file to the Filename which has different name than the Key from where you are downloading, e.g. Key = 'hello.txt', Filename = '/tmp/MylocalCopyOfTheHello.txt'
I'm using google-cloud-platform data fusion.
Assuming that the bucket's path is as follows:
test_buk/...
In the test_buk bucket there are four files:
20190901, 20190902
20191001, 20191002
Let's say there is a directory inside test_buk called dir.
I have a prefix-based bundle based on 201909(e.g, 20190901, 20190902)
also, I have a prefix-based bundle based on 201910(e.g, 20191001, 20191002)
I'd like to complete the data-pipeline for 201909 and 201910 bundles.
Here's what I've tried:
with regex path filter
gs://test_buk/dir//2019 to run the data pipeline.
If regex path filter is inserted, the Input value is not read, and likewise there is no Output value.
When I want to create a data pipeline with a specific directory in a bundle, how do I handle it in a datafusion?
If using directly the raw path (gs://test_buk/dir/), you might be getting an error when escaping special characters in the regex. That might be the reason for which you do not get any input file into the pipeline that matches your filter.
I suggest instead that you use ".*" to math the initial part (given that you are also specifying the path, no additional files in other folders will match the filter).
Therefore, I would use the following expressions depending on the group of files you want to use (feel free to change the extension of the files):
path = gs://test_buk/dir/
regex path filter = .*201909.*\.csv or .*201910.*\.csv
If you would like to know more about the regex used, you can take a look at (1)
I'm trying to process a file in a Kettle transformation. The targeted file has a static name, let's say TARGETED.LOG and it's in a subdirectory which contains a date component (variable) in his name. So, the whole path name will be something like:
c:\username\kettleworkspace\report_[DDMMYYYY]\TARGETED.LOG.
Any advice?
Use the Get File Names step with the include subfolders option, and drop the resulting list of files in your Text File Input with the Accept filenames from previous step option.
Of course between these two step you would probably want to add some Filter step.
Whenever I create an ImageField using a file called moon.png, Django, correctly following my configuration settings, puts the file in:
campaign/primary-banner/2015/11/25/moon.png
or
campaign/primary-banner/2015/11/25/moon_RcJ3FuD.png
And that is the value of imagefield.name, which I can show to the user, but is not really user friendly.
I would like to show the name of ImageField.name, but in a human readable format. Is it possible to extract the original file name (moon.png) from the ImageField? The workarounds that I can think of are:
add an extra field to my model to hold the human readable file name. Extra work, which I would like to avoid: DRY.
process the imagefield.name value to extract the original filename, but this seems too complex (I would need to exactly understand how django is generating the filename in the first place, to make sure I cover corner cases)
I had similar problem and solved it by creating my own subdirectry under MEDIA_ROOT, in which I made directory structure involvind the date and also some unique identifier. Then I moved my file to that final subdirectory and put the name to FileField.name. Everything works like charm - files are unique (thanks to unique directories) and the final name is exactly what i want user to see (as there is no need to rename it - no conflict is possible)
p.filename=get_filename_of_existing_file_to_be_stored()
p.originalfilename=get_how_it_should_be_named()
masterobject=MyNewMaster.objects.get(pk=some_id)
uniq_name=masterobject.make_something_unique()
new_object=MyObject()
new_object.master=masterobject
new_object.some_fields=some_values
daystr=date.today().strftime('%y-%m-%d')
directory='MyDir/%s/%s/%s/'%(daystr,masterobject.id,uniq_name)
if not os.path.exists(os.path.join(settings.MEDIA_ROOT,directory)):
os.makedirs(os.path.join(settings.MEDIA_ROOT,directory))
fname=os.path.join(directory,p.originalfilename)
os.rename(p.filename,os.path.join(settings.MEDIA_ROOT,fname))
new_object.filename = fname
new_object.save()
Now everthing works for me and no conflicts are possible and even Django seems happy with this hack and provides the correct filename and url for everything i tried to do with MyObject :)
Now I ended with path like:
MyDir/2015-11-29/12345/child_7/moon.png
I need to download an xml file from AWS-S3.
I tried using get_contents_to_filename(fname) , it worked.
But i need to download the file without specifying fname, because if i specify the fname my downloaded file gets saved tofname.
I want to save the file as it is, with its name.
this is my current code
k = Key(bucket)
k.set_contents_from_filename(fname)
can someone please help me to download and fetch the file without using key.
Thanks in advance!
I'm not sure which library you're using, but if k is the AWS key you want to download, then k.name is probably the key name, so k.get_contents_to_filename(k.key) would probably do more or less what you want.
The one problem is that the key name might not be a legal file name, or it may have file path separators. So if the key name were something like '../../../../somepath/somename' the file would be saved somewhere you don't expect. So copy k.name to a string and either sanitize it by changing all dangerous characters to safe ones, or just extract the part of the key name you want to use for the file name.