I am writing some unit tests for a piece of code that takes a path and attempts to load the file if it has a known extension, then does more careful checking.
In the unit test, I would like to create a temporary file that has the correct extension, but incorrect contents, in my case an empty file posing as test.tif.
How can I create a temporary file while specifying the extension (or the entire name), using the tempfile module?
I have looked at the NamedTemporaryFile class, as well as the suffix and prefix parameters, but I still cannot set the extension. I suppose I could manually create a file in a temporary directory, but then I loose the self-deleting capability that I am after.
This doesn't work for you?
In [2]: tempfile.NamedTemporaryFile(suffix='.tif').name
Out[2]: '/var/folders/gq/swc6jtld5853skyq_xc2lpc40000gn/T/tmplrtwvxg7.tif'
did you try?
fd, path =tempfile.mkstemp(suffix = '.tif')
print(path)
Related
In the Flask documentation for file uploads, they recommend use of secure_filename() to sanitize a file's name before storing it.
Here's their example:
uploaded_file = request.files['file']
if uploaded_file:
filename = secure_filename(uploaded_file.filename) # <<<< note the use of secure_filename() here
file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
return redirect(url_for('display_file',
filename=filename))
The documentation says:
Now the problem is that there is that principle called “never trust user
input”. This is also true for the filename of an uploaded file. All
submitted form data can be forged, and filenames can be dangerous. For
the moment just remember: always use that function to secure a
filename before storing it directly on the filesystem.
With offsite storage (S3 or Google Cloud), I will not be using Flask to store the file on the web server. Instead, I'll rename the upload file (with my own UUID), and then upload it elsewhere.
Example:
blob = bucket.blob('prompts/{filename}'.format(filename=uuid.uui4()))
blob.upload_from_string(uploaded_file.read(), content_type=uploaded_file.content_type)
Under this scenario, am I right that you do you not need to invoke secure_filename() first?
It would seem that because I (a) read the contents of the file into a string and then (b) use my own filename, that my use case is not vulnerable to directory traversal or rogue command-type attacks (e.g. "../../../../home/username/.bashrc") but I'm not 100% sure.
You are correct.
You only need to use the secure_filename function if you are using the value of request.files['file'].filename to build a filepath destined for your filesystem - for example as an argument to os.path.join.
As you're using a UUID for the filename, the user input value is disregarded anyway.
Even without S3, it would also be safe NOT to use secure_filename if you used a UUID as the filename segment of the filepath on your local filesystem. For example:
uploaded_file = request.files['file']
if uploaded_file:
file_uuid = uuid.uuid4()
file.save(os.path.join(app.config['UPLOAD_FOLDER'], file_uuid))
# Rest of code
In either scenario you'd then store the UUID somewhere in the database. Whether you store the originally provided request.files['file'].filename value alongside that is your choice.
This might make sense if you want the user to see the original name of the file when they uploaded it. In that case it's definitey wise to run the value through secure_filename anyway, so there's never a situation where the frontend displays a listing to a user which includes a file called ../../../../ohdear.txt
the secure_filename docstring also points out some other functionality:
Pass it a filename and it will return a secure version of it. This
filename can then safely be stored on a regular file system and passed
to :func:os.path.join. The filename returned is an ASCII only string
for maximum portability.
On windows systems the function also makes sure that the file is not
named after one of the special device files.
>>> secure_filename("My cool movie.mov")
'My_cool_movie.mov'
>>> secure_filename("../../../etc/passwd")
'etc_passwd'
>>> secure_filename(u'i contain cool \xfcml\xe4uts.txt')
'i_contain_cool_umlauts.txt'
The docs say this should work:
bool did=pixmap.save( "hoppy.png" );
qDebug("did is: %d",did);
My logging returns 1 suggesting, as per docs, that the save was successful. However, no file appears on my drive. According to the docs, this save() should indeed save a file, so what is missing?
According to the docs, this save() should indeed save a file, so what is missing?
Nothing missing; it is simply saved to the folder of where you run the qt executable from. If it is a direct invocation, then it is beside the executable, otherwise it is beside the script or other program that calls the qt executable (sure, the other program could modify the current working directory, but let us forget about that for a bit).
If that is not what you would like to do, you better use an absolute path to the location where you wish to save it. However, if it is some common location, consider using QStandardPaths.
Another answer notes:
Nothing missing; it is simply saved to the folder of where you run the qt executable from
However my question indicated that this is actually not the case here.
Using the full path rather than a relative path or instead of using the ~ character, resolved it.
I have a mapping like
SA-->SQ--->EXPR--->TGT
The source will be of the same structure and the tartget also.
There are a bunch of files(with the same structure) which will go through this mapping .
So i want to use a parameter file through which i will give the file names for every run manually.
How to use the param file in session for Source filename attribute
Please suggest..
you could use indirect source type, wherein your source file is basically a list of files, and in turn the session reads each of the files one by one.
the parameter file could reference a source file name (the list) as
$InputFile_myName=/a/b/c.list
In line with what Raghav says, indicate the name of a file that will hold a list of input files in the 'Source filename' property box for the SQ in question in the Mapping tab, making the file 'Source filetype' be 'Indirect', specified in the Session Properties. If you already know ahead of time the names of the input files, you can specify them in that file and deploy that file with the workflow to the location you indicate in the 'Source file directory' property box. However if you won't know the names of the input files until run-time but know the files' naming standard (e.g: "Input_files_name_ABC_" where "" represents variable text, such as a numeric value incremented per input file generated by some other process), then one way to deal with that is to use a Pre-Session Command specifiable in the 'Components' tab of the Session. Create one that will build a new file at the location and with the name specified for the Indirect input file referenced above by using the Unix shell (or if running on Windows, the cmd shell) to list the files conforming to the naming standard for them and redirect the listing output to that file.
Tricky thing is that there must be one or more files listed in that Indirect type of input file. If that file is empty, the workflow will fail (abend). An Indirect file type must have in it listed at least one file (even if that file is empty) and that file must exist. The workflow fails if the indirect file reader gets no files to read from or if a file listed in it is not present on the server to be read from. One way to get around this is to make sure an empty file is present at all times that conforms to the naming standard. This can be assured by creating a "touchfile" before executing the listing command to build the Indirect file type listing file. In Unix, you'd use the 'touch {path}/{filename}' command ({filename} could be, for example, "Input_files_name_ABC_TOUCHFILE"), or on Windows you'd redirect an empty string to a file likewise named via cmd shell process. Either way, that will help you avoid an abend. Cleaning up that file is easy to do: a Post-Session command can be used to delete the empty touchfile. Likewise, you can do the same for the Indirect type of file if desired.
at the moment I'm writing a kind of lib, which gets from outside the file name 'sFilename'. With it data were written to a file it will be created, data were append to an existing file with data, data were updated in an existing file with data or the data were read from an existing data.
The user of the application with my lib should get as much as possible on information about errors of file handling.
For this purpose I code a method FileExists(...) which uses _stat() to determine if a file exists or not and a method "bool checkPermission(std::string sFilename, CFile::EOpenmode iOpenmode)" which gives back a bool if the specified file (sFilename) with the iOpenmode (Read, Write, Readwrite) have the permission to be read, written or read and written.
This method works with _stat(sFilename.c_str(), &buf) too and gives the desired information of the file back in buf.
Before checking any file I want to check if the directory containing the specified file has the desired permissions and for that I want to use the checkPermission method [works with _stat()] for the directory!
Now the problem: how can I determine easyly the containing directory? If the user just give a filename "test.txt" the file will be created or read in working directory. So its easy to get the up-directory. Its the same like the working directory. (And there its simple to use checkPermission to get detailed information about the directory).
But what about when the user not only give the file name? For exaample "....\test.txt" or "dir1\dir2\test.txt". How to combine the working directory with a specific path to gain the up-directory of the file and then to check the permissions?
Phew, I hope all is clear and it was'nt too long ;-)
Rumo
I'd suggest using the Boost FileSystem library at www.boost.org. In particular, check out the path class, which has methods such as make_absolute and parent_path.
This is Windows example code GetFileNameFromHandle to show you how to get the path from a HANDLE. I think it is what you are looking for.
http://msdn.microsoft.com/en-us/library/aa366789%28v=vs.85%29.aspx
I found out that _stat() and _access() doesn't really works for the permissions of the directories. See this stackoverflow page.
With _stat() you can't use ".\" to get information about the current directory. But _access() at least can check if a directory exists as well ".\" or "..\".
In conclusion I use _access() to check the existence of a directory and _stat() to check the permissions of an existing file. If a file should be created I'll check it by doing.
And by the way ;-) I don't need to combine working directory with the user specified file because I can use the specified file alone in _access() to determine if directory exists.
Rumo
I'm trying to generate some LaTeX code which from thereon should generate PDF documents.
Currently, I'm using the Django templating system for dynamically creating the code, but I have no idea on as how to move on from here. I understand that I could save the code in a .tex file, and use subprocess to run pdflatex for generating the PDF. But I had so much trouble escaping the LaTeX code in "plain" Python that I decided to use the Django templating system. Is there a way that I could somehow maybe pipe the output produced by Django to pdflatex? The code produced is working properly, it's just that I do not know what to do with it.
Thanks in advance
I was tackling the same issue in a project I had previously worked on, and instead of piping the output, I created temporary files in a temporary folder, since I was worried about handling the intermediate files LaTeX produces. This is the code I used (note that it's a few years old, from when I was still new to Python/Django; I'm sure I could come up with something better if I was writing this today, but this definitely worked for me):
import os
from subprocess import call
from tempfile import mkdtemp, mkstemp
from django.template.loader import render_to_string
# In a temporary folder, make a temporary file
tmp_folder = mkdtemp()
os.chdir(tmp_folder)
texfile, texfilename = mkstemp(dir=tmp_folder)
# Pass the TeX template through Django templating engine and into the temp file
os.write(texfile, render_to_string('tex/base.tex', {'var': 'whatever'}))
os.close(texfile)
# Compile the TeX file with PDFLaTeX
call(['pdflatex', texfilename])
# Move resulting PDF to a more permanent location
os.rename(texfilename + '.pdf', dest_folder)
# Remove intermediate files
os.remove(texfilename)
os.remove(texfilename + '.aux')
os.remove(texfilename + '.log')
os.rmdir(tmp_folder)
return os.path.join(dest_folder, texfilename + '.pdf')
The dest_folder variable is usually set to somewhere in the media directory, so that the PDF can then be served statically. The value returned is the path to the file on disk. The logic of what its URL would be is handled by whatever function sets the dest_folder.