In django version 3.2.12, I try to download an image for which I have an URL and store it inside an ImageField. It looks like this:
def load_image_from_url(self):
image_file = NamedTemporaryFile(delete=True, dir='project/media')
with urlopen(self.extra_large_url) as uo:
assert uo.status == 200
image_file.write(uo.read())
image_file.flush()
image = File(image_file)
self.image.save(image.name, image)
When I create the NamedTemporaryFile, the path generated is an absolute path.
In django.core.files.utils, it get blocked in this condition:
if allow_relative_path:
# Use PurePosixPath() because this branch is checked only in
# FileField.generate_filename() where all file paths are expected to be
# Unix style (with forward slashes).
path = pathlib.PurePosixPath(name)
if path.is_absolute() or '..' in path.parts:
raise SuspiciousFileOperation(
"Detected path traversal attempt in '%s'" % name
)
With a dir argument set for the NamedTemporaryFile, the issue remains even if the folder is a subdirectory of the project.
Is there a better solution than doing this:
self.image.save(os.path.basename(image.name), image)
Related
I'm using Django and django-storages[azure] as backend.
But i can't to find out to list dir the files instead I have to use snippets like this:
block_blob_service.list_blobs(container_name='media', prefix=folderPath)]
this is not working:
listdir(absoluteFolderPath)
In storages/backend/azure_storage.py I found this part:
def listdir(self, path=''):
"""
Return directories and files for a given path.
Leave the path empty to list the root.
Order of dirs and files is undefined.
"""
files = []
dirs = set()
for name in self.list_all(path):
n = name[len(path):]
if '/' in n:
dirs.add(n.split('/', 1)[0])
else:
files.append(n)
return list(dirs), files
But how can I use it?
regards
Christopher.
Assuming you have set DEFAULT_FILE_STORAGE you could access it through storage
from django.core.files.storage import default_storage
dirs = default_storage.listdir(absoluteFolderPath)
The API for listdir returns a tuple of two lists, the first being the directories (folders) in the given path, the second being the files.
The correct way to unpack this would be to assign two variables during the call.
Examples:
from django.core.files.storage import default_storage
# list the contents of the root directory of the storage
directories, files = default_storage.listdir('')
# list the contents of a directory named 'images' inside 'media'
directories, files = default_storage.listdir('media/images')
# now you can iterate through just the files
for filename in files:
print(filename)
I'm using django-storages with S3, but Azure looks to work exactly the same.
I have the following function that gives me the list of files(complete path) in a given list of directories:
from os import walk
from os.path import join
# Returns a list of all the files in the list of directories passed
def get_files(directories = get_template_directories()):
files = []
for directory in directories:
for dir, dirnames, filenames in walk(directory):
for filename in filenames:
file_name = join(dir, filename)
files.append(file_name)
return files
I'am adding some files to the template directories in Django. But this function always return the same list of files even though some are added/deleted in the run time. These changes are reflected only when I do a server restart. Is that because of some caching that os.walk() performs or is it required that we need to restart the server after adding/removing some files ?
It is not django problem, your behaviour is result of python interpreter specific:
Default arguments may be provided as plain values or as the result of a function call, but this latter technique need a very big warning. Default values evaluated once at start application and never else.
I' m sure this code will solve your problem:
def get_files(directories = None):
if not directories:
directories = get_template_directories()
files = []
for directory in directories:
for dir, dirnames, filenames in walk(directory):
for filename in filenames:
file_name = join(dir, filename)
files.append(file_name)
return files
You can find same questions on Stackoverflow Default Values for function parameters in Python
I am trying to get PrettyTables to work with the following script. I can get it almost to look right but it keeps separating my tables so it is printing 16 separate tables. I need all information in one table that I can sort. I appreciate all the help i can get.
import sys
import os
import datetime
import hashlib
import logging
def getScanPath(): #12
# Prompt User for path to scan
path = raw_input('Please enter the directory to scan: ')
# Verify that the path is a directory
if os.path.isdir(path):
return path
else:
sys.exit('Invalid File Path ... Script Aborted')
def getFileList(filePath):
# Create an empty list to hold the resulting files
pathList =[]
# Get a list of files, note these will be just the names of the files
# NOT the full path
simpleFileNameList = os.listdir(filePath)
# Now process each filename in the list
for eachFile in simpleFileNameList:
# 1) Get the full path by join the directory with the filename
fullPath = os.path.join(filePath, eachFile)
# 2) Make sure the full path is an absolute path
absPath = os.path.abspath(fullPath)
# 3) Make sure the absolute path is a file i.e. not a folder or directory
if os.path.isfile(absPath):
# 4) if all is well, add the absolute path to the list
pathList.append(absPath)
else:
logging.error('A Non-File has been identified')
# 5) Once all files have been identified, return the list to the caller
return pathList
def getFileName(theFile):
return os.path.basename(theFile)
def getFileSize(theFile):
return os.path.getsize(theFile)
def getFileLastModified(theFile):
return os.path.getmtime(theFile)
def getFileHash(theFile):
hash_md5 = hashlib.md5()
with open(theFile, "rb") as f:
for chunk in iter(lambda: f.read(4096), b""):
hash_md5.update(chunk)
return hash_md5.hexdigest()
# Main Script Starts Here
if __name__ == '__main__':
#Welcome Message
print "\nWelcome to the file scanner\n"
# prompt user for directory path
scanPath = getScanPath()
# Get a list of files with full path
scanFileList = getFileList(scanPath)
# Output Filenames
print "Files found in directory"
for eachFilePath in scanFileList:
fileName = getFileName(eachFilePath)
fileSize = getFileSize(eachFilePath)
lastModified = getFileLastModified(eachFilePath)
hashValue = getFileHash(eachFilePath)
fileModified = (datetime.datetime.fromtimestamp(lastModified))
from prettytable import PrettyTable
pTable = PrettyTable()
pTable.field_names = ["File Name", "File Size", "Last Modified", "Md5 Hash Value"]
pTable.add_row ([fileName, fileSize, fileModified, hashValue])
print (pTable)enter code here
This should show me one big table using all the values from a set directory that the user chooses. This will allow me to sort the table later using prettytables.
I have no experience with prettyTables, but I noticed you have lastModified and fileModified yet only fileModified is used for a column in your table. Are you sure pretty table doesn't have some kind of row limit?
I have looked extensively on this site and I can't see an example that fits the bill. I have 4 directories each of which contains a number of files and another directory called 'Superseded'. I am trying to write a script that will move all files in each folder into the 'Superseded' folder but I'm not having any luck.
import os, shutil
source = r'U:\Data\All\Python_Test\Exports\GLA'
dest = r'U:\Data\All\Python_Test\Exports\Superseded'
listofFiles = os.listdir(source)
for f in listofFiles:
fullPath = source + "/" + f
shutil.move(fullPath, dest)
I can only get this to work for one directory and even then only when I've made the destination directory outside of the GLA directory if that makes sense.
I know there is a a os.path.isfile() module so that I can only move the files but I can't seem to get it to work. Does anybody have any ideas?
This works for me:
import os
#from:
# https://stackoverflow.com/questions/1158076/implement-touch-using-python
# I use this to create some empty file to move around later
def touch(fname, times=None):
fhandle = open(fname, 'a')
try:
os.utime(fname, times)
finally:
fhandle.close()
# this function is only to create the folders and files to be moved
def create_files_in_known_folders():
nameList=["source_dir_{:02d}".format(x) for x in range(4)]
for name in nameList:
path=os.path.expanduser(os.path.join("~",name))
if not os.path.exists(path):
os.mkdir(path)
ssPath=os.path.join(path,"superseded")
if not os.path.exists(ssPath):
os.mkdir(ssPath)
for i in range(3):
filename="{}_{:02d}.dat".format(name,i)
filepath=os.path.join(path, filename)
if not os.path.exists(filepath):
touch(filepath)
# THIS is actually the function doing what the OP asked for
# there many details that can be tweaked
def move_from_known_to_dest():
# here my given names from above
nameList=["source_dir_{:02d}".format(x) for x in range(4)]
# and my destination path
destPath=os.path.expanduser(os.path.join("~","dest"))
# not interested in files that are in subfolders
# if those would exist change to os.walk and
# exclude the destination folder with according if...:
for name in nameList:
path=os.path.expanduser(os.path.join("~",name))
dirList=os.listdir(path)
print path
for fileName in dirList:
filePath=os.path.join(path, fileName)
print filePath
if os.path.isfile(filePath):
destPath=os.path.join(path,"superseded",fileName)
print destPath
#alternatively you can chose to 1) overwrite ()might not work 2)delete first 3) not copy
# another option is to check for existence and if
# present add a number to the dest-file-name
# use while loop to check for first non-present number
assert not os.path.exists(destPath), "file {} already exits".format(destPath)
#https://stackoverflow.com/questions/8858008/how-to-move-a-file-in-python
os.rename( filePath, destPath)
if __name__=="__main__":
create_files_in_known_folders()
#break here and check that filestructure and files have been created
move_from_known_to_dest()
But, think carefully what to do if the file already exits in your destination folder.
os.walk might also be something you want to look at.
Implementing several options for the copy behaviour may look like this:
import warnings
#from:
# https://stackoverflow.com/questions/2187269/python-print-only-the-message-on-warnings
formatwarning_orig = warnings.formatwarning
warnings.formatwarning = lambda message, category, filename, lineno, line=None: \
formatwarning_orig(message, category, filename, lineno, line='')
def move_from_known_to_dest_extra(behaviour='overwrite'):
assert behaviour in ['overwrite','leave','accumulate'], "unknown behaviour: {}".format(behaviour)
nameList=["source_dir_{:02d}".format(x) for x in range(4)]
destPath=os.path.expanduser(os.path.join("~","dest"))
for name in nameList:
path=os.path.expanduser(os.path.join("~",name))
dirList=os.listdir(path)
for fileName in dirList:
filePath=os.path.join(path, fileName)
if os.path.isfile(filePath):
destPath=os.path.join(path,"superseded",fileName)
# simplest case...does not exist so copy
if not os.path.exists(destPath):
os.rename( filePath, destPath)
else:
if behaviour=='leave':
warnings.warn( "Warning! Not copying file: {}; file {} already exists!".format(filePath, destPath))
elif behaviour =='overwrite':
os.remove(destPath)
# documentation states:
# On Windows, if dst already exists, OSError will be raised even if it is a file.
os.rename( filePath, destPath)
warnings.warn( "Warning!Overwriting file: {}.".format(destPath))
elif behaviour=='accumulate': #redundant but OK
addPost=0
while True:
newDestPath=destPath+"{:04d}".format(addPost)
if not os.path.exists(newDestPath):
break
addPost+=1
assert addPost < 10000, "Clean up the mess!"
os.rename( filePath, newDestPath)
else:
assert 0, "Unknown copy behaviour requested."
Additionally one might check for file permissions as, e.g., os.remove() may raise an exception. In this case, however, I assume that permissions are properly set by the OP.
I think I am pretty familiar with how os.mkdir and os.makedirs work but somehow I'm missing something here. I have a function named check_for_dir() in my script myscript.py that is located in this path Users/myuser/Projects/myProject/myscript.py. I want to create a directory named mydirectory in my home folder. It looks like this:
import os
path = 'Users/myuser/mydirectory/'
def check_for_dir(path):
if not os.path.exists(path):
os.makedirs(path)
check_for_dir(path)
But for some reason the whole structure defined in variable path is created in script's location. What this means is that directory mydirectory/ is created in this path:
Users/myuser/Projects/myproject/Users/myuser/mydirectory/
What am I doing wrong?
import os
path = '/Users/myuser/mydirectory/'
def check_for_dir(path):
if not os.path.exists(path):
os.makedirs(path)
check_for_dir(path)
I think you have to change path as path = '/Users/myuser/mydirectory/' Hope this will help
If you specify path = 'Users/myuser/mydirectory/' then it will be consider as a relative path then creates a folder as you have mentioned in the question