I've got a python script that deletes an entire directory and its subfolders, and I'd like to print out the number of files and folders removed. Currently, I have found some code from a different question posed 2010, but the answer I receive back is 16... If I right-click on the the folder it states that theres 152 files, 72 folders...
The code I currently have for checking the directory;
import os, getpass
user = getpass.getuser()
copyof = 'Copy of ' + user
directory = "C:/Documents and Settings/" + user
print len([item for item in os.listdir(directory)])
How can I extend this to show the same number of files and folders that there actually are?
To perform recursive search you may use os.walk.
os.walk(top, topdown=True, onerror=None, followlinks=False)
Generate the file names in a directory tree by walking the tree either top-down or bottom-up. For each directory in the tree rooted at
directory top (including top itself), it yields a 3-tuple (dirpath,
dirnames, filenames).
Sample usage:
import os
dir_count = 0
file_count = 0
for _, dirs, files in os.walk(dir_to_list_recursively):
dir_count += len(dirs)
file_count += len(files)
I was able to solve this issue by using the following code by octoback (copied directly);
import os
cpt = sum([len(files) for r, d, files in os.walk("G:\CS\PYTHONPROJECTS")])
Related
I have the following function that gives me the list of files(complete path) in a given list of directories:
from os import walk
from os.path import join
# Returns a list of all the files in the list of directories passed
def get_files(directories = get_template_directories()):
files = []
for directory in directories:
for dir, dirnames, filenames in walk(directory):
for filename in filenames:
file_name = join(dir, filename)
files.append(file_name)
return files
I'am adding some files to the template directories in Django. But this function always return the same list of files even though some are added/deleted in the run time. These changes are reflected only when I do a server restart. Is that because of some caching that os.walk() performs or is it required that we need to restart the server after adding/removing some files ?
It is not django problem, your behaviour is result of python interpreter specific:
Default arguments may be provided as plain values or as the result of a function call, but this latter technique need a very big warning. Default values evaluated once at start application and never else.
I' m sure this code will solve your problem:
def get_files(directories = None):
if not directories:
directories = get_template_directories()
files = []
for directory in directories:
for dir, dirnames, filenames in walk(directory):
for filename in filenames:
file_name = join(dir, filename)
files.append(file_name)
return files
You can find same questions on Stackoverflow Default Values for function parameters in Python
So, my main goal is to have this make directories based on the amount of (files//2000)+1 and put 2000 files in each directory and whatever is leftover into the final directory (assuming it'll never be divisible by 2000 evenly).
So far, my scripting looks like this:
import os
import shutil
def createDirs():'
src='P:\\stuff
folderNumber=0
filesNumber=0
for files in os.listdir(src):
filesNumber=filesNumber+1
print filesNumber
totalFolders=(filesNumber//2000)+1
print totalFolders
for folders in range(0, totalFolders):
os.mkdir('P:\\Project\\User\\TEST\\folder' + str(folderNumber))
folderNumber=folderNumber+1
def group():
fileType='.txt'
src='P:\\Project\\User\\files'
folderCount=0
fileCount=0
for folders in os.listdir('P:\\Project\\User\\TEST'):
folderCount=folderCount+1
for files in os.listdir('P:\\Project\\User\\TEST\\folder' + str(folderCount)):
fileCount=fileCount+1
while fileCount <= 2000:
for file in os.listdir(src):
if file.endswith(filetype):
path = os.path.join(src, file):
shutil.move(path, 'P:\\Project\\User\\TEST\\folder' + str(folderCount))
The directories get made, the files move to folder one, but files just keep flowing into folder 1 endlessly.
Any help and/or ways to improve this process would be greatly appreciated.
in your while statement you never increase fileCount or folderCount. I would probably remove the while statement and replace it with:
for name in os.listdir(src):
if name.endswith(filetype):
fileCount += 1
if fileCount > 2000:
fileCount = 0
folderCount += 1
path = os.path.join(src, file):
shutil.move(path, 'P:\\Project\\User\\TEST\\folder' + str(folderCount))
The code below searches within a directory for any PDFs and for each one it finds it moves into the corresponding folder which has '_folder' appended.
Could it be expressed in simpler terms? It's practically unreadable. Also if it can't find the folder, it destroys the PDF!
import os
import shutil
for root, dirs, files in os.walk(folder_path_variable):
for file1 in files:
if file1.endswith('.pdf') and not file1.startswith('.'):
filenamepath = os.path.join(root, file1)
name_of_file = file1.split('-')[0]
folderDest = filenamepath.split('/')[:9]
folderDest = '/'.join(folderDest)
folderDest = folderDest + '/' + name_of_file + '_folder'
shutil.move(filenamepath2, folderDest)
Really I want to traverse the same directory after constructing the variable name_of_file and if that variable is in a folder name, it performs the move. However I came across issues trying to nest another for loop...
I would try something like this:
for root, dirs, files in os.walk(folder_path_variable):
for filename in files:
if filename.endswith('.pdf') and not filename.startswith('.'):
filepath = os.path.join(root, filename)
filename_prefix = filename.split('-')[0]
dest_dir = os.path.join(root, filename_prefix + '_folder')
if not os.path.isdir(dest_dir):
os.mkdir(dest_dir)
os.rename(filepath, os.path.join(dest_dir, filename))
The answer by John Zwinck is correct, except it contains a bug where if the destination folder already exists, a folder within that folder is created and the pdf is moved to that location. I have fixed this by adding a 'break' statement within the inner for loop (for filename in files).
The code below now executes correctly. Looks for folder named as the pdf's first few characters (taking the prefix split at '-') with '_folder' at the tail, if it exists the pdf is moved into it. If it doesn't, one is created with the prefix name and '_folder' and pdf is moved into it.
for root, dirs, files in os.walk(folder_path_variable):
for filename in files:
if filename.endswith('.pdf') and not filename.startswith('.'):
filepath = os.path.join(root, filename)
filename_prefix = filename.split('-')[0]
dest_dir = os.path.join(root, filename_prefix + '_folder')
if not os.path.isdir(dest_dir):
os.mkdir(dest_dir)
os.rename(filepath, os.path.join(dest_dir, filename))
break
I quite new in Python programming and i try to rename 100 files with ".jpg" extention, located in specific folder using pyhthon.
I need that the files will be renamed by running order start from number 1. This is the code i start writing:
import os,glob,fnmatch
os.chdir(r"G:\desktop\Project\test")
for files in glob.glob("*.jpg"):
print files
When i run it, i get:
>>>
er3.jpg
IMG-20160209-ssdeWA0000.jpg
IMG-20160209-WA0000.jpg
sd4.jpg
tyu2.jpg
uj7.jpg
we3.jpg
yh7.jpg
>>>
so the code, till now is OK.
For example my folder is:
and i need that all the files name will be:
1,2,3,4 - with running order names. Is it possible with python 2.7?
If you simply want to rename all files as 1.jpg, 2.jpg etc. you can do this:
import os
import glob
os.chdir(r"G:\desktop\Project\test")
for index, oldfile in enumerate(glob.glob("*.jpg"), start=1):
newfile = '{}.jpg'.format(index)
os.rename (oldfile,newfile)
enumerate() is used to get get the index of each file from the list returned by glob(), so that it can be used to create the new filename. Note that it allows you to specify the start index, so I've started from 1, rather than Python Standard, zero
If you want this list of files to be sortable properly, you'll want the filename to be padded with zero's as well (001.jpg, etc.). In which case simply replace newfile = '{}.jpg'.format(index)' with newfile = '{:03}.jpg'.format(index).
See the the docs for more on str.format()
To rename all the JPG files from a particular folder First, get the list of all the files contain in the folder.
os.listdir will give you list all the files in images path.
use enumerate to get the index numbers to get the new name for
images.
import os
images_path = r"D:\shots_images"
image_list = os.listdir(images_path)
for i, image in enumerate(image_list):
ext = os.path.splitext(image)[1]
if ext == '.jpg':
src = images_path + '/' + image
dst = images_path + '/' + str(i) + '.jpg'
os.rename(src, dst)
import os
from os import path
os.chdir("//Users//User1//Desktop//newd//pics")
for file in os.listdir():
name,ext=path.splitext(file)
if ext == '.jpeg':
dst= '{}.jpg'.format(name)
os.rename(file,dst)
I want to copy too long paths with python using shutil.copyfile.
Now I read this Copy a file with a too long path to another directory in Python page to get the solution. I used:
shutil.copyfile(r'\\\\?\\' + ErrFileName,testPath+"\\"+FilenameforCSV+"_lyrErrs"+timestrLyr+".csv")
to copy the file but it gives me an error : [Errno 2] No such file or directory: '\\\\?\\C:\\...
Can anyone please let me know how to incorporate longs paths with Shutil.copyfile, the method I used above should allow 32k characters inside a file path, but I cant even reach 1000 and it gives me this error.
Since the \\?\ prefix bypasses normal path processing, the path needs to be absolute, can only use backslash as the path separator, and has to be a UTF-16 string. In Python 2, use the u prefix to create a unicode string (UTF-16 on Windows).
shutil.copyfile opens the source file in 'rb' mode and destination file in 'wb' mode, and then copies from source to destination in 16 KiB chunks. Given a unicode path, Python 2 opens a file by calling the C runtime function _wfopen, which in turn calls the Windows wide-character API CreateFileW.
shutil.copyfile should work with long paths, assuming they're correctly formatted. If it's not working for you, I can't think of any way to "force" it to work.
Here's a Python 2 example that creates a 10-level tree of directories, each named u'a' * 255, and copies a file from the working directory into the leaf of the tree. The destination path is about 2600 characters, depending on your working directory.
#!python2
import os
import shutil
work = 'longpath_work'
if not os.path.exists(work):
os.mkdir(work)
os.chdir(work)
# create a text file to copy
if not os.path.exists('spam.txt'):
with open('spam.txt', 'w') as f:
f.write('spam spam spam')
# create 10-level tree of directories
name = u'a' * 255
base = u'\\'.join([u'\\\\?', os.getcwd(), name])
if os.path.exists(base):
shutil.rmtree(base)
rest = u'\\'.join([name] * 9)
path = u'\\'.join([base, rest])
os.makedirs(path)
print 'src directory listing (tree created)'
print os.listdir(u'.')
dest = u'\\'.join([path, u'spam.txt'])
shutil.copyfile(u'spam.txt', dest)
print '\ndest directory listing'
print os.listdir(path)
print '\ncopied file contents'
with open(dest) as f:
print f.read()
# Remove the tree, and verify that it's removed:
shutil.rmtree(base)
print '\nsrc directory listing (tree removed)'
print os.listdir(u'.')
Output (line wrapped):
src directory listing (tree created)
[u'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaa', u'spam.txt']
dest directory listing
[u'spam.txt']
copied file contents
spam spam spam
src directory listing (tree removed)
[u'spam.txt']