Python: copy long file path Shutil.copyfile - python-2.7

I want to copy too long paths with python using shutil.copyfile.
Now I read this Copy a file with a too long path to another directory in Python page to get the solution. I used:
shutil.copyfile(r'\\\\?\\' + ErrFileName,testPath+"\\"+FilenameforCSV+"_lyrErrs"+timestrLyr+".csv")
to copy the file but it gives me an error : [Errno 2] No such file or directory: '\\\\?\\C:\\...
Can anyone please let me know how to incorporate longs paths with Shutil.copyfile, the method I used above should allow 32k characters inside a file path, but I cant even reach 1000 and it gives me this error.

Since the \\?\ prefix bypasses normal path processing, the path needs to be absolute, can only use backslash as the path separator, and has to be a UTF-16 string. In Python 2, use the u prefix to create a unicode string (UTF-16 on Windows).
shutil.copyfile opens the source file in 'rb' mode and destination file in 'wb' mode, and then copies from source to destination in 16 KiB chunks. Given a unicode path, Python 2 opens a file by calling the C runtime function _wfopen, which in turn calls the Windows wide-character API CreateFileW.
shutil.copyfile should work with long paths, assuming they're correctly formatted. If it's not working for you, I can't think of any way to "force" it to work.
Here's a Python 2 example that creates a 10-level tree of directories, each named u'a' * 255, and copies a file from the working directory into the leaf of the tree. The destination path is about 2600 characters, depending on your working directory.
#!python2
import os
import shutil
work = 'longpath_work'
if not os.path.exists(work):
os.mkdir(work)
os.chdir(work)
# create a text file to copy
if not os.path.exists('spam.txt'):
with open('spam.txt', 'w') as f:
f.write('spam spam spam')
# create 10-level tree of directories
name = u'a' * 255
base = u'\\'.join([u'\\\\?', os.getcwd(), name])
if os.path.exists(base):
shutil.rmtree(base)
rest = u'\\'.join([name] * 9)
path = u'\\'.join([base, rest])
os.makedirs(path)
print 'src directory listing (tree created)'
print os.listdir(u'.')
dest = u'\\'.join([path, u'spam.txt'])
shutil.copyfile(u'spam.txt', dest)
print '\ndest directory listing'
print os.listdir(path)
print '\ncopied file contents'
with open(dest) as f:
print f.read()
# Remove the tree, and verify that it's removed:
shutil.rmtree(base)
print '\nsrc directory listing (tree removed)'
print os.listdir(u'.')
Output (line wrapped):
src directory listing (tree created)
[u'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaa', u'spam.txt']
dest directory listing
[u'spam.txt']
copied file contents
spam spam spam
src directory listing (tree removed)
[u'spam.txt']

Related

Executing an .exe file on files in another folder

I have my python code that runs a C++ code, which takes files in another folder as input.
I have my codes in folder A, and the input files are in folder B, and I have been trying this:
path = 'C:/pathToInputFiles'
dirs = os.listdir(path)
for path in dirs:
proc = subprocess.Popen([fullPathtoCppCode, inputFiles])
However, I keep receiving WindowsError: [Error 2] The system cannot find the file specified
The only way it works is when I put the C++ executable file in the same folder of the input files, which I am avoiding to do.
How can I make python reads the file path properly?
Try using os.path.join after your for statement.
path = os.path.join(directory, filename)
for example
def test(directory):
for filename in os.listdir(directory):
filename = os.path.join(directory, filename)
proc = subprocess.Popen([fullPathtoCppcode, inputFiles])

Python shutil file move in os walk for loop

The code below searches within a directory for any PDFs and for each one it finds it moves into the corresponding folder which has '_folder' appended.
Could it be expressed in simpler terms? It's practically unreadable. Also if it can't find the folder, it destroys the PDF!
import os
import shutil
for root, dirs, files in os.walk(folder_path_variable):
for file1 in files:
if file1.endswith('.pdf') and not file1.startswith('.'):
filenamepath = os.path.join(root, file1)
name_of_file = file1.split('-')[0]
folderDest = filenamepath.split('/')[:9]
folderDest = '/'.join(folderDest)
folderDest = folderDest + '/' + name_of_file + '_folder'
shutil.move(filenamepath2, folderDest)
Really I want to traverse the same directory after constructing the variable name_of_file and if that variable is in a folder name, it performs the move. However I came across issues trying to nest another for loop...
I would try something like this:
for root, dirs, files in os.walk(folder_path_variable):
for filename in files:
if filename.endswith('.pdf') and not filename.startswith('.'):
filepath = os.path.join(root, filename)
filename_prefix = filename.split('-')[0]
dest_dir = os.path.join(root, filename_prefix + '_folder')
if not os.path.isdir(dest_dir):
os.mkdir(dest_dir)
os.rename(filepath, os.path.join(dest_dir, filename))
The answer by John Zwinck is correct, except it contains a bug where if the destination folder already exists, a folder within that folder is created and the pdf is moved to that location. I have fixed this by adding a 'break' statement within the inner for loop (for filename in files).
The code below now executes correctly. Looks for folder named as the pdf's first few characters (taking the prefix split at '-') with '_folder' at the tail, if it exists the pdf is moved into it. If it doesn't, one is created with the prefix name and '_folder' and pdf is moved into it.
for root, dirs, files in os.walk(folder_path_variable):
for filename in files:
if filename.endswith('.pdf') and not filename.startswith('.'):
filepath = os.path.join(root, filename)
filename_prefix = filename.split('-')[0]
dest_dir = os.path.join(root, filename_prefix + '_folder')
if not os.path.isdir(dest_dir):
os.mkdir(dest_dir)
os.rename(filepath, os.path.join(dest_dir, filename))
break

Zipping a file from a directory and placing it in another Directory

I am trying to set up a program to put my minecraft server world into a zip and place it into another directory on another drive (/media/500gb/MinecraftWorldBackups)
But I keep getting this error
Although the folder doesn't contain a folder or file called 'h'
What do I need to do to fix this I believe it is due to file and folder?
#!/usr/bin/env python
import time, zipfile
while True:
FileName = 'MinecraftBackup_' + str(int(time.time()))
Path = '/home/bertie/Desktop/FeedTheBeastServer/world/'
print(FileName)
Zip = zipfile.ZipFile('/media/500gb/MinecraftWorldBackups/'+FileName+'.zip','w')
for each in Path:
print(each)
try: Zip.write(Path + each)
except IOError: None
Zip.Close()
print('Done')
time.sleep(60)

Python - Counting the number of files and folders in a directory

I've got a python script that deletes an entire directory and its subfolders, and I'd like to print out the number of files and folders removed. Currently, I have found some code from a different question posed 2010, but the answer I receive back is 16... If I right-click on the the folder it states that theres 152 files, 72 folders...
The code I currently have for checking the directory;
import os, getpass
user = getpass.getuser()
copyof = 'Copy of ' + user
directory = "C:/Documents and Settings/" + user
print len([item for item in os.listdir(directory)])
How can I extend this to show the same number of files and folders that there actually are?
To perform recursive search you may use os.walk.
os.walk(top, topdown=True, onerror=None, followlinks=False)
Generate the file names in a directory tree by walking the tree either top-down or bottom-up. For each directory in the tree rooted at
directory top (including top itself), it yields a 3-tuple (dirpath,
dirnames, filenames).
Sample usage:
import os
dir_count = 0
file_count = 0
for _, dirs, files in os.walk(dir_to_list_recursively):
dir_count += len(dirs)
file_count += len(files)
I was able to solve this issue by using the following code by octoback (copied directly);
import os
cpt = sum([len(files) for r, d, files in os.walk("G:\CS\PYTHONPROJECTS")])

using os.walk cannot open the file from the list

My problem is to read '.csv' files in catalogs and do some calculations on them.
I have calculations working but my for loop seem not to work as I want to.
d = 'F:\MArcin\Experiments\csvCollection\'
for dirname, dirs, files in os.walk(d):
for i in files:
if i.endswith('.csv'):
data1 = pd.read_csv(i, sep=",")
data = data1['x'][:, np.newaxis]
target = data1['y']
The error Iam getting is:
IOError: File 1.csv does not exist
files is list of all '.csv' files inside dirname
i is str of size 1 and contains 1.csv (that is first of the files in catalog)
Any ideas why this is not working?
Thanks for any help.
Because 1.csv is somewhere on the filesystem and when you call read_csv() it opens file relative to current directory.
Just open it using absolute path:
data1 = pd.read_csv(os.path.join(dirname, i), sep=",")
dirname in os.walk represents actual directory where file 1.csv is located.