I am trying to find situations when os.path.isfile() returns false. I have the following code which checks if the content is not a file (should have used isdir()) and does structured zip.
for fpath in glob(os.path.join(local_dir, glob_wildcard)):
dirpath, filename = os.path.split(fpath)
try:
if not os.path.isfile(fpath):
//code for structured zipping and move
else:
//move the file as is
This was working fine until now but recently found some of the files are getting zipped, than getting moved. One cause was, I have found another instance running at the same time which stole the file and isfile() for this instance returned false. So, it created an empty zip file.
Are there any other scenarios when it will return false? like file getting created or copied etc.
Related
I am learning about exceptions and so performing some file operations and testing various parts of code that can possibly generate exceptions while working with files in Python. I am executing this Python 2.7 code on Canopy.
#!/usr/bin/python
import os
try:
fp = open('testfile', 'r')
except IOError:
print 'File not opened successfully'
else:
print 'File opened successfully'
try:
fp.write('Hello!')
except IOError:
print 'Write not allowed on this file'
else:
print 'Write successful'
try:
fp.close()
except IOError:
print 'File not closed properly'
else:
print 'File closed successfully'
finally:
if os.path.exists(fp.name):
os.remove(fp.name)
When I execute this code, I get the following output:
File not opened properly
NameErrorTraceback (most recent call last)
/home/sr/Python/tcs.py in ()
--> 185 if os.path.exists(fp.name)
NameError: name 'fp' is not defined
But if I change the access mode of file to 'w', Then everything seems to work properly with the correct output as:
File opened successfully
Write successful
File closed successfully
I cannot understand why the 'r' mode is not making the file open properly and thus the fp file object is not created. Please help me figure the problem out.
P.S.: Also I would like to know if there is a better way of implementing the same thing. But this is optional.
Explanation
The error combined with your printout should be pretty self-explanatory: the variable fp does not exist if you can't open the file.
The mode 'r' indicates that you want to open the file for reading. You can not read something that is not there, so you end up going to the finally block in your code after processing the IOError. But the error occurs before fp was set, so there is no variable fp, hence the error. [Solutions below]
The mode 'w' indicates that you want to open for writing, but from scratch. There is also an 'a' mode to append if the file already exists. You can write to a non-existent file just fine, so your code does not fail. In fact, if the file did exist in 'w' mode, it would be trucated and any previous contents would be lost.
Try creating an empty file and running with mode 'r'. You should get an exception that prints 'Write not allowed on this file'. That is because, as your error message correctly indicates, writing to a file opened in read mode is not allowed.
Improvements
There are two major improvements that you can make to your code. One is fixing the logical flaws, the other is a major stylistic improvement using with statements.
You have two major logic errors. The first is in the outermost finally block that you already saw. The simplest fix is moving the contents of the finally block into the else, since you don't have any action to take if the file was not opened. Another solution is to refer to the file name you are trying to open in the first place. For example, you could store the file name into a variable and use that:
filename = 'testfile'
try:
fp = open(filename, 'r')
...
finally:
if os.path.exists(filename):
os.remove(filename)
The second major logic error is that you do not close the file if your write fails. Notice that you call fp.close() only in the else clause of your try block. If should instead appear in a finally block. The print statement should of course stay in the else. Change
else:
print 'Write successful'
try:
fp.close()
...
to
else:
print 'Write successful'
finally:
try:
fp.close()
...
The entire code can be improved stylistically by using with blocks to manage your file operations. The obvious way to do this is as follows:
fname = 'testfile'
with open(fname, 'r') as fp:
fp.write('Hello!')
if os.path.exists(fname):
os.remove(fname)
You will not get as many detailed messages when things fail, but overall, this code is cleaner, shorter and more robust than what you have. It guarantees that the file will be closed whether or not an exception occurred anywhere along the way. If you need the detailed error output that you currently have, keep using the current try blocks. The reason that most people will prefer with is that any error that occurs will have a detailed desciption and a line number that it happened on, so you basically get all the necessary information with a lot less work.
Here are some additional resources to help you understand with and context managers:
Understanding Python's "with" statement (from effbot.org)
Official documentation for with
SO 1, 2, 3, 4
So trying to finish a very simple script that has given me a unbelievably hard time. It's supposed to iterate through specified directories and open all text files in them and append them all with the same specified string.
The issue is it's not doing anything to the files at all. Using print to test my logic I've replaced lines 10 and 11 with print f (the write and close functions), and get the following output:
<open file '/Users/russellculver/documents/testfolder/.DS_Store', mode 'a+' at
So I think it is storing the correct files in the f variable for the write function, however I am not familiar with how Mac's handle DS_STORE or the exact role it plays in temporary location tracking.
Here is the actual script:
import os
x = raw_input("Enter the directory path here: ")
def rootdir(x):
for dirpaths, dirnames, files in os.walk(x):
for filename in files:
try:
with open(os.path.join(dirpaths, filename), 'a+') as f:
f.write('new string content')
f.close()
except:
print "Directory empty or unable to open file."
return x
rootdir(x)
And the exact return in Terminal after execution:
Enter the directory path here: /Users/russellculver/documents/testfolder
Exit status: 0
logout
[Process completed]
Yet nothing written to the .txt files in the provided directory.
The way the indentation is in the question, you return from the function right after writing the first file; either of the for-loops never finish. Which is relatively easy to surmise from the fact that you only get one output file printed.
Since you're not doing anything with the result of the rootdir function, I would just remove the return statement entirely.
An aside: there is no need to use f.close() when you open a file with the with statement: it will automatically be closed (even upon an exception). That is in fact what the with statement was introduced for (see the pep on context managers if necessary).
To be complete, here's the function the way I would have (roughly) written it:
def rootdir(x):
for dirpaths, dirnames, files in os.walk(x):
for filename in files:
path = os.path.join(dirpaths, filename)
try:
with open(path, 'a+') as f:
f.write('new string content')
except (IOError, OSError) as exc:
print "Directory empty or unable to open file:", path
(Note that I'm catching only the relevant I/O errors; any other exceptions (though unlikely) will not be caught, as they are likely not to be related to non-existing/unwritable file.)
Return was indented wrong, ending the iteration after a single loop. Wasn't even necessary so was removed entirely.
In the code I maintain I run across:
from win32com.shell import shell, shellcon
# ...
result,nAborted,mapping = shell.SHFileOperation(
(parent,operation,source,target,flags,None,None))
In Python27\Lib\site-packages\win32comext\shell\ (note win32comext) I just have a shell.pyd binary.
What is the return value of shell.SHFileOperation for a deletion (operation=FO_DELETE in the call above) ? Where is the code for the shell.pyd ?
Can I get the list of files actually deleted from this return value or do I have to manually check afterwards ?
EDIT: accepted answer answers Q1 - having a look at the source of pywin32-219\com\win32comext\shell\src\shell.cpp I see that static PyObject *PySHFileOperation() delegates to SHFileOperation which does not seem to return any info on which files failed to be deleted - so I guess answer to Q2 is "no".
ActiveState Python help contains SHFileOperation description:
shell.SHFileOperation
int, int = SHFileOperation(operation)
Copies, moves, renames, or deletes a file system object.
Parameters
operation : SHFILEOPSTRUCT
Defines the operation to perform.
Return Value
The result is a tuple containing int result of the
function itself, and the result of the fAnyOperationsAborted member
after the operation. If Flags contains FOF_WANTMAPPINGHANDLE, returned
tuple will have a 3rd member containing a sequence of 2-tuples with
the old and new file names of renamed files. This will only have any
content if FOF_RENAMEONCOLLISION was specified, and some filename
conflicts actually occurred.
Source code can be downloaded here: http://sourceforge.net/projects/pywin32/files/pywin32/Build%20219/ (pywin32-219.zip)
Just unpack and go to .\pywin32-219\com\win32comext\shell\src\
Currently I have the following part code in my Sync:
...
int index = file.find(remoteDir);
if(index >= 0){
file.erase(index, remoteDir.size());
file.insert(index, localDir);
}
...
// Uses PUT command on the file
Now I want to do the following instead:
If a file is the same as before, except for a rename, don't use the PUT command, but use the Rename command instead
TL;DR: Is there a way to check whether a file is the same as before except for a rename that occurred? So a way to compare both files (with different names) to see if they are the same?
check the md5sum, if it is different then the file is modified.
md5 check sum of a renamed file will remain same. Any change in content of file will give a different value.
I first tried to use Renjith method with md5, but I couldn't get it working (maybe it's because my C++ is for windows instead of Linux, I dunno.)
So instead I wrote my own function that does the following:
First check if the file is the exact same size (if this isn't the case we can just return false for the function instead of continuing).
If the sizes do match, continue checking the file-buffer per BUFFER_SIZE (in my case this is 1024). If the entire buffer of the file matches, return true.
PS: Make sure to close any open streams before returning.. My mistake here was that I had the code to close one stream after the return-statement (so it was never called), and therefore I had errno 13 when trying to rename the file.
I am using QSettings to try and figure out if an INI is valid.(using status() to check) I made a purposefully invalid INI file and loaded it in. The first time the code is called, it returns invalid, but every time after that, it returns valid. Is this a bug in my code?
It's a Qt bug caused by some global state. Note that the difference in results happens whether or not you call delete on your QSettings object, which you should. Here's a brief summary of what happens on the first run:
The result code is set to NoError.
A global cache is checked to see if your file is present
Your file isn't present the first time, so it's parsed on qsettings.cpp line 1530 (Qt-4.6.2)
Parsing results in an error and the result code is set (see qsettings.cpp line 1552).
The error result code is returned.
And the second run is different:
The result code is set to NoError.
A global cache is checked, your file is present.
The file size and timestamp are checked to see if the file has changed (see qsettings.cpp line 1424).
The result code is returned, which happens to be NoError -- the file was assumed to have been parsed correctly.
Checked your code, you need to delete the file object before returning.
Apart from that, your code uses the QSettings::QSettings(fileName, format) c'tor to open an ini-file. That call ends in the function QConfFile::fromName (implemented in qsettings.cpp). As I read it (there are a few macros and such that I decided not to follow), the file is not re-opened if the file already is open (i.e. you have not deleted the object since the last time). Thus the status will be ok the second time around.