When I was downloading streaming video, it was interrupted due to some network reasons.
At this time, there was a .part file that saved the downloaded part.
When I resumed the download, the program will continue to download with another file name, but the previous one .part file is invalid.
Can I independently convert this .part file into a usable video file?
Related
I'm opening a file for a video I'm creating and writing to disk with fopen in C++, I'm able to write to disk. But when I try to read it as I'm writing it, it will throw errors saying that it doesn't have permission to read the file as soon as I close the file or stop the program, I can suddenly read from it.
Not an issue with not finishing writing the write as if I crash the program, can still read it. Also, VLC's logs tell me it's a permission issue.
Any idea how to change that permission?
Response to William asking for code snippets or if open happened before the file existed:
Thanks William, here's what I've got. I waited a few minutes and could see the file with windows explorer by that point and waited until after I'd flushed and data was there, couldn't open with VLC or Notepad++ or Notepad or Windows Media Player
Notepad says cannot access because it is being used by another process, others too.
Here is the VLC log while it tries to open this:
http://snippi.com/s/g4cbu23
Here is where I create the file with fopen:
http://snippi.com/s/cyajw4h
At the very end is where I write to the file using fwrite and flush:
http://snippi.com/s/oz27m0g
You need to use _fsopen with _SH_DENYNO if you want the file to be shareable.
We have File like Zip, txt Files in Windows SFTP Server and we use Informatica for our ETL Jobs , but our concern is the vendors who drop Files in SFTP Server they drop files in random times , and the files are of different sizes so How can we detect a File transfer is complete or Not??
Unfortunately, you can't. When your client is asking the FTP server for a list of files, it receives the current state on the server. There's no way of telling if one of the files is currently being written to or not.
So the only way here is to work out some kind of protocol with the vendor. You would need to work with some kind of lock file. If the vendor is writing to the file, they first have to create the lock file. Same goes for your ETL job when reading the file, you would first have to create the lock file and the vendor is not allowed to start a new file writing process until the lock is removed. You get the idea.
It all depends on how fool prove the solution needs to be. Another option is to let the vendor write to a temporary file first. Only when the upload process is finished, they rename the file to its final name.
Writing to temp file and renaming once write is complet is one option, as Socken23 indicated. Others might be:
create empty .ready file once write to target file is done. Your process should then check for the existence of .ready file and read data from the other one.
perform a sequence of file size checks on FTP before starting the ETL. E.g. check file size, wait one minute, check again, wait & check third time. If all sizes match, you may assume the write is complete.
I'm creating a program which downloads a file from an FTP server. I want to display the progress of how much bytes the user has downloaded. I have tried searching for it but I couldn't find anything. First , my plan was to get the file size using fstream but then I realized that I can't share 2 processes at the same time.
I assume you're using FtpGetFile(..) to download the file from the FTP server. This is the simple way of downloading a file which gives you no information about how many bytes are already downloaded and you have very little control how the function behaves (e.g. cancelling of a download upon user request?). For quick test it is very helpful, though.
In order to monitor the download progress do the following:
Determine size of remote file using FtpFindFirstFile.
Open remote file using FtpOpenFile.
Create local file using CreateFile.
Read some bytes from remote file using InternetReadFile.
Evaluate return value and error code of InternetReadFile the detect errors and if transmittion is complete.
Write these bytes to local file using WriteFile.
Calculate/Update your progress information.
Go back to step 4 if file is not downloaded entirely and there is no error.
Close local file using CloseHandle.
Close remote file using InternetCloseHandle.
If you need more help I can offer some code doing this...
I am using Live555/openRTSP based code to stream an H.264 video source to an mp4 file, and would like to concurrently read the earlier parts of the file (from a different application). Changing the fopen to be fid = _fsopen(fileName, "wb", _SH_DENYWR) (from OutputFile.cpp) clearly isn't enough, because it makes no difference and the media players still won't open it (is that the write line for the mp4 output?)
On a related note if I simulate an unclean shutdown of the software (e.g. power failure) the unfinished files are not readable by any media players. I assume that what is written to the file on file-close is what allows a media player to understand the file. This is also a situation I'd like to code for, if possible, and is quite likely the really the same problem as above?
Any pointers/answers/thoughts greatly received :-)
You normally don't read from MP4 (the same is valid for many other formats) files using fopen-like API, and you use libraries instead. Which, in turn, expect a completed file and don't attempt to recover a broken file, or read the file still being written.
So while technically it is possible to read while it is still being written, you are unlikely to succeed in this with regular libraries, applications and players. You need to complete write first to make the file valid, readable and playable - because the completion step writes indices required for playback.
In case anybody ever needs the same, this is how I did it, and it was easier than I imagined:
Change fopen in OutputFile.cpp to the file-sharing call _fsopen
(share read access)
Every 10 seconds in the
QuickTimeFileSink::continuePlaying() function I call completeOutputFile(), thus keeping the header up to date (with video length, etc).
Windows Media Player didn't like it (probably tried to get exclusive file access), but VLC was quite happy to read the file whilst I was still streaming into it.
A file is copied from machine1/dir1 to machine2/dir2. I have added a inotify watch on the dir2 for any new files created. Now if the file is large, it might take a few seconds to be fully on the new machine. If I'm not wrong, inotify will notify the application as soon as it detects an event. So if the file status has to be checked, How should it be done ?
Save downloaded file with temporary filename (or to other directory) and rename it to expected filename when file moved successfully.
Nginx for example use this method to store cached data
Caching data is first written to the temporary file which is then moved to the final location in a cache directory. Cheap and atomic rename syscall is performed instead of a full file copy So it's better to use the same file system in both locations
There's no way to answer this because it depends on the application environment and requirements. It might do to see that the file hasn't been modified for 60 seconds. It might require checking every few seconds. It depends.
Using IN_CLOSE_WRITE works if its only a scp from one machine to another. Otherwise, it depends on the way the file is uploaded from one machine to another. If its a one time open and close , IN_CLOSE_WRITE is the way to do it.
Both the answers above make sense depending on how we do it.