I want to implement an progress bar in my C++ windows application when downloading a file using WinHTTP. Any idea how to do this? It looks as though the WinHttpSetStatusCallback is what I want to use, but I don't see what notification to look for... or how to get the "percent downloaded"...
Help!
Thanks!
Per the docs:
WINHTTP_CALLBACK_STATUS_DATA_AVAILABLE
Data is available to be retrieved with
WinHttpReadData. The
lpvStatusInformation parameter points
to a DWORD that contains the number of
bytes of data available. The
dwStatusInformationLength parameter
itself is 4 (the size of a DWORD).
and
WINHTTP_CALLBACK_STATUS_READ_COMPLETE
Data was successfully read from the
server. The lpvStatusInformation
parameter contains a pointer to the
buffer specified in the call to
WinHttpReadData. The
dwStatusInformationLength parameter
contains the number of bytes read.
There may be other relevant notifications, but these two seem to be the key ones. Getting "percent" is not necessarily trivial because you may not know how much data you're getting (not all downloads have content-length set...); you can get the headers with:
WINHTTP_CALLBACK_STATUS_HEADERS_AVAILABLE
The response header has been received
and is available with
WinHttpQueryHeaders. The
lpvStatusInformation parameter is
NULL.
and if Content-Length IS available then the percentage can be computed by keeping track of the total number of bytes at each "data available" notification, otherwise your guess is as good as mine;-).
Related
I'm evaluating Chilkat and working with CkZip component to see if it feets our requirements. I have some questions derived of my tests:
When I put an event callback object, in: void FileZipped(const char
*path, _int64 fileSize, _int64 compressedSize, bool *abort);
I always get the same value for fileSize and compressedSize
(compression level was put to 9 and algo to deflate) Is it
intentionally / normal? Maybe it's a bug...
It seems that ProgressInfo event is received for the whole zip, so
when compressing a single large file and it tooks a bit of time, we
have no feedback about compression progress (ToBeZipped and
FileZipped received, with a difference of minutes).
I see the method AppendCompressed. So compressing a file with
CkCompression I can obtain compressed data and apply to
AppendCompressed directly. But documentation says CkCompression
handles "ppmd", "deflate", "zlib", "bzip2", or "lzw", and
AppendCompressed says that data should be unencrypted deflate data.
When we are building zipx files with lzma algo, AppendCompressed
data will took deflate compressed data and recompress with lzma? or
AppendCompressed data only takes deflate data so we cannot make a
lzma zipx file using AppendCompressed?.
Thanks in advance!
PD: Sorry, had to post here because chilkat forum says "This forum is closed. Post instead to stackoverflow.com with tag "chilkat""
Due lack of support/answer from Chilkat we have chosen to use another library that for now conforms to what we want/expect and does not have the mentioned failures.
Thanks!
I need a library for my C++ program.
But the problem, I don't know the name of this data type I want.
I have NPAPI plugin (I know this API is deprecated and removed from modern browsers) which issues to a server
HTTP range requests. Request is asyncronious and the data may arraive in any order with any chunks size.
So I need to track ranges I already have requested from a server.
For example, if initially I requested bytes [10-20] (inclusevely), then I requested [30-40] the data type I need should keep it as two intervals:
[10-20],[30-40]
But if I request [21-29] or even [15-35] it should be merged in one interval:
[10-20],[30-40] + [15-35] = [10-40]
Also I need a substraction when a requested block arrives:
[10-40] - [20-30] = [10-19],[31-40]
(requested - arrived = we're still waiting for)
I had a look at boost::numeric::intervals library but at first glance it is too big for this task (1583 files, 13 Mb of sources after './dist/bin/bcp numeric/interval ~/boost').
Also, GNU ddrescue has some similar arithmetics inside but the code isn't a library there, it coupled too much with the applications specifics.
UPDATE:
Here is what I've found on my way:
A container for integer intervals, such as RangeSet, for C++
https://en.wikipedia.org/wiki/Interval_tree
Boost.ICL
NCBI C++ Toolkit, CIntervalTree
I have a server sending a multi-dimensional character array
char buff1[][3] = { {0xff,0xfd,0x18} , {0xff,0xfd,0x1e} , {0xff,0xfd,21} }
In this case the buff1 carries 3 messages (each having 3 characters). There could be multiple instances of buffers on server side with messages of variable length (Note : each message will always have 3 characters). viz
char buff2[][3] = { {0xff,0xfd,0x20},{0xff,0xfd,0x27}}
How should I store the size of these buffers on client side while compiling the code.
The server should send information about the length (and any other structure) of the message with the message as part of the message.
An easy way to do that is to send the number of bytes in the message first, then the bytes in the message. Often you also want to send the version of the protocol (so you can detect mismatches) and maybe even a message id header (so you can send more than one kind of message).
If blazing fast performance isn't the goal (and you are talking over a network interface, which tends to be slower than computers: parsing may be cheap enough that you don't care), using a higher level protocol or format is sometimes a good idea (json, xml, whatever). This also helps with debugging problems, because instead of debugging your custom protocol, you get to debug the higher level format.
Alternatively, you can send some sign that the sequence has terminated. If there is a value that is never a valid sequence element (such as 0,0,0), you could send that to say "no more data". Or you could send each element with a header saying if it is the last element, or the header could say that this element doesn't exist and the last element was the previous one.
I know I can find other answers about this on SO, but I want clarifications from somebody who really knows MPEG-1/MPEG-2 (or MP3, obviously).
The start of an MPEG-1/2 frame is 12 set bits starting at a byte boundary, so bytes ff f*, where * is any nibble. Those 12 bits are called a sync word. This is a useful characteristic to find the start of a frame in any MPEG-1/2 stream.
My first question is: formally, can a false sync word be found or not in the payload of an MPEG-1/2 frame, outside its header?
If so, here's my second question: why does the sync word mechanism even exist then? If we cannot make sure that we found a new frame when reading fff, what is the purpose of this sync word?
Please do not even consider ID3 in your answer; I already know about sync words that can be found in ID3v2 payloads, but that's well documented.
I worked on MPEG-2 streams, more precisely Transport Streams (TS): I guess we can find similarities.
A TS is composed of Transport Packets, which have a header, starting with a sync byte 0x47.
We also can found 0x47 within the payload of the TP, but we know that it is not a sync byte because it is not aligned (TP have a fixed size of 188 bytes).
The sync word gives an entry point to someone that looks at the stream, and allows a program to synchronize his process with the stream, hence the name.
It also allows a fast browsing and parsing of the stream: in a TS you can jump from a packet to another (inspect header, check sync byte, skip 188 bytes and so on)
Finally it is a safety measure that helps you to spot errors (in the stream during transmission for example or in the process if a bug caused a bad alignment)
These argument are about TS but I think the same goes with your case : finding a sync word within a payload should not be an issue because you should always able to distinguish payload and header, most of the time with a length information (either because the size is fixed like in TP or because you have a TLV format).
can a false sync word be found or not in the payload of an MPEG-1/2
frame, outside its header?
According to this, "frame sync can be easily (and very frequently) found in any binary file." See the section titled "MPEG Audio Frame Header"
I confirmed this with an .mp3 song that I chose at random (stripped of ID3 tags). It had 5193 sync words, of which only 4898 were found to be valid (using code too long to be included here).
>>> f = open('notag.mp3', 'rb')
>>> r=f.read()
>>> r.count(b'\xff\xfb')
5193
why does the sync word mechanism even exist then? If we cannot make
sure that we found a new frame when reading fff, what is the purpose
of this sync word?
We can be (relatively) sure if we are checking the rest of the frame header, and not just the sync word. There are bits following the sync which can be used to:
identify a false positive or
give you useful info
With .mp3, you have to use those useful bits to calculate the size of the frame. By skipping ahead <frame-size> bytes before looking for the next sync word, you avoid any false syncs that may be present in the payload. See the section titled "How to calculate frame length" in that same link.
I've written a small program which is able to upload files to a server via ftp. Because of the large size of some files I want to create a progress bar for the user. So during the upload I need to know at certain intervals how many bytes have been sent to the server in order to derive the percentage of the file that has been uploaded. What I have tried so far:
While I call the function FtpPutFile() to upload the file, I spawn a thread with the following code:
hInternet = InternetOpen(NULL,INTERNET_OPEN_TYPE_DIRECT,NULL,NULL,0);
hFtpSession = InternetConnect(hInternet, ftpserver, port, user, pass, INTERNET_SERVICE_FTP, 0, 0);
int filesize = 0; // 2GB max
hFile = FtpOpenFile(hFtpSession,szFileTitle,GENERIC_READ,FTP_TRANSFER_TYPE_BINARY,0);
filesize = FtpGetFileSize(hFile,0);
cout << "Size: " << filesize << endl;
However this doesn't seem to work as filesize keeps returning a value of -1. I think this is due to the fact that I'm writing to a file (uploading part) and at the same time I'm trying to read it to get the file size. And I think this is not possible (please correct me if I'm wrong).
My main question: is there another way to create a progress bar for ftp uploading? Perhaps counting the bytes before they are uploaded using the function readBytesCount() (not sure if this is possible at all).
You want to:
Call InternetSetStatusCallback to set a function that will be called periodically during the transfer.
Pass a (non-zero) value the last parameter to FtpOpenFile. This will be passed back to your status callback function during the transfer.
Then, during the FTP operation, your callback function will be invoked periodically with information about the progress of the transfer, which it can then display to the user.
I don't believe this will let you show actual bytes as they're being transferred though -- if memory serves it mostly shows the discrete steps in a transfer, like opening the handle, resolving names, sending/receiving cookies, and finally closing the handle.
To deal with the actual bytes being written during the transfer of the file itself, you'd typically read a buffer-full of data from the local file, then write that buffer with InternetWriteFile. With that, you can compute the percentage transferred as the number of bytes written so far divided by the total size of the file (and multiply by 100).
Well, I resolved such a problem by sending the file by chunks and updating the progress in place of sending the whole file by one call to FtpPutFile
I mean:
FtpOpenFile(...)
for( ... )
{
InternetWriteFile(... dwChunkSize...)
UpdateProgressBar(dwChunkSize)
}
InternetCloseHandle(...)