how i can send gzip instade deflate by skip header - c++

i write a c++ http server ( Microsoft http server api )
that send html page file in gzip format
and gzip file is static
for example file page1.htm and page1.htm.gz are in same directory
according to
http://en.wikipedia.org/wiki/Gzip
i know that gzip is deflate with extra header
and deflate is part of gzip
how i can sending gzip instade deflate by skip header
fileHandle=CreateFile( "page1.htm.gz" ,dwAccess,dwShare,NULL,dwCreationFlags,dwAttributes,NULL);
....
ADD_KNOWN_HEADER(response, HttpHeaderContentEncoding, "deflate" );
HTTP_DATA_CHUNK dataChunk;
{
HTTP_DATA_CHUNK dataChunk;
response.EntityChunkCount = 1;
dataChunk.DataChunkType = HttpDataChunkFromFileHandle;
dataChunk.FromFileHandle.FileHandle =fileHandle;
dataChunk.FromFileHandle.ByteRange.StartingOffset.QuadPart =9;// 9 is gzip header len
dataChunk.FromFileHandle.ByteRange.Length.QuadPart = HTTP_BYTE_RANGE_TO_EOF;
response.pEntityChunks=&dataChunk;
}
.....

The deflate and gzip encoding are not quite the same thing, although the differences are minor.
When you are sending gzip, change your code to:
ADD_KNOWN_HEADER(response, HttpHeaderContentEncoding, "gzip" );
You should do that of course if gzip is listed in Accept-Encoding.
Here's an excerpt from the gzip FAQ:
“ What's the difference between the "gzip" and "deflate" HTTP 1.1
encodings?
"gzip" is the gzip format, and "deflate" is the zlib format. They
should probably have called the second one "zlib" instead to avoid
confusion with the raw deflate compressed data format. While the HTTP
1.1 RFC 2616 correctly points to the zlib specification in RFC 1950 for the "deflate" transfer encoding, there have been reports of
servers and browsers that incorrectly produce or expect raw deflate
data per the deflate specficiation in RFC 1951, most notably
Microsoft. So even though the "deflate" transfer encoding using the
zlib format would be the more efficient approach (and in fact exactly
what the zlib format was designed for), using the "gzip" transfer
encoding is probably more reliable due to an unfortunate choice of
name on the part of the HTTP 1.1 authors.”
http://www.gzip.org/zlib/zlib_faq.html#faq38

Related

Some questions about CkZip

I'm evaluating Chilkat and working with CkZip component to see if it feets our requirements. I have some questions derived of my tests:
When I put an event callback object, in: void FileZipped(const char
*path, _int64 fileSize, _int64 compressedSize, bool *abort);
I always get the same value for fileSize and compressedSize
(compression level was put to 9 and algo to deflate) Is it
intentionally / normal? Maybe it's a bug...
It seems that ProgressInfo event is received for the whole zip, so
when compressing a single large file and it tooks a bit of time, we
have no feedback about compression progress (ToBeZipped and
FileZipped received, with a difference of minutes).
I see the method AppendCompressed. So compressing a file with
CkCompression I can obtain compressed data and apply to
AppendCompressed directly. But documentation says CkCompression
handles "ppmd", "deflate", "zlib", "bzip2", or "lzw", and
AppendCompressed says that data should be unencrypted deflate data.
When we are building zipx files with lzma algo, AppendCompressed
data will took deflate compressed data and recompress with lzma? or
AppendCompressed data only takes deflate data so we cannot make a
lzma zipx file using AppendCompressed?.
Thanks in advance!
PD: Sorry, had to post here because chilkat forum says "This forum is closed. Post instead to stackoverflow.com with tag "chilkat""
Due lack of support/answer from Chilkat we have chosen to use another library that for now conforms to what we want/expect and does not have the mentioned failures.
Thanks!

decompressing IMAP deflated message

I have an issue trying to decompress an imap message compressed using deflate method. The things I've tryed so far were isolating one of the directions of an IMAP conversation (using wireshark's follow tcp function) and saving the message data in an raw format that I hope it contains only the deflated message part. I then found some programs like tinf (1st and 3rd example) and miniz (tgunzip example) and tryed to inflate back that file, but with no succes.
I am missing something? Thank you in advance.
tinf - http://www.ibsensoftware.com/download.html
Miniz - https://code.google.com/archive/p/miniz/source/default/source
Try piping that raw data to:
perl -MCompress::Zlib -pe 'BEGIN{$i = inflateInit(-WindowBits => -15)}
$_=$i->inflate($_)'
The important part is the -WindowBits => -15 that changes the expected format into a raw one without adler checksum.
(that's derived from the dovecot source, works for me on Thunderbird to gmail network capture).
From RFC4978 that specifies IMAP compression (emphasis mine):
When using the zlib library (see RFC1951), the functions
deflateInit2(), deflate(), inflateInit2(), and inflate() suffice to
implement this extension. The windowBits value must be in the range
-8 to -15, or else deflateInit2() uses the wrong format.
deflateParams() can be used to improve compression rate and resource
use. The Z_FULL_FLUSH argument to deflate() can be used to clear the
dictionary (the receiving peer does not need to do anything).

Windows 8 (Store App) in C - How to disable gzip compression?

I have an app that appears to enable gzip encoding by default while sending data to the server.
We tried disabling the gzip compression by explicitly using:
IXMLHttpRequest2::SetRequestHeader(L"Accept-Encoding", L"") (on the HTTP Request Object, of course)
This still doesn't seem to help. Is there anyway to disable GZIP being enabled in the HTTP-Request headers from the C++ App?
Thanks!
To ask a server to do not use a specific encoding you should provide a list of Accept-Encoding values. From section 14.11 of RFC2616 (HTTP/1.1) you see that it has one of forms (values are examples):
Accept-Encoding: compress, gzip
Accept-Encoding:
Accept-Encoding: *
Accept-Encoding: compress;q=0.5, gzip;q=1.0
Accept-Encoding: gzip;q=1.0, identity; q=0.5, *;q=0
If the content-coding is one of the content-codings listed in Accept-Encoding field, then it is acceptable, unless it is by a qvalue of 0. (As defined in section 3.9, a of 0 means "not acceptable.")
Then to ask the server to do not use gzip compression you should provide, instead of an empty string, this value for Accept-Encoding:
gzip;q=0
This will require the server to do not use it and but you have to provide another encoding. See section 3.5 for available encodings. Use the quality q parameter to inform the server about your preferences (do not forget that if it can't provide that encoding for your request it'll reply with error 406).
identity;q=1.0, gzip=0.5
In this way you ask to use identity encoding and, in case it's not available, you can accept a gzip encoding too (this will prevent the server to reply with an error if it, for any reason, can't use any other encoding for your request). You may try performance of other encodings too (compress and deflate, for example).
Code
Then, finally, you have to use IXMLHttpRequest2::SetRequestHeader(L"Accept-Encoding", L"identity;q=1.0, gzip=0.5"). In SetRequestHeader you see that it's an append to headers sent by default so if you specify an empty string actually the value won't be changed (actually how it is interpreted may depends on the server, I didn't find any proper specification about this, you may inspect both your HTTP request and response to check what is actually sent/received).
Old value: Accept-Encoding: compress
Call: IXMLHttpRequest2::SetRequestHeader(L"Accept-Encoding", L"")
New value: Accept-Encoding: compress

Decompression of payload in my webservice

How to process an incoming Payload(SOAP) which is COMPRESSED, in my webservices which I have created using Axis ???
Assuming you'r content is GZip Encoded
1.Check the content encoding. If it is gzip then
2.wrapper the input stream as a GZIPInputStream
read as usual from input stream.

URLOpenPullStream and gzip content download - need uncompressed data

I am using URLOpenPullStream along with a IBindStatusCallback and IHttpNegotiate callbacks to handle the negotiate, status, and data messages. Problem that I have is when the content is gzip (e.g. Content-Encoding: gzip). The data that I am receiving via OnDataAvailable is compressed. I need the uncompressed data. I am using BINDF_PULLDATA | BINDF_GETNEWESTVERSION | BINDF_NOWRITECACHE binding flags. I have read some posts that says it should support gzip format.
I initially tried to change the Accept-Encoding request header to specify that I did not want gzip but was unsucessful with this. I can change or add headers in BeginningTransaction, but it fails to change Accept-Content. I was able to change the User-Agent, and was able to add a new header, so the process works, but it would not override the Accept-Content for some reason.
Other option is to un-gzip the data myself. In a quick test using a C++ gzip library, I was able to ungzip the content. So, this may be an option. If this is what I need to do, what is the best method to detect it is gzip. I noticed that I got an OnProgress event with BINDSTATUS_MIMETYPEAVAILABLE and the text set to "application/x-gzip-compressed". Is this how I should detect it?
Looking for any solution to get around this problem! I do want to stay with URLOpenPullStream. This is a product that has been released and wish to keep changes to the minimum.
I will answer my own question after more research. It seems that the website that I having the issue with is returning something incorrect where IE, FF, and URLOpenPullStream do not recognize it as valid gzip content. The headers appear to be fine, e.g.
HTTP/1.1 200 OK
Content-Type: text/html; charset=iso-8859-1
Content-Encoding: none
Server: Microsoft-IIS/6.0
MSNSERVER: H: COL102-W41 V: 15.4.317.921 D: 2010-09-21T20:29:43
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 4258
Date: Wed, 27 Oct 2010 20:48:15 GMT
Connection: keep-alive
Set-Cookie: xidseq=4; domain=.live.com; path=/
Set-Cookie: LD=; domain=.live.com; expires=Wed, 27-Oct-2010 19:08:15 GMT; path=/
Cache-Control: no-cache, no-store
Pragma: no-cache
Expires: -1
Expires: -1
but URLOpenPullStream just downloaded it in raw compressed format, IE reports an error if you try to access the site, and FF shows garbage.
After doing a test with a site that does return valid gzip content, e.g. www.webcompression.org, then IE, FF, and URLOpenPullStream worked fine. So, it appears that URLOpenPullStream does support gzip content. In this case, it was transparent. In OnDataAvailable, I received the uncompressed data, and in the OnResponse, the headers did not show the Content-Encoding as gzip.
Unfortunately, this still did not solve my problem. I resolved by checking the response headers in OnResponse event. If the Content-Encoding was gzip, then I set a flag and when the download was complete, then used zlib gzip routines to uncompress the content. This seemed to work fine. This should be fine for my rare case since typically I should never receive a Content-Encoding : gzip in the OnResponse headers since the URLOpenPullStream handles the uncompress transparently.
Dunno :)