Deflating POCO HttpResponse with gzip cuts the content - c++

I am using POCO 1.7.8 to write a HTTP server. Problem is when using gzip for deflating the response data:
std::string content = "HELLO WORLD, THIS IS LONGISH STRING THAT IS CUT";
response->set("Content-Encoding", "gzip");
std::ostream& responseStream = response->send();
Poco::DeflatingOutputStream deflater(responseStream, Poco::DeflatingStreamBuf::STREAM_GZIP);
deflater << content;
deflater.close();
Response for the client is:
HELLO WORLD, THIS IS LONGISH STRING
Response headers:
Access-Control-Allow-Headers: origin, x-csrftoken, content-type, accept
Access-Control-Allow-Methods:POST, GET, OPTIONS
Access-Control-Allow-Origin: *
Access-Control-Max-Age: 86400
Connection: Close
Content-Encoding: gzip
Content-Language: en
Content-Length: 45
Content-Type: text/plain
Date: Tue, 09 Jan 2018 07:52:17 GMT
If I change this to use ZLIB and set the Content-Encoding to deflate, the whole response is correctly returned from the server:
std::string content = "HELLO WORLD, THIS IS LONGISH STRING THAT IS CUT";
response->set("Content-Encoding", "deflate");
std::ostream& responseStream = response->send();
Poco::DeflatingOutputStream deflater(responseStream, Poco::DeflatingStreamBuf::STREAM_ZLIB);
deflater << content;
deflater.close();
Response for the client is:
HELLO WORLD, THIS IS LONGISH STRING THAT IS CUT
Response headers:
Access-Control-Allow-Headers: origin, x-csrftoken, content-type, accept
Access-Control-Allow-Methods: POST, GET, OPTIONS
Access-Control-Allow-Origin: *
Access-Control-Max-Age: 86400
Connection: Close
Content-Encoding: deflate
Content-Language: en
Content-Length: 45
Content-Type: text/plain
Date: Tue, 09 Jan 2018 08:07:36 GMT
I tried to find examples how this should be done in the POCO server but couldn't find any and I am a bit stuck now with this. Any help is appreciated!

Are you sure you're setting the Content-Length header correctly for your compressed response? Alternatively try enabling chunked transfer encoding before calling send().
response->setChunkedTransferEncoding(true);

Related

OAuth2 with beast boost returns temporary redirect 307

I'm trying to implement an app with access to google drive in beast boost C++ usingoauth2 authentication.
https://developers.google.com/identity/protocols/OAuth2ForDevices
I try to get the user code in Postman with the following POST request:
POST /o/oauth2/device/code HTTP/1.1
Host: accounts.google.com
Content-Type: application/x-www-form-urlencoded
Cache-Control: no-cache
scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive.file&client_id=610490019085-l1v2mv7lv95lu7cr111vbtqmp1bigv42.apps.googleusercontent.com
And it works perfectly fine, returning:
{
"verification_url": "https://www.google.com/device",
"expires_in": 1800,
"interval": 5,
"device_code": "AH-1Ng0IgBnIXIUeltwDoL7AwNExNTT0rozdxD5FMnP8dip4DaDi8_XtzK2aVT92YKYmYa7KWqHRVqw5AmJCDtalzK3k6pvbFw",
"user_code": "LWZY-BDXD"
}
Now I want to do the same request in C++ using boost, with the following code snippet for the request:
http::request<http::string_body> req{http::verb::post, "/o/oauth2/device/code", 11};
req.set(http::field::host, "accounts.google.com");
req.set("Cache-Control", "no-cache");
req.set(http::field::content_type, "application/x-www-form-urlencoded");
req.body() = "scope=https://www.googleapis.com/auth/drive.file&client_id=610490019085-l1v2mv7lv95lu7cr111vbtqmp1bigv42.apps.googleusercontent.com";
req.prepare_payload();
This one returns:
HTTP/1.0 307 Temporary Redirect
Content-Type: text/html; charset=UTF-8
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Date: Mon, 14 May 2018 11:06:01 GMT
Location: https://accounts.google.com/o/oauth2/device/code
Content-Length: 232
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Server: GSE
<HTML>
<HEAD>
<TITLE>Temporary Redirect</TITLE>
</HEAD>
<BODY BGCOLOR="#FFFFFF" TEXT="#000000">
<H1>Temporary Redirect</H1>
The document has moved here.
</BODY>
</HTML>
Any ideas how I can get the JSON returned as with Postman?
Thank You!
Beast is a low-level protocol library, it doesn't know anything about resolving domain names or connecting sockets. It doesn't even know about TCP/IP, just how to serialize and deserialize HTTP/1 messages over objects which meet Asio's stream concept requirements (examples: SyncReadStream, or AsyncWriteStream). You have to handle redirects yourself. If you get a redirect response, extract the Location field value and parse the URI, resolve the domain, and then issue another GET request for the specified resource.
It is my hope that other folks (maybe you?) will build on top of beast and provide higher-level functionality like this in the form of open source libraries.

how to capture the Cookies in HttpQueryInfo using wininet c++

Currently I'm facing an issue in capturing the Cookies which is coming as part of response. I'm using WinInet for my connection.
After sending the request using HttpSendRequest, I'm using HttpQueryInfo to query the response in the below order
HTTP_QUERY_STATUS_CODE
HTTP_QUERY_SET_COOKIE
HTTP_QUERY_CONTENT_TYPE
HTTP_QUERY_CONTENT_LENGTH
Finally I'm reading body content using InternetReadFile.
Server has the loadbalance installed. all the response sent from LB has "Set-Cookies" enabled.
If I get any response without "Set-Cookies" in header then my code is reading the content successfully.
When there is "Set-Cookies" field in header then the issue comes. It is resulting out in "Header Not Found" when I query for "HTTP_QUERY_CONTENT_TYPE"
Below is the sample response header which is success
HTTP/1.1 200 OK
Cache-Control: private
Server: Microsoft-IIS/8.5
X-Powered-By: ASP.NET
Date: Thu, 17 Dec 2015 10:05:00 GMT
Content-Length: 107
Content-Type: text/html; Charset=utf-8
Via: 1.1 TestServer:80 (Cisco-WSA/8.8.0-085)
Connection: keep-alive
Below is the sample response header which is error out.
HTTP/1.1 200 OK
Cache-Control: private
Server: Microsoft-IIS/8.5
Set-Cookie: MYID=B3C5D2B2%2D0426%2D473F%2DACF6%2DF19BAC3F5A6C%3A%3A88757401; path=/
X-Powered-By: ASP.NET
Date: Thu, 17 Dec 2015 10:07:20 GMT
Content-Length: 277
Content-Type: text/html; Charset=utf-8
Via: 1.1 TestServer:80 (Cisco-WSA/8.8.0-085)
Connection: keep-alive
Work around found,
While querying the response header, if I add HTTP_QUERY_COOKIE in the sequence then the issue is getting resolved but I'm not sure on this is correct or not.
Can any one help me?
Why SET_COOKIE is not able to catch the "set-cookies"
What is the difference between HTTP_QUERY_SET_COOKIE and HTTP_QUERY_COOKIE?
What will be the impact if I use HTTP_QUERY_COOKIE to overcome this issue?
Thanks
Vijay

Not getting custom header value exposed in Access-Control-Expose-Headers

I am using LibCurl/C++ to send and receive html requests. One of the servers I access returns back this header information Access-Control-Expose-Headers: X-Custom1, where X-Custom1 is a custom header to provide data that I need to parse. But I am not seeing any value for X-Custom1.
I tried requesting it specifically by adding
Access-Control-Request-Headers: X-Custom1
header in my curl c++ request. But no use.
Response header data:
HTTP/1.1 200
Server: <server name>
Date: Thu, 01 Oct 2015 06:43:06 GMT
Content-Type: application/json
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: <cookie data>; expires=Fri, 30-Sep-16 06:43:05 GMT; path=/; domain=<domain name>; HttpOnly
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: Authorization,Content-Type,Accept,Origin,User-Agent
Access-Control-Allow-Methods: GET, POST, OPTIONS, PUT, DELETE
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: X-Custom1
Access-Control-Max-Age: 1728000
Etag: W/"<tag data>"
CF-RAY: <ray data>
Content-Encoding: gzip
Anyone know how to get the X-Custom1 data ?
I have been stuck at this for a long time now and need help.
Forgive me if this could be a cross post or wrongly tagged. I am not sure where else to post this question.
Thanks in advance.
Looks like I have been asking the wrong question after all.The response merely says X-Custom1 will be accessible from the server. It turned out that I had to request an entirely different url to get the actual valid data for that header.
Posting it here to let others seeking out similar questions know how it turned out.

How to invoke REST Webservice from the javascript by Preflight Request?

I am trying to invoke the service which was in another domain from the javascript itself. I could able to request the cross domain service . But I cant retrieve the information from the service. Some how I have been blocked by the same origin policy. Please help me to find the errors in the code.
My Client side Javascript Code :
var requestJsonData;
function crossDomainCall(){ ** It will be called by button click **
requestJsonData = createCORSRequest('POST', 'IPAddress/servicePath');
if (requestJsonData){
requestJsonData.onreadystatechange = handler;
requestJsonData.send();
}
else {
alert('Cross Domain Call is not invoked');
}
}
function handler(evtXHR) {
if(requestJsonData.readyState == 4) {
if(requestJsonData.status == 200) {
var response = requestJsonData.responseText;
}
else {
alert(" Invocation Errors Occured " + requestJsonData.readyState + " and the status is " + requestJsonData.status);
}
}
else {
alert("currently the application is at " + requestJsonData.readyState);
}
}
function createCORSRequest(method, url){
var xhr;
xhr = new XMLHttpRequest();
if ("withCredentials" in xhr){
xhr.open(method, url, true);
xhr.setRequestHeader('X-PINGOTHER', 'pingpong');
} else if (typeof XDomainRequest != "undefined"){
xhr = new XDomainRequest();
xhr.open(method, url);
} else {
xhr = null;
}
return xhr;
}
Service code :
#OPTIONS
#Path("/servicePath")
#Produces("*/*")
#Consumes("*/*")
public Response corsRequest() {
Response response = null;
ResponseBuilder builder = null;
builder = Response.ok();
builder.header("Access-Control-Allow-Headers", "X-PINGOTHER");
builder.header("Access-Control-Max-Age","1728000");
builder.header("Access-Control-Allow-Origin","Origin_Ip_Address");
builder.header("Access-Control-Allow-Methods", "POST, GET, OPTIONS");
builder.header("Content-Type","text/plain");
builder.header("Connection", "Keep-Alive");
response = builder.build();
System.out.println("Exited from Options method");
return response;
}
#POST
#Path("/servicePath")
#Produces("application/json")
public String drawRegions() {
System.out.println("Entered inside Post method");
// Some calculation to arrive jsonObject.
return jsonObject;
}
From the code, I have received the following as a results.
OPTIONS Method Request and Response Headers
Request Headers :
OPTIONS /SolartisGeoCodeLookUpService/Service/drawRegions HTTP/1.1
Host: Cross_Domain_IP_Address
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:25.0) Gecko/20100101 Firefox/25.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Origin: Origin_IP_Address
Access-Control-Request-Method: POST
Access-Control-Request-Headers: x-pingother
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Response Headers
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Access-Control-Allow-Headers: X-PINGOTHER
Connection: Keep-Alive
access-control-allow-origin: Origin_IP_Address
Access-Control-Max-Age: 1728000
Access-Control-Allow-Methods: POST, GET, OPTIONS
Content-Type: text/plain
Content-Length: 0
Date: Thu, 12 Dec 2013 12:39:27 GMT
Response Cache Header
Response Headers From Cache
Access-Control-Allow-Head... X-PINGOTHER
Access-Control-Allow-Meth... POST, GET, OPTIONS
Access-Control-Max-Age 1728000
Connection Keep-Alive
Content-Length 0
Content-Type text/plain
Date Thu, 12 Dec 2013 12:39:27 GMT
Server Apache-Coyote/1.1
access-control-allow-original Origin_IP_Address
POST Method Request and Response Headers
Request Headers
POST /servicePath HTTP/1.1
Host: crossDomain_IP_Address
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:25.0) Gecko/20100101 Firefox/25.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
X-PINGOTHER: pingpong
Origin: Origin_IP_Address
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Content-Length: 0
Response Headers
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: text/json
Content-Length: 128
Date: Thu, 12 Dec 2013 12:39:27 GMT
ADDITIONAL INFO
From the javascript two times the handler method has been called. At the First time, It is comeup with "currently the application is at 2" - readyState value. At the Second time, It is comeup with "Invocation Errors Occured 4(readyState value) and status code is 0 (response status code)". The second time response clearly says, invoking the service has been stopped by the same origin policy. But I dont know How to overcome from this problem and have to access the resource. Please help me by correcting my code.
Instead of dealing with X domain calls in javascript, why don't you develop a service local to your application that consumes the web service in the other domain, then you can call you local service from javascript.
I would suggest also, and alternatively, that you use jQuery to perform that Cross Domain Ajax call, see this link: http://www.pureexample.com/jquery/cross-domain-ajax.html.
There is no need to deal with XHR directly since you have jQuery to do it for you.
Hope this helps,
Regards.

determining HTTP version using raw sockets in C

I am trying to create raw sockets in C/C++, then create a request message and then send that message to a target server. If the port I specify happens to be 80 I want to send a HTTP request to determine the HTTP version that the target server is using. For e.g I send GET HTTP/1.0 to www.google.com.
For some servers it returns HTTP/1.1 400 bad request. While in some cases it responds with an XML message. I know the GET command is wrong since I am not specifying any object to actually GET. So is there a generic way to do this?
Try:
HEAD / HTTP/1.0\r\n
\r\n
Or:
GET / HTTP/1.0\r\n
\r\n
The first line of the servers response should contain the HTTP version. Note that some servers will return 400 Bad Request if the Host: <hostname> is omitted from the header (which is not required in 1.0, but in 1.1). I would do:
Try:
HEAD / HTTP/1.0\r\n
Host: <hostname>\r\n
\r\n
Or:
GET / HTTP/1.0\r\n
Host: <hostname>\r\n
\r\n
If you don't require the message body, you should use HEAD as it will require less data to receive.
You will have to progressively try each version of HTTP. For example, if I query google.com with HTTP 1.0, it will respond with HTTP 1.0:
$ printf "HEAD / HTTP/1.0\nHost: google.com\n\n" | nc google.com 80
HTTP/1.0 301 Moved Permanently
Location: http://www.google.com/
Content-Type: text/html; charset=UTF-8
Date: Tue, 26 Nov 2013 19:44:42 GMT
Expires: Thu, 26 Dec 2013 19:44:42 GMT
Cache-Control: public, max-age=2592000
Server: gws
Content-Length: 219
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Alternate-Protocol: 80:quic
If I query it with HTTP 1.1, it will respond with HTTP 1.1:
$ printf "HEAD / HTTP/1.1\nHost: google.com\n\n" | nc google.com 80
HTTP/1.1 301 Moved Permanently
Location: http://www.google.com/
Content-Type: text/html; charset=UTF-8
Date: Tue, 26 Nov 2013 19:44:47 GMT
Expires: Thu, 26 Dec 2013 19:44:47 GMT
Cache-Control: public, max-age=2592000
Server: gws
Content-Length: 219
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Alternate-Protocol: 80:quic
GET / HTTP/1.0\r\n
Host: www.google.com\r\n
\r\n
This is a basic HTTP request. Alternatively, you can use HEAD instead of GET, sinse you only want the headers, but some basic servers don't recognize HEAD as a valid method.
Not every server will support HTTP/1.0, they will most likely ignore this and answer as HTTP/1.1, others will just ape the version in your request and not really mean it.
It can be frustrating to try to determine the HTTP version in a random server. Perhaps you should instead use 1.1 in the request and see if the server answers with 1.0, I believe it may be the safest way to know if the server is at least giving a damn.