I ran into following problem which I have no clue about the how and why:
I have written a webservice in C++. I have also used an example for a client that should test for the response.
client.cpp
#include <memory>
#include <future>
#include <cstdio>
#include <cstdlib>
#include <restbed>
using namespace std;
using namespace restbed;
void print( const shared_ptr< Response >& response )
{
fprintf( stderr, "*** Response ***\n" );
fprintf( stderr, "Status Code: %i\n", response->get_status_code( ) );
fprintf( stderr, "Status Message: %s\n", response->get_status_message( ).data( ) );
fprintf( stderr, "HTTP Version: %.1f\n", response->get_version( ) );
fprintf( stderr, "HTTP Protocol: %s\n", response->get_protocol( ).data( ) );
for ( const auto header : response->get_headers( ) )
{
fprintf( stderr, "Header '%s' > '%s'\n", header.first.data( ), header.second.data( ) );
}
auto length = response->get_header( "Content-Length", 0 );
Http::fetch( length, response );
fprintf( stderr, "Body: %.*s...\n\n", length, response->get_body( ).data( ) );
}
int main( ){
auto request = make_shared<Request>(
Uri("http://localhost:3030/apps/17?start_date=2017-02-01&end_date=2017-01-31&kpis=foo,bar"));
request->set_header("Accept", "application/json");
request->set_header("Host", "localhost");
auto response = Http::sync(request);
print(response);
auto future = Http::async(request, [](const shared_ptr<Request>, const shared_ptr<Response> response){
fprintf(stderr, "Printing async response\n");
print(response);
});
future.wait();
return EXIT_SUCCESS;
}
And my service streams the response in chunks (or is supposed to stream the response in chunks)
First, it streams the parameters of the request such as start_date, end_date and kpis. Second, the requested data is streamed.
Here is the stream_result_parameter function:
void stream_result_parameter(std::shared_ptr<restbed::Session> session, const Parameter params,
const std::string endpoint, const std::string mime_type)
{
std::stringstream stream;
if(mime_type.compare("application/json") == 0)
{
std::vector<std::string> kpis = params.get_kpis();
stream << "\n{\n"
<< "\"result_parameter\":{\n"
<< "\"App\":" << params.get_app_id() << ",\n"
<< "\"start_date\":" << params.get_start_date() << ",\n"
<< "\"end_date\":" << params.get_end_date() << ",\n"
<< "\"Kpis\":[";
for(std::vector<std::string>::iterator kpi = kpis.begin(); kpi != kpis.end(); ++kpi)
{
if(kpi == kpis.end()-1)
{
stream << *kpi << "]\n},";
}
else
{
stream << *kpi << ",";
}
}
}
else
{
if(endpoint.compare("app") == 0 )
{
stream << "Called App Endpoint App: "
<< std::to_string(params.get_app_id())
<< "\r\nStart Date: "
<< params.get_start_date()
<< "\r\nEnd Date: "
<< params.get_end_date()
<<"\n";
}
else
{
stream << "Called Cohorts Endpoint App: "
<< std::to_string(params.get_app_id())
<< "\r\nStart Date: "
<< params.get_start_date()
<< "\r\nEnd Date: "
<< params.get_end_date()
<<"\n";
}
}
session->yield(200, "\r"+stream.str()+"\r",
{ { "Content-Length", std::to_string( stream.str().length())},
{ "Content-Type", mime_type },
{ "Connection", "keep-alive" } });
}
Now, my problem occured after I added the Content-Length to it where it simply stops and closes the conversation between client and him(the service). curl gives me following error
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 3030 (#0)
> GET /apps/17?start_date=2017-02-01&end_date=2017-01-31&kpis=foo,bar HTTP/1.1
> Host: localhost:3030
> User-Agent: curl/7.54.0
> Accept:text/csv
>
< HTTP/1.1 200 OK
< Connection: keep-alive
< Content-Length: 94
< Content-Type: text/csv
<
* Excess found in a non pipelined read: excess = 2, size = 94, maxdownload = 94, bytecount = 0
Called App Endpoint App: 17
Start Date: Wed. February 1 2017
* Connection #0 to host localhost left intact
End Date: Tue. January 31 2017
Does the excess have anything to do with it?
Lastly, I want to show you the output from my test client and curl if I take the content-length away.
client.cpp output:
*** Response ***
Status Code: 200
Status Message: OK
HTTP Version: 1.1
HTTP Protocol: HTTP
Header 'Connection' > 'keep-alive'
Header 'Content-Type' > 'application/json'
Body: ...
Printing async response
*** Response ***
Status Code: 200
Status Message: OK
HTTP Version: 1.1
HTTP Protocol: HTTP
Header 'Connection' > 'keep-alive'
Header 'Content-Length' > '64'
Header 'Content-Type' > 'application/json'
Body:
"result_set":{
"i":1,
"j values": [
{"j":1,"kpi_values":[1,1]}...
But curl gives me everything I need:
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 3030 (#0)
> GET /apps/17?start_date=2017-02-01&end_date=2017-01-31&kpis=foo,bar HTTP/1.1
> Host: localhost:3030
> User-Agent: curl/7.54.0
> Accept:text/csv
>
< HTTP/1.1 200 OK
< Connection: keep-alive
< Content-Type: text/csv
* no chunk, no close, no size. Assume close to signal end
<
Called App Endpoint App: 17
Start Date: Wed. February 1 2017
End Date: Tue. January 31 2017
1,1,1,1
1,2,1,2
1,3,1,3
1,4,1,4
1,5,1,0
1,6,1,1
1,7,1,2
1,8,1,3
1,9,1,4
1,10,1,0
2,1,2,1
2,2,2,2
2,3,2,3
2,4,2,4
2,5,2,0
2,6,2,1
2,7,2,2
2,8,2,3
2,9,2,4
2,10,2,0
(Please note I did not want to copy 17400 lines so its just part of the complete and rightful output)
Maybe I am violating some rule or missing something else but I just can't think of it. Thanks in advance
UPDATE:
The excess message is gone once I accounted for the "/r"s but still the response is sent and no more chunks can follow:
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 3030 (#0)
> GET /apps/17?start_date=2017-02-01&end_date=2017-01-31&kpis=foo,bar HTTP/1.1
> Host: localhost:3030
> User-Agent: curl/7.54.0
> Accept:text/csv
>
< HTTP/1.1 200 OK
< Connection: keep-alive
< Content-Length: 96
< Content-Type: text/csv
<
Called App Endpoint App: 17
Start Date: Wed. February 1 2017
End Date: Tue. January 31 2017
* Connection #0 to host localhost left intact
Once the response is received your application will end.
auto future = Http::async(request, [](const shared_ptr<Request>, const shared_ptr<Response> response){
fprintf(stderr, "Printing async response\n");
print(response);
});
future.wait();
Please see github issue for final result.
Related
I am using an Adafruit Feather Huzzah ESP8266 and the ArduinoJson library to parse the response of an HTTP request. I am successfully getting the expected responses but the deserialization process fails every other time it iterates through the loop. I think it may be a memory allocation issue but I cannot resolve the problem. Any help would be greatly appreciated.
I have tried using dynamic vs static documents and initialising them inside/outside the loop without success. I have also tried the doc.clear() method to release the memory but still no luck. Below is my loop with the connection parameters missing:
void loop() {
WiFiClientSecure client;
for (int i = 0; i <= numOfInstallations - 1; i++) {
String url = "/v2/installations/" + String(idSites[i]) +
"/widgets/Graph?attributeCodes[]=SOC&instance=" + String(instance[i]) +
"&start=" + String(startTime) +
"&end=" + String(endTime);
client.print(String("GET ") + url + " HTTP/1.1\r\n" +
"Host: " + host + "\r\n" +
"X-Authorization: Token " + token + "\r\n" +
"Connection: keep-alive\r\n\r\n");
Serial.println(String("GET ") + url + " HTTP/1.1\r\n" +
"Host: " + host + "\r\n" +
"X-Authorization: Token " + token + "\r\n" +
"Connection: keep-alive\r\n\r\n");
Serial.println("request sent");
delay(500);
// Ignore the response headers
char endOfHeaders[] = "\r\n\r\n";
if (!client.find(endOfHeaders)) {
Serial.println(F("No headers"));
return;
}
const size_t capacity = 5*JSON_ARRAY_SIZE(2) + JSON_ARRAY_SIZE(5) + 2*JSON_OBJECT_SIZE(1) + 2*JSON_OBJECT_SIZE(2) + JSON_OBJECT_SIZE(4) + 130;
DynamicJsonDocument doc(capacity); // Json document setup
// Get the Json data from the response
DeserializationError error = deserializeJson(doc, client);
if (error) {
Serial.print(F("deserializeJson() failed: "));
Serial.println(error.c_str());
Serial.print("\r\n");
}
else {
Serial.println("deserializeJson() successful\r\n");
}
}
}
I am expecting the deserialization process to be successful each time but here is the output:
GET /v2/installations/xxxxx/widgets/Graph?attributeCodes[]=SOC&instance=215&start=1555070100&end=1555070400 HTTP/1.1
Host: vrmapi.victronenergy.com
X-Authorization: Token c324f8876e672ad1797cd69a9d9f62611507d25aa5a0b1ff40f9fb524d96f2fc
Connection: keep-alive
request sent
deserializeJson() successful
GET /v2/installations/xxxxx/widgets/Graph?attributeCodes[]=SOC&instance=258&start=1555070100&end=1555070400 HTTP/1.1
Host: vrmapi.victronenergy.com
X-Authorization: Token XXXXXXXXXX
Connection: keep-alive
request sent
deserializeJson() failed: InvalidInput
GET /v2/installations/xxxxx/widgets/Graph?attributeCodes[]=SOC&instance=258&start=1555070100&end=1555070400 HTTP/1.1
Host: vrmapi.victronenergy.com
X-Authorization: Token XXXXXXXXXX
Connection: keep-alive
request sent
deserializeJson() successful
GET /v2/installations/xxxxx/widgets/Graph?attributeCodes[]=SOC&instance=258&start=1555070100&end=1555070400 HTTP/1.1
Host: vrmapi.victronenergy.com
X-Authorization: Token XXXXXXXXXX
Connection: keep-alive
request sent
deserializeJson() failed: InvalidInput
GET /v2/installations/xxxxx/widgets/Graph?attributeCodes[]=SOC&instance=258&start=1555070100&end=1555070400 HTTP/1.1
Host: vrmapi.victronenergy.com
X-Authorization: Token XXXXXXXXXX
Connection: keep-alive
request sent
deserializeJson() successful
I am trialling a little project for Philips Hue and learning c++ at the same time. What I am trying to do is act a proxy between a client and the bridge. The app runs as a bridge as far as any clients are concerned, receives the request and passes this on to the bridge.
I have a weird problem where if the app runs normally, I only get a HTTP 200 OK response, nothing else, if I step through the code, I get the full response.
Below is my code, there is no thread, there are no classes, its all being done in the main method.
WindowsSocket socketManager(&bitsLibrary);
if (!socketManager.createSocket(AF_INET, SOCK_STREAM, IPPROTO_TCP, 80, 1024))
{
cout << "Failed to create socket" << endl;
}
socketManager.bindAndStartListening();
WindowsSocket bridgeSocketManager(&bitsLibrary);
while (true)
{
sockaddr_in clientAddr;
memset(&clientAddr, 0, sizeof(sockaddr_in));
SOCKET clientSocket = socketManager.acceptClientAndReturnSocket(&clientAddr);
this_thread::sleep_for(chrono::milliseconds(500));
string received = socketManager.receiveDataOnSocket(&clientSocket);
bitsLibrary.writeToLog(received);
struct sockaddr_in server;
server.sin_family = AF_INET;
//server.sin_addr.s_addr = inet_addr("139.162.223.149");
server.sin_addr.s_addr = inet_addr("192.168.1.67");
server.sin_port = htons(80);
bridgeSocketManager.createSocket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
this_thread::sleep_for(chrono::milliseconds(500));
int result = connect(*bridgeSocketManager.returnSocket(), (struct sockaddr *)&server, sizeof(server));
if (result < 0)
{
cout << "Connect Failed: " << WSAGetLastError() << endl;
return EXIT_FAILURE;
}
this_thread::sleep_for(chrono::milliseconds(500));
SOCKET * bridgeSocket = bridgeSocketManager.returnSocket();
this_thread::sleep_for(chrono::milliseconds(500));
socketManager.sendToSocket(bridgeSocket, received);
string reply = socketManager.receiveDataOnSocket(bridgeSocket);
//boost::replace_all(reply, "Host: 192.168.1.70", "Host: 192.168.1.67");
bitsLibrary.writeToLog(reply);
this_thread::sleep_for(chrono::milliseconds(1000));
int sent = socketManager.sendToSocket(&clientSocket, reply);
this_thread::sleep_for(chrono::milliseconds(500));
bridgeSocketManager.closeSocket();
this_thread::sleep_for(chrono::milliseconds(500));
socketManager.closeSocket(&clientSocket);
/*while (true)
{
SOCKET clientSocket = socketManager.acceptClientAndReturnSocket(&clientAddr);
SocketProcessor socketProcessor;
socketProcessor.startThread(socketManager, clientSocket);
}*/
}
socketManager.closeSocket();
My socket receive method is as follows:
std::string WindowsSocket::receiveDataOnSocket(SOCKET *socket)
{
if (*socket != -1)
{
string receivedData = "";
char *temp = NULL;
int bytesReceived = 0;
do
{
bytesReceived = recv(*socket, this->buffer, this->bufferLength, 0);
if (bytesReceived == SOCKET_ERROR)
{
string socketError = this->getErrorStringFromErrorCode(WSAGetLastError()).c_str();
stringstream logstream;
logstream << "Failed to receive data on socket.The socket will now be closed and cleanup performed. Error: " << socketError;
this->bitsLibrary->writeToLog(logstream.str(), "WindowsSocket", "receiveDataOnSocket");
closesocket(*socket);
WSACleanup();
throw SocketException(socketError.c_str());
return "";
}
//If we got here, then we should be able to get some data
temp = new char[bytesReceived + 1];
//memset(&temp, 0, bytesReceived + 1);
strncpy(temp, this->buffer, bytesReceived);
temp[bytesReceived] = '\0'; //Add a null terminator to the end of the string
receivedData.append(temp);
temp = NULL;
//Now clear the buffer ready for more data
memset(this->buffer, 0, this->bufferLength);
cout << "Bytes Received: " << bytesReceived << " BUffer Length: " << this->bufferLength << endl;
} while (bytesReceived == this->bufferLength && bytesReceived >= 0); //Keep going until the received bytes is less than the buffer length
return receivedData;
}
else
{
stringstream logstream;
logstream << "Can't receive on socket as already be closed";
throw SocketException(logstream.str().c_str());
}
}
The send method looks as follows:
int WindowsSocket::sendToSocket(SOCKET *clientSocket, string dataToSend)
{
//dataToSend.append("\r\n");
int sentBytes = send(*clientSocket, dataToSend.c_str(), dataToSend.length(), 0);
if (sentBytes == SOCKET_ERROR)
{
throw SocketException(this->getErrorStringFromErrorCode(WSAGetLastError()).c_str());
}
return sentBytes;
}
When I run the app normally in Visual Studio with no break points, I get the following output:
03/02/2017 22:08:16: WindowsSocket/bindAndStartListening: Socket has binded and is now listening
Bytes Received: 413 BUffer Length: 1024
Receiving data
GET /api/nouser/config HTTP/1.1
Host: 192.168.1.70
Connection: keep-alive
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-GB,en;q=0.8
03/02/2017 22:08:22: BaseSocket/createSocket: Creating buffer of length 1024
Bytes Received: 17 BUffer Length: 1024
03/02/2017 22:08:23: HTTP/1.1 200 OK
Sent: 17
If I set a breakpoint and then step through the code I get the following:
03/02/2017 22:09:03: WindowsSocket/bindAndStartListening: Socket has binded and is now listening
Bytes Received: 413 BUffer Length: 1024
GET /api/nouser/config HTTP/1.1
Host: 192.168.1.70
Connection: keep-alive
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-GB,en;q=0.8
03/02/2017 22:09:09: BaseSocket/createSocket: Creating buffer of length 1024
Bytes Received: 630 BUffer Length: 1024
03/02/2017 22:09:17: HTTP/1.1 200 OK
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Expires: Mon, 1 Aug 2011 09:00:00 GMT
Connection: close
Access-Control-Max-Age: 3600
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
Access-Control-Allow-Methods: POST, GET, OPTIONS, PUT, DELETE, HEAD
Access-Control-Allow-Headers: Content-Type
Content-type: application/json
{"name":"Philips hue","datastoreversion":"59","swversion":"01036659","apiversion":"1.16.0","mac":"00:17:88:1a:1f:43","bridgeid":"001788FFFE1A1F43","factorynew":false,"replacesbridgeid":null,"modelid":"BSB001"}
Sent: 630
Notice when I don't do a break point I receive only 17 bytes of the HTTP 200 OK but when I breakpoint and step through the code I then get over 600 bytes and receive everything that I was expecting.
I can't see what the problem is, I've put sleeps in expecting this would "fix" the issue but even the sleeps make no difference.
send()/recv() are low-level calls to work with sockets. To properly handle HTTP, which is quite a complex protocol, you should be aware of HTTP specification, see about-to-become-obsolete RFC2616.
Your best bet is to set condition in do-while loop of receiveDataOnSocket() to while (bytesReceived > 0). It will only work, if server closes connection after sending the response.
If persistent connection is used, then the next best thing you can do is to use non-blocking sockets (at least for accepting data from both client and server) and as data arrives, forward them to other side. It may work, but may also fail.
The next thing is actually implementing HTTP, which is too complex for C++ tutorial.
I've been trying to send HTTP POST Request to facebook without success i get this response from the server :
HTTP/1.1 400 Bad Request Content-Type: text/html; charset=utf-8 Date:
Sat, 10 Dec 2016 21:28:17 GMT Connection: close Content-Length: 2959
Facebook | Error
Sorry, something went wrong We're working on it and we'll get it fixed
as soon as we can
My code
#include <iostream>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <stdlib.h>
#include <unistd.h>
#include <netdb.h>
#include <fstream>
using namespace std;
int main()
{
int s, error;
struct sockaddr_in addr;
s = socket(AF_INET, SOCK_STREAM, 0);
if(s <0)
{
cout<<"Error 01: creating socket failed!\n";
close(s);
return 1;
}
addr.sin_family = AF_INET;
addr.sin_port = htons(80);
inet_aton("31.13.90.36",&addr.sin_addr);
error = connect(s,(sockaddr*)&addr,sizeof(addr));
if(error!=0)
{
cout<<"Error 02: conecting to server failed!\n";
close(s);
return 1;
}
const int msgSize = 1042;
char msg[] = "POST /login.php?login_attempt=1 \r\n"
"HTTP/1.1\r\n"
"HOST: facebook.com\r\n\r\n"
"Content-type: application/x-www-form-urlencoded\r\n"
"Content-Length: 41\r\n"
"email=lel#gmail.com&pass=test123&submit=1\r\n" ;
char answ[1042];
//cin.getline(&msg[0],256);
send(s,msg,strlen(msg),0);
ssize_t len;
while((len = recv(s, msg, strlen(msg), 0)) > 0)
{
cout.write(msg, len);
std::cout << std::flush;
}
if(len < 0)
{
cout << "error";
}
close(s);
}
What I did wrong?
There are several errors in your message. This is what you send according to your code:
1 POST /login.php?login_attempt=1 \r\n
2 HTTP/1.1\r\n
3 HOST: facebook.com\r\n\r\n
4 Content-type: application/x-www-form-urlencoded\r\n
5 Content-Length: 41\r\n
6 email=lel#gmail.com&pass=test123&submit=1\r\n
Instead it should be like this:
1 POST /login.php?login_attempt=1 HTTP/1.1\r\n
2 HOST: facebook.com\r\n
3 Content-type: application/x-www-form-urlencoded\r\n
4 Content-Length: 41\r\n
5 \r\n
6 email=lel#gmail.com&pass=test123&submit=1
In detail:
Line 1 and 2 should be a single line, i.e. "method path HTTP-version"
Line 3 should not contain multiple \r\n
instead the empty line \r\n should be after all HTTP headers (new line 5)
the body in line 6 should contain only the data covered by the content-length. The 41 bytes you set there don't include the \r\n you send
Apart from that you don't properly parse the response and instead expect the server to close the connection once the response is done. Since you are using HTTP/1.1 you implicitly use persistent connections (HTTP keep-alive) so the server might actually wait for more requests within the same TCP connection and not close the connection immediately.
I really recommend that you study the standard of HTTP instead of guessing how the protocol might work.
i'm developing a small application.
For testing purpouse i'm tryng to "ping" google using libcurl while behind an ntlm proxy.
This is my c++ code:
CURLcode testConnection(void)
{
CURL *curl;
CURLcode res = CURLE_OK;
res = curl_global_init(CURL_GLOBAL_ALL);
curl = curl_easy_init();
if (curl)
{
cout << "Url: " << curl_easy_strerror(curl_easy_setopt(curl, CURLOPT_URL, "http://www.google.com")) << "\n";
cout << "T-out: " << curl_easy_strerror(curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, 3)) << "\n";
cout << "No_body: " << curl_easy_strerror(curl_easy_setopt(curl, CURLOPT_NOBODY, true)) << "\n";
cout << "Proxy Url: " << curl_easy_strerror(curl_easy_setopt(curl, CURLOPT_PROXY, "proxyrm.wind.root.it")) << "\n";
cout << "Proxy Port: " << curl_easy_strerror(curl_easy_setopt(curl, CURLOPT_PROXYPORT, 8080)) << "\n";
cout << "Ntml: " << curl_easy_strerror(curl_easy_setopt(curl, CURLOPT_HTTPAUTH, CURLAUTH_NTLM)) << "\n";
cout << "Verbose: " << curl_easy_strerror(curl_easy_setopt(curl, CURLOPT_VERBOSE, true)) << "\n";
res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
}
else
res = CURLE_FAILED_INIT;
curl_global_cleanup();
return res;
}
All thi went good, since the verbose output is the following.
Url: No error
T-out: No error
No_body: No error
Proxy Url: No error
Proxy Port: No error
Ntml: No error
Verbose: No error
* About to connect() to proxy myroxy port 8080 (#0)
* Trying IP...
* Connected to myroxy (IP) port 8080 (#0)
> HEAD http://www.google.com HTTP/1.1
Host: www.google.com
Accept: */*
Proxy-Connection: Keep-Alive
< HTTP/1.1 407 Proxy Authentication Required
< Proxy-Authenticate: NTLM
< Proxy-Authenticate: BASIC realm="windroot"
< Cache-Control: no-cache
< Pragma: no-cache
< Content-Type: text/html; charset=utf-8
* HTTP/1.1 proxy connection set close!
< Proxy-Connection: close
< Set-Cookie: BCSI-CS-602d36a7505d346e=2; Path=/
< Connection: close
< Content-Length: 989
<
* Closing connection 0
Curl Final: No error
But, if i try to ping https://google.com the result become this
Url: No error
T-out: No error
No_body: No error
Proxy Url: No error
Proxy Port: No error
Ntml: No error
Ntml: No error
Ntml: No error
Verbose: No error
* About to connect() to proxy myroxy port 8080 (#0
* Trying IP...
* Connected to myroxy (IP) port 8080 (#0)
* Establish HTTP proxy tunnel to www.google.com:443
> CONNECT www.google.com:443 HTTP/1.1
Host: www.google.com:443
Proxy-Connection: Keep-Alive
< HTTP/1.1 407 Proxy Authentication Required
< Proxy-Authenticate: NTLM
< Proxy-Authenticate: BASIC realm="windroot"
< Cache-Control: no-cache
< Pragma: no-cache
< Content-Type: text/html; charset=utf-8
< Proxy-Connection: close
< Set-Cookie: BCSI-CS-602d36a7505d346e=2; Path=/
< Connection: close
< Content-Length: 1135
<
* Ignore 1135 bytes of response-body
* Received HTTP code 407 from proxy after CONNECT
* Connection #0 to host myroxy left intact
Curl Final: Failure when receiving data from the peer
Using curl in command line allow me to ping https (i have to specify the -k argument) but i don't know if this is really relevant. Someone can help me to figure out what's happening? And how to avoid that?
Solved: i was missing this instruction
curl_easy_setopt(ctx, CURLOPT_PROXYUSERPWD, ":");
This (i think) tell curl to use Username an Password retrieved from SO passing througt ntlm auth
I am writing an application for work that needs to perform HTTP requests to a server, and get a response, in JSON back.
At the moment, my code connects to the server, and gets a response back, which is great. However, I also need to send data to the server, which will process it, and send me the jSON.
My problem is that I am able to send, thanks to POSTFIELDS, only a single "field", and won't get any response if I insert more that one.
The code is the following:
// writefunc works, so there is no point adding its code in here
size_t Simulator::writefunc(void *ptr, size_t size, size_t nmemb, struct string *buffer_in);
void Simulator::move(Player *player)
{
std::ostringstream ss_url;
ss_url << API_URL << "functionCalled";
char const *url = ss_url.str().c_str();
std::ostringstream ss_postfields;
// This will not work - And all values do NOT contain any space
// The server do NOT receive any data (in POST and GET)
ss_postfields << "user_name=" << player->getName()
<< "&user_secret=" << player->secret()
<< "&app_id=" << player->getApp_id()
<< "&lat=" << player->getLocation()->getLat()
<<"&lon=" << player->getLocation()->getLon();
// This will not work either
// ss_postfields << "user_name=nicolas&app_id=2";
// This works and will send the data to the server, which will receive and process it.
// ss_postfields << "user_name=" << player->getName();
const char *postfields = ss_postfields.str().c_str();
CURL *curl_handle;
curl_global_init(CURL_GLOBAL_ALL);
curl_handle = curl_easy_init();
if(curl_handle){
struct string s;
init_string(&s);
curl_easy_setopt(curl_handle, CURLOPT_URL, url);
curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, writefunc);
curl_easy_setopt(curl_handle, CURLOPT_NOPROGRESS, 1L);
curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, &s);
curl_easy_setopt(curl_handle, CURLOPT_POSTFIELDS, postfields);
curl_easy_setopt(curl_handle, CURLOPT_POSTFIELDSIZE, strlen(postfields));
curl_easy_setopt(curl_handle, CURLOPT_POST, 1L);
CURLcode res = curl_easy_perform(curl_handle);
if(CURLE_OK != res)
{
printf("Error: %s\n", strerror(res));
exit(0);
}
printf("%s\n", s.ptr);
curl_easy_cleanup(curl_handle);
curl_global_cleanup();
}
}
I thought I would also give the output of CURLOPT_HEADER and CURLOPT_VERBOSE for when I send only 1 value, or multiple values:
When I Send One value only:
* About to connect() to localhost port 8888 (#0)
* Trying ::1...
* connected
* Connected to localhost (::1) port 8888 (#0)
> POST [api_url] HTTP/1.1
Host: localhost:8888
Accept: */*
Content-Length: 22
Content-Type: application/x-www-form-urlencoded
* upload completely sent off: 22 out of 22 bytes
< HTTP/1.1 200 OK
< Date: Sun, 24 Nov 2013 17:25:14 GMT
< Server: Apache
< X-Powered-By: PHP/5.3.20
< Cache-Control: no-cache
< X-Debug-Token: a41727
< Transfer-Encoding: chunked
< Content-Type: application/json
<
* Connection #0 to host localhost left intact
0
HTTP/1.1 200 OK
Date: Sun, 24 Nov 2013 17:25:14 GMT
Server: Apache
X-Powered-By: PHP/5.3.20
Cache-Control: no-cache
X-Debug-Token: a41727
Transfer-Encoding: chunked
Content-Type: application/json
[ OUTPUT FROM SERVER HERE ]
* Closing connection #0
And when I send multiple values:
* About to connect() to localhost port 8888 (#0)
* Trying ::1...
* connected
* Connected to localhost (::1) port 8888 (#0)
> POST [api_url] HTTP/1.1
Host: localhost:8888
Accept: */*
Content-Length: 1
Content-Type: application/x-www-form-urlencoded
* upload completely sent off: 1 out of 1 bytes
< HTTP/1.1 200 OK
< Date: Sun, 24 Nov 2013 17:30:13 GMT
< Server: Apache
< X-Powered-By: PHP/5.3.20
< Cache-Control: no-cache
< X-Debug-Token: d1947e
< Transfer-Encoding: chunked
< Content-Type: application/json
<
* Connection #0 to host localhost left intact
0
HTTP/1.1 200 OK
Date: Sun, 24 Nov 2013 17:30:13 GMT
Server: Apache
X-Powered-By: PHP/5.3.20
Cache-Control: no-cache
X-Debug-Token: d1947e
Transfer-Encoding: chunked
Content-Type: application/json
[ OUTPUT FROM SERVER HERE ]
* Closing connection #0
Well you seem to be missing an equals sign!
ss_postfields << "user_name=" << player->getName()
<< "&user_secret=" << player->secret()
<< "&app_id=" << player->getApp_id()
<< "&lat=" << player->getLocation()->getLat()
<<"&lon=" << player->getLocation()->getLon();
instead of
ss_postfields << "user_name=" << player->getName()
<< "&user_secret" << player->secret()
<< "&app_id=" << player->getApp_id()
<< "&lat=" << player->getLocation()->getLat()
<<"&lon=" << player->getLocation()->getLon();
Since I was in a rush, I rewrote everything using HTTP sockets. But I could not be done with this issue, so I went back to it, and finally found the problem. It seems like cUrl does not really like streams.
I thought I would update this post in the case anyone is having the same issue:
Therefore, something like the following will NOT work:
std::ostringstream ss_postfields;
// This will not work - And all values do NOT contain any space
// The server do NOT receive any data (in POST and GET)
ss_postfields << "user_name=" << player->getName()
<< "&user_secret=" << player->getSecret();
const char *postfields = ss_postfields.str().c_str();
// ...
curl_easy_setopt(curl_handle, CURLOPT_POSTFIELDS, postfields);
CURLcode res = curl_easy_perform(curl_handle);
However, and bizarrely, this will work:
std::string str_postfields;
std::string user_name = player->getName();
std::string user_secret = player->getSecret();
std::string str_postfields = "user_name=" +user_name +"&user_secret=" +user_secret;
const char *postfields = str_postfields.c_str();
// ...
curl_easy_setopt(curl_handle, CURLOPT_POSTFIELDS, postfields);
CURLcode res = curl_easy_perform(curl_handle);