I am new to using the libcurl libraries in C++ just to learn some new stuff, however i cant seem to find to much useful info on the subject with good practical examples.
I am trying to retrieve some stats from this website:
http://www.squawka.com/football-player-rankings#performance-score#player-stats#spanish-la-liga|season-2014/2015#all-teams#all-player-positions#16#39#0#0#90#23/08/2014#28/12/2014#season#1#all-matches#total#desc#total
For this, after the proper includes i am pulling the webpage using:
int main()
{
CURL *curl;
CURLcode res;
curl = curl_easy_init();
if(curl)
{
// Set URL
curl_easy_setopt(curl, CURLOPT_URL, "http://www.squawka.com/football-player-rankings#performance-score#player-stats#spanish-la-liga|season-20
14/2015#r-madrid#all-player-positions#16#34#0#0#90#23/08/2014#14/11/2014#season#1#all-matches#total#desc#total");
// Perform the request, res will get the return code
res = curl_easy_perform(curl);
// Check for errors
if(res != CURLE_OK)
{
fprintf(stderr, "curl_easy_perform() failed: %s\n", curl_easy_strerror(res));
}
// Print the code
cout << res << endl;
// Always cleanup each call to curl_easy_init
curl_easy_cleanup(curl);
}
return 0;
}
From the websites source code i can check for example that the name Messi is in:
<div class="stats-player-name">Messi</div> in <tr style class="ranking-data-row" data-id="1569" data-row="1">.
However if i run a search on the printed code i can´t see the name Messi, nor his stats anywhere. What am i missing so i can tell the site to download all the players data? Shouldnt this be done automatically when i give the URL?
I tried using something like this: Add paramethers to libcurl GET in c++ with no success.
Thanks in advance for some basic guidelines so i can continue with this.
Related
Basically I have two concerns, however the focus is on how to get around the certification.
I am having a hard time understanding how to make an HTTP post request in WinRT|C++. I have an ASP.Net-6-Web-Api-Project, which I have already been able to communicate with via a Python project and a C++ project (via Curl). I also tested the api via Postman. Every time doing so I had to ignore the validation of certification and it worked fine.
But now I have a WinRT/C++ project and have thrown together the following code. I want to be able to upload a file. In my case it is a point cloud in a .ply syntax as a string.
My Concerns:
In WinRT I got the expected error for invalid/untrusted certification, so I looked up what to do and ended up using IgnorableServerCertificateErrors from HttpBaseProtocolFilter, like you can see at the end of my code. But that did not fix my error. What am I missing? I still get the errors:
WinRT originate error - 0x80072F0D : 'The certificate authority is invalid or incorrect'.
WinRT originate error - 0x80190190 : 'The response status code does not indicate success: 400 ().'.
From the point of view of a developer familiar with WinRT, is the implementation correct in terms of an HTTP post request? Especially the lines
binaryContent.Headers().Append(L"Content-Type", L"image/jpeg"); andHttpContentDispositionHeaderValue disposition{ L"form-data" }; And what are the follow lines for? Is this just about assigning arbitrary names?
disposition.Name(L"fileForUpload");
disposition.FileName(L"test.ply");
Code:
void HL2ResearchMode::SendPLY(std::wstring const& pointCloud)
{
OutputDebugString(L"--- SendPLY()\n");
if (pointCloud.size() == 0)
return;
init_apartment();
auto buffer{
winrt::Windows::Security::Cryptography::CryptographicBuffer::ConvertStringToBinary(
pointCloud,
winrt::Windows::Security::Cryptography::BinaryStringEncoding::Utf8
)
};
winrt::Windows::Web::Http::HttpBufferContent binaryContent{ buffer };
// binaryContent.Headers().Append(L"Content-Type", L"text/plain;charset=utf8");
binaryContent.Headers().Append(L"Content-Type", L"image/jpeg");
winrt::Windows::Web::Http::Headers::HttpContentDispositionHeaderValue disposition{ L"form-data" };
//winrt::Windows::Web::Http::Headers::HttpContentDispositionHeaderValue disposition{ L"multipart/form-data" };
binaryContent.Headers().ContentDisposition(disposition);
disposition.Name(L"fileForUpload");
disposition.FileName(L"test.ply");
winrt::Windows::Web::Http::HttpMultipartFormDataContent postContent;
postContent.Add(binaryContent);
winrt::Windows::Web::Http::HttpResponseMessage httpResponseMessage;
std::wstring httpResponseBody;
try
{
// Send the POST request.
winrt::Windows::Foundation::Uri requestUri{ L"https://192.168.178.41:5001/api/meshes/uploadPointCloud" };
winrt::Windows::Web::Http::Filters::HttpBaseProtocolFilter myFilter;
auto fu = myFilter.IgnorableServerCertificateErrors();
fu.Append(ChainValidationResult::Expired);
fu.Append(ChainValidationResult::Untrusted);
fu.Append(ChainValidationResult::InvalidName);
fu.Append(ChainValidationResult::InvalidSignature);
fu.Append(ChainValidationResult::InvalidCertificateAuthorityPolicy);
winrt::Windows::Web::Http::HttpClient httpClient(myFilter);
httpResponseMessage = httpClient.PostAsync(requestUri, postContent).get();
httpResponseMessage.EnsureSuccessStatusCode();
httpResponseBody = httpResponseMessage.Content().ReadAsStringAsync().get();
}
catch (winrt::hresult_error const& ex)
{
httpResponseBody = ex.message();
}
std::wcout << httpResponseBody;
}
I am a newbie in http and I am using a embedded http server called mongoose.
Using a web root in ~/web_root, I would like to do :
if uri == "/hello"
show "hello world"
else
show the file in web_root like index.html
I tried to use mongoose like this
struct mg_server *server;
server = mg_create_server(p, HttpEventHandler);
mg_set_option(server, "document_root", "~/web_root");
mg_set_option(server, "listening_port", "8080");
for(;;)
mg_poll_server(server, 1000);
and this is HttpEvenHandler
int HttpEventHandler(struct mg_connection *conn, mg_event ev) {
if(ev == MG_AUTH)
return MG_TRUE;
else if(ev == MG_REQUEST) {
if(strcmp("/hello", conn->uri) == 0) {
mg_printf_data(conn, "%s\n", "hello world");
}
return MG_TRUE;
}
// I don't know what to write here
return MG_FALSE;
}
else if(ev == MG_REQUEST) {
if(strcmp("/hello", conn->uri) == 0) {
mg_printf_data(conn, "%s\n", "hello world");
return MG_TRUE;
}
return MG_FALSE; // Let Mongoose serve the request
}
Also, I don't think that is going to work:
mg_set_option(server, "document_root", "~/web_root");
Specify a full path, e.g. /home/joe/web_root.
Is that REALLY what you want to achieve?
The document states that you can configure what kind of URIs are recognized as CGI invocation by command line options like -cgi_pattern /cgi-bin/*.cgi.
Then you only need to put a CGI executable named hello which outputs "hello world" under ~/web_root and tell the Mongoose server to use it as the only possible CGI: -cgi_pattern /hello$ (I haven't tested it by myself, though)
Hm, it seems that mg_set_option() is nowhere to find any more (ver. 7.3).
The correct API call for setting the web root directory would be: mg_http_serve_dir( connection, http_event_object, options);
The last parameter "options" has a member mg_http_serve_opts::root_dir. That would be the way to specify the web root directory for serving.
From design point of view this recent approach is more flexible, allowing to serve different directories based on different endpoints.
I'm new to cURL and needed it for my assignment. And I'm using C++ for this.
I have this particular line which works fine.
curl_easy_setopt(curl, CURLOPT_URL, "http://www.google.com");
But my problem occurs when I modify the URL into variable. i.e
string URL = "http://www.google.com";
curl_easy_setopt(curl, CURLOPT_URL, URL);
My program crashes. Anyone can point to me what's my mistakes?
CURLOPT_URL: Pass in a pointer to the actual URL to deal with. The parameter should be a char * to a zero terminated string...
If you hold the URL in a std::string variable you should use std::string::c_str().
std::string URL = "http://www.google.com";
curl_easy_setopt(curl, CURLOPT_URL, URL.c_str());
I’m currently working on a Web C++ application using FastCGI with Apache and mod_fcgid.
I’m trying to retrieve the headers of a request, but I didn’t find how to do so. After some researches, I thought the headers were in the attribute “envp” of “FCGX_Request”, but it contains environment variables such as:
REMOTE_ADDR: 192.168.0.50
SERVER_SOFTWARE: Apache/2.2.21 (Unix) mod_ssl/2.2.21 OpenSSL/1.0.0f DAV/2 mod_fcgid/2.3.6
REDIRECT_UNIQUE_ID: TxytP38AAAEAABpcDskAAAAE
FCGI_ROLE: RESPONDER
HTTP_ACCEPT_LANGUAGE: fr
SERVER_SIGNATURE: <address>Apache/2.2.21 [etc.]
These variables offer me useful informations, but I need the real HTTP headers, and especially “Cookie”. I tried to read on the stream “in” of the “FCGX_Request” but it seems to be for the request body (POST datas). As my application is intended to be multi-threaded, I use “FCGX_Accept_r()”, like this:
while(true)
{
FCGX_Init();
FCGX_Request* fcgiRequest = new FCGX_Request;
FCGX_InitRequest(fcgiRequest, 0, 0);
if(FCGX_Accept_r(fcgiRequest) < 0)
break;
Request* request = new Request(fcgiRequest);
request->process();
}
But actually, I don’t use threads. Requests are executed one after the other.
How can I get the request headers?
Thank you.
Try the following code. It should print out the entire environment so you can find the variable you are looking for.
while(true)
{
FCGX_Init();
FCGX_Request* fcgiRequest = new FCGX_Request;
FCGX_InitRequest(fcgiRequest, 0, 0);
if(FCGX_Accept_r(fcgiRequest) < 0)
break;
char **env = fcgiRequest->envp;
while (*(++env))
puts(*env);
Request* request = new Request(fcgiRequest);
request->process();
}
I would like to send an http post request in c++. It seems like libcurl (Curlpp) is the way to go.
Now, here is a typical request that am sending
http://abc.com:3456/handler1/start?<name-Value pairs>
The name values pairs will have:
field1: ABC
field2: b, c, d, e, f
field3: XYZ
etc.
Now, I would like to know how to achieve the same using curlpp or libcurl.
Code snippets will really help.
Don't have experience with Curlpp but this is how I did it with libcurl.
You can set your target url using
curl_easy_setopt(m_CurlPtr, CURLOPT_URL, "http://urlhere.com/");
POST values are stored in a linked list -- you should have two variables to hold the begin and the end of that list so that cURL can add a value to it.
struct curl_httppost* beginPostList;
struct curl_httppost* endPostList;
You can then add this post variable using
curl_formadd(&beginPostList, &endPostList, CURLFORM_COPYNAME, "key", CURLFORM_COPYCONTENTS, "value", CURLFORM_END);
Submitting then works like this
curl_easy_setopt(m_CurlPtr, CURLOPT_POST, true);
curl_easy_setopt(m_CurlPtr, CURLOPT_HTTPPOST, beginPostList);
curl_easy_perform(m_CurlPtr);
Hope this helps!