Curl - checking connection - c++

I used Curl 7.2.9 and checked connection this way:
Here's example:
curl = curl_easy_init();
bool result = false;
if(curl)
{
curl_easy_setopt(curl, CURLOPT_URL, m_checkConnectionUrl);
CURLcode res = curl_easy_perform(curl);
}
if(res != CURLE_OK)
{
}
else
{
// connection is available
}
Now I switched to curl-7.33.0 and got *CURLE_WRITE_ERROR* error,
and to make it work I must code it like
std::string output;
char* encodedUrl = curl_easy_escape(curl, m_checkConnectionUrl, 0);
curl_easy_setopt(curl, CURLOPT_POST, 0);
curl_easy_setopt(curl, CURLOPT_URL, encodedUrl);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writeMemoryCurlCallbackStub);
CURLcode res = curl_easy_perform(curl);
But I don't need to write anything. Any ideas?

Manily the Curl option *CURLOPT_WRITEFUNCTION* is used to have a certain amount of data periodically(at the callback functoin) to handle a large file download. I don't see any reason to use this with your curl purpose, regardless the version.
Remove the *CURLOPT_POST*(by default its 0) and *CURLOPT_WRITEFUNCTION* from the code and it should work. If it doesn't, then you are doing something wrong at other places in your code!
Also, if you are checking whether the URL is ok or not, then using CURL is ok. But to only check for connection, you can only check whether the port 80 of the domain is on or not.

You need to write a writecallback as well
size_t CurlWriteCallback(char* buf, size_t size, size_t nmemb, void* up)
{
TRACE("CURL - Response received:\n%s", buf);
TRACE("CURL - Response handled %d bytes:\n%s", size*nmemb);
// tell curl how many bytes we handled
return size*nmemb;
}
if(curl)
{
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, &CurlWriteCallback);
curl_easy_setopt(curl, CURLOPT_URL, m_checkConnectionUrl);
CURLcode res = curl_easy_perform(curl);
}

Old question, but I have just encountered a similar problem. After some more googling this is the solution:
curl_easy_setopt(curl, CURLOPT_CONNECT_ONLY, 1L);
curl_easy_perform(curl);
// OK, now we are connected (if nothing bad happened),
// but it would be nice to communicate with the server:
curl_easy_setopt(curl, CURLOPT_CONNECT_ONLY, 0L);
//now we can do the actual communication
I used this to separate authentication from the actual emails sending.

Related

libcurl - "curl_easy_cleanup()" is causing malloc error (C++)

When running my code (pertinent sections pasted below), I periodically get the following error:
program(34010,0x70000e58b000) malloc: *** error for object
0x7fc43d93fcf0: pointer being freed was not allocated set a breakpoint
in malloc_error_break to debug Signal: SIGABRT (signal SIGABRT)
I am running multi-threaded C++ code on a Macbook (OS-10.13) wherein different threads make use of the code in question simultaneously. To my knowledge, libcurl is indeed thread safe as long as I do not utilize the same "curl handle" (which I understand to be an instance of "CURL" aka "CURL *curl = curl_easy_init();") in two different threads at the same time. In my case, since each thread calls the function separately and initializes a new instance of a CURL object, I should be "safe", right? Hopefully there is something obvious that I'm missing that is causing me (or lib curl in this case) to attempt to free memory that has already been freed. If there is any more information I should have included (below) please don't hesitate to let me know.
The function that seg faults is
string http_lib::make_get_request(string url)
on the line that reads
curl_easy_cleanup(curl);
and sometimes (less often) on the line that reads
res = curl_easy_perform(curl);
Below is what I think would be the pertinent sections of my code:
size_t http_lib::CurlWrite_CallbackFunc_StdString(void *contents, size_t size, size_t nmemb, std::string *s)
{
size_t newLength = size*nmemb;
size_t oldLength = s->size();
try
{
s->resize(oldLength + newLength);
}
catch(std::bad_alloc &e)
{
//handle memory problem
return 0;
}
std::copy((char*)contents,(char*)contents+newLength,s->begin()+oldLength);
return size*nmemb;
}
string http_lib::make_post_request(string url, vector<string> headers, string post_params) {
CURL *curl;
CURLcode res;
curl = curl_easy_init();
string s;
if(curl)
{
struct curl_slist *chunk = NULL;
for(int i=0; i<headers.size(); i++){
/* Add a custom header */
chunk = curl_slist_append(chunk, headers[i].c_str());
}
/* set our custom set of headers */
res = curl_easy_setopt(curl, CURLOPT_HTTPHEADER, chunk);
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, post_params.c_str());
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L); //only for https
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0L); //only for https
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, CurlWrite_CallbackFunc_StdString);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &s);
if(networking_debug){
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L); //verbose output
}
/* Perform the request, res will get the return code */
res = curl_easy_perform(curl);
/* Check for errors */
if(res != CURLE_OK)
{
fprintf(stderr, "curl_easy_perform() failed: %s\n",
curl_easy_strerror(res));
}
/* always cleanup */
curl_easy_cleanup(curl);
}
// Debug output
if (networking_debug){
cout<<"Response: " << s <<endl;
}
return s;
}
string http_lib::make_get_request(string url) {
//SslCurlWrapper sslObject;
CURL *curl;
CURLcode res;
curl = curl_easy_init();
string s;
if (curl) {
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
//tell libcurl to follow redirection
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L); //only for https
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0L); //only for https
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, CurlWrite_CallbackFunc_StdString);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &s);
if(networking_debug){
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L); //verbose output
}
/* Perform the request, res will get the return code */
res = curl_easy_perform(curl);
/* Check for errors */
if (res != CURLE_OK)
fprintf(stderr, "curl_easy_perform() failed: %s\n", curl_easy_strerror(res));
/* always cleanup */
curl_easy_cleanup(curl);
}
if (networking_debug){
cout << "Response: " << s << endl;
}
return s;
}
In main() I have
int main(int argc, char *argv[]){
// Initialize http_lib (curl)
curl_global_init(CURL_GLOBAL_DEFAULT);
... spin up 10 or so threads that make get/post requests to https site (some requests utilize the make_post_request() function and others utilize make_get_requet() function).
}
CMAKE doesn't/didn't seem to want to use anything other than CURL_ROOT_DIR of "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/usr/include" for libcurl (aka curl).
Thus it was using the curl lib that mac (and/or Xcode) ships with. I haven't figured out what version that is, but I can say that not using it and instead using CURL version 7.57 is what fixed my issue.
I used "brew" package manager to
brew install curl
Doing so created /usr/local/Cellar/curl/7.57.0 directory and put all libs/includes in there.
Then I added
-I/usr/local/Cellar/curl/7.57.0/include -L/usr/local/Cellar/curl/7.57.0/lib
to my CMAKE CMAKE_CXX_FLAGS.
TLDR; Solution was to ensure I was using the newest version of the curl lib. Now that I am, no problem.

How to login on a page and read url content from it?

I have a problem.
I wanted to login e.g. in ogame and read the url source code from it.
But everytime I start my code, my file returns the login page and not the second page after login.
#include <stdio.h>
#include <curl/curl.h>
FILE *fptr;
static size_t WriteCallback(void *contents, size_t size, size_t nmemb, void *userp)
{
fptr = fopen("ogametest.html", "w");
fputs((const char*)contents, (FILE*)fptr);
fclose (fptr);
//printf("%s", (char *) contents);
return size * nmemb;
}
int main(int argc, char **argv){
CURL *curl;
CURLcode res;
char strlist[16000]="";
char* str= &strlist[0];
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_URL, "https://de.ogame.gameforge.com/");
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "usernameLogin=myLogin&passwordLogin=myPassw&serverLogin=s146-de.ogame.gameforge.com");
res = curl_easy_perform(curl);
//curl_easy_reset(curl);
curl_easy_setopt(curl, CURLOPT_URL, "https://s146-de.ogame.gameforge.com/game/index.php?page=overview&relogin=1");
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteCallback);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, str);
res = curl_easy_perform(curl);
fptr = fopen("ogametest.html", "r");
fgets(str, 16000, fptr);
if(res != CURLE_OK)
fprintf(stderr, "curl_easy_perform() failed: %s\n",
curl_easy_strerror(res));
/* always cleanup */
curl_easy_cleanup(curl);
}
curl_global_cleanup();
return 0;
}
It would be fine if you can help me.
Have a good day
There are several issues with your code.
The url is wrong for login. The right url is "https://de.ogame.gameforge.com/main/login"
The post message is not the right format.
You need to handle the cookies. The cookies are used to save the login session. Without the cookies you'll be redirected to main page everytime you make any request at ogame.
Here is some curl code that works to login ogame
CURL* curl;
//Local initializing of curl object
curl = curl_easy_init();
//Defining parameters
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L); //print html header request/responses in console
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, TRUE);//Follow redirects
curl_easy_setopt(curl, CURLOPT_URL, "https://de.ogame.gameforge.com/main/login");//Url
curl_easy_setopt(curl, CURLOPT_COOKIEJAR, "cookie.txt");//Save cookies here
curl_easy_setopt(curl, CURLOPT_COOKIEFILE, "cookie.txt");//Load cookies here
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "kid=&login=my%40email.here&pass=mypassword&uni=s148-de.ogame.gameforge.com");//Post message
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, &writeCallback);//html code response function
curl_easy_setopt(curl, CURLOPT_USERAGENT, "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.30 (KHTML, like Gecko) Chrome/12.0.742.112 Safari/534.30");//Setting agent. Just do this
//Executing curl object
curl_easy_perform(curl);
where
string htmlcode;
size_t writeCallback(char* buf, size_t size, size_t nmemb, void* up){
for (int c = 0; c<size*nmemb; c++)
{
htmldata.push_back(buf[c]);
}
return size*nmemb;}
is globally defined. So in this code the html code will be store in the string htmlcode.
Now the "my%40email.here" should be changed to your email. The %40 is #. "mypassword" should be changed to your password. The "s148-de.ogame.gameforge.com" depends the universe you play on. 148 is Virgo.
I don't know what you're trying to do, but I suggest you use a program to intercept the html requests/responses. In chrome you can go to DevTools Overview (shortcut f12). Under network you can track the exchange between the browser and server, which you can then try to mimic.

CURL - simple example returning "CURLE_WRITE_ERROR"

I am trying to run a simple example using libcurl, but just running this simple example gives me CURLE_WRITE_ERROR when I execute the curl_easy_perform(...) command. Does anyone have any idea what I am doing wrong? I have also tried other sites besides example.com.
CURL *curl = curl_easy_init();
if(curl)
{
CURLcode res;
curl_easy_setopt(curl, CURLOPT_URL, "http://example.com/");
res = curl_easy_perform(curl); // returns CURLE_WRITE_ERROR always!
curl_easy_cleanup(curl);
}
OK turns out Joachim is right. I did need a write callback
size_t CurlWriteCallback(char* buf, size_t size, size_t nmemb, void* up)
{
TRACE("CURL - Response received:\n%s", buf);
TRACE("CURL - Response handled %d bytes:\n%s", size*nmemb);
// tell curl how many bytes we handled
return size*nmemb;
}
// ...
CURL *curl = curl_easy_init();
if(curl)
{
CURLcode res;
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, &CurlWriteCallback);
curl_easy_setopt(curl, CURLOPT_URL, "http://example.com/");
res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
}

CURL finish executing and timeout

I'm performing a server request with curl in C++ which return responses in pieces and those pieces's size may also vary.
At the time of arrival of each piece, the callback function is being called. The problem is I can't detect when the connection finished in order to perform an another callback to my parent class.
And by the way, I want to know if we can set and detect timeout for a curl?
Here is my code in short:
CURL *curl = curl_easy_init();
curl_global_init(CURL_GLOBAL_ALL);
curl_easy_setopt(curl, CURLOPT_URL, "My URL");
curl_easy_setopt(curl, CURLOPT_POST, 1);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "My Postfields");
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writeCallback);
curl_easy_perform(curl);
curl_easy_cleanup(curl);
curl_global_cleanup();
The default callback:
size_t writeCallback(char* buf, size_t size, size_t nmemb, void* up)
{
//do something
//But how can I detect the last callback when connection finished
//in order to call an another one?
return size*nmemb;
}
The data you want can be saved off during the callback, then used once curl_easy_perform returns. Example:
CURL *curl = curl_easy_init();
curl_global_init(CURL_GLOBAL_ALL);
// NOTE: added to accumulate data.
std::string result;
curl_easy_setopt(curl, CURLOPT_URL, "My URL");
curl_easy_setopt(curl, CURLOPT_POST, 1);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "My Postfields");
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writeCallback);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &result); // NOTE: added
curl_easy_perform(curl);
// TODO: do something with your data stored in result
curl_easy_cleanup(curl);
curl_global_cleanup();
And in your write callback:
size_t writeCallback(char* buf, size_t size, size_t nmemb, void* up)
{
std::string* pstr = static_cast<std::string*>(up);
std::copy(buf, buf+size*nmemb, std::back_inserter(*pstr));
return size*nmemb;
}
or something along those lines. I leave all the error checking to you (and sorry for any typos; I don't have a compiler to validate this on immediately available to me).
Regarding timeout length, there are a multitude of timeout options available to a easy-mode curl request. Too many to mention here, in fact. See the documentation for curl_easy_setopt, in particular the connection options approximately 2/3rd of the way down the page.
Best of luck.

CURLE_COULDNT_CONNECT - while Internet is ok and server is available

I get this error from Curl, yet, the site is available, and I see nothing going out on wireshark.
What might cause this error ?
I've try to run it against www.google.com and got the same error.
this very code was working a few hours ago. I have no idea what might cause this.
here is the code:
CURL *curl;
CURLcode res = CURLE_OK;
struct curl_slist *headers=NULL;
headers = curl_slist_append(headers, "Content-Type: text/xml");
curl_global_init(CURL_GLOBAL_ALL);
curl = curl_easy_init();
if(curl) {
struct rcvdstring s;
init_string(&s);
string FullAddress = URL+Method;
curl_easy_setopt(curl, CURLOPT_URL, FullAddress.c_str());
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
res = curl_easy_setopt(curl, CURLOPT_POSTFIELDS, DATA.c_str());
res = curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
char buf[1024];
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writefunc);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &s);
res = curl_easy_perform(curl);
Respons.assign(s.ptr);
return res;
}
* Update - took the exe to another computer, it works there. This is local to my computer....
Ok, Moving the code to another computer - showed me the code is OK.
The problem was with Norton Internet Security that after a few days of developing decided to block my communication DLL.