cURL couldn't resolve host How to deal with that? - c++

Question

I am working on a C++ program, which uses cURL lib. When I try to connect to URL, that is incorrect or the page is down I get following error :
getaddrinfo(3) failed for www.wp.pl#:80
Couldn't resolve host 'www.wp.pl#'
And the program terminates. And that is the problem, because I it to go futher and connect to other pages. I just won't parse and analyze the broken page I don't want to end my program after come across such one.
I posted similar topic, but it contained only snippets of code so it was diffcult for you to say what is wrong, now I will post whole code.
int PageHandler::getCleanAndRepairedPage(string pageURL, string& outDoc)
throw (exception) {
if (find(visitedPages.begin(), visitedPages.end(), pageURL)
!= visitedPages.end()) {
// url has already been visited
outDoc = "";
return VISITED_PAGE_ERROR;
} else { // new url
visitedPages.push_back(pageURL);
char *charURL;
charURL = const_cast<char*> (pageURL.c_str());
CURL *curl;
char curl_errbuf[CURL_ERROR_SIZE];
TidyBuffer output = { 0 };
TidyBuffer errbuf = { 0 };
TidyBuffer docbuf = { 0 };
TidyDoc tdoc = tidyCreate(); // Initialize "document"
tidyOptSetBool(tdoc, TidyForceOutput, yes);
tidyOptSetInt(tdoc, TidyWrapLen, 4096);
tidyBufInit(&docbuf);
int rc = -1;
Bool ok;
curl = curl_easy_init();
curl_easy_setopt( curl, CURLOPT_URL, charURL );
curl_easy_setopt( curl, CURLOPT_ERRORBUFFER, curl_errbuf );
curl_easy_setopt( curl, CURLOPT_WRITEFUNCTION, write_cb );
curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0L);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
curl_easy_setopt( curl , CURLOPT_WRITEDATA, &docbuf );
int err;
err = curl_easy_perform(curl);
cout << "curl_easy_perfom return value = " << err << endl;
if (!err) {
ok = tidyOptSetBool(tdoc, TidyXhtmlOut, yes); // Convert to XHTML
if (ok) {
rc = tidySetErrorBuffer(tdoc, &errbuf); // Capture diagnostics
if (rc >= 0) {
rc = tidyParseBuffer(tdoc, &docbuf); // parse the buffer
}
if (rc >= 0) {
rc = tidyCleanAndRepair(tdoc); // Tidy it up!
}
if (rc >= 0) {
rc = tidyRunDiagnostics(tdoc); // Kvetch
}
if (rc > 1) { // If error, force output.
rc = (tidyOptSetBool(tdoc, TidyForceOutput, yes) ? rc : -1);
}
if (rc >= 0) {
rc = tidySaveBuffer(tdoc, &output); // Pretty Print
}
}
if (rc >= 0) {
if (rc > 0) {
//printf("\nDiagnostics:\n\n%s", errbuf.bp);
}
} else {
printf("A severe error (%d) occurred.\n", rc);
}
} else {
printf("%s\n", curl_errbuf);
}
if (err == NO_ERROR) {
string tmp(reinterpret_cast<char const*> (output.bp));
outDoc = tmp;
} else {
outDoc = "";
}
curl_easy_cleanup(curl);
tidyBufFree(&docbuf);
tidyBufFree(&output);
tidyBufFree(&errbuf);
tidyRelease(tdoc);
return err; // err == 0 <=> everything ok!
}
}
And the console output :
EDIT : I forgot to say what is going on on console output. I connect to many pages and the invalid URL is just one of them. At the beginning you can see a succesful connection and cURL messages about it, the result of curl_easy_perform equals 0, which indicate that everything went good. Next message is about connection to the invalid URL, as u can see return value of curl_easy_perform is 6, which is not good.

Answer 1

"www.wp.pl#" is not a legal URL and it causes libcurl to (mistakenly) use the entire string as a host name, and you don't have any such and the operation fails.