curl_easy_perform buffer overflow - c++

My problem is that in one part of my program a double variable gets incorrectly set to 2.71179e-308 on one specific (virtual) computer (not any other). No crashes or anything like that. After much work I narrowed the problem down to a call to curl_easy_perform (the call has no other connection with said variable). If I fake the curl_easy_perform call and return before it executes the variable is not modified.
My first thought was that there was some problem with the callback write function, but having looked long and hard at it I can't find anything wrong, and replacing it with an empty function still didn't help the error.
My second thought was that perhaps my curl settings strings went out of scope before being used, but in my version of curl (7.43.0) they should be copied and stored by curl itself.
Now I'm at wits end and so turn to SO for some sort of hint at what could be wrong. This is the code for the class I use for all communication;
Header:
#include <string>
class HTTPClient
{
public:
enum Verb
{
V_GET,
V_POST,
V_PUT,
V_DELETE,
};
// Constructor is not thread safe. Make sure to initialize in main thread.
HTTPClient(const std::string& userAgent, const std::string& certPath, const std::string& cookieFile, const std::string& proxy = "");
~HTTPClient(void);
int MakeRequest(Verb verb, const std::string& url, int& responseCode, std::string& response);
private:
static size_t WriteFunction(void *ptr, size_t size, size_t nmemb, void *custom);
void* m_curl;
};
Source:
#include "HTTPClient.h"
#include <curl/curl.h>
HTTPClient::HTTPClient(const std::string& userAgent, const std::string& certPath, const std::string& cookieFile, const std::string& proxy)
{
curl_global_init(CURL_GLOBAL_DEFAULT);
m_curl = curl_easy_init();
if (m_curl)
{
// Indicate where the certificate is located
curl_easy_setopt(m_curl, CURLOPT_CAINFO, certPath.c_str());
// Indicate where the cookie file is located
curl_easy_setopt(m_curl, CURLOPT_COOKIEFILE, cookieFile.c_str());
curl_easy_setopt(m_curl, CURLOPT_COOKIEJAR, cookieFile.c_str());
// Set user agent
curl_easy_setopt(m_curl, CURLOPT_USERAGENT, userAgent.c_str());
// Set the response function
curl_easy_setopt(m_curl, CURLOPT_WRITEFUNCTION, &HTTPClient::WriteFunction);
// Set proxy if specified
if (!proxy.empty())
curl_easy_setopt(m_curl, CURLOPT_PROXY, proxy.c_str());
}
}
HTTPClient::~HTTPClient(void)
{
if (m_curl)
curl_easy_cleanup(m_curl);
curl_global_cleanup();
}
int HTTPClient::MakeRequest(Verb verb, const std::string& url, int& responseCode, std::string& response)
{
std::string protocol, server, path, parameters;
if (!m_curl)
return CURLE_FAILED_INIT;
// Set response data
response.clear();
curl_easy_setopt(m_curl, CURLOPT_WRITEDATA, &response);
switch (verb)
{
case V_GET:
curl_easy_setopt(m_curl, CURLOPT_CUSTOMREQUEST, "GET");
curl_easy_setopt(m_curl, CURLOPT_URL, url.c_str());
break;
// Other cases removed for brevity
}
// Execute command
CURLcode res = curl_easy_perform(m_curl); // <-- When this executes the variable gets damaged
// Get response code
long code;
curl_easy_getinfo(m_curl, CURLINFO_RESPONSE_CODE, &code);
responseCode = code;
return res;
}
size_t HTTPClient::WriteFunction(void *ptr, size_t size, size_t nmemb, void *custom)
{
size_t full_size = size * nmemb;
std::string* p_response = reinterpret_cast<std::string*>(custom);
p_response->reserve(full_size);
for (size_t i=0; i<full_size; ++i)
{
char c = reinterpret_cast<char*>(ptr)[i];
p_response->push_back(c);
}
return full_size;
}
Edit:
I have now run tests with WinDbg and have found out pretty much what happens (but not how to solve the problem).
The double that is trashed is in the release build put in the mmx7 register. It then sits there without being changed until curl_easy_perform is called but before the WriteFunction is called by curl_easy_perform.
Other than compiling my own version of curl so I can debug deeper I don't know how to get past this problem.

Related

It is necessary to send alerts to Telegram bot from Teams using webhook_url

The program will hang on the server and work around the clock. It is necessary to catch the received message in the Microsoft-Teams group (channel) via webhook_url and send at least one message (preferably a whole message) to the bot's Telegram. I write code on the pluses as I don’t know anything other than them. Below is the code. The code did not work for me, if there are any ideas or suggestions on how to improve what I am certainly happy about. Thanks in advance.
#include <iostream>
#include <string>
#include <curl/curl.h> // Library for working with HTTP requests
using namespace std;
// Incoming webhook URL created in Teams
const string webhook_url = "Webhook from Tims";
// Telegram bot access token
const string bot_token = "Bot token";
// Telegram chat ID where to send notifications
const string chat_id = "Bot name";
size_t write_callback(void* ptr, size_t size, size_t nmemb, void* userdata) {
// The callback function used to store the server's response
string* response = (string*)userdata;
response->append((char*)ptr, size * nmemb);
return size * nmemb;
}
void send_webhook_message(const string& text, const string& bot_token, const string& chat_id) {
// Function for sending a webhook message through a bot in Telegram
CURL* curl = curl_easy_init();
if (curl) {
string response;
curl_easy_setopt(curl, CURLOPT_URL, ("https://t.me/ITSys****" + bot_token + "/sendMessage").c_str());
curl_easy_setopt(curl, CURLOPT_POST, 1);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, ("chat_id=" + chat_id + "&text=" + text).c_str());
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_callback);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &response);
curl_easy_perform(curl);
curl_easy_cleanup(curl);
}
}
int main ()
{for ( ; ; )
{
}
return 0;
}

libcurl FTP-Download incomplete in the Unreal Engine

I am currently trying to download a File from a public git-Repository using curl in my Unreal C++ Project. Here is the code I'm trying to execute that I derived from the FTP-Example:
// This is in the .h file
struct FFtpFile {
FILE* File;
const char* Filename;
};
void FtpFetch(const std::string URL, const char* Filename) {
CURL* Curl = curl_easy_init();
const FFtpFile FtpFile {
nullptr,
Filename
};
if (!Curl) {
UE_LOG(LogTemp, Warning, TEXT("Error Initiating cURL"));
return;
}
curl_easy_setopt(Curl, CURLOPT_URL, URL.c_str());
curl_easy_setopt(Curl, CURLOPT_VERBOSE, 1);
curl_easy_setopt(Curl, CURLOPT_FOLLOWLOCATION, 1);
curl_easy_setopt(Curl, CURLOPT_SSL_VERIFYPEER, false);
// Data Callback
const auto WriteCallback = +[](void* Contents, const size_t Size, const size_t NumMem, FFtpFile* FileStruct) -> size_t {
if (!FileStruct->File) {
fopen_s(&FileStruct->File, FileStruct->Filename, "wb");
if (!FileStruct->File) {
return CURLE_WRITE_ERROR;
}
}
return fwrite(Contents, Size, NumMem, FileStruct->File);
};
curl_easy_setopt(Curl, CURLOPT_WRITEFUNCTION, DownloadCallback);
curl_easy_setopt(Curl, CURLOPT_WRITEDATA, FtpFile);
const CURLcode Result = curl_easy_perform(Curl);
if (Result != CURLE_OK) {
const FString Message(curl_easy_strerror(Result));
UE_LOG(LogTemp, Warning, TEXT("Error Getting Content of the Model File: %s"), *Message);
return;
}
curl_easy_cleanup(Curl);
// Close the Stream after Cleanup
UE_LOG(LogTemp, Log, TEXT("Successfully Fetched FTP-File. Closing Write Stream"))
if (FtpFile.File) {
fclose(FtpFile.File);
}
}
Note that this is executed on a separate Thread using the Unreal Async function:
void AsyncFetchModelFile(const std::string URL) {
std::string Path = ...
TFunction<void()> Task = [Path, URL]() {
FtpFetch(URL, Path.c_str());
};
UE_LOG(LogTemp, Log, TEXT("Fetching FTP on Background Thread"))
Async(EAsyncExecution::Thread, Task, [](){UE_LOG(LogTemp, Warning, TEXT("Finishied FTP on Background Thread!"))});
}
I already removed the curl_global calls, as the documentation states those are not thread-safe. I also tried running the code on the main thread, but the same error happens there too.
To the error itself: The download runs almost flawlessly, but the downloaded file (in this case a .fbx file) always misses the last ~800 Bytes and is therefore incomplete. Also, the file keeps being open in Unreal, so I can't delete/move the file unless I close the Editor.
Before writing this Unreal code I tried running the same code in a pure C++ setting and there it worked flawlessly. But for some reason doing the same in Unreal doesn't work.
I also tried using a private method instead of the lambda-Function, but that didn't make any difference.
Any help would be appreciated
~Okaghana
I didn't manage to get it working in the end (I suspect it's an Unreal-Related Bug), but I found another way using the included Unreal HTTP Module:
FString Path = ...
// Create the Callback when the HTTP-Request has finished
auto OnRequestComplete = [Path](FHttpRequestPtr Request, FHttpResponsePtr Response, bool bWasSuccessful) {
if (bWasSuccessful) {
FFileHelper::SaveArrayToFile(Response->GetContent(), *Path);
UE_LOG(LogTemp, Log, TEXT("Successfully downloaded the file to '%s'"), *Path)
} else {
UE_LOG(LogTemp, Warning, TEXT("Error downloading the file (See EHttpResponseCodes): %s"), Response->GetResponseCode())
}
};
// Create a HTTP-Request and Fetch the file
TSharedRef<IHttpRequest, ESPMode::ThreadSafe> Request = FHttpModule::Get().CreateRequest();
Request->SetVerb("GET");
Request->SetURL(URL);
Request->OnProcessRequestComplete().BindLambda(OnRequestComplete);
Request->ProcessRequest();

Posting String To Web Server, Get Response

I have a long base64 encoded text string. It's about 1024 characters. From my Objective C code, I want to send it to my PHP script, have it dump it to a log, and return an "OK" response back. I tried this cookbook example, but it only has an example of upload and download (not both combined), and it doesn't even work in my case.
I'd be willing to switch this to a C++ solution if I knew how.
The Objective C Client Code (command line client)
NSString *sMessage = #"My Long Base64 Encoded Message";
NSString *sURL = "http://example.com/request.php";
NSURL *oURL = [NSURL URLWithString:sURL];
NSData *data = [NSData dataWithBytes:sMessage.UTF8String length:sMessage.length];
NSURLSessionDataTask *downloadTask = [[NSURLSession sharedSession]
dataTaskWithURL:oURL completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
NSLog(#"\n\nDATA\n\n%#",data);
NSLog(#"\n\nRESPONSE\n\n%#",response);
NSLog(#"\n\nERROR\n\n%#",error);
}];
[downloadTask resume];
The PHP Web Server Code
<?php
error_reporting(E_ALL);
ini_set('display_errors','On');
$sRaw = file_get_contents('php://input');
file_put_contents('TEST.TXT',$sRaw);
die('OK');
There's a far easier route using ordinary C++. You'll have to convert your .m file to a .mm file in order to be able to mix Objective C and C++ code.
The PHP code is good and doesn't require a revision. Here's the C++ example I used that worked. It used the STL and curl. I was doing this on a Mac, and by default OSX has the curl libraries pre-installed. Note that the example below is synchronous -- it jams program execution until the server call is completed. (I desired this in my case -- you may not.)
The C++ Client Code (class)
#pragma once
#include <string>
#include <sstream>
#include <iostream>
#include <curl/curl.h>
class Webby {
public:
static size_t write_data(void *ptr, size_t size, size_t nmemb, void *stream) {
std::string buf = std::string(static_cast<char *>(ptr), size * nmemb);
std::stringstream *response = static_cast<std::stringstream *>(stream);
response->write(buf.c_str(), (std::streamsize)buf.size());
return size * nmemb;
}
static std::string sendRawHTTP(std::string sHostURL, std::string &sStringData) {
CURL *curl;
CURLcode res;
curl = curl_easy_init();
if (curl) {
std::stringstream response;
curl_easy_setopt(curl, CURLOPT_URL, sHostURL.c_str());
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, sStringData.c_str());
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, Webby::write_data);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &response);
res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
return response.str();
}
return "";
}
}; // end class

Curl causes openssl error

My current curl setup to call a webpage, save it into a string, and reiterate the process after sleeping for a second. This is the code to write into the string:
#include <curl/curl.h>
#include <string>
#include <iostream>
#include <thread>
#include <chrono>
size_t curl_writefunc(void* ptr, size_t size, size_t nmemb, std::string* data)
{
data->append((const char*)ptr, size * nmemb);
return size * nmemb;
}
void curl_handler(std::string& data)
{
int http_code = 0;
CURL* curl;
// Initialize cURL
curl = curl_easy_init();
// Set the function to call when there is new data
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, curl_writefunc);
// Set the parameter to append the new data to
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &data);
// Set the URL to download; just for this question.
curl_easy_setopt(curl, CURLOPT_URL, "http://www.example.com/");
// Download
curl_easy_perform(curl);
// Get the HTTP response code
curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &http_code);
// Clean up
curl_easy_cleanup(curl);
curl_global_cleanup();
}
int main()
{
bool something = true;
std::string data;
while (something)
{
curl_handler(data);
std::cout << data << '\n';
data.clear();
std:: this_thread:: sleep_for (std:: chrono:: seconds(1));
}
}
However it runs into a problem about 20 minutes into runtime and this is the message it confronts me with:
140377776379824:error:02001018:system library:fopen:Too many open files:bss_file.c:173:fopen('/etc/ssl/openssl.cnf','rb')
140377776379824:error:2006D002:BIO routines:BIO_new_file:system lib:bss_file.c:178:
140377776379824:error:0E078002:configuration file routines:DEF_LOAD:system lib:conf_def.c:199:
It seems to stem from an openssl file, that does not close once it has fullfilled its task in the single iteration. If iterated more than once, the open files add up and are bound to enter into an error at some point.
I am still much of a beginner programmer, and therefore don't want to start messing with openSSL, so I came here to ask, wether there is a solution for this kind of problem. Could it be solved by declaring the curl object outside of the recalled function?
What has to be done is simply declaring the handle and its settings before getting the data. Only the actual download and its accompanying response is then reiterated in the loop. It is encouraged to re-use a handler as often as needed, since part of its resources (like the files opened in this session), may need to redeployed again.

Incomplete response string from server in implementing POST-requests

I’m writing the program sending POST-requests to the server and getting answers from it. (using curl library) (OS Linux, Red Hat Linux 3.2.2.-5). Sometimes I see , that response from server contains only second part of the message. ( I print _sResponse and sometimes I see full message(more often) and sometimes broken message(only last part of it)).
Class CurlSoapHandler
{
……..
static std::string _sResponse;
static std::string GetResponse() {return _sResponse;}
static size_t write_data(char *ptr, size_t size, size_t count, std::string *buffer)
{
int result = 0;
if (buffer != NULL)
{
std::string tmp_buffer(ptr, size * count);
_sResponse = tmp_buffer;
result = size * count;
}
else
{
std::cout<<"Buffer is not OK!"<<std::endl;
}
return result;
}
};
void CurlSoapHandler::DoRequest(const std::string& sRequest, std::string& sResponse)
{
CURL* _CURL;
CURLcode res;
struct curl_slist *headerlist=NULL;
headerlist = curl_slist_append(headerlist, "Content-Type:text/xml");
curl_global_init(CURL_GLOBAL_ALL);
_CURL = curl_easy_init();
if(_CURL)
{
curl_easy_setopt( _CURL, CURLOPT_URL, ttp://10.10.10.11:8083/Server/Server.asmx);
curl_easy_setopt( _CURL, CURLOPT_USERPWD, "user#password");
curl_easy_setopt( _CURL, CURLOPT_POSTFIELDS, sRequest.c_str());
curl_easy_setopt( _CURL, CURLOPT_HTTPHEADER, headerlist);
curl_easy_setopt( _CURL, CURLOPT_TIMEOUT, 20);
curl_easy_setopt( _CURL, CURLOPT_WRITEFUNCTION, CurlSoapHandler::write_data);
res = curl_easy_perform(_CURL);
sResponse = GetResponse();
if(res != CURLE_OK)
{
std::cerr<<"CURL message: "<<curl_easy_strerror(res)<<std::endl;
}
curl_easy_cleanup(_CURL);
}
else
{
std::cout<<"Curl initialization problem!"<<std::endl;
}
curl_global_cleanup();
curl_slist_free_all (headerlist);
}
I have no ideas why it occurs. Does someone have ideas? (May be it’s necessary to set some curl options, for example). How can I determine the reason for it and understand if it’s problem of my applications or not. I'd caught traffic using tcpdump utility (I run it on my Virtual Machine (VMWARe)) and saw full message in the dump. So, I think that server sends the correct response. Thanks in advance!
Just replace _sResponse = tmp_buffer; by _sResponse += tmp_buffer; shoulds work.