Same code works/gives error in different projects - c++

I am using visual studio 2019.
When I try to run this from a new project it works totally fine. Sends the email and doesn't give any error.
But when I try to run this from my server code it gives this error:
To be more precise:
Project1/DEBUG works fine.
Project1/Release works fine.
Project2/DEBUG works fine.
Project2/Release gives runtime error.
"Exception thrown: read access violation.
this was 0xF. occurred"
And compiler is redirected to "xstring" code page, line 2873.
part from xstring
basic_string& assign(_In_reads_(_Count) const _Elem* const _Ptr, _CRT_GUARDOVERFLOW const size_type _Count) {
// assign [_Ptr, _Ptr + _Count)
if (_Count <= _Mypair._Myval2._Myres) { // LINE 2873
_Elem* const _Old_ptr = _Mypair._Myval2._Myptr();
_Mypair._Myval2._Mysize = _Count;
_Traits::move(_Old_ptr, _Ptr, _Count);
_Traits::assign(_Old_ptr[_Count], _Elem());
return *this;
}
I've tried with totally same code (with copy paste) but still no use.
Project properties are the same. I've checked it several times for any change in properties but there is not even a single difference.
email.h
#pragma warning(disable : 4996) //_CRT_SECURE_NO_WARNINGS
#ifndef EMAIL_H
#define EMAIL_H
#include <vector>
#include <string>
#include "curl/curl.h"
// Enable one of the below depending on your operating system
#define WINDOWS_H
//#define LINUX_H
class Email
{
public:
// default constructor
Email();
// sets who the email is going to
void setTo(const std::string to);
// sets who the email came from
void setFrom(const std::string from);
// sets the cc
void setCc(const std::string to);
// sets the subject of the email
void setSubject(const std::string subject);
// set the body of the email
void setBody(const std::string body);
// sets the smtp username
void setSMTP_username(const std::string username);
// sets the smtp password
void setSMTP_password(const std::string password);
// sets the SMTP HOST
void setSMTP_host(const std::string host);
// adds a binary attachment to the email
void addAttachment(const std::string filepath);
// removes an attachment from the email (Not implemented yet)
void removeAttachment(const std::string filepath);
// removes all attachments
void removeAllAttachments();
// contructs the final email
void constructEmail();
// clears the contents of the email
void clearEmailContents();
// void send email
int send() const; // returns a CURL error code
// dumps the email contents for debugging
void dump() const;
private:
// smtp information
std::string smtp_user;
std::string smtp_password;
std::string smtp_host;
// email data
std::string to;
std::string from;
std::string cc;
std::string subject;
std::string body;
// vector which stores the email data
std::vector<std::string> email_contents;
std::vector<std::string> attachments;
// length of the above vector
int numberOfLines;
};
#endif
email.cpp
#include <iostream>
#include <ctime>
#include <string.h>
// #include "stdafx.h"
#include "email.h"
#define MAX_LEN 255 // this must be divisible by 3 otherwise the SMTP server won't be able to decode the attachment properly
#define ENCODED_LEN 341
// offsets into the email template
#define BOUNDARY 5
#define END_LINE 8
#define ATTACH_TYPE 10
#define ATTACH_TRANSFER 11
#define ATTACH_DEPOSITION 12
#define END_OF_TRANSMISSION_BOUNDARY 15
#define END_OF_TRANSMISSION 16
#ifdef WINDOWS_H
#define DIR_CHAR '\\'
#else
#define DIR_CHAR '/'
#endif
using namespace std;
// base64 encoding functions
void base64_encode(char* input_buf, char* output_buf, size_t input_size);
void encodeblock(char* in, char* out, int len);
// callback function
static size_t payload_source(void* ptr, size_t size, size_t nmemb, void* userp);
// base64 encoding table
const char b64_table[65] = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
// global vector this is TEMP
vector<string> global_vec;
// struct used by the CURL callback
struct upload_status {
int lines_read;
};
static const char* email_template[] = {
/* i = 0 */
"User-Agent: DXF Viewer agent\r\n",
"MIME-Version: 1.0\r\n",
"Content-Type: multipart/mixed;\r\n",
" boundary=\"------------030203080101020302070708\"\r\n",
"\r\nThis is a multi-part message in MIME format.\r\n",
"--------------030203080101020302070708\r\n", // use this type of boundary for subsequent attachments
"Content-Type: text/plain; charset=utf-8; format=flowed\r\n",
"Content-Transfer-Encoding: 7bit\r\n",
/* i = 8 (BODY) */
"\r\n", /* empty line to divide headers from body, see RFC5322 */
"--------------030203080101020302070708\r\n", /* i = 9 */
/* Template for binary attachments i = 10 */
"Content-Type: application/octet-stream\r\n",
"Content-Transfer-Encoding: base64\r\n",
"Content-Disposition: attachment;\r\n",
/* Filename here */
"\r\n",
/* Encoded base64 contents here */
"\r\n",
"--------------030203080101020302070708--\r\n", // this type of boundary indicates that there are no more parts i = 15
"\r\n.\r\n",
NULL
};
Email::Email()
{
// empty constructor
}
void Email::setTo(const string to)
{
this->to = to;
}
void Email::setFrom(const string from)
{
this->from = from;
}
void Email::setCc(const string cc)
{
this->cc = cc;
}
void Email::setSubject(const string subject)
{
this->subject = subject;
}
void Email::setBody(const string body)
{
this->body = body;
}
void Email::setSMTP_username(const string user)
{
this->smtp_user = user;
}
void Email::setSMTP_password(const string pass)
{
this->smtp_password = pass;
}
void Email::setSMTP_host(const string hostname)
{
this->smtp_host = hostname;
}
void Email::addAttachment(const string file_path)
{
/*FILE* fp = NULL;
char buffer[MAX_LEN + 1] = { 0 };
char encodedBuffer[ENCODED_LEN] = { 0 };
char tempBuffer[MAX_LEN] = { 0 };
char* filename = NULL;
unsigned int fileSize = 0;
unsigned int bytesCopied = 0;
int bytesToCopy = 0;
int bytesRead = 0;
fp = fopen(file_path.c_str(), "rb");
if (fp) {
// get the file size
fseek(fp, 0, SEEK_END);
fileSize = ftell(fp);
fseek(fp, 0, SEEK_SET);
// copy the attachment header
string type(email_template[ATTACH_TYPE]);
this->attachments.push_back(type);
string encodingType(email_template[ATTACH_TRANSFER]);
this->attachments.push_back(encodingType);
string deposition(email_template[ATTACH_DEPOSITION]);
this->attachments.push_back(deposition);
// extract the filename from the path
filename = (char*)strrchr(file_path.c_str(), DIR_CHAR);
if (filename != NULL) {
filename += 1;
// push the filename
snprintf(tempBuffer, MAX_LEN, " filename=\"%s\"\r\n", filename);
string filename(tempBuffer);
this->attachments.push_back(filename);
string endLine(email_template[END_LINE]);
this->attachments.push_back(endLine);
// copy the file MAX_LEN bytes at a time into the attachments vector
while (bytesCopied < fileSize) {
// determine how many bytes to read
if (bytesCopied + MAX_LEN > fileSize) {
bytesToCopy = fileSize - bytesCopied;
}
else {
bytesToCopy = MAX_LEN;
}
// read from the file
memset(buffer, 0, MAX_LEN + 1);
bytesRead = fread(buffer, sizeof(char), bytesToCopy, fp);
// encoded the data read
memset(encodedBuffer, 0, ENCODED_LEN);
base64_encode(buffer, encodedBuffer, bytesRead);
// setup the encoded string so that we can push it to attachments
string line(encodedBuffer);
line += endLine;
this->attachments.push_back(line);
// update the number of bytes we have copied
bytesCopied += bytesToCopy;
}
this->attachments.push_back(endLine);
string boundary(email_template[BOUNDARY]);
this->attachments.push_back(boundary);
}
else {
removeAllAttachments();
cout << "Failed to extract filename!" << endl;
}
// close the file
fclose(fp);
}
else {
cout << "Unable to open file." << endl;
}
*/
}
void Email::constructEmail(void)
{
int i = 0;
char buffer[MAX_LEN];
string boundary(email_template[BOUNDARY]);
// time stuff
time_t rawtime;
struct tm* timeinfo;
// store all the email contents in a vector
// TO:
snprintf(buffer, MAX_LEN, "To: %s\r\n", this->to.c_str());
string line1(buffer);
this->email_contents.push_back(line1);
// FROM:
memset(buffer, 0, MAX_LEN);
snprintf(buffer, MAX_LEN, "From: %s\r\n", this->from.c_str());
string line2(buffer);
this->email_contents.push_back(line2);
// CC:
memset(buffer, 0, MAX_LEN);
snprintf(buffer, MAX_LEN, "Cc: %s\r\n", this->cc.c_str());
string line3(buffer);
if (this->cc.length() > 0) {
this->email_contents.push_back(line3);
}
// Subject:
memset(buffer, 0, MAX_LEN);
snprintf(buffer, MAX_LEN, "Subject: %s\r\n", this->subject.c_str());
string line4(buffer);
this->email_contents.push_back(line4);
// time:
time(&rawtime);
timeinfo = localtime(&rawtime);
memset(buffer, 0, MAX_LEN);
strftime(buffer, sizeof(buffer), "%d/%m/%Y %I:%M:%S +1100\r\n", timeinfo);
string time_str(buffer);
this->email_contents.push_back(time_str);
cout << time_str << endl;
for (i = 0; i < END_LINE; i++) { // other stuff e.g user-agent etc
string line(email_template[i]);
this->email_contents.push_back(line);
}
// add in the body
string endLine(email_template[END_LINE]);
this->email_contents.push_back(endLine);
memset(buffer, 0, MAX_LEN);
snprintf(buffer, MAX_LEN, "%s", this->body.c_str()); // Body:
string line5(buffer);
this->email_contents.push_back(line5); // body
this->email_contents.push_back(endLine); // \r\n
this->email_contents.push_back(boundary); // boundary
// add in the attachments
for (i = 0; i < attachments.size(); i++) {
this->email_contents.push_back(this->attachments[i]);
}
// add the last boundary with the two hyphens
string lastBoundary(email_template[END_OF_TRANSMISSION_BOUNDARY]);
this->email_contents.push_back(lastBoundary);
// specify that we don't want to send any more data
string endTransmission(email_template[END_OF_TRANSMISSION]);
this->email_contents.push_back(endTransmission);
}
/*
This function was taken and modified from:
https://curl.haxx.se/libcurl/c/smtp-ssl.html
*/
int Email::send(void) const
{
CURL* curl;
CURLcode res = CURLE_OK;
struct curl_slist* recipients = NULL;
struct upload_status upload_ctx;
upload_ctx.lines_read = 0;
curl = curl_easy_init();
global_vec = this->email_contents;
if (curl) {
/* Set username and password */
curl_easy_setopt(curl, CURLOPT_USERNAME, this->smtp_user.c_str());
curl_easy_setopt(curl, CURLOPT_PASSWORD, this->smtp_password.c_str());
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0); // allows emails to be sent
/* This is the URL for your mailserver. Note the use of smtps:// rather
* than smtp:// to request a SSL based connection. */
curl_easy_setopt(curl, CURLOPT_URL, this->smtp_host.c_str());
/* If you want to connect to a site who isn't using a certificate that is
* signed by one of the certs in the CA bundle you have, you can skip the
* verification of the server's certificate. This makes the connection
* A LOT LESS SECURE.
*
* If you have a CA cert for the server stored someplace else than in the
* default bundle, then the CURLOPT_CAPATH option might come handy for
* you. */
#ifdef SKIP_PEER_VERIFICATION
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
#endif
/* If the site you're connecting to uses a different host name that what
* they have mentioned in their server certificate's commonName (or
* subjectAltName) fields, libcurl will refuse to connect. You can skip
* this check, but this will make the connection less secure. */
#ifdef SKIP_HOSTNAME_VERIFICATION
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0L);
#endif
/* Note that this option isn't strictly required, omitting it will result
* in libcurl sending the MAIL FROM command with empty sender data. All
* autoresponses should have an empty reverse-path, and should be directed
* to the address in the reverse-path which triggered them. Otherwise,
* they could cause an endless loop. See RFC 5321 Section 4.5.5 for more
* details.
*/
curl_easy_setopt(curl, CURLOPT_MAIL_FROM, this->from.c_str());
/* Add two recipients, in this particular case they correspond to the
* To: and Cc: addressees in the header, but they could be any kind of
* recipient. */
recipients = curl_slist_append(recipients, this->to.c_str());
if (this->cc.length() > 0) {
recipients = curl_slist_append(recipients, this->cc.c_str());
}
curl_easy_setopt(curl, CURLOPT_MAIL_RCPT, recipients);
/* We're using a callback function to specify the payload (the headers and
* body of the message). You could just use the CURLOPT_READDATA option to
* specify a FILE pointer to read from. */
curl_easy_setopt(curl, CURLOPT_READFUNCTION, payload_source);
curl_easy_setopt(curl, CURLOPT_READDATA, &upload_ctx);
curl_easy_setopt(curl, CURLOPT_UPLOAD, 1L);
/* Since the traffic will be encrypted, it is very useful to turn on debug
* information within libcurl to see what is happening during the
* transfer */
// curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
/* Send the message */
res = curl_easy_perform(curl);
/* Check for errors */
if (res != CURLE_OK)
fprintf(stderr, "curl_easy_perform() failed: %s\n",
curl_easy_strerror(res));
/* Free the list of recipients */
curl_slist_free_all(recipients);
/* Always cleanup */
curl_easy_cleanup(curl);
}
return (int)res;
}
void Email::removeAllAttachments()
{
this->attachments.clear();
}
void Email::clearEmailContents()
{
this->email_contents.clear();
this->attachments.clear();
}
void Email::dump(void) const
{
int i = 0;
cout << "Email contents: " << endl;
for (i = 0; i < this->email_contents.size(); i++) {
cout << this->email_contents[i];
if (i == 20) {
break;
}
}
cout << "\n\nEmail attachments: " << endl;
for (i = 0; i < this->attachments.size(); i++) {
cout << this->attachments[i];
if (i == 5) {
break;
}
}
}
void base64_encode(char* input_buf, char* output_buf, size_t input_size)
{
char in[3];
char out[4];
size_t len = input_size;
*output_buf = 0;
for (size_t i = 0; i < len; )
{
int buf3_len = 0;
for (int j = 0; j < 3; ++j)
{
in[j] = input_buf[i++];
if (i > len)
in[j] = 0;
else
buf3_len++;
}
if (len > 0)
{
encodeblock(in, out, buf3_len);
strncat(output_buf, out, 4);
}
}
}
void encodeblock(char* in, char* out, int len)
{
out[0] = (char)b64_table[(int)(in[0] >> 2)];
out[1] = (char)b64_table[(int)(((in[0] & 0x03) << 4) | ((in[1] & 0xf0) >> 4))];
out[2] = (char)(len > 1 ? b64_table[(int)(((in[1] & 0x0f) << 2) | ((in[2] & 0xc0) >> 6))] : '=');
out[3] = (char)(len > 2 ? b64_table[(int)(in[2] & 0x3f)] : '=');
}
static size_t payload_source(void* ptr, size_t size, size_t nmemb, void* userp)
{
struct upload_status* upload_ctx = (struct upload_status*)userp;
const char* data;
if ((size == 0) || (nmemb == 0) || ((size * nmemb) < 1)) {
return 0;
}
if (upload_ctx->lines_read >= 0 && upload_ctx->lines_read < global_vec.size())
{
data = global_vec[upload_ctx->lines_read].c_str();
if (data) {
size_t len = strlen(data);
memcpy(ptr, data, len);
upload_ctx->lines_read++;
return len;
}
}
return 0;
}
main.cpp
#include <iostream>
#include "email.h"
using namespace std;
int main(void)
{
Email e;
int curlError = 0;
// e.dump();
e.setTo("receiver#example.com");
e.setFrom("sender#example.com");
e.setSubject("Test Email");
e.setCc("");
e.setBody("Do not reply to this email");
e.setSMTP_host("smtp://smtp.mail.com:587");
e.setSMTP_username("sender#example.com");
e.setSMTP_password("sender_password");
e.addAttachment("");
// e.addAttachment("email.h");
// e.addAttachment("main.cpp");
e.constructEmail();
e.dump();
curlError = e.send();
if (curlError) {
cout << "Error sending email!" << endl;
}
else {
cout << "Email sent successfully!" << endl;
}
return 0;
}
What is the reason this same code runs perfectly on one project but doesn't work on another?
EDIT:
error happens in Email::send(void) on the line :
global_vec2 = this->email_contents;
but I cannot try/catch it. it doesn't throws exception.
EDIT2 :
Problem is caused by static vector. It doesn't exist, doesn't initialize. I don't understand what's causing this project to prevent static std::vectors from being created.

Related

write access violation with c++ libcurl [closed]

Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed last month.
Improve this question
I'm trying to create c++ code that download data from some URLs, but it's throwing a write access violation:
#include <errno.h>
#include <stdlib.h>
#include <string.h>
#ifndef WIN32
#endif
#include <curl/curl.h>
#include <string>
static const char* urls[] = {
"http://www.example.com",
"http://www.example1.com",
};
#define MAX_PARALLEL 10 /* number of simultaneous transfers */
#define NUM_URLS sizeof(urls)/sizeof(char *)
static size_t write_cb(void* ptr, size_t size, size_t nmemb, void* buffer)
{
((std::string*)buffer)->append((char*)ptr, nmemb);
return nmemb;
}
static void add_transfer(CURLM* cm, int i, int* left)
{
CURL* eh = curl_easy_init();
curl_easy_setopt(eh, CURLOPT_WRITEFUNCTION, write_cb);
curl_easy_setopt(eh, CURLOPT_URL, urls[i]);
curl_easy_setopt(eh, CURLOPT_PRIVATE, urls[i]);
curl_easy_setopt(eh, CURLOPT_WRITEDATA, &write_cb);
curl_easy_setopt(eh, CURLOPT_VERBOSE, 1L);
curl_multi_add_handle(cm, eh);
(*left)++;
}
int main(void)
{
CURLM* cm;
unsigned int transfers = 0;
int msgs_left = -1;
int left = 0;
curl_global_init(CURL_GLOBAL_ALL);
cm = curl_multi_init();
/* Limit the amount of simultaneous connections curl should allow: */
curl_multi_setopt(cm, CURLMOPT_MAXCONNECTS, (long)MAX_PARALLEL);
for (transfers = 0; transfers < MAX_PARALLEL && transfers < NUM_URLS;
transfers++)
add_transfer(cm, transfers, &left);
do {
int still_alive = 1;
curl_multi_perform(cm, &still_alive);
CURLMsg* msg;
int queued;
CURLMcode mc = curl_multi_perform(cm, &still_alive);
if (cm)
/* wait for activity, timeout or "nothing" */
mc = curl_multi_poll(cm, NULL, 0, 1000, NULL);
if (mc)
break;
do {
msg = curl_multi_info_read(cm, &queued);
if (msg) {
if (msg->msg == CURLMSG_DONE) {
/* a transfer ended */
fprintf(stderr, "Transfer completed\n");
}
}
} while (msg);
if (left)
curl_multi_wait(cm, NULL, 0, 1000, NULL);
} while (left);
curl_multi_cleanup(cm);
curl_global_cleanup();
return EXIT_SUCCESS;
}
Its crashing on the line:
_Mypair._Myval2._Mysize = _Old_size + _Count;
The full error message is:
Exception thrown: write access violation.
this was 0x7FF7941D39D0.
How can I make this code download each Url data without any error?
In your write_cb() callback, you are expecting the buffer parameter to point at a std::string object, but in add_transfer() you are setting CURLOPT_WRITEDATA to point at write_cb itself rather than at a std::string object.
Try something more like this instead:
struct url_info {
const char* url;
std::string data;
};
static url_info urls[] = {
{"http://www.example.com", ""},
{"http://www.example1.com", ""}
};
static const int NUM_URLS = sizeof(urls)/sizeof(urls[0]);
static size_t write_cb(void* ptr, size_t size, size_t nmemb, void* buffer)
{
size_t result = size * nmemb;
static_cast<std::string*>(buffer)->append(static_cast<char*>(ptr), result);
return result;
}
static void add_transfer(CURLM* cm, int i, int* left)
{
...
curl_easy_setopt(eh, CURLOPT_URL, urls[i].url);
curl_easy_setopt(eh, CURLOPT_WRITEDATA, &urls[i].data);
curl_easy_setopt(eh, CURLOPT_PRIVATE, &urls[i]);
...
}
int main(void)
{
...
while ((msg = curl_multi_info_read(cm, &queued)) != NULL) {
if (msg->msg == CURLMSG_DONE) {
/* a transfer ended */
url_info *info;
curl_easy_getinfo(msg->easy_handle, CURLINFO_PRIVATE, reinterpret_cast<char**>(&info));
// use info->url and info->data as needed...
std::cerr << "Transfer completed from " << info->url << ", bytes received: " << info->data.size() << "\n";
}
}
...
}

Capturing the output from libcurl in a std::string in c++ under Windows

I want to capture the libcurl error output that is normally sent to stderr into a std::string instead.
This is the code I have come up with so far:
#include <windows.h>
#include <streambuf>
#include <iostream>
#include <io.h>
#include <sstream>
#include <fstream>
#include <stdlib.h>
#include <stdio.h>
#include <fstream>
#include <string>
#include "curl\curl.h"
std::string data; //will hold the url's contents
// callback function for curl
size_t writeCallback(char* buf, size_t size, size_t nmemb, void* up)
{ //callback must have this declaration
//buf is a pointer to the data that curl has for us
//size*nmemb is the size of the buffer
for (int c = 0; c < size * nmemb; c++)
{
data.push_back(buf[c]);
}
return size * nmemb; //tell curl how many bytes we handled
}
int main()
{
CURL* curl; //our curl object
curl_global_init(CURL_GLOBAL_ALL); //pretty obvious
curl = curl_easy_init();
CURLcode res;
char errbuf[CURL_ERROR_SIZE];
curl_easy_setopt(curl, CURLOPT_URL, "http://ww.example.com/path");
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, &writeCallback);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L); //tell curl to output its progress
int old;
FILE* DataFile;
old = _dup(2); // redirect stderr to "old"
if (fopen_s(&DataFile, "data", "w") != 0)
{
puts("Can't open file 'Datafile'\n");
exit(1);
}
// stderr now refers to file "Datafile"
if (-1 == _dup2(_fileno(DataFile), 2))
{
perror("Can't _dup2 stderr");
exit(1);
}
res = curl_easy_perform(curl);
// Flush stderr stream buffer so it goes to correct file
fflush(stderr);
fclose(DataFile);
// Restore original stderr
_dup2(old, 2);
curl_easy_cleanup(curl);
curl_global_cleanup();
std::ifstream filetoread(DataFile);
if (filetoread)
{
// get length of file:
filetoread.seekg(0, filetoread.end);
int length = filetoread.tellg();
filetoread.seekg(0, filetoread.beg);
char* buffer = new (std::nothrow) char[length];
if (buffer == nullptr)
{
std::cout << "Error allocating buffer.\nvariable 'length' is: " << length << std::endl;
return 0;
}
std::cout << "Reading " << length << " characters... ";
// read data as a block:
filetoread.read(buffer, length);
if (filetoread)
std::cout << "all characters read successfully.";
else
std::cout << "error: only " << filetoread.gcount() << " could be read";
filetoread.close();
// ...buffer contains the entire file...
std::string buffertext(buffer);
delete[] buffer;
}
_flushall();
}
It seems to me that the file Datafile is empty since the variable length is -1. Can anybody help with this code? The examples in redirect stdout/stderr to a string haven't proved to be a solution to my question so far.
I'd use the FUNCTION/DATA pairs in a wrapper class to be able to create event handlers that are called automatically for the events you'd like to handle. Here's an example with comments in the code:
#include "curl/curl.h"
#include <iostream>
#include <memory>
#include <stdexcept>
#include <utility>
// A simple wrapper class for the C functions in libcurl
class EasyCurly {
public:
// default constructor
EasyCurly() : handle(curl_easy_init()) {
if(handle == nullptr) throw std::runtime_error("curl_easy_init failed");
// Set "this" as data pointer in callbacks to be able to make a call to the
// correct EasyCurly object. There are a lot more callback functions you
// could add here if you need them.
setopt(CURLOPT_WRITEDATA, this);
setopt(CURLOPT_DEBUGDATA, this);
setopt(CURLOPT_XFERINFODATA, this);
// Setup of proxy/callback functions. There should be one for each function
// above.
setopt(CURLOPT_WRITEFUNCTION, write_callback);
setopt(CURLOPT_DEBUGFUNCTION, debug_callback);
setopt(CURLOPT_XFERINFOFUNCTION, progress_callback);
// some default options, remove those you usually don't want
setopt(CURLOPT_NOPROGRESS, 0); // turn on progress callbacks
setopt(CURLOPT_FOLLOWLOCATION, 1L); // redirects
setopt(CURLOPT_HTTPPROXYTUNNEL, 1L); // corp. proxies etc.
// setopt(CURLOPT_REDIR_PROTOCOLS, CURLPROTO_HTTP | CURLPROTO_HTTPS);
setopt(CURLOPT_VERBOSE, 1L); // we want it all
}
// copy constructor
EasyCurly(const EasyCurly& other) : handle(curl_easy_duphandle(other.handle)) {
if(handle == nullptr) throw std::runtime_error("curl_easy_duphandle failed");
// State information is not shared when using curl_easy_duphandle. Only the
// options you've set (so you can create one CURL object, set its options and
// then use as a template for other objects. The document and debug data are
// therefor also not copied.
}
// move constructor
EasyCurly(EasyCurly&& other) :
handle(std::exchange(other.handle, nullptr)),
m_document(std::move(other.m_document)), m_debug(std::move(other.m_debug)) {}
// copy assignment
EasyCurly& operator=(const EasyCurly& other) {
CURL* tmp_handle = curl_easy_duphandle(other.handle);
if(handle == nullptr) throw std::runtime_error("curl_easy_duphandle failed");
// dup succeeded, now destroy any handle we might have and copy the tmp
curl_easy_cleanup(handle);
handle = tmp_handle;
return *this;
}
// move assignment
EasyCurly& operator=(EasyCurly&& other) {
std::swap(handle, other.handle);
std::swap(m_document, other.m_document);
std::swap(m_debug, other.m_debug);
return *this;
}
virtual ~EasyCurly() { curl_easy_cleanup(handle); }
// To be able to use an instance of EasyCurly with C interfaces if you don't add
// a function to this class for it, this operator will help
operator CURL*() { return handle; }
// generic curl_easy_setopt wrapper
template<typename T>
CURLcode setopt(CURLoption option, T v) {
return curl_easy_setopt(handle, option, v);
}
// perform by supplying url
CURLcode perform(std::string_view url) {
setopt(CURLOPT_URL, url.data());
return perform();
}
// perform with a previously supplied url
CURLcode perform() {
m_document.clear();
m_debug.clear();
return curl_easy_perform(handle);
}
// get collected data
std::string const& document() const { return m_document; }
std::string const& debug() const { return m_debug; }
// callbacks from proxy functions
virtual size_t on_write(char* ptr, size_t total_size) {
m_document.insert(m_document.end(), ptr, ptr + total_size);
// std::cout << "<write_callback> size " << total_size << "\n";
return total_size;
}
virtual int on_debug(curl_infotype /*type*/, char* data, size_t size) {
m_debug.insert(m_debug.end(), data, data + size);
// std::cout << "<debug>\n";
return 0; // must return 0
}
// override this to make a progress bar ...
virtual int on_progress(curl_off_t /*dltotal*/, curl_off_t /*dlnow*/,
curl_off_t /*ultotal*/, curl_off_t /*ulnow*/) {
// std::cout << "<progress>\n";
return 0;
}
private:
// a private class to initialize and cleanup once
class CurlGlobalInit {
public:
CurlGlobalInit() {
// std::cout << "CurlGlobalInit\n";
CURLcode res = curl_global_init(CURL_GLOBAL_ALL);
if(res) throw std::runtime_error("curl_global_init failed");
}
~CurlGlobalInit() {
// std::cout << "~CurlGlobalInit\n";
curl_global_cleanup();
}
};
// callback functions - has to be static to work with the C interface in curl
// use the data pointer (this) that we set in the constructor and cast it back
// to a EasyCurly* and call the event handler in the correct object.
static size_t write_callback(char* ptr, size_t size, size_t nmemb,
void* userdata) {
EasyCurly* ecurly = static_cast<EasyCurly*>(userdata);
return ecurly->on_write(ptr, nmemb * size); // size==1 really
}
static int debug_callback(CURL* /*handle*/, curl_infotype type, char* data,
size_t size, void* userptr) {
EasyCurly* ecurly = static_cast<EasyCurly*>(userptr);
return ecurly->on_debug(type, data, size);
}
static int progress_callback(void* clientp, curl_off_t dltotal, curl_off_t dlnow,
curl_off_t ultotal, curl_off_t ulnow) {
EasyCurly* ecurly = static_cast<EasyCurly*>(clientp);
return ecurly->on_progress(dltotal, dlnow, ultotal, ulnow);
}
// resources
CURL* handle;
std::string m_document{};
std::string m_debug{};
// a static initializer object
static CurlGlobalInit setup_and_teardown;
};
// This must be defined in your .cpp file if you use this and split it into header
// and implementation parts.
EasyCurly::CurlGlobalInit EasyCurly::setup_and_teardown{};
// end of wrapper class
// --------------------------------------------------------------------------
// with that in place, the rest becomes something like this:
int main() {
EasyCurly curl; // our curl object
CURLcode res = curl.perform("http://www.google.com/");
std::cout << "result: " << res << "\n";
std::cout << "--- document size " << curl.document().size() << " captured ---\n"
<< curl.document() << "---------\n\n";
std::cout << "--- debug size " << curl.debug().size() << " captured ---\n"
<< curl.debug() << "---------\n";
}

Generate an OAuth access token with openssl and c++

I need to get an access token (for a service account) for the google's OAuth authentication service. I tried several things an studied a lot of on the web but don't succeed.
Basically i followed https://developers.google.com/accounts/docs/OAuth2ServiceAccount
What i have done (VS2013):
int _tmain(int argc, _TCHAR* argv[])
{
Json::Value jwt_header;
Json::Value jwt_claim_set;
std::string jwt_b64;
std::time_t t = std::time(NULL);
Json::FastWriter jfw;
Json::StyledWriter jsw;
/* Create jwt header */
jwt_header["alg"] = "RS256";
jwt_header["typ"] = "JWT";
std::cout << jsw.write(jwt_header);
/* Create jwt claim set */
jwt_claim_set["iss"] = "myid#developer.gserviceaccount.com"; /* service account email address */
jwt_claim_set["scope"] = "https://www.googleapis.com/auth/plus.me" /* scope of requested access token */;
jwt_claim_set["aud"] = "https://accounts.google.com/o/oauth2/token"; /* intended target of the assertion for an access token */
jwt_claim_set["iad"] = std::to_string(t); /* issued time */
jwt_claim_set["exp"] = std::to_string(t+3600); /* expire time*/
std::cout << jsw.write(jwt_claim_set);
/* create http POST request body */
/* for header */
std::string json_buffer;
std::string json_buffer1;
json_buffer = jfw.write(jwt_header);
json_buffer = json_buffer.substr(0, json_buffer.size() - 1);
json_buffer = base64_encode(reinterpret_cast<const unsigned char*>(json_buffer.c_str()), json_buffer.length(), true); /* urlsafeBasic64 encode*/
json_buffer1.clear();
std::remove_copy(json_buffer.begin(), json_buffer.end(), std::back_inserter(json_buffer1), '=');
jwt_b64 = json_buffer1;
jwt_b64 += ".";
/* for claim set */
json_buffer = jfw.write(jwt_claim_set);
json_buffer = json_buffer.substr(0, json_buffer.size() - 1);
json_buffer = base64_encode(reinterpret_cast<const unsigned char*>(json_buffer.c_str()), json_buffer.length(), true); /* urlsafeBasic64 encode*/
json_buffer1.clear();
std::remove_copy(json_buffer.begin(), json_buffer.end(), std::back_inserter(json_buffer1), '=');
jwt_b64 += json_buffer1;
/* for signature */
std::string jwt_signature = jws_sign(jwt_b64, "key.p12");
if (!jwt_signature.empty())
{
jwt_b64 += ".";
json_buffer1.clear();
std::remove_copy(jwt_signature.begin(), jwt_signature.end(), std::back_inserter(json_buffer1), '=');
jwt_b64 += json_buffer1;
write2file("jwt.bat", jwt_b64); /* for test purpose calling with curl */
}
else
std::cout << "Error creating signature";
return 0;
}
int write2file(std::string filename, std::string data)
{
std::ofstream f(filename);
f << "%curl% -d \"grant_type=urn%%3Aietf%%3Aparams%%3Aoauth%%3Agrant-type%%3Ajwt-bearer&assertion=";
f << data;
f << "\" https://accounts.google.com/o/oauth2/token";
f.close();
return 0;
}
std::string jws_sign(std::string data, std::string pkcs12_path) {
SHA256_CTX mctx;
unsigned char hash[SHA256_DIGEST_LENGTH];
size_t hlen = SHA256_DIGEST_LENGTH;
const char *buf = data.c_str();
int n = strlen((const char*) buf);
SHA256_Init(&mctx);
SHA256_Update(&mctx, buf, n);
SHA256_Final(hash, &mctx);
std::string signature_b64;
unsigned char *sig = NULL;
size_t slen = 0;
EVP_PKEY_CTX *kctx;
EVP_PKEY *key = getPkey(pkcs12_path);
kctx = EVP_PKEY_CTX_new(key, NULL);
if (!kctx) goto err;
if (EVP_PKEY_sign_init(kctx) <= 0) goto err;
if (EVP_PKEY_CTX_set_rsa_padding(kctx, RSA_PKCS1_PADDING) <= 0) goto err;
if (EVP_PKEY_CTX_set_signature_md(kctx, EVP_sha256()) <= 0) goto err;
/* Determine buffer length */
if (EVP_PKEY_sign(kctx, NULL, &slen, hash, hlen) <= 0) goto err;
sig = (unsigned char *) OPENSSL_malloc(slen);
if (!sig) goto err;
if (EVP_PKEY_sign(kctx, sig, &slen, hash, hlen) <= 0) goto err;
signature_b64 = base64_encode(sig, (unsigned int)slen, true);
return signature_b64;
err:
/* Clean up */
EVP_cleanup();
signature_b64.clear();
return signature_b64;
}
All i receive back is
{
"error" : "invalid_grant"
}
So if someone can point me into the right direction would be great.
It would also help, if someone can point me to get the thing working by manually generating the jwt request out of openssl commands.
I'm working with VS2013
I found my mistake - was simply a typo :(
jwt_claim_set["iad"] = std::to_string(t); /* issued time */
needs to be
jwt_claim_set["iat"] = std::to_string(t); /* issued time */
The code works and generate valid token requests.
I've made a class for authentication on C++, will leave it here, may be someone may need it.
// YOU SHOULD GO TO Credentials SECTION FOR YOUR PROJECT AT https://console.developers.google.com/
// MAKE Service Account AND GET AUTHENTICATION JSON FROM IT,
// PLACE IT TO BUILD FOLDER AND CALL IT google_service_account.json
#include <string>
#include <fstream>
#include <sstream>
#include <iostream>
#include <iomanip>
// SSL INCLUDES
#include <openssl/aes.h>
#include <openssl/evp.h>
#include <openssl/pem.h>
#include <openssl/bio.h>
// https://github.com/nlohmann/json
#include <nlohmann/json.hpp>
using json = nlohmann::json;
class TGoogleAuthCpp {
json serviceAccountJSON;
bool serviceAccountExists;
void readServiceAccountJson();
RSA* createPrivateRSA(std::string key);
bool RSASign( RSA* rsa,
const unsigned char* Msg,
size_t MsgLen,
unsigned char** EncMsg,
size_t* MsgLenEnc);
std::string signMessage(std::string privateKey, std::string plainText);
std::string url_encode(const std::string &value);
std::string base64_encode(const std::string &in);
public:
TGoogleAuthCpp();
int createRequest();
};
TGoogleAuthCpp::TGoogleAuthCpp() {
serviceAccountExists = false;
readServiceAccountJson();
}
RSA* TGoogleAuthCpp::createPrivateRSA(std::string key) {
RSA *rsa = NULL;
const char* c_string = key.c_str();
BIO * keybio = BIO_new_mem_buf((void*)c_string, -1);
if (keybio==NULL) {
return 0;
}
rsa = PEM_read_bio_RSAPrivateKey(keybio, &rsa,NULL, NULL);
return rsa;
}
bool TGoogleAuthCpp::RSASign( RSA* rsa,
const unsigned char* Msg,
size_t MsgLen,
unsigned char** EncMsg,
size_t* MsgLenEnc) {
EVP_MD_CTX* m_RSASignCtx = EVP_MD_CTX_create();
EVP_PKEY* priKey = EVP_PKEY_new();
EVP_PKEY_assign_RSA(priKey, rsa);
if (EVP_DigestSignInit(m_RSASignCtx,NULL, EVP_sha256(), NULL,priKey)<=0) {
return false;
}
if (EVP_DigestSignUpdate(m_RSASignCtx, Msg, MsgLen) <= 0) {
return false;
}
if (EVP_DigestSignFinal(m_RSASignCtx, NULL, MsgLenEnc) <=0) {
return false;
}
*EncMsg = (unsigned char*)malloc(*MsgLenEnc);
if (EVP_DigestSignFinal(m_RSASignCtx, *EncMsg, MsgLenEnc) <= 0) {
return false;
}
EVP_MD_CTX_cleanup(m_RSASignCtx);
return true;
}
std::string TGoogleAuthCpp::signMessage(std::string privateKey, std::string plainText) {
RSA* privateRSA = createPrivateRSA(privateKey);
unsigned char* encMessage;
size_t encMessageLength;
RSASign(privateRSA, (unsigned char*) plainText.c_str(), plainText.length(), &encMessage, &encMessageLength);
std::string str1((char *)(encMessage), encMessageLength);
free(encMessage);
return base64_encode(str1);
}
void TGoogleAuthCpp::readServiceAccountJson() {
std::string fname = "google_service_account.json";
std::string line;
std::ifstream myfile (fname);
if (myfile.good()) {
std::stringstream ss;
if (myfile.is_open()) {
while (getline(myfile, line)) {
ss << line << '\n';
}
myfile.close();
serviceAccountJSON = json::parse(ss.str());
serviceAccountExists = true;
}
}
}
std::string TGoogleAuthCpp::base64_encode(const std::string &in) {
std::string out;
std::string base64_encode_b = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";//=
int val=0, valb=-6;
for (unsigned char c : in) {
val = (val<<8) + c;
valb += 8;
while (valb>=0) {
out.push_back(base64_encode_b[(val>>valb)&0x3F]);
valb-=6;
}
}
if (valb>-6) out.push_back(base64_encode_b[((val<<8)>>(valb+8))&0x3F]);
while (out.size()%4) out.push_back('=');
return out;
}
std::string TGoogleAuthCpp::url_encode(const std::string &value) {
std::ostringstream escaped;
escaped.fill('0');
escaped << std::hex;
for (std::string::const_iterator i = value.begin(), n = value.end(); i != n; ++i) {
std::string::value_type c = (*i);
// Keep alphanumeric and other accepted characters intact
if (isalnum(c) || c == '-' || c == '_' || c == '.' || c == '~') {
escaped << c;
continue;
}
// Any other characters are percent-encoded
escaped << std::uppercase;
escaped << '%' << std::setw(2) << int((unsigned char) c);
escaped << std::nouppercase;
}
return escaped.str();
}
int TGoogleAuthCpp::createRequest() {
if (!serviceAccountExists) return 0;
json jwt_header;
json jwt_claim_set;
std::time_t t = std::time(NULL);
// Create jwt header
jwt_header["alg"] = "RS256";
jwt_header["typ"] = "JWT";
// Create jwt claim set
jwt_claim_set["iss"] = serviceAccountJSON["client_email"]; /* service account email address */
jwt_claim_set["scope"] = "https://www.googleapis.com/auth/androidpublisher" /* scope of requested access token */;
jwt_claim_set["aud"] = serviceAccountJSON["token_uri"]; /* intended target of the assertion for an access token */
jwt_claim_set["iat"] = t; /* issued time */
jwt_claim_set["exp"] = t+3600; /* expire time*/
// web token
std::stringstream jwt_ss;
// header
jwt_ss << base64_encode(jwt_header.dump());
jwt_ss << ".";
// claim set
jwt_ss << base64_encode(jwt_claim_set.dump());
// signature
std::string signed_msg = signMessage(serviceAccountJSON["private_key"], jwt_ss.str());
jwt_ss << "." << signed_msg;
std::stringstream post_body_ss;
post_body_ss << "curl -d '";
post_body_ss << "grant_type=" << url_encode("urn:ietf:params:oauth:grant-type:jwt-bearer");
post_body_ss << "&assertion=" << url_encode(jwt_ss.str());
post_body_ss << "' https://oauth2.googleapis.com/token";
std::string post_body = post_body_ss.str();
std::cout << post_body << std::endl;
return 1;
}
int main() {
TGoogleAuthCpp auth;
int res = auth.createRequest();
}

cURL: Handle multiple asynchronous requests

I've never really done anything multithreaded or asynchronous in c++, I only used cURL to do single synchronous requests so far.
In order to better visualize what I'm trying to do, I wrote a simple Javascript which would do what I want to do with cURL in C++.
function AddRequest( method, url, data, id ) {
var httpObj = new ActiveXObject("Msxml2.XMLHTTP.6.0"); //new XMLHttpRequest();
httpObj.onreadystatechange = function() {
if (httpObj.readyState == 4)
ResponseCallback( httpObj, id );
};
httpObj.Open( method, url, true );
httpObj.Send( data );
}
function ResponseCallback( httpObj, id ) {
WScript.Echo( id ); //alert( id );
WScript.Echo( httpObj.ResponseText ); //alert( httpObj.ResponseText );
}
//It could now be used like this:
AddRequest("GET","http://example.com/","",1);
AddRequest("GET","https://www.facebook.com","",2);
WScript.Echo( "all requests sent" ); //alert( "all requests sent" );
//these requests are all done at the same time
//and every time a request has finished it calls the ResponseCallback() function,
//telling it which request has finished
CURL just seems to be COMPLETELY different and unnecessary more complicated than XmlHttpRequest, even though both are just sending http requests...
Here is my first approach (based on the answer of hogren):
#include "stdafx.hpp"
#include <iostream> //#include <stdio.h>
#include <curl.h>
#include <pthread.h>
#include <map>
#include <string>
using namespace std;
bool printing = false; //will allow us to prevent prints overlapping each other
struct requestStruct { //will allow us to pass more than one argument to the threaded functions
int id;
const char* url;
const char* method;
const char* body;
map<const char*, const char*> headers;
const char* proxy;
int timeout;
};
struct responseStruct { //will allow us to return more than one value from the Request function
long statusCode;
//map<const char*, const char*> headers;
const char* body;
};
size_t writeToString(void *ptr, size_t size, size_t count, void *stream) {
((string*)stream)->append((char*)ptr, 0, size* count);
return size* count;
}
static void *ResponseCallback(int id, struct responseStruct *response) {
long statusCode = response -> statusCode;
//map<const char*, const char*> headers = response -> headers;
const char* body = response -> body;
//while (printing) {} //wait for other threads to stop printing
printing = true; //tell other threads to not print anything
cout << id << " response received! Code: " << statusCode << endl << body << endl;
printing = false; //tell other threads printing is okay again
return NULL;
}
struct responseStruct HttpRequest(const char* url, const char* method, const char* body, map<const char*, const char*> &headers, const char* proxy, long timeout) {
CURL *curl;
curl = curl_easy_init();
long statusCode = 0;
map<const char*, const char*> respHeaders;
string respBody;
string _url(url);
string _method(method);
string _proxy(proxy);
struct curl_slist *headerList = NULL;
string headerString;
curl_easy_setopt(curl, CURLOPT_URL, url); //set url
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, method); //set method
for (auto header=headers.begin(); header!=headers.end(); ++header) { //make header list
headerString = header->first;
headerString.append(": ").append(header->second);
headerList = curl_slist_append(headerList, headerString.c_str());
//cout << headerString << '\n';
}
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headerList); //set headers
if (_method == "POST" || _method == "PUT" || _method == "DELETE") //set body if the request method would allow it
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, body);
if (_url.find(string("https://")) != string::npos) //set ssl verifypeer if it's an https url
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
if (_proxy != "") //set proxy
curl_easy_setopt(curl, CURLOPT_PROXY, proxy);
if (timeout != 0) //set timeout
curl_easy_setopt(curl, CURLOPT_TIMEOUT, timeout);
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L); //follow redirects
//curl_easy_setopt(curl, CURLOPT_HEADERFUNCTION, writeToString);
//curl_easy_setopt(curl, CURLOPT_WRITEHEADER, &respHeaders); //to receive response headers
//??
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writeToString);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &respBody); //to receive response body
curl_easy_perform(curl); //send the request
curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &statusCode); //get status code
struct responseStruct response;
response.statusCode = statusCode;
//response.headers;
response.body = respBody.c_str();
curl_easy_cleanup(curl);
return response;
}
static void *AddRequest( void *arguments ) {
// get arguments:
struct requestStruct *args = (struct requestStruct*)arguments;
int id = args->id;
const char* url = args->url;
const char* method = args->method;
const char* body = args->body;
map<const char*, const char*> headers = args->headers;
const char* proxy = args->proxy;
int timeout = args->timeout;
// print arguments:
//while (printing) {} //wait for other threads to stop printing
//printing = true; //tell other threads to not print anything
// cout << id << endl << url << endl << method << endl;
//printing = false; //tell the other threads it's okay to print again now
struct responseStruct response = HttpRequest(url, method, body, headers, proxy, timeout);
ResponseCallback(id,&response);
pthread_exit(0);
return NULL;
}
int main() {
//map<const char*, const char*> headers;
//headers["User-Agent"] = "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:32.0) Gecko/20100101 Firefox/32.0";
//struct responseStruct response = HttpRequest("https://facebook.com", "GET", "", headers, "localhost:8888", 6000);
//cout << response.body << endl;
pthread_t threads[3];
struct requestStruct reqArguments[3];
map<const char*, const char*> headers;
headers["User-Agent"] = "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:32.0) Gecko/20100101 Firefox/32.0";
const char* proxy = "";
reqArguments[0].id = 0;
reqArguments[0].url = "https://www.facebook.com/";
reqArguments[0].method = "GET";
reqArguments[0].headers = headers;
reqArguments[0].body = "";
reqArguments[0].proxy = proxy;
reqArguments[0].timeout = 6000;
pthread_create(&threads[0], NULL, &AddRequest, (void *)&reqArguments[0]); //create a thread on AddRequest() passing a full struct of arguments
reqArguments[1].id = 1;
reqArguments[1].url = "https://www.facebook.com/";
reqArguments[1].method = "GET";
reqArguments[1].headers = headers;
reqArguments[1].body = "";
reqArguments[1].proxy = proxy;
reqArguments[1].timeout = 6000;
pthread_create(&threads[1], NULL, &AddRequest, (void *)&reqArguments[1]); //create a thread on AddRequest() passing a full struct of arguments
reqArguments[2].id = 2;
reqArguments[2].url = "https://www.facebook.com/";
reqArguments[2].method = "GET";
reqArguments[2].headers = headers;
reqArguments[2].body = "";
reqArguments[2].proxy = proxy;
reqArguments[2].timeout = 6000;
pthread_create(&threads[2], NULL, &AddRequest, (void *)&reqArguments[2]); //create a thread on AddRequest() passing a full struct of arguments
getchar(); //prevent console from closing instantly
return 0;
}
I'm not really sure if I'm doing the whole pthread thing correctly..
There are some issues:
1. For some reason only the first request succeeds the others aren't even sent.
UNLESS I uncomment the first 4 lines of the main function which will do a direct request without a new thread, but I obviously don't want to use that code.
2. The HttpRequest() function doesn't return the response html code properly, I only receive garbage.
I think issue 2 might be a pointer related issue with the return struct of HttpRequest(), but I wasn't able to fix it. :(
3. My last and not that important problem is that I don't know how to receive the response headers and put them in a map.
Btw: I'm compiling with Visual C++ 2010 and I'm debugging the http traffic with Fiddler.
EDIT : This is your code that I corrected.
There was not really an error. But after several tests, I saw that to launch several curl_perform in the same time cause issues. So I added a delay (5000ms is large, you can reduce it).
And pthread_exit() caused prolems with response error.
#include "stdafx.hpp"
#include <iostream> //#include <stdio.h>
#include <curl/curl.h>
#include <pthread.h>
#include <map>
#include <string>
using namespace std;
bool printing = false; //will allow us to prevent prints overlapping each other
#if defined(__WIN32__) || defined(_WIN32) || defined(WIN32) || defined(__WINDOWS__) || defined(__TOS_WIN__)
#include <windows.h>
inline void delay( unsigned long ms )
{
Sleep( ms );
}
#else /* presume POSIX */
#include <unistd.h>
inline void delay( unsigned long ms )
{
usleep( ms * 1000 );
}
#endif
struct requestStruct { //will allow us to pass more than one argument to the threaded functions
int id;
const char* url;
const char* method;
const char* body;
map<const char*, const char*> headers;
const char* proxy;
int timeout;
};
struct responseStruct { //will allow us to return more than one value from the Request function
long statusCode;
//map<const char*, const char*> headers;
const char* body;
};
size_t writeToString(void *ptr, size_t size, size_t count, void *stream) {
((string*)stream)->append((char*)ptr, 0, size* count);
return size* count;
}
static void *ResponseCallback(int id, struct responseStruct *response) {
long statusCode = response -> statusCode;
//map<const char*, const char*> headers = response -> headers;
const char* body = response -> body;
//while (printing) {} //wait for other threads to stop printing
printing = true; //tell other threads to not print anything
cout << id << " response received! Code: " << statusCode << endl << body << endl;
printing = false; //tell other threads printing is okay again
return NULL;
}
struct responseStruct HttpRequest(const char* url, const char* method, const char* body, map<const char*, const char*> &headers, const char* proxy, long timeout) {
CURL *curl;
curl = curl_easy_init();
long statusCode = 0;
map<const char*, const char*> respHeaders;
string respBody;
string _url(url);
string _method(method);
string _proxy(proxy);
struct curl_slist *headerList = NULL;
string headerString;
curl_easy_setopt(curl, CURLOPT_URL, url); //set url
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, method); //set method
for (std::map<const char*, const char*>::iterator header=headers.begin(); header!=headers.end(); ++header) { //make header list
headerString = header->first;
headerString.append(": ").append(header->second);
headerList = curl_slist_append(headerList, headerString.c_str());
//cout << headerString << '\n';
}
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headerList); //set headers
if (_method == "POST" || _method == "PUT" || _method == "DELETE") //set body if the request method would allow it
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, body);
if (_url.find(string("https://")) != string::npos) //set ssl verifypeer if it's an https url
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
if (_proxy != "") //set proxy
curl_easy_setopt(curl, CURLOPT_PROXY, proxy);
if (timeout != 0) //set timeout
curl_easy_setopt(curl, CURLOPT_TIMEOUT, timeout);
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L); //follow redirects
//curl_easy_setopt(curl, CURLOPT_HEADERFUNCTION, writeToString);
//curl_easy_setopt(curl, CURLOPT_WRITEHEADER, &respHeaders); //to receive response headers
//??
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writeToString);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &respBody); //to receive response body
static int i=0;
delay(5000*(i++));
std::cout << "url: " << _url << ";" << std::endl;
curl_easy_perform(curl); //send the request
curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &statusCode); //get status code
struct responseStruct response;
response.statusCode = statusCode;
//response.headers;
response.body = respBody.c_str();
curl_easy_cleanup(curl);
return response;
}
static void *AddRequest( void *arguments ) {
// get arguments:
struct requestStruct *args = (struct requestStruct*)arguments;
int id = args->id;
const char* url = args->url;
const char* method = args->method;
const char* body = args->body;
map<const char*, const char*> headers = args->headers;
const char* proxy = args->proxy;
int timeout = args->timeout;
// print arguments:
//while (printing) {} //wait for other threads to stop printing
//printing = true; //tell other threads to not print anything
// cout << id << endl << url << endl << method << endl;
//printing = false; //tell the other threads it's okay to print again now
struct responseStruct response = HttpRequest(url, method, body, headers, proxy, timeout);
ResponseCallback(id,&response);
/* this code cause trouble (no response code) */
//pthread_exit(0);
return NULL;
}
int main() {
//map<const char*, const char*> headers;
//headers["User-Agent"] = "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:32.0) Gecko/20100101 Firefox/32.0";
//struct responseStruct response = HttpRequest("https://facebook.com", "GET", "", headers, "localhost:8888", 6000);
//cout << response.body << endl;
pthread_t threads[3];
struct requestStruct reqArguments[3];
map<const char*, const char*> headers;
headers["User-Agent"] = "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:32.0) Gecko/20100101 Firefox/32.0";
const char* proxy = "";
reqArguments[0].id = 0;
reqArguments[0].url = "https://www.duckduckgo.com/";
reqArguments[0].method = "GET";
reqArguments[0].headers = headers;
reqArguments[0].body = "";
reqArguments[0].proxy = proxy;
reqArguments[0].timeout = 6000;
pthread_create(&threads[0], NULL, &AddRequest, (void *)&reqArguments[0]); //create a thread on AddRequest() passing a full struct of arguments
reqArguments[1].id = 1;
reqArguments[1].url = "https://www.google.com/";
reqArguments[1].method = "GET";
reqArguments[1].headers = headers;
reqArguments[1].body = "";
reqArguments[1].proxy = proxy;
reqArguments[1].timeout = 6000;
pthread_create(&threads[1], NULL, &AddRequest, (void *)&reqArguments[1]); //create a thread on AddRequest() passing a full struct of arguments
reqArguments[2].id = 2;
reqArguments[2].url = "https://www.facebook.com/";
reqArguments[2].method = "GET";
reqArguments[2].headers = headers;
reqArguments[2].body = "";
reqArguments[2].proxy = proxy;
reqArguments[2].timeout = 6000;
pthread_create(&threads[2], NULL, &AddRequest, (void *)&reqArguments[2]); //create a thread on AddRequest() passing a full struct of arguments
// getchar();
// that is cleaner
for (int i=0; i<3; ++i) {
int rc = pthread_join(threads[i], NULL);
printf("In main: thread %d is complete\n", i);
}
return 0;
}
For the last question about headers, please post an other question on stackoverflow. Because there are yet many subjects in one (I think).
And a little advice, work with objects is very more easy to write and to read code sources.
END EDIT
This is a copy of the official example to make multi-threading with libcurl :
http://curl.haxx.se/libcurl/c/multithread.html
/***************************************************************************
* _ _ ____ _
* Project ___| | | | _ \| |
* / __| | | | |_) | |
* | (__| |_| | _ <| |___
* \___|\___/|_| \_\_____|
*
* Copyright (C) 1998 - 2011, Daniel Stenberg, <daniel#haxx.se>, et al.
*
* This software is licensed as described in the file COPYING, which
* you should have received as part of this distribution. The terms
* are also available at http://curl.haxx.se/docs/copyright.html.
*
* You may opt to use, copy, modify, merge, publish, distribute and/or sell
* copies of the Software, and permit persons to whom the Software is
* furnished to do so, under the terms of the COPYING file.
*
* This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
* KIND, either express or implied.
*
***************************************************************************/
/* A multi-threaded example that uses pthreads extensively to fetch
* X remote files at once */
#include <stdio.h>
#include <pthread.h>
#include <curl/curl.h>
#define NUMT 4
/*
List of URLs to fetch.
If you intend to use a SSL-based protocol here you MUST setup the OpenSSL
callback functions as described here:
http://www.openssl.org/docs/crypto/threads.html#DESCRIPTION
*/
const char * const urls[NUMT]= {
"http://curl.haxx.se/",
"ftp://cool.haxx.se/",
"http://www.contactor.se/",
"www.haxx.se"
};
static void *pull_one_url(void *url)
{
CURL *curl;
curl = curl_easy_init();
curl_easy_setopt(curl, CURLOPT_URL, url);
curl_easy_perform(curl); /* ignores error */
curl_easy_cleanup(curl);
return NULL;
}
/*
int pthread_create(pthread_t *new_thread_ID,
const pthread_attr_t *attr,
void * (*start_func)(void *), void *arg);
*/
int main(int argc, char **argv)
{
pthread_t tid[NUMT];
int i;
int error;
/* Must initialize libcurl before any threads are started */
curl_global_init(CURL_GLOBAL_ALL);
for(i=0; i< NUMT; i++) {
error = pthread_create(&tid[i],
NULL, /* default attributes please */
pull_one_url,
(void *)urls[i]);
if(0 != error)
fprintf(stderr, "Couldn't run thread number %d, errno %d\n", i, error);
else
fprintf(stderr, "Thread %d, gets %s\n", i, urls[i]);
}
/* now wait for all threads to terminate */
for(i=0; i< NUMT; i++) {
error = pthread_join(tid[i], NULL);
fprintf(stderr, "Thread %d terminated\n", i);
}
return 0;
}
You can, for a interactive use, transform the urls Array to a vector.
I hope that it will help you !

Adding to char array isn't working

I'm trying to read a text file line by line, and add each line to a char array. But the lines aren't added, at all.
//This is the default char array that comes with the cURL code.
char *text[]={
"one\n",
"two\n",
"three\n",
" Hello, this is CURL email SMTP\n",
NULL
};
/*Now we're going to replace that char array, with an array that holds the contents of a textfile.
We'll read a textfile out line by line, and add each line to the char array.
*/
void makemailmessage()
{
text[0] = '\0'; //Clear text
text[0] = "testy\n"; //First line in new char array
//Read the text file, add each line to the char array.
string line;
ifstream myfile ("C:\\Users\\admin\\Downloads\\bbb.txt");
int counter;
counter = 1;
if (myfile.is_open())
{
while ( myfile.good() )
{
getline (myfile,line);
//Convert the string variable "line" to a char (a)
char *a=new char[line.size()+1];
a[line.size()]=0;
memcpy(a,line.c_str(),line.size());
//Add \n to the end of "a" (new char will be "str")
char str[80];
strcpy (str,a);
strcat (str,"\n");
//Add "str" to the char array "text"
text[counter] = str;
text[counter+1] = "test\n"; //Also added this for testing purposes
write_data("C:\\Users\\admin\\Downloads\\checkit.txt", str); //Also for testing purposes
//Increase counter by 2 because we added two new items to the char array "text"
counter++;
counter++;
}
myfile.close();
text[counter-1] = "testy2\n"; //Ad another text line
text[counter] = NULL; //End char array
}
Each str is written correctly to checkit.txt but for some reason it is not added to the char array because I end up with the char array looking like this:
testy
test
test
testy2
What am I doing wrong?
UPDATE2:
The reason I am trying to make a char array is because the cURL function I am using needs a char array to form the email body. This is the important part of the cURL code.
static size_t read_callback(void *ptr, size_t size, size_t nmemb, void *userp)
{
struct WriteThis *pooh = (struct WriteThis *)userp;
const char *data;
if(size*nmemb < 1)
return 0;
data = text[pooh->counter]; //This part is using the char array.
if(data) {
size_t len = strlen(data);
memcpy(ptr, data, len);
pooh->counter++;
return len;
}
return 0;
}
Here's the full code
Okay, after chatting on this a bit more, here is a fix:
C++ version
Full code file here: https://gist.github.com/1342118#file_test.cpp
Replace the relevant code with:
#include <vector>
#include <fstream>
// ...
std::vector<std::string> text;
static int read_text(char* fname)
{
//Read the text file, add each line to the char array.
std::ifstream myfile (fname);
std::string line;
while (std::getline(myfile, line))
text.push_back(line + '\n');
return 0;
}
static size_t read_callback(void *ptr, size_t size, size_t nmemb, void *userp)
{
/* This was already in. */
struct WriteThis *pooh = (struct WriteThis *)userp;
if(size*nmemb < 1)
return 0;
if (pooh->counter < text.size())
{
const std::string& data = text[pooh->counter];
memcpy(ptr, data.data(), data.length());
pooh->counter++; /* advance pointer */
return data.length();
}
return 0; /* no more data left to deliver */
}
Pure C version
Full code file here: https://gist.github.com/1342118#file_test.c
Replace
//This is the default char array that comes with the cURL code.
char *text[]={
"one\n",
"two\n",
"three\n",
" Hello, this is CURL email SMTP\n",
NULL
};
With
char **text = 0;
static int read_text(char* fname)
{
unsigned capacity = 10;
int linecount = 0;
// free_text(); see below
text = realloc(text, capacity*sizeof(*text));
FILE* file = fopen(fname, "r");
if (!file)
{ perror("Opening file"); return 1; }
char buf[2048];
char* line = 0;
while (line = fgets(buf, sizeof(buf), file))
{
if (linecount>=capacity)
{
capacity *= 2;
text = realloc(text, capacity*sizeof(*text));
}
text[linecount++] = strdup(line);
}
fclose(file);
return 0;
}
Hook it up in you main function, e.g. like so
if (argc<2)
{
printf("Usage: %s <email.eml>\n", argv[0]);
exit(255);
} else
{
printf("Reading email body from %s\n", argv[1]);
if (0 != read_text(argv[1]))
exit(254);
}
Or, if you so prefer, just calling read_text("C:\\Users\\admin\\Downloads\\bbb.txt") :)
To really top things off, don't forget to reclaim memory when you're done - properly:
#include "curl/curl.h"
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <memory.h>
#include <string.h>
#define GetCurrentDir getcwd
#define USERNAME "obscured#gmail.com"
#define PASSWORD "obscured"
#define SMTPSERVER "smtp.gmail.com"
#define SMTPPORT ":587"
#define RECIPIENT "<obscured#gmail.com>"
#define MAILFROM "<obscured#gmail.com>"
#define MULTI_PERFORM_HANG_TIMEOUT 60 * 1000
/* Note that you should include the actual meta data headers here as well if
you want the mail to have a Subject, another From:, show a To: or whatever
you think your mail should feature! */
char **text = 0;
void free_text()
{
if (text)
{
char** it;
for (it = text; *it; ++it)
free(*it);
free(text);
text = 0;
}
}
static int read_text(char* fname)
{
unsigned capacity = 10;
int linecount = 0;
free_text();
text = realloc(text, capacity*sizeof(*text));
FILE* file = fopen(fname, "r");
if (!file)
{ perror("Opening file"); return 1; }
char buf[2048];
char* line = 0;
while (line = fgets(buf, sizeof(buf), file))
{
if (linecount>=capacity)
{
capacity *= 2;
text = realloc(text, capacity*sizeof(*text));
}
text[linecount++] = strdup(line);
}
if (linecount>=capacity)
text = realloc(text, (++capacity)*sizeof(*text));
text[linecount] = 0; // terminate
fclose(file);
return 0;
}
struct WriteThis {
int counter;
};
static size_t read_callback(void *ptr, size_t size, size_t nmemb, void *userp)
{
/* This was already in. */
struct WriteThis *pooh = (struct WriteThis *)userp;
const char *data;
if(size*nmemb < 1)
return 0;
data = text[pooh->counter];
if(data) {
size_t len = strlen(data);
memcpy(ptr, data, len);
pooh->counter++; /* advance pointer */
return len;
}
return 0; /* no more data left to deliver */
}
static struct timeval tvnow(void)
{
/*
** time() returns the value of time in seconds since the Epoch.
*/
struct timeval now;
now.tv_sec = (long)time(NULL);
now.tv_usec = 0;
return now;
}
static long tvdiff(struct timeval newer, struct timeval older)
{
return (newer.tv_sec-older.tv_sec)*1000+
(newer.tv_usec-older.tv_usec)/1000;
}
int main(int argc, char** argv)
{
if (argc<2)
{
printf("Usage: %s <email.eml>\n", argv[0]);
exit(255);
} else
{
printf("Reading email body from %s\n", argv[1]);
if (0 != read_text(argv[1]))
exit(254);
}
CURL *curl;
CURLM *mcurl;
int still_running = 1;
struct timeval mp_start;
char mp_timedout = 0;
struct WriteThis pooh;
struct curl_slist* rcpt_list = NULL;
pooh.counter = 0;
curl_global_init(CURL_GLOBAL_DEFAULT);
curl = curl_easy_init();
if(!curl)
return 1;
mcurl = curl_multi_init();
if(!mcurl)
return 2;
rcpt_list = curl_slist_append(rcpt_list, RECIPIENT);
/* more addresses can be added here
rcpt_list = curl_slist_append(rcpt_list, "<others#example.com>");
*/
curl_easy_setopt(curl, CURLOPT_URL, "smtp://" SMTPSERVER SMTPPORT);
curl_easy_setopt(curl, CURLOPT_USERNAME, USERNAME);
curl_easy_setopt(curl, CURLOPT_PASSWORD, PASSWORD);
curl_easy_setopt(curl, CURLOPT_READFUNCTION, read_callback);
curl_easy_setopt(curl, CURLOPT_MAIL_FROM, MAILFROM);
curl_easy_setopt(curl, CURLOPT_MAIL_RCPT, rcpt_list);
curl_easy_setopt(curl, CURLOPT_USE_SSL, CURLUSESSL_ALL);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER,0);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0);
curl_easy_setopt(curl, CURLOPT_READDATA, &pooh);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1);
curl_easy_setopt(curl, CURLOPT_SSLVERSION, 0);
curl_easy_setopt(curl, CURLOPT_SSL_SESSIONID_CACHE, 0);
curl_multi_add_handle(mcurl, curl);
mp_timedout = 0;
mp_start = tvnow();
/* we start some action by calling perform right away */
curl_multi_perform(mcurl, &still_running);
while(still_running) {
struct timeval timeout;
int rc; /* select() return code */
fd_set fdread;
fd_set fdwrite;
fd_set fdexcep;
int maxfd = -1;
long curl_timeo = -1;
FD_ZERO(&fdread);
FD_ZERO(&fdwrite);
FD_ZERO(&fdexcep);
/* set a suitable timeout to play around with */
timeout.tv_sec = 1;
timeout.tv_usec = 0;
curl_multi_timeout(mcurl, &curl_timeo);
if(curl_timeo >= 0) {
timeout.tv_sec = curl_timeo / 1000;
if(timeout.tv_sec > 1)
timeout.tv_sec = 1;
else
timeout.tv_usec = (curl_timeo % 1000) * 1000;
}
/* get file descriptors from the transfers */
curl_multi_fdset(mcurl, &fdread, &fdwrite, &fdexcep, &maxfd);
/* In a real-world program you OF COURSE check the return code of the
function calls. On success, the value of maxfd is guaranteed to be
greater or equal than -1. We call select(maxfd + 1, ...), specially in
case of (maxfd == -1), we call select(0, ...), which is basically equal
to sleep. */
//rc = select(maxfd+1, &fdread, &fdwrite, &fdexcep, &timeout);
if (tvdiff(tvnow(), mp_start) > MULTI_PERFORM_HANG_TIMEOUT) {
fprintf(stderr, "ABORTING TEST, since it seems "
"that it would have run forever.\n");
break;
}
switch(rc) {
case -1:
/* select error */
break;
case 0: /* timeout */
default: /* action */
curl_multi_perform(mcurl, &still_running);
break;
}
}
curl_slist_free_all(rcpt_list);
curl_multi_remove_handle(mcurl, curl);
curl_multi_cleanup(mcurl);
curl_easy_cleanup(curl);
curl_global_cleanup();
free_text();
return 0;
}
I'm trying to read a text file line by line, and add each line to a
char array.
Since this is C++, why not use an std::vector<string> and use the std::string version of getline?
The std::string class will look after the memory needed to hold a string of any sort of length, and the std::vector class will worry about the memory needed to hold an "array", so to speak, of strings.
EDIT: Actually looking at your code again, you do use an std::string and then allocate memory to store it as an array of chars, and then store pointers to those strings in some fixed sized array, test. Why go to all that trouble when, as I mentioned above, you can use an std::vector<string> to hold all your std::string objects? Mind = boggled.
EDIT2: Couldn't you also use cURLpp as a C++ wrapper for cURL? I haven't used either so I can't comment on the effectiveness of it.
What am I doing wrong?
For one, this:
char str[80];
strcpy (str,a);
strcat (str,"\n");
//Add "str" to the char array "text"
text[counter] = str;
str is allocated on the stack, with block-wide scope. Then you enter that pointer in an array with a greater scope. This is usually a recipe for disaster - a rather impressive segmentation fault or whatever the equivalent is on your platform.
In this case, due to its use in a loop, your program will either crash, or - if the stars have the proper alignment - you will end up with all pointers in your array pointing to the same out-of-scope string, namely the one that was last read.
Why do you even go into that trouble, when you have already dynamically allocated a in the heap?
By the way, mixing char[] arrays (and the associated standard C library functions) with C++ strings is NOT a good idea. Not even an acceptable one. OK, it a bad idea. Just stick to C++ strings...