I am transferring a binary file (.exe) with FTP using libcurl, and saving it to a local file. The problem is that after the file is transferred, it is altered and is no longer a valid Win32 application, and doesn't run. Here's how I'm doing it:
CURL *curl;
curl = curl_easy_init();
FILE* f = fopen("C:\\blah.exe", "w");
if(curl) {
curl_easy_setopt(curl, CURLOPT_URL, "ftp://ftp.mysite.com");
curl_easy_setopt(curl, CURLOPT_USERPWD, "blah:blah");
curl_easy_setopt(curl, CURLOPT_FTP_FILEMETHOD, CURLFTPMETHOD_SINGLECWD);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, NULL);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &f);
} else {
fclose(f);
return CURL_EASY_INIT_FAIL;
}
fclose(f);
The file is written but is bigger than it is on the FTP server. Like I said, trying to run it results in the "%1 is not a valid Win32 application" error. Did I forget to set an option or something?
You forgot the binary flag.
This is the correct code:
FILE* f = fopen("C:\\blah.exe", "wb");
The reason is that you transfer as ASCII and not as binary. So your end of lines might get broken. Of there are CRs in the binary they might turn into CR LF or the other way around. Tune CURL to make a binary transfer.
Related
My Question is closed so I have to update this.
1- My purpose is to send my compressed file google cloud storage URL.
2- To do that I have generated a postman request. I have stored my file to my google cloud storage by using the postman tool and the tool has generated the following code.
CURL *curl;
CURLcode res;
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "PUT");
curl_easy_setopt(curl, CURLOPT_URL, "my URL");
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
curl_easy_setopt(curl, CURLOPT_DEFAULT_PROTOCOL, "https");
struct curl_slist *headers = NULL;
headers = curl_slist_append(headers, "Content-Type:
application/octet-stream");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
curl_easy_setopt(curl,CURLOPT_POSTFIELDS,"<file contents here>");
res = curl_easy_perform(curl);
}
curl_easy_cleanup(curl);
3- Then I have copied the code above into my c++ project to send my compressed file to the URL.
4- To create CURLOPT_POSTFIELDS content I did implement the following code;
std::ifstream ifs;
ifs.open ("./compressed.gz", std::ios::binary |
std::ios::ate);
PRINT("Size ->", ifs.tellg());
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, &ifs);
And when I compile and run my code, the request returns me the 200 success response.
But when I checked the google storage dashboard, it just contains 6 bytes of data. Actually, the size of my ifstream data is 1090.
So my problem is that why my request uploads all bytes of the compressed file to cloud storage? Whats wrong in my code ?
How to read compressed files c++
Compressed files are generally binary formats, they are not null terminated text. You cannot use strlen to get their length because that requires a null terminated text as input.
You can use any UnformattedInputFunction to read binary data. Don't forget to open any stream in binary mode.
I need to put the compressed file in a char pointer to put it to the server by using libcurl.
You don't need to read the file in order to do that. You can let libcurl take care of reading the file by passing a FILE* to CURLOPT_READDATA.
This way it won't be necessary to store the entire file in memory. Reading the entire file could be a problem if the file is very large.
You can do something like that when you want to read binary data:
std::ifstream file("./compressed.gz", std::ios::binary);
if (!file.is_open())
ERROR();
std::vector<unsigned char> buffer(std::istreambuf_iterator<char>(file), {});
Then you will have it in the vector buffer. Access the size via buffer.size() and to get raw data use buffer.data() Note that since buffer is an std::vector if it goes outside of scope it will be destructed so the data will be deleted.
Using the CURL at the command line:
curl -d #%1 -X GET https://blah-blah
This is in a Windows batch file and I pass the name of the file on the command line. Using this, I can issue a service call sending in a file with a whole lot of input parameters and I receive a substantial output.
For the life of me, when I try pro \grammatically, I can't make it work. It must be possible, since it can be done on the command. However, when I set
curl_easy_setopt(m_Curl, CURLOPT_HTTPGET, 1);
I can't upload a file, even if I set the callback
If I use:
curl_easy_setopt(m_Curl, CURLOPT_UPLOAD, 1L);
the call becomes 'PUT', even though I try and force the header to be a GET. I follow the documentation and can see that this is the documented behavior. However, what is the pathway for avoiding this default?
Any guidance would be most appreciated.
Thanks,
Stan
AFAIK it's not strictly forbidden but you shouldn't send a body with a GET request. On the other side a server should be able to handle a GET request with body but the response shouldn't be dependent of the content of the body.
If you really need to send a request containing a body with GET you can change the value of the method with CURLOPT_CUSTOMREQUEST. This won't change the behavior of curl. This snippet will upload data with GET:
CURL* curl = curl_easy_init();
if (curl) {
curl_easy_setopt(curl, CURLOPT_UPLOAD, 1L);
curl_easy_setopt(curl, CURLOPT_URL, url);
curl_easy_setopt(curl, CURLOPT_PORT, port);
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "GET");
const auto* file = fopen(filename.c_str(), "r");
curl_easy_setopt(curl, CURLOPT_READDATA, file);
curl_easy_setopt(
curl, CURLOPT_INFILESIZE_LARGE, static_cast<curl_off_t>(filesize));
curl_easy_perform(curl);
}
You can also use the CURLOPT_POSTFIELDS command, to paste the body as a string. Afterwards you change the request to GET like in #Thomas Sablik s answer.
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, chunk);
// Set target URL
curl_easy_setopt(curl, CURLOPT_URL, requestURL.c_str());
// Set HTTP version
curl_easy_setopt(curl, CURLOPT_HTTP_VERSION, (long) CURL_HTTP_VERSION_3);
// Set POST fields
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, requestBody.c_str());
// Set request to get
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "GET");
// Set response target
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, callback);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &response);
// Perform CURL request
CURLcode res = curl_easy_perform(curl);
I am writing an simple file downloader with a help of libcurl. Here's the code for downloading the file from HTTP server:
static size_t WriteCallback(void *contents, size_t size, size_t nmemb, void *userp) {
((std::string*)userp)->append((char*)contents, size * nmemb);
return size * nmemb;
}
std::wstring result; //result with polish letters (ą, ę etc.)
CURL *curl;
CURLcode res;
std::string readBuffer;
curl = curl_easy_init();
ERROR_HANDLE(curl, L"CURL could not been inited.", MOD_INTERNET);
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteCallback);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &readBuffer);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0L);
curl_easy_setopt(curl, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
curl_easy_setopt(curl, CURLOPT_USERPWD, (login + ":" + password).c_str()); //e.g.: "login:password"
curl_easy_setopt(curl, CURLOPT_POST, true);
//curl_easy_setopt(curl, CURLOPT_ENCODING, "UTF-8"); //does not change anything
res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
result = C::toWString(readBuffer);
return res == 0; //0 = OK
It works fine when the file I want to download is encoded as ANSI (according to e.g. Notepad++). But when I try to download the UTF-8 file (UTF-8 without BOM), I get an error with some characters (e.g. polish letters) due to encoding problem.
For example, I run the code for two files with the same text ("to jest teść to") and saved it to std::wstring. The result is from ANSI file and result2 (problematic) from UTF-8 version:
Both files opened on server with e.g. Notepad++ displays the right text.
So, how can I get the UTF-8 file content with libcurl and save it to std::wstring with the proper encoding (so the debugger of Visual Studio will show it as to jest teść to)?
This is not a libcurl issue. You are storing the raw data in a std::string and then converting that to a std::wstring after the download is finished. You have to look at the charset reported in the HTTP response and decode the data to std::wstring accordingly. C::toWString() has no concept of charsets, so you should use something else, like ICONV or ICU. Or, if you know the data is always UTF-8, do the conversion manually (UTF conversions are easy to code by hand), or use C++11's built in UTF conversions using the std::wstring_convert class.
libcurl won't convert or translate the contents for you. It will deliver the exact bytes to your application that the server sent out.
You can use HTTP Accept headers etc to affect what the server responds, but then you need to check the received charset and convert accordingly by yourself if you're not satisfied with what you get.
I'm using curl with c++ to list all the bulbs successfully
curl_easy_setopt(curl,CURLOPT_USERNAME, MY_API_key);
curl_easy_setopt(curl, CURLOPT_URL, "https://api.lifx.com/v1beta1/lights/all/");
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteCallback);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &Data);
res = curl_easy_perform(curl);
To toggle power to all light the documentation http://developer.lifx.com/#toggle-power says to use
curl -u "c87c73a896b554367fac61f71dd3656af8d93a525a4e87df5952c6078a89d192:" -X POST "https://api.lifx.com/v1beta1/lights/all/toggle"
I've tested this via the pre-built curl binary it works fine. I can't figure out how to construct the POST format in the C++ code.
curl_easy_setopt(curl,CURLOPT_USERNAME, MY_API_key);
curl_easy_setopt(curl,CURLOPT_POST,"https://api.lifx.com/v1beta1/lights/all/toggle");
curl_easy_setopt(curl,CURLOPT_SSL_VERIFYPEER, FALSE);
curl_easy_setopt(curl,CURLOPT_WRITEFUNCTION, WriteCallback);
curl_easy_setopt(curl,CURLOPT_WRITEDATA, &Data);
res = curl_easy_perform(curl);
However, res returns CURLE_URL_MALFORMAT, I think this is because I haven't set the CURLOPT_URL property... but I'm not sure what it needs to be set to.
I tried using a similar format to this PHP question (PHP HTTP CURL PUT request for LIFX Power On/Off) but with no luck, it still returns CURLE_URL_MALFORMAT.
CURLOPT_POST is wrongly used there. It should be set to 0 or 1 only. You set the URL with CURLOPT_URL.
You could use --libcurl sample.c added to your (working) curl command line to get a good sample source code to start from.
To mimic that command line closer, you can probably skip CURLOPT_POST and just have CURLOPT_CUSTOMREQUEST set to "POST"
Using the below code, I'm trying to get any of the libcurl IMAP commands to work.
Currently, regardless of the command set via CURLOPT_CUSTOMREQUEST, in my callback function the only data that is given is the oldest email (1st) in my inbox. I can even put something like "dfsafdasfasfaf" in the CURLOPT_CUSTOMREQUEST, and no error will be shown, and the oldest email will be printed from the callback.
I've tried using the sample codes on libcurl's site, to list folders, LSUB, etc and it's always the same - the only thing returned is the contents of the 1st email in my inbox.
I'm using curl 7.40 mingw32 on win32 g++ (-lcurldll).
Surely I must be doing something wrong. If you could take a moment to correct my error, I would be most appreciative. Thank you.
EDIT - Even if you don't know the answer, could you please leave a comment if you have successfully gotten libcurl IMAP to work before? Because if no one has gotten libcurl imap to work before I'll stop wasting my time with it and move on to VMime or another option..
EDIT2- My principal question is how can I list folders via libcurl?
size_t writeCallback(char* buf, size_t size, size_t nmemb, void* up)
{
printf("%s\n", buf);
return size*nmemb; //tell curl how many bytes we handled
}
int main(void)
{
CURL *curl;
CURLcode res = CURLE_OK;
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_USERNAME, "gmailuser");
curl_easy_setopt(curl, CURLOPT_PASSWORD, "password");
curl_easy_setopt(curl, CURLOPT_URL, "imaps://imap.gmail.com/INBOX");
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, &writeCallback);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "LIST");
res = curl_easy_perform(curl);
if(res != CURLE_OK)
{
fprintf(stderr, "curl_easy_perform() failed: %s\n", curl_easy_strerror(res));
}
curl_easy_cleanup(curl);
}
_getch ();
return (int)res;
}
In order to get list of folders in a given GMail inbox, you should use:
curl_easy_setopt(curl, CURLOPT_URL, "imaps://imap.gmail.com/");
Also, I believe you don't need this line to perform LIST request:
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "LIST");
I have tested it on Linux, on libcurl version 7.35.0, though I believe the problems you are encountering are not OS-specific and are rather caused by the current state of the implementation of IMAP support in the library. You can find source code for libcurl version 7.35.0 here.
You can also find more examples of current libcurl IMAP support on the examples page (see the links on the right for more detailed examples).