I've created a program that downloads subtitles from INTERNET using curl
How to know the extension of the file I downloaded (.zip or .rar)?
here is my code (it is a part of function)
FILE* download=fopen("download.zip","wb");//i assume it's a zip
curl = curl_easy_init();
curl_easy_setopt(curl, CURLOPT_POST,1);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS,post_data.c_str());
curl_easy_setopt(curl, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)");
curl_easy_setopt(curl, CURLOPT_URL,url.c_str());
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write2file);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, download);
res = curl_easy_perform(curl);
if(res != CURLE_OK)
std::cout<<"Download Failed "<<std::endl;
curl_easy_cleanup(curl);
fclose(download);
You can use the curl_easy_getinfo() function, it has the content-type info:
CURLINFO_CONTENT_TYPE
Pass a pointer to a char pointer to receive the content-type of the
downloaded object. This is the value read from the Content-Type:
field. If you get NULL, it means that the server didn't send a valid
Content-Type header or that the protocol used doesn't support this.
Using the info you can give the right extension to the downloaded file by renaming it on disk.
Related
I am using libcurl.dll version 7.72.0.0 in Visual Studio for a C++ application.
When I am posting a JSON request to an HTTPS url on one Windows 7 PC with Service Pack 1, the request is posted successfully. But on other PC with same OS as Windows 7 Service Pack 1, I am getting response code 35 from curl_easy_perform(): SSL connect error.
What could be the reason of the same C++ code working on one Windows 7 PC and getting an error on other Windows 7 PC? Both PCs are on the same network.
Following is the C++ code:
curl_easy_setopt(curl, CURLOPT_URL, Url.c_str());
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, header);
// send all data to this function
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteMemoryCallback);
// we pass our 'chunk' struct to the callback function
curl_easy_setopt(curl, CURLOPT_WRITEDATA, (void *)&sResponse);
//some servers don't like requests that are made without a user-agent
// field, so we provide one
curl_easy_setopt(curl, CURLOPT_USERAGENT, "libcurl-agent/1.0");
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, postthis);
printf("\npostthisData=%s\n", postthis);
if (itsHTTPSRequest)
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0);
// if we don't provide POSTFIELDSIZE, libcurl will strlen() by
// itself
curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, (long)strlen(postthis));
curl_easy_setopt( curl, CURLOPT_CONNECTTIMEOUT, m_RWTimeOut );
curl_easy_setopt( curl, CURLOPT_LOW_SPEED_TIME, m_RWTimeOut);
curl_easy_setopt( curl, CURLOPT_LOW_SPEED_LIMIT, 30L);
I'm trying to send POST request to a specific website:
curl_easy_setopt(curl, CURLOPT_URL, web.c_str());
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
curl_easy_setopt(curl, CURLOPT_VERBOSE, true);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &response);
curl_easy_setopt(curl, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)");
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, false);
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, true);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, data);
res = curl_easy_perform(curl);
And the website won't show me the result of the POST request, I think it is because the curl request didn't act like a "real browser".
Is there a better way to make a curl request that mimics a browser the best?
Or do you think the problem is different.
Even though I set in header Content-Lenght I'm getting 411 error. I'm trying to send PUT request.
struct curl_slist *headers = NULL;
curl = curl_easy_init();
std::string paramiters =
"<data_file><archive>false</archive><data_type_id>0a7a184a-dcc6-452a-bcd3-52dbd2a83ea2</data_type_id><data_file_name>backwardstep.stt</data_file_name><description>connectionfile</description><job_id>264cf297-3bc7-42e1-8edc-5e2948ee62b6</job_id></data_file>";
if (curl) {
headers = curl_slist_append(headers, "Accept: */*");
headers = curl_slist_append(headers, "Content-Length: 123");
headers = curl_slist_append(headers, "Content-Type: application/xml");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
curl_easy_setopt(curl, CURLOPT_VERBOSE, true);
curl_easy_setopt(curl, CURLOPT_UPLOAD, 1L);
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "PUT");
curl_easy_setopt(curl, CURLOPT_URL,
"..url/data_files/new/link_upload.xml");
curl_easy_setopt(curl, CURLOPT_USERPWD, "kolundzija#example.ch:PASS");
curl_easy_setopt(curl, CURLOPT_HEADER, 1L);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, paramiters.c_str());
curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE,
strlen(paramiters.c_str()));
curl_easy_setopt(curl, CURLOPT_FAILONERROR, 1L);
res = curl_easy_perform(curl);
and this is response from SERVER:
Host: cloud...
Transfer-Encoding: chunked
Accept: */*
Content-Length: 123
Content-Type: application/xml
Expect: 100-continue
* The requested URL returned error: 411 Length Required
* Closing connection #0
Ok, I honestly can not find your error. But you should have an example from the curl website (first google hit for "curl put c code"): http://curl.haxx.se/libcurl/c/httpput.html
Maybe mixing the easy and advanced interface confuses curl.
What confuses me are the options CURLOPT_POSTFIELDS and CURLOPT_POSTFIELDSIZE. This is a put request, so why are they even there? With PUT the arguments are in the URL. The body is opaque, at least from the perspective of HTTP.
You DON'T need to use a file and do NOT use custom requests, INstead set the UPLOAD and PUT options as it is specified in the documentation here:
http://curl.haxx.se/libcurl/c/httpput.html
Unlike the example above where they use a file as your data structure you can USE ANYTHING to hold your data.It's all on using a callback function with this option:
CURLOPT_READFUNCTION
The difference is made on how you set your callback function which only has to do two things:
1.-measure the size of your payload (your data) in bytes
2.-copy the data to the memory address that curl passes to the callback (that is the first argument on your call back function, the FIRST void pointer in this definition)
static size_t read_callback(void *ptr, size_t size, size_t nmemb, void *stream)
That is the ptr argument.
Use memcpy to copy the data.
Take a look at this link. I ran into the same problem as you and was able to solve it using this approach,one thing YOU need to keep in mind is that you ALSO need to set the file size before sending the curl request.
How do I send long PUT data in libcurl without using file pointers?
Use CURLOPT_INFILESIZE or CURLOPT_INFILESIZE_LARGE for that.
I am using Curl (libcurl) in a C++ aplication, and am unable to send cookies (I think).
I have Fiddler, TamperData and LiveHTTP Headers installed, but they are only useful for viewing browser traffic, and are (it would seem) unable of monitoring general network traffic on a machine, so when I run my machine, I cant see the header information being sent. However, when I view the page in a browser, when succesfully logged on, I can see that cookie information is being sent.
When running my app, I succesfully log onto the page, when I subsequently, try to fetch another page, the (page) data suggests that I am not logged on - i.e. "state" has somehow being lost.
My C++ code looks alright, so I dont know what is going wrong - this is why I need to:
First be able to view my machines network traffic (not just browser traffic) - which (free) tool?
Assuming I am using Curl incorrectly, whats wrong with my code? (the cookies are being retrieved and stored ok, it seems they are just not being sent with requests for some reason.
Here is the section of my class that deals with the cookie side of Http requests:
curl_easy_setopt(curl, CURLOPT_TIMEOUT, long(m_timeout));
curl_easy_setopt(curl, CURLOPT_USERAGENT,
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; WOW64; SV1; .NET CLR 2.0.50727)");
curl_easy_setopt(curl, CURLOPT_COOKIEFILE, "cookies.txt");
curl_easy_setopt(curl, CURLOPT_COOKIEJAR, "cookies.txt");
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
curl_easy_setopt(curl, CURLOPT_AUTOREFERER, 1L);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, CurlCallback);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, this);
Is there anything wrong with the above code?
You can use Wireshark (the former Ethereal) to view all the network traffic a machine is sending and receiving.
As Sean Carpenter said, Wireshark is the right tool to view network traffic. Start a capture and use http as a filter to see only HTTP traffic. If you just want to see HTTP requests/responses sent/received by Curl, set the CURL_VERBOSE option and look at stderr: curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L).
I believe you are using Curl correctly. Compile and run the following (complete) example; you will see that, the second time you run it (when cookies.txt exists) cookies are sent to the server.
Example code:
#include <stdio.h>
#include <curl/curl.h>
int main()
{
CURL *curl;
CURLcode success;
char errbuf[CURL_ERROR_SIZE];
int m_timeout = 15;
if ((curl = curl_easy_init()) == NULL) {
perror("curl_easy_init");
return 1;
}
curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, errbuf);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
curl_easy_setopt(curl, CURLOPT_TIMEOUT, long(m_timeout));
curl_easy_setopt(curl, CURLOPT_URL, "http://www.google.com/");
curl_easy_setopt(curl, CURLOPT_USERAGENT,
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; WOW64; SV1; .NET CLR 2.0.50727)");
curl_easy_setopt(curl, CURLOPT_COOKIEFILE, "cookies.txt");
curl_easy_setopt(curl, CURLOPT_COOKIEJAR, "cookies.txt");
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
curl_easy_setopt(curl, CURLOPT_AUTOREFERER, 1L);
if ((success = curl_easy_perform(curl)) != 0) {
fprintf(stderr, "%s: %s\n", "curl_easy_perform", errbuf);
return 1;
}
curl_easy_cleanup(curl);
return 0;
}
I am attempting to write an application that uses libCurl to post soap requests to a secure web service. This Windows application is built against libCurl version 7.19.0 which, in turn, is built against openssl-0.9.8i. The pertinent curl related code follows:
FILE *input_file = fopen(current->post_file_name.c_str(), "rb");
FILE *output_file = fopen(current->results_file_name.c_str(), "wb");
if(input_file && output_file)
{
struct curl_slist *header_opts = 0;
CURLcode rcd;
header_opts = curl_slist_append(header_opts, "Content-Type: application/soap+xml; charset=utf8");
curl_easy_reset(curl_handle);
curl_easy_setopt(curl_handle, CURLOPT_NOPROGRESS, 1);
curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, output_file);
curl_easy_setopt(curl_handle, CURLOPT_READDATA, input_file);
curl_easy_setopt(curl_handle, CURLOPT_URL, fs_service_url);
curl_easy_setopt(curl_handle, CURLOPT_POST, 1);
curl_easy_setopt(curl_handle, CURLOPT_HTTPHEADER, header_opts);
rcd = curl_easy_perform(curl_handle);
if(rcd != 0)
{
current->curl_result = rcd;
current->curl_error = curl_easy_strerror(rcd);
}
curl_slist_free_all(header_opts);
}
When I attempt to execute the URL, curl returns an CURLE_OUT_OF_MEMORY error which appears to be related to a failure to allocate an SSL context. Has anyone else encountered this problem before?
I had the same problem, just thought I'd add the note that rather than calling the OpenSsl export SSL_library_init directly it can be fixed by adding the flag CURL_GLOBAL_SSL to curl_global_init
After further investigation, I found that this error was due to a failure to initialise the openSSL library by calling SSL_library_init().
I encountered the same symptom after upgrading to Ubuntu 16.04 as described in this answer. The solution was to Use TLS like so.
curl_easy_setopt(curl_, CURLOPT_SSLVERSION, CURL_SSLVERSION_TLSv1_2));
Apparently SSLv3 was disabled on Ubuntu 16.04.