I get redirected to a page with address like http://example.com#foo=bar. I want to get foo=bar part of it. The whole thing would be ok too.
I found this thing:
char * url;
curl_easy_getinfo(myHandle, CURLINFO_EFFECTIVE_URL, &url);
I don't know english well to find information myself. Every time I want to find it, I find information on getting the page into string variable.
Code:
std::string readBuffer;
curl_global_init( CURL_GLOBAL_ALL);
CURL * myHandle;
CURLcode result;
myHandle = curl_easy_init();
curl_easy_setopt(myHandle, CURLOPT_COOKIEJAR, "coo.txt");
curl_easy_setopt(myHandle, CURLOPT_COOKIEFILE, "coo.txt");
curl_easy_setopt(myHandle, CURLOPT_URL, "https://www.google.ru/#q=stack");
curl_easy_setopt(myHandle, CURLOPT_WRITEFUNCTION, WriteCallback);
curl_easy_setopt(myHandle, CURLOPT_WRITEDATA, &readBuffer);
curl_easy_setopt(myHandle, CURLOPT_FOLLOWLOCATION, 1L);
result = curl_easy_perform(myHandle);
char * ch_cur_url;
result = curl_easy_getinfo(myHandle, CURLINFO_EFFECTIVE_URL,
&ch_cur_url);
printf("%s\n", ch_cur_url);
Outputs https://www.google.ru/
When I wanted https://www.google.ru/#q=stack
cURL removes the "fragment identifier" from the URL before making a request, as per the bug reports (1, 2). See also this patch. Thus the "fragment identifier" is not available as part of the CURLINFO_EFFECTIVE_URL.
If the "fragment identifier" is returned as part of a redirect (e.g. the Location HTTP header) and you can't get it any other way, then you may use the debug modes to peek on the communications between the cURL and the servers and extract the "fragment identifier" yourself. To that end you'll need to setup either CURLOPT_DEBUGFUNCTION or CURLOPT_HEADERFUNCTION.
P.S. A bit of advise: Googling the relevant information was very easy. First thing I did was to learn the "official" name of the #foo=bar. To get it I visited Wikipedia at URL and was brought to Fragment identifier. After that, Googling with the "curl fragment" netted the relevant parts. If you're looking for something, learn it's proper name.
Related
I'm using curl with c++ to list all the bulbs successfully
curl_easy_setopt(curl,CURLOPT_USERNAME, MY_API_key);
curl_easy_setopt(curl, CURLOPT_URL, "https://api.lifx.com/v1beta1/lights/all/");
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteCallback);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &Data);
res = curl_easy_perform(curl);
To toggle power to all light the documentation http://developer.lifx.com/#toggle-power says to use
curl -u "c87c73a896b554367fac61f71dd3656af8d93a525a4e87df5952c6078a89d192:" -X POST "https://api.lifx.com/v1beta1/lights/all/toggle"
I've tested this via the pre-built curl binary it works fine. I can't figure out how to construct the POST format in the C++ code.
curl_easy_setopt(curl,CURLOPT_USERNAME, MY_API_key);
curl_easy_setopt(curl,CURLOPT_POST,"https://api.lifx.com/v1beta1/lights/all/toggle");
curl_easy_setopt(curl,CURLOPT_SSL_VERIFYPEER, FALSE);
curl_easy_setopt(curl,CURLOPT_WRITEFUNCTION, WriteCallback);
curl_easy_setopt(curl,CURLOPT_WRITEDATA, &Data);
res = curl_easy_perform(curl);
However, res returns CURLE_URL_MALFORMAT, I think this is because I haven't set the CURLOPT_URL property... but I'm not sure what it needs to be set to.
I tried using a similar format to this PHP question (PHP HTTP CURL PUT request for LIFX Power On/Off) but with no luck, it still returns CURLE_URL_MALFORMAT.
CURLOPT_POST is wrongly used there. It should be set to 0 or 1 only. You set the URL with CURLOPT_URL.
You could use --libcurl sample.c added to your (working) curl command line to get a good sample source code to start from.
To mimic that command line closer, you can probably skip CURLOPT_POST and just have CURLOPT_CUSTOMREQUEST set to "POST"
Using the below code, I'm trying to get any of the libcurl IMAP commands to work.
Currently, regardless of the command set via CURLOPT_CUSTOMREQUEST, in my callback function the only data that is given is the oldest email (1st) in my inbox. I can even put something like "dfsafdasfasfaf" in the CURLOPT_CUSTOMREQUEST, and no error will be shown, and the oldest email will be printed from the callback.
I've tried using the sample codes on libcurl's site, to list folders, LSUB, etc and it's always the same - the only thing returned is the contents of the 1st email in my inbox.
I'm using curl 7.40 mingw32 on win32 g++ (-lcurldll).
Surely I must be doing something wrong. If you could take a moment to correct my error, I would be most appreciative. Thank you.
EDIT - Even if you don't know the answer, could you please leave a comment if you have successfully gotten libcurl IMAP to work before? Because if no one has gotten libcurl imap to work before I'll stop wasting my time with it and move on to VMime or another option..
EDIT2- My principal question is how can I list folders via libcurl?
size_t writeCallback(char* buf, size_t size, size_t nmemb, void* up)
{
printf("%s\n", buf);
return size*nmemb; //tell curl how many bytes we handled
}
int main(void)
{
CURL *curl;
CURLcode res = CURLE_OK;
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_USERNAME, "gmailuser");
curl_easy_setopt(curl, CURLOPT_PASSWORD, "password");
curl_easy_setopt(curl, CURLOPT_URL, "imaps://imap.gmail.com/INBOX");
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, &writeCallback);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "LIST");
res = curl_easy_perform(curl);
if(res != CURLE_OK)
{
fprintf(stderr, "curl_easy_perform() failed: %s\n", curl_easy_strerror(res));
}
curl_easy_cleanup(curl);
}
_getch ();
return (int)res;
}
In order to get list of folders in a given GMail inbox, you should use:
curl_easy_setopt(curl, CURLOPT_URL, "imaps://imap.gmail.com/");
Also, I believe you don't need this line to perform LIST request:
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "LIST");
I have tested it on Linux, on libcurl version 7.35.0, though I believe the problems you are encountering are not OS-specific and are rather caused by the current state of the implementation of IMAP support in the library. You can find source code for libcurl version 7.35.0 here.
You can also find more examples of current libcurl IMAP support on the examples page (see the links on the right for more detailed examples).
I'm trying to download a file needed for my application off the internet (as part of installation) so that the first time the app starts up, the needed files get downloaded. For now I'm putting them on Google Drive and making them public, then I'm going to use libcURL to download them. The problem is, I just can't get the data.
I use the following link: https://docs.google.com/uc?id=documentID&export=download and replace documentID with the id. When I try connecting to the site though, it keeps giving me a small snippet of HTML code that basically says "Moved Temporarily" and gives me a link to the new URL. When I use the new link in my program, I get no output whatsoever. However, both links work just fine in my web browser, even when I'm not signed in. So Why don't they work in my program? Am I not setting up SSL options correctly, or is Google Drive simply not meant for this kind of thing?
Here's my code:
#include <curl/curl.h>
int main()
{
curl_global_init(CURL_GLOBAL_ALL);
CURL *curl;
CURLcode res;
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_URL, "https://docs.google.com/uc?id=documentID&export=download");
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0L);
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
res = curl_easy_perform(curl);
}
curl_easy_cleanup(curl);
return 0;
}
Any help would be appreciated.
You'll need to set the CURLOPT_FOLLOWLOCATION option to tell cURL to follow redirects.
I do not know if this helps directly but I have always made the call
curl_global_init(CURL_GLOBAL_ALL);
which I see you don't use. I have seen this call made here in the threaded SSL code example http://curl.haxx.se/libcurl/c/threaded-ssl.html. This `curl_global_init() call will perform SSL initialisation amongst other things. It is discussed in this link http://curl.haxx.se/libcurl and also in the libcurl tutorial here http://curl.haxx.se/libcurl/c/libcurl-tutorial.html
This issue/quirk/side-effect is driving me crazy. Near the bottom the code, the response code of the HTTP interaction is passed by reference into responseCode_. However it often comes out as 0 even though the site can otherwise be accessed, and returns too quickly to be a timeout...
All variables are defined, the code below is just a snippet of a C++ method in a class. Any var_ variables are instance based. It runs on several threads, but that should not be a problem. Each class that uses libcurl has its own instance on the respective threads.
Thanks in advance for any ideas or advice...
CURL *curl;
curl = curl_easy_init();
//The URL
curl_easy_setopt(curl, CURLOPT_URL, url.getURLString().c_str());
//Timeout
curl_easy_setopt(curl, CURLOPT_TIMEOUT, &timeout_);
//disable signals to use with threads
curl_easy_setopt(curl, CURLOPT_NOSIGNAL, 1);
//Redirecting
curl_easy_setopt(curl, CURLOPT_MAXREDIRS, 5);
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1);
//Writing callback
curl_easy_setopt(curl, CURLOPT_HEADERFUNCTION, &writerh);
curl_easy_setopt(curl, CURLOPT_HEADERDATA, &head_);
//Writing callback
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, &writerb);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &body_);
//Headers
struct curl_slist *headers = NULL;
for (std::map<std::string, std::string>::iterator itr = requestHeaders_.begin(); itr != requestHeaders_.end(); itr++) {
std::stringstream header;
header << itr->first << ": " << itr->second;
headers = curl_slist_append(headers, header.str().c_str());
}
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
//UA
curl_easy_setopt(curl, CURLOPT_USERAGENT, "RDFaS-Bot/1.0 (+http://www.rdfas.com/bot)");
curl_easy_perform(curl); /* ignores error */
//Response code
curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &responseCode_);
//clean headers
curl_slist_free_all(headers);
curl_easy_cleanup(curl);
Update:
curl_easy_perform was not returning CURLE_OK when the response code was 0, as the marked answer explains. However debug hooks are very useful too and an excellent suggestion
The response code is only going to be set if curl_easy_perform() returns CURLE_OK so you should check that first to make sure curl actually performed the request successfully. Are you sure the callback functions for writing the header and body are set up correctly?
Also, make sure curl_global_init(CURL_GLOBAL_ALL) is called before these easy_perform threads start.
Assuming nothing in the curl handle returned by curl_easy_init() is shared across threads, then the code looks correct.
Use the debugging hooks built into libcurl.
Seriously. libcurl is a "C" nightmare of void*s and error codes. Everything can go wrong when using libcurl. Write your libcurl debugging hooks once and don't remove them from your code. You'll need them again, and again,... and again.
I've found there is really very little information around on this topic. I already have a dll making successful posts using libcurl.
I've compiled libcurl with openssl for ssl functionality.
Here is an exert of my original curl setup.
curl_easy_setopt(handle, CURLOPT_ERRORBUFFER, errorBuffer);
//curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER , 1);
//curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST , 1);
//curl_easy_setopt(curl, CURLOPT_CAINFO , "./ca.cert");
curl_easy_setopt(handle, CURLOPT_POSTFIELDS, cParam);
curl_easy_setopt(handle, CURLOPT_POSTFIELDSIZE, strlen(cParam));
curl_easy_setopt(handle, CURLOPT_FOLLOWLOCATION, 1);
curl_easy_setopt(handle, CURLOPT_WRITEFUNCTION, Request::writer);
curl_easy_setopt(handle, CURLOPT_WRITEDATA, &buffer);
curl_easy_setopt(handle, CURLOPT_URL, cURL);
My question to those who've done this before, is it as easy as just adding those lines above to get SSL to work (as long as the certificate exists)? Or is it more complicated?
The funny thing is I'm not completely sure how SSL works. I've never worked with it before. Do I need to store a key in my application and send it with each request? Anyway my main question was the first. Thank you in advance.
Yes, it is that simple. Just make sure that the "ca.cert" file you have is a true CA cert that can verify your server's certificate.
All you need to do to use SSL with libcurl is give an https url instead of an http url. The only option you need to set with curl_easy_setopt is CURLOPT_URL, although it will just print the received data to stdout if you don't specify a write callback.
CURL *handle = curl_easy_init();
char url[] = "https://google.com";
curl_easy_setopt(handle, CURLOPT_URL, url);
curl_easy_perform(handle);
Make sure that when using CURLOPT_SSL_VERIFYHOST you set the actual value to 2L (which is the default) instead of 1 (as shown as a comment in that example), if you really want to check the hostname matches, otherwise it would just check for the existence of a "Common name" (CN) in the certificate.