Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
Is it possible to use Winsock to create a program that will login to a website with http form submission?
I am currently reading some Winsock tutorials (listed below) but I would like to see an example of sending login data to a website.
http://msdn.microsoft.com/en-us/library/ms738545(v=vs.85).aspx
http://www.planet-source-code.com/vb/scripts/ShowCode.asp?txtCodeId=2241&lngWId=3
http://tangentsoft.net/wskfaq/
I have done this with LibCurl. Would like some tips to apply what I have done with curl to Winsock.
curl_easy_setopt(myHandle, CURLOPT_USERAGENT, "Mozilla/4.0");
curl_easy_setopt(myHandle, CURLOPT_AUTOREFERER, 1);
curl_easy_setopt(myHandle, CURLOPT_FOLLOWLOCATION, 1);
curl_easy_setopt(myHandle, CURLOPT_COOKIEFILE, "");
// Visit the login page once to obtain a PHPSESSID cookie
curl_easy_setopt(myHandle, CURLOPT_URL, result);
curl_easy_perform(myHandle);
// Now, can actually login. First we forge the HTTP referer field, or HTS will deny the login
curl_easy_setopt(myHandle, CURLOPT_REFERER, result);
// Next we tell LibCurl what HTTP POST data to submit
char *data = "username=username&password=password";
curl_easy_setopt(myHandle, CURLOPT_POSTFIELDS, data);
curl_easy_perform(myHandle);
Perfectly possible and very educational but I do not recommended writing your own HTTP client for any software you intend using or distributing. The problem is that although basic HTTP is very simple, things like web proxies and differences in server configuration make writing a client that works in all situations very difficult.
Using a library like libCurl that handles all the annoyances for you is much easier.
Related
To donwload the file i am currently using
curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, Read_Cb);
curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, &ReadBuffer);
curl_easy_perform(curl_handle)
Can CURLOPT_RESUME_FROM be used for download ? So that i can resume the next transfer from that location.
Also does it internally send range request ?
The option CURLOPT_RESUME_FROM is more suitable for uploads, HTTP POST and HTTP PUT requests. You need to use CURLOPT_RANGE for downloads, HTTP GET and HTTP HEAD request. If you use CURLOPT_RESUME_FROM for downloads, it sets Range: <From>- HTTP header.
I am using libcurl with C++ to send an HTTP request to another host (my friend's computer) over the Internet. Now, the other end uses a Node.js script to receive and handle the request.
I have tried the code (both C++ and Node) on my local network between my computers, and everything works just fine. This is the code, in case you wanna look at it:
CURL *handle;
string response;
curl_global_init(CURL_GLOBAL_WIN32);
handle = curl_easy_init();
curl_easy_setopt(handle, CURLOPT_URL, "10.10.10.10:7890"); //<= IP:PORT of the remote host
curl_easy_setopt(handle, CURLOPT_POSTFIELDS, "This is my POST request data!!");
curl_easy_setopt(handle, CURLOPT_WRITEFUNCTION, write_to_string);
curl_easy_setopt(handle, CURLOPT_WRITEDATA, &response);
curl_easy_perform(handle);
curl_easy_cleanup(handle);
and the Node.js code:
var http = require("http");
var server = http.createServer(function(request, response) {
request.on('data', function(chunk) {
console.log("We got a connection!");
console.log("Received POST data:");
console.log(chunk.toString());
});
});
server.listen(7890);
console.log("Server is listening");
As I said, on my local network it works very well, but why it doesn't my friend receive the request? I noticed that my code (C++ part) takes some time, before closing, as if it was trying to connect. Is this some kind of firewall problem? We have tried disabling it, but nothing changed. We are almost total noobs with networking, and we're trying to get into it.
If anybody could help us, thanks in advance!
Your friend's home router needs to know what to do with a connection being requested on port 7890 from the internet. It needs to be configured to forward any communication on that port to the specific internal IP address of your friend's computer.
Not all home routers are capable of port forwarding and many manufacturers have different ways of getting to the settings.
Any software firewall on the friend's computer also needs to know that communication over port 7890 is safe.
I am trying to develop the auto mail retrieval application in C++.
I just want to know how can I use libcurl to log in the following webpage.
https://webmail.stanford.edu/
where it is redirected to some log in webpage for the authentication.
I am not sure how I can fill the form and send a request through the libcurl. Here's what I have tried.
I realized that log in web page consists following input fields;
2 hidden input fields named "RT" and "ST" (with value keys) --> I stored them in input_field map.
1 text input field for username
1 password input field for password
So I tried the following methods to send a POST request to the server.
post_field.append("RT=").append(input_field["RT"]).append("&");
post_field.append("ST=").append(input_field["ST"]).append("&");
post_field.append("username=MYUSERNAME&password=MYPASSWORD");
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, post_field);
curl_easy_setopt(curl, CURLOPT_POST, 1);
res = curl_easy_perform(curl);
However it seems it is not working (I enabled cookie, SSL connection). All I just want to do is fill the form and press submit button through libcurl.
Edit;
After sending the POST request, the retrieved web page says
Error: The URL given is incomplete.
To try again, please return to the preceding web page (or you may need to start your request for this web service again). For help or to report a service problem, contact...
I want to use Curl to download and parse data from this website:
http://xetra.com/xetra/dispatch/en/xetraCSV/navigation/xetra/100_market_structure_instruments/100_instruments/100_all_tradable_instruments/
I have used my Curl code on different websites before and it works without issue but this site is different in that it returns a redirect response with an actual link containing the data.
I enabled this setting:
curl_easy_setopt(m_pCurl, CURLOPT_FOLLOWLOCATION, TRUE));
but I get caught in an infinite loop of redirects filling the log file.
To avoid this, I then parsed the initial HTTP response to get the redirect location, and attempt to download using that link. However, Curl tells me the headers and body are empty (CURLE_GOT_NOTHING) and throws. When I visit using a browser I can see the data loading so I know that there is something there, Curl just doesn't seem to be able to see it.
Any help on this issue would be greatly appreciated.
Many thanks,
pma07pg
Many thanks to Captain Giraffe for this answer!
If you have have a redirect link and need to store the cookies then add these options:
curl_easy_setopt(m_pCurl, CURLOPT_MAXREDIRS , 5); // Stop redirecting ad infinitum
curl_easy_setopt(m_pCurl, CURLOPT_COOKIEFILE, "");
You need the JSESSIONID cookie to not get redirected.
Add the cookie you receive on the first request (302 Found) to your headers, repeat the request et viola.
Sample dealing with libcurl Cookies here
hopefully someone can answer this for me as I'm beating my head against the wall. I'm using LibCurl to do a simple file post to an HTTPS URL. Overall, it works, however contacting the site receiving the post, they apparently have an issue resolved, and I just don't get it.
Apparently, I prepare my request, put the URL, provide USER and PASSWORD for the site. The post then includes a CURL_FORMADD( for the parameters to add a single file attachment via multi-part form).
I had problems working out the CURLOPT_SSL_VERIFYPEER and CURLOPT_SSL_VERIFYHOST and finally got it working with the following settings.
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, false );
curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 1);
Here's the problem. Apparently, when I issue the curl_easy_perform(), it tries to send the request, the response comes back and says ... send me the certificate information and then apparently re-submits the entire package WITH the cert info. And then is accepted by the server I'm sending to.
So, how can I tell LibCurl to submit EVERYTHING... User, Password AND Certificate credentials all up front so the server I'm sending to doesn't reject the first request only to get the credentials the subsequent cycle.
Thanks.
---- EDITING via feedback.
While looking into more as mentioned by Eugen...
I've disabled the "CURLOPT_SSL_VERIFYPEER" from other readings that by doing so can open a transaction up to a "man-in-the-middle" compromise. I don't want that. By finding the method of authentication, it is BASIC, so when I am issuing the connection, I am sending along
curl_easy_setopt(curl, CURLOPT_HTTPAUTH, CURLAUTH_BASIC );
Now, it comes back with CURLE_SSL_CONNECT_ERROR. I'm trying to get the sdterror, but that's crashing via...
FILE *pLocalFile;
if( fopen_s( &pLocalFile, "MyHTTPS.Log", "w" ))
return -1;
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L );
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteFileCallback );
curl_easy_setopt(curl, CURLOPT_STDERR, pLocalFile );
As soon as I do the perform, I get the DLL caused an exception. So, I cant' even see what the connection failure is about .. which I strongly suspect is certificates.
You need to keep things apart to make the solving of this easier.
First you fix your SSL connection/handshake so that you don't get any SSL related problems or errors.
Then you make sure you provide the necessary HTTP authentication stuff and set basic, and you should only need to do a single HTTP request to get the correct response (assuming that is truly what the server wants).
The "DLL caused an exception" seems to be a third problem and is independent of the other stuff. It's hard to tell why you get that as we don't have a complete picture of what you're doing.