Currently, I'm working on a HTTP Proxy using libcurl in C++.
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
Above is my code to customize HTTP request header. The headers parameter is a curl_slist pointer variable. I have append all necessary heads information like Accept, Keep Alive, Referer and so on using curl_slist_append() method. However, when I used wireshark to observe the network traffic, I found that CURL didn't use my customized headers while it used "Accept: * / *\r\n".
Does anyone know how to disable the internal header of CURL?
try CURLOPT_ENCODING, don't know about C++ but in PHP:
The contents of the "Accept-Encoding: " header. This enables decoding
of the response. Supported encodings are "identity", "deflate", and
"gzip". If an empty string, "", is set, a header containing all
supported encoding types is sent.
CURLOPT_HTTPHEADER is the way to add, remove or replace internally used headers. If you have problems with this, show us a full program that repeats the problem and we might be able to help out.
Related
I have a python test script that performs an API streaming using requests.post(). It looks like so:
response = requests.post(url_events, data="XYZ", stream=True, headers = {"A":"B"})
if (response.ok):
for chunk in response.iter_content(chunk_size=256):
print chunk
I'm trying to figure out how can I have the same logic but using C++. From what I found the curl library may help, however I cannot find how to pass data field. This is the code I have so far:
CURL* connection = curl_easy_init();
// set url
curl_easy_setopt(connection, CURLOPT_URL, url_events);
// set header
struct curl_slist* headers = NULL;
headers = curl_slist_append(headers, "A:B");
code = curl_easy_setopt(connection, CURLOPT_HTTPHEADER, headers);
// set streaming callback that will print every received message
curl_easy_setopt(connection, CURLOPT_WRITEFUNCTION, printCallback);
// start connection
code = curl_easy_perform(connection);
// ...
curl_easy_cleanup(connection);
curl_slist_free_all(headers);
I was looking through the curl.h file trying to find how to specify the data field, but nothing seems to fit (based on the name)?
Am I on the right track? Would using curl be the right approach for my task, or should I be looking into some other C,C++ libraries? An example that does the same task as above request.post() is appreciated, or a suggestion how to achieve the same using curl.
I am trying to login to betfair via the betfair api using curl. I have already succeeded in doing this via the command line version of curl from a bash script but I now want to do this from my C++ code directly with libcurl. Sadly libcurl is not supported by betfair so there is no direct documentation - but if command line curl works then it should all be doable.
My libcurl is failing straight away during an attempt to login (which should get a "session token" as a response). I tell curl about my certificate and key with the lines:
curl_easy_setopt(curl, CURLOPT_SSLCERT,"client-2048.crt");
curl_easy_setopt(curl, CURLOPT_SSLKEY,"client-2048.key");
Later on I call curl_easy_perform(curl); I get a response of:
{"loginStatus":"CERT_AUTH_REQUIRED"}
According to betfair documentation this means: "Certificate required or certificate present but could not authenticate with it"
So I guess that somehow the SSL authentication failed. As an experiment I tried deliberatly putting garbage certificate and key file names (e.g. "client-2048_XXX.crt") but saw no difference in the response from betfair or any of the curl diagnostics (set up via curl_easy_setopt(curl, CURLOPT_DEBUGFUNCTION, my_trace); and curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);). So I have no way of knowing for sure the libcurl has processed my cert and key files properly. Indeed I am even suspecting that perhaps it is not processing them at all - perhaps I'm missing some other option along the lines of CURLOPT_PLEASE_USE_SSL_AUTHENTICATION which is resulting in the files being ignored?
EDIT: So for the record, the full sequence looks like this (some text hidden):
curl_easy_setopt(curl, CURLOPT_URL, "https://identitysso.betfair.com/api/certlogin");
curl_easy_setopt(curl, CURLOPT_SSLCERT,"client-2048.crt");
curl_easy_setopt(curl, CURLOPT_SSLKEY,"client-2048.key");
curl_easy_setopt(curl, CURLOPT_POST, 1);
struct curl_slist *chunk = NULL;
chunk = curl_slist_append(chunk, "X-Application: _HIDDEN_");
chunk = curl_slist_append(chunk,"Content-Type: application/x-www-form-urlencoded");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, chunk);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS,"username=_HIDDEN_&password=_HIDDEN_");
curl_easy_perform(curl);
EDIT: I thought that perhaps I somehow compiled my libcurl without SSL. To check this I called curl_version_info, which amongst other things gives you:
char *ssl_version; /* human readable string */
Printing this out gave the string "WinSSL" which I assume means that SSL is included.
EDIT: FYI, I have four "client-2048" files, a .crt, .pem, .key and .csr. They all sit on the same directory as my exe file. I also print out the current working directory (as reported by _getcwd()) at run time just do double check that my exe is running from the right location.
EDIT: Here is my working login.sh (please excuse it's inelegance, I was hacking with cut and paste code and scarcely knew what I was doing). getlog.exe was a tiny utility I wrote to do some string manipulation on some JSON.
#!/bin/bash
# uses rescript
APP_KEY=_HIDDEN_
#SESSION_TOKEN=$2
HOST=https://api.betfair.com/exchange/betting
AUTHURL=https://identitysso.betfair.com/api/certlogin
CURL=curl
function bflogin()
{
echo "bflogin()"
OUT=$($CURL -s -k --cert client-2048.crt --key client-2048.key --data "username=_HIDDEN_&password=_HIDDEN_" -H "X-Application: $APP_KEY" -H "Content-Type: application/x-www-form-urlencoded" $AUTHURL)
echo $OUT
SESSION_TOKEN=$(echo $OUT | getlog.exe)
echo
echo "Here -> "$SESSION_TOKEN
echo $SESSION_TOKEN > st.txt
}
Double check the paths to you crt and key files.
Also, since you haven't specified a key encoding type, PEM is the default. You can change the encoding type with the CURLOPT_SSLCERTTYPE option, or try converting your keys to PEM.
I have this code
struct curl_slist *pCURL_List = NULL;
pCURL_List = curl_slist_append(pCURL_List, "Content-type: application/x-amf");
curl_easy_setopt(m_pCURL, CURLOPT_HTTPHEADER, pCURL_List);
curl_easy_perform(m_pCURL);
and right after curl_easy_perform(m_pCURL); I need to reset the headers (the first part of the code). If I do curl_slist_free_all(pCURL_List); the next curl_easy_perform(m_pCURL); crashes the program.
I sure have the other parameters, the program is quite big and everything works EXCEPT calls after these lines.
Also, when I said that I need to reset the headers, I mean that I'd like curl to set back the old default values of Content-type. If I do
pCURL_List = curl_slist_append(pCURL_List, "Content-type:");
afaik it will delete the "Content-type" header.
For the crash, did you set CURLOPT_HTTPHEADER back to null before calling curl_easy_perform the second time? I'm thinking the list itself was freed but the CURL handle still has a pointer to the now invalid memory.
I am playing around writing some HTTP communication in C++ using the Winsock APIs. I have no trouble performing GET requests and receiving the response, however I am having a problem when trying to perform a file upload via a POST request.
So first of all, I will share the code of my PHP file which receives the upload request (upload.php):
<?php
if(isset($_FILES['file'])){
$errors= array();
$file_name = $_FILES['file']['name'];
$file_size =$_FILES['file']['size'];
$file_tmp =$_FILES['file']['tmp_name'];
$file_type=$_FILES['file']['type'];
$file_ext=strtolower(end(explode('.',$_FILES['file']['name'])));
if(empty($errors)==true){
move_uploaded_file($file_tmp,"uploads/".$file_name);
echo "Success";
}
}
else{
echo "Was no file";
}
?>
<form action="" method="POST" enctype="multipart/form-data">
<input type="file" name="file"/>
<input type="submit"/>
</form>
Now I know that there is nothing wrong with this code, because I succeed to perform a file upload to it by using WinInet APIs (HttpSendRequest). My WinInet code which works on this PHP file is comprised of the following main steps:
HttpOpenRequest(..., "POST", "uploader/upload.php", "HTTP/1.0", NULL, NULL, NULL, NULL)
HttpSendRequest with headers I set to: "Content-Type: multipart/form-data; boundary=---------FILE_BOUNDARY----------". I printed out the request body that was built by my program as you can see here, sorry I didn't print it out in a way which I could copy and paste here: http://i.imgur.com/mMmo7Xd.png
This works beautifully, the file is uploaded properly. However my problem arises when I try to "port" this code to use winsock API's instead of wininet. With winsock, as you may know, I must completely manually construct the whole request (headers and body). I assume this must be where my problem is, because the main body of the request itself is the same as when I am using wininet APIs. Here is a printout of my winsock request that is being sent: http://i.imgur.com/TkLNGrq.png
PS I have no idea why MessageBox put the boundary part of the header on another line, the is no "\r\n" there in my code. Could this have something to do with my issue? Here is my code for building the entire request string. Please don't give me pointers on security and buffer overruns, this is not production code:
wsprintfA(FullReqStr,
// Headers
"POST %s HTTP/1.0\r\n"
"Host: %s\r\n"
"Content-Type: multipart/form-data; boundary=---------FILE_BOUNDARY----------\r\n\r\n"
// Body
"-----------FILE_BOUNDARY----------\r\n"
"Content-Disposition: form-data; name=\"file\"; filename=\"file.log\"\r\nContent-Type: application/octet-stream\r\n\r\n"
"%s\r\n"
"-----------FILE_BOUNDARY----------\r\n"
"Content-Disposition: form-data; name=\"submit\"\r\n\r\n"
"Submit\r\n"
"-----------FILE_BOUNDARY------------\r\n",
GatePath, Server, FileBody
);
And yes the file bytes in the winsock request are different, I used different file bytes for testing. That is besides the point. Basically, when I send that request and receive the response, it hits the "Was no file" else in the PHP file. I feel like I must be missing some required field in the headers of my winsock request, one that is otherwise automatically added when using WinInet APIs. I cannot think of anything else, however I'm not sure what I'm missing.
Could anybody point me to what is wrong in my Winsock code? I would greatly appreciate it. Thank you. By the way please not I am not looking for any security or anything in my upload script, obviously. I'm just playing around trying to get this to work. It is not being applied to any system that needs to be secure or anything.
Thanks.
Looks like I didn't read the RFC closely enough. http://www.ietf.org/rfc/rfc1867.txt
I needed to add the "Content-Length" field to my HTTP header. Now I construct the body of the request first so I can take the length, and then construct the entire request using that. Working fine.
I want to create a cURL request in some C++ code which will get me the length of a file in a server without downloading the file. For that, I use some cURL options to tell I only want headers in the request response, and then I examine the response to get the file length.
I'm setting the following request options:
curl_easy_setopt(_curl_handle, CURLOPT_HEADER, 1);
curl_easy_setopt(_curl_handle, CURLOPT_NOBODY, 1);
Then processing the request, waiting for the response, which shows a OK=200, and finally enquiring about the file length:
curl_easy_getinfo(_curl_handle, CURLINFO_CONTENT_LENGTH_UPLOAD, &dResult);
But I get a file length of -1. According to cURL documentation, that means size is unknown. How can it happen that cURL doesn't get the file length information from the server?
CURLINFO_CONTENT_LENGTH_UPLOAD is the number of bytes uploaded. You need to use CURLINFO_CONTENT_LENGTH_DOWNLOAD instead.
Note that if the server dynamically generates the data, the length may be different when you actualy download the file versus just downloading its headers.
Also note that if the server sends data as compressed when downloaded, there may not be any size available in the headers (if the Transfer-Encoding header is used instead of the Content-Length header), so CURLINFO_CONTENT_LENGTH_DOWNLOAD would still return -1. The only way to know the size in that situation would be to download it in full.
Have you tried with CURLINFO_CONTENT_LENGTH_DOWNLOAD instead?
need call perform()
curl_easy_setopt(_curl_handle, CURLOPT_HEADER, 1);
curl_easy_setopt(_curl_handle, CURLOPT_NOBODY, 1);
curl_easy_perform(_curl_handle);
curl_easy_getinfo(_curl_handle, CURLINFO_CONTENT_LENGTH_UPLOAD,
&dResult);