Just a simple file download test. Not working. Probaby because of the proxy at a Company. How can I configure the proxy?
Before adding "127.0.0.1:8080" I always got error 7: Could not connect to server. Now I always get return code 0, but the test file json.txt contains only a 404. But the url is obviously working in my browser. I also tried to get proxy settings from system via this, but I failed.
HTTP/1.1 404 Not Found
Connection: Keep-Alive
Server: Embedthis-http
ETag: "0-0-56d6b6ea"
Cache-Control: no-cache
Last-Modified: Wed, 02 Mar 2016 09:48:26 GMT
Date: Fri, 19 May 2017 14:33:57 GMT
Content-Length: 167
Keep-Alive: timeout=60, max=199
<!DOCTYPE html>
<html><head><title>Not Found</title></head>
<body>
<h2>Access Error: 404 -- Not Found</h2>
<pre>Cannot locate document: /</pre>
</body>
</html>
Here is my cURL attempt:
CURL *curl;
FILE *fp;
char url[] = "http://www.carqueryapi.com/api/0.3/?callback=?&cmd=getMakes&year=2010&sold_in_us=1";
char outfilename[FILENAME_MAX] = "C:\\temp\\json.txt";
curl = curl_easy_init();
if (curl) {
fp = fopen(outfilename,"wb");
CURLcode res;
curl_easy_setopt(curl, CURLOPT_URL, url );
curl_easy_setopt(curl, CURLOPT_NOBODY, true);
curl_easy_setopt(curl, CURLOPT_HEADER, true);
curl_easy_setopt(curl, CURLOPT_HTTPGET, true);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, fp);
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1 );
curl_easy_setopt(curl, CURLOPT_PROXY, "127.0.0.1:8080" );
res = curl_easy_perform(curl);
cout << "Curl response code " << res << ": " << curl_easy_strerror( res ) << endl;
curl_easy_cleanup(curl);
fclose(fp);
}
UPDATE
The internet settings are configured to use the wpad.dat file to detect proxies. If I enter one of those, I get a response from the server, but it says the access is denied because the website is blocked. But my browser can display this site.
On the contrary, if I enter netsh.exe winhttp show proxy it says that the current winhttp proxy settings are "direct access (no proxy server)".
UPDATE 2
I finally got it working with those options: Used Verbose to get more helpful output; output showed some auth method like negotiate and ntlm; the ntlm option did not work but the negotiateone together with the userpwd : which takes the credentials from the system, if libcurl has the sspi module.
curl_easy_setopt(curl, CURLOPT_URL, url );
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1 );
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1 );
curl_easy_setopt(curl, CURLOPT_HEADER, 1);
curl_easy_setopt(curl, CURLOPT_HTTPAUTH, CURLAUTH_ANY );
curl_easy_setopt(curl, CURLOPT_PROXY, <"host:port"> );
curl_easy_setopt(curl, CURLOPT_PROXYAUTH, CURLAUTH_NEGOTIATE );
curl_easy_setopt(curl, CURLOPT_PROXYUSERPWD, ":" );
Try user agent and define header.
#define HEADER_ACCEPT "Accept:text/html,application/xhtml+xml,application/xml"
#define HEADER_USER_AGENT "User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.70 Safari/537.17"
struct curl_slist *list = NULL;
list = curl_slist_append(list, HEADER_USER_AGENT);
list = curl_slist_append(list, HEADER_ACCEPT);
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, list);
You may need a real proxy, find a proxy sites, and retry with another ip
Related
I am doing a post request using CURL in c++ and I wanted to know if theres a way to ignore the response body that gets printed to the output. This is what my code looks like currently
if (curl) {
// Set url
curl_easy_setopt(curl, CURLOPT_URL, str_url);
// Set header
list = curl_slist_append(list, header);
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, list);
// Set request type and data
curl_easy_setopt(curl, CURLOPT_POST, 1);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, body);
curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, (long)strlen(body));
res = curl_easy_perform(curl);
everything works but I keep getting the response body posted to the output and I just want to ignore it since all I need is the response code. With the command line I can make a curl request with the following curl --silent --output to ignore it but is there a way to do this in c++?
when posting data to zendesk, all of my umlauts are saved as "?" characters.
the same post from the postmann app works fine.
i even copied the whole postman C - libcurl code, compiled and fired it up:
same result, umlauts are shown as questionsmarks.
Whats the difference between postman posting the json and visual studio posting it?
here is the code:
CURL *curl;
CURLcode res;
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "POST");
curl_easy_setopt(curl, CURLOPT_URL, "https://xx.zendesk.com/api/v2/users/create_or_update.json");
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
curl_easy_setopt(curl, CURLOPT_DEFAULT_PROTOCOL, "https");
struct curl_slist *headers = NULL;
headers = curl_slist_append(headers, "id: 1");
headers = curl_slist_append(headers, "Authorization: Basic eXaMpLe1234");
headers = curl_slist_append(headers, "Content-Type: application/json");
headers = curl_slist_append(headers, "Cookie: __cfruid=2c9eb500bc16cc34cd7390604b24ece125e7e8ad-187187187");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
const char *data = "{ \"user\": {\"external_id\": \"160102\", \"name\" : \"TestÜÄÖ\", \"phone\": \"014567-0\", \"user_fields\": { \"anrede\": \"Herr\"}, \"organization\":{\"name\":\"testorg\"}, \"email\" : \"\"} }";
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, data);
res = curl_easy_perform(curl);
}
curl_easy_cleanup(curl);
is there a configuration in VS that i have to do to tell him i want utf-8 encoding when posting data?
"charset utf-8" in header yields the same results
Edit: sending a basic curl with the same data using windows cmd behaves the same and saved umlauts as questionmarks, even tho postman generated that curl statement which works when postman sends it.
for people googling:
the u8 prefix as commented by Botje worked
TEST(ApiTest, TestPostMethod) {
CURL *curl;
CURLcode res;
curl_global_init(CURL_GLOBAL_ALL);
curl = curl_easy_init();
if (curl) {
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "POST");
std::string url = "https://postman-echo.com/post";
curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
curl_easy_setopt(curl, CURLOPT_DEFAULT_PROTOCOL, "https");
struct curl_slist *headers = NULL;
// curl_slist_append(headers, "HOST: postman-echo.com");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
ASSERT_EQ(CURLE_OK, res);
}
}
Error: CURLE_COULDNT_RESOLVE_HOST (6)
When I tested this POST request on postman it worked as expected. Also curl command
curl --location --request POST 'https://postman-echo.com/post'
works fine.
Expected equality of these values:
CURLE_OK
Which is: 0
res
Which is: 6
Run NSLOOKUP as suggested by #Andreas Wenzel
> postman-echo.com
Server: 75.75.75.75
Address: 75.75.75.75#53
Non-authoritative answer:
Name: postman-echo.com
Address: 3.210.84.203
Name: postman-echo.com
Address: 52.22.211.29
https://github.com/alexzk1/ed_highway/blob/master/utils/restclient.cpp
this works for many sites last 3-4 years. you can pick those 2 files only.
It uses CURLOPT_POSTFIELDS for post.
Btw, in your code u're following redirects, are u sure website don't redirect you to non-existing link?
Sorry. I figured this out. My server on which I was run this has restrictions in connecting with outside world. Thanks everyone for looking into this.
I'm trying to download an internet page using curl c++.
Using http analyzer I can see the header and use it in cURL.
When I execute my GET request by curl I have a crypted response content, something like this:
‹í=ksÛ8’Ÿ“ªùíÞÊ©1%‘Ô3‰=å8NÆ»ñccMfo§æX IH(BÇmewÿÏÝ߸ûc×
ð%‰’(Kžä®&›ÐÝèn¼^~ûúê´ÿï×gdNÜã§/ñq©7:ªðP;ïWÈýÄõ‚£Ê8§Ïëõ»»»ÚYþ¨®÷z½ú=f¨`FFã§O_NXH‰=¦~À£Jµ.|†02öÙp÷†"}êÐë
R¢ðè„UFÌc>
…_!¶ðBæN ÷Ê$ÍZ‡œ^Ü`Ñ9x€åÞ(¬¸‹`ì–P?:Yu‡8t /]î}">s*A8sY0f,¬p6…ì!»ëvTâú#ª0C6™Öm1™ú,„_Ÿ0ÄÍh;–ÑèÒv‡1Çh9tØëtMÖ`FË6›5Àó½Þlµ›ÝN§Õ¨ s8=ªPW½
CîB'L¸WGZ\²?ק>÷BUŽÙ6;³aši9òã>JúÔ‡Ô³gq_ÿøkÄüY-y×ôšYKjj6M½g6Ú)í3æí«²»’º¶;F«?RI~Èâ‚:Íf·e˜’l¼8¬ã‡»¡ó%sÙ~A‰Ü¦øßàeuà´Å´¸‘F+ßÝ…æÉ {‰ž;—Z®qÏrøˆ‡Ô•©”©˜FӤ˻V«Û4öGMAãÌDFf’ˆô-&A‡ÆÐÛ9®{asêZˆ3Täò„t½G%$¤ƒs’d‹àé3•TüšØêšFÓ0¹ÂPØ L#ásºHâʯ ‰ºÙê5ÌîoH¢±žFc‘Η¥Ԙ¦&
A»¹Ç¾æ[SN=(Å÷ùpê(p¹Ãüú{ˆÈ·œëhàr»~
j³oýY®1GŸÆÎndŽd0îuÑùrD
#Ÿi [äu¿Âî¸çUСb$¾ª‚´AÀ _
e('Cá ‹{¶×dE¢9
¨Â¹¡…V‘؇|KFÇv»ÛíöZ_”kœÖz”DKKtB=MÙʼn:2ÛF¯Ñx³êY`&Fؾ–ÙyŒ¢†B
If I execute my request when the sniffer is enabled all works correctly.
I can't understand why... The website is in http and not in https.
This is my code:
curl_easy_reset(curl);
/* Headers ed altri parametri validi per ogni cURL*/
struct curl_slist *chunk = NULL;
chunk = curl_slist_append(chunk, "Host: www.*****.it");
chunk = curl_slist_append(chunk, "User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:43.0) Gecko/20100101 Firefox/43.0");
chunk = curl_slist_append(chunk, "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
chunk = curl_slist_append(chunk, "Accept-Language: it-IT,it;q=0.8,en-US;q=0.5,en;q=0.3");
chunk = curl_slist_append(chunk, "Accept-Encoding: gzip, deflate");
chunk = curl_slist_append(chunk, "Connection: Keep-Alive");
curl_easy_setopt(curl, CURLOPT_COOKIEFILE, "cookie.txt");
curl_easy_setopt(curl, CURLOPT_COOKIEJAR, "cookie.txt");
curl_easy_setopt(curl, CURLOPT_TIMEOUT, 90);
curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, 90);
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, chunk);
curl_easy_setopt(curl,CURLOPT_FOLLOWLOCATION, true);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteCallback);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, &readBuffer);
curl_easy_setopt(curl, CURLOPT_URL, "http://www.******.it/");
res = curl_easy_perform(curl);
writeFile("a.html", readBuffer);
What can change the sniffer when it is enabled?
Why I receive a strange response content?
With this line:
chunk = curl_slist_append(chunk, "Accept-Encoding: gzip, deflate");
You're saying that you accept gzip, deflate, so the server is sending it back compressed. Try removing that line.
I am currently using pre-compiled minGW32 libraries for libcurl-7.21.6, C++, QT libraries, minGW32 windows compiler (in QT creator) and QT creator as my IDE.
I am trying to post some http information but keep running into a problem. I need to remove Expect: 100... from my header. From everything i have seen online,
headerlist = curl_slist_append(headerlist, "Expect:");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headerlist);
should do it, but it seems to get nullified by:
curl_easy_setopt(curl, CURLOPT_HTTPPOST, formpost);
If i call the HTTPPOST before the HTTPHEADER, it seems like the HTTPHEADER nullifies everything from the HTTPPOST, and vice versa.
Am I doing something wrong? Should the Expect: be somehow included in the HTTPPOST instead of separately in HTTPHEADER?
I am trying to get rid of the Expect header because the server keeps responding to my request with Expectation Failed.
Here is the curl command used with curl.exe to do the samething i'm trying to do with libcurl:
system("curl --referer http://192.168.16.23/upthefile.html -F uploadfile=#instructions.xml -F config=on http://192.168.16.23/cgi-bin/upload.cgi -H \"Expect:\">nul");
Any help will be appreciated.
Using cURL 7.25.0 and this snippet:
curl_global_init(CURL_GLOBAL_ALL);
struct curl_httppost *formpost = NULL;
struct curl_httppost *lastptr = NULL;
struct curl_slist *headerlist = curl_slist_append(NULL, "Expect:");
curl_formadd(&formpost, &lastptr, CURLFORM_COPYNAME, "field", CURLFORM_COPYCONTENTS, "value", CURLFORM_END);
CURL *curl = curl_easy_init();
if (curl)
{
curl_easy_setopt(curl, CURLOPT_URL, "http://www.example.com/");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headerlist);
curl_easy_setopt(curl, CURLOPT_HTTPPOST, formpost);
CURLcode res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
curl_formfree(formpost);
curl_slist_free_all(headerlist);
}
I obtain the following request (Wireshark'd):
POST / HTTP/1.1
Host: www.example.com
Accept: */*
Content-Length: 145
Content-Type: multipart/form-data; boundary=----------------------------523d686b5061
------------------------------523d686b5061
Content-Disposition: form-data; name="field"
value
------------------------------523d686b5061--
While omitting the line curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headerlist); the header contains also:
HTTP/1.1 100 Continue