I have been working on uploading a simple XML file to a specialized Web Server. This webserver only supports the use of POST or GET, so it eliminates the use of CURLOPT_UPLOAD which only provides for PUT.
I can use the command line to send the file using the following command line
curl -H 'Content-Type: text/xml; charset=utf-8' -H 'SOAP Action:'
-T igs_meters2.xml -X POST //url//
I tried to use CURLOPT_POST and CURLOPT_READDATA, but received a 413 Request Entity Too Large.
I know this cannot be this hard, but I am out in areas I have never work in. Thanks in advance for an idea.
SOLVED:
the tip about CUSTOMREQUEST was what I needed to find. That helped, and I also needed to add in an HTTPHEADER for CONTENT-TYPE and SOAPAction:
Related
I'm looking for a secure way to directly upload large files to S3 (for performance issue).
After few hours of research, I've come to the (maybe incorrect) conclusion that I should use "Browser-Based Uploads Using POST".
As referenced in this thread:
Amazon S3 direct file upload from client browser - private key disclosure
Before trying this directly, I thought about making a cURL POC with direct upload and signature computation.
I've failed to make it work, and haven't found a successful POC on the web using Authentification Version 4 manual settings.
My signature is OK.
The only issue is that Amazon is double SHA256ing my file content and is thus not validation my x-amz-content-sha-256 header.
lower(SHA256(e8379a31b13fb9423928fe28dd41a5e3204a52072634503c31e8b3ea42605b46))
= 4fa84cd7d18e0d33dbd62d0492eca4a159e122391ae0a3e636bd3cf527680c87
I'm not sure of understanding what should I put in my cURL and canonical request payload (and the linked content-length value) and the x-amz-content-sha-256 header.
Should it all be the same values ?
If yes, then the Amazon doc specifies it should all be encrypted to SHA256, so I've no idea why Amazon reSHA256 my already SHA256 payload...
Error:
<?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>XAmzContentSHA256Mismatch</Code>
<Message>The provided 'x-amz-content-sha256' header does not match what was computed.</Message><ClientComputedContentSHA256>e8379a31b13fb9423928fe28dd41a5e3204a52072634503c31e8b3ea42605b46</ClientComputedContentSHA256><S3ComputedContentSHA256>4fa84cd7d18e0d33dbd62d0492eca4a159e122391ae0a3e636bd3cf527680c87</S3ComputedContentSHA256>
<RequestId>419A185269B0F891</RequestId><HostId>QHWxK0Mzz6AfG44ypXBti3W0tYx1xkG9lZGqc2kUKyMF9STwP18M3racio0k06aH5+1ok/Irdn8=</HostId>
</Error>
cURL command:
curl
-v https://??.s3.amazonaws.com/recordtest/test.jpg
-H "Authorization: AWS4-HMAC-SHA256 Credential=??/20170228/eu-west-1/s3/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date, Signature=43750caa762314eb70aace1f7f8ae34633b93352aa25646433ef21e48dd79429" -H "Content-Length: 64"
-H "Content-Type: application/octet-stream"
-H "x-amz-content-sha256: e8379a31b13fb9423928fe28dd41a5e3204a52072634503c31e8b3ea42605b46"
-H "x-amz-date: 20170228T111828Z"
-d "e8379a31b13fb9423928fe28dd41a5e3204a52072634503c31e8b3ea42605b46"
-X PUT
Generated canonical request:
PUT
/recordtest/test.jpg
content-length:64
content-type:application/octet-stream
host:??.s3.amazonaws.com
x-amz-content-sha256:e8379a31b13fb9423928fe28dd41a5e3204a52072634503c31e8b3ea42605b46
x-amz-date:20170228T111200Z
content-length;content-type;host;x-amz-content-sha256;x-amz-date
e8379a31b13fb9423928fe28dd41a5e3204a52072634503c31e8b3ea42605b46
I'm using a curl command to send a large file (100+ MB) to a web service. I'm noticing that only when I send the file to the web service using curl the file gets mangled and data is lost.
Here is the command I'm using to send the file:
curl -v --raw -X POST -H "Transfer-Encoding: chunked" -H "Content-Type: text/xml; charset=UTF-8" -d #medline16n0736.xml "http://localhost:2323/TestWebService"
Am I missing something? I thought telling it to use text/xml and charset=UTF-8 would keep it UTF-8 once received by the web service.
You are asking curl to post the XML file using the -d option, which will post the file as if it were being submitted via an HTTP webform in application/x-www-webform-urlencoded format. To post the file by itself, use the -T option instead. Also, you are using the --raw option, which will disable handling of HTTP transfer encodings, even though you are sending a Transfer-Encoding: chunked header. Remove --raw and -T will detect the header to enable chunking.
You are also asking curl to send a Content-Type header to tell the WebService that the uploaded data is UTF-8 encoded XML. It is your responsibility to make sure the XML file is actually UTF-8 encoded. Curl won't check that for you.
I'm using the curl api from http://curl.haxx.se/ in a c++ application. I'm trying to write the equivalent of:
curl -v -X POST -H "Content-type: application/json; charset=utf-16" --data-binary #some data.json
I'm using curl_easy_setopt() to set up the various options but the bit I can't figure out is the --data-binary flag. Which CURLOPT setting would I use to set this?
Thanks in advance
You should use CURLOPT_POSTFIELDS:
specify data to POST to server [...] Pass a char * as parameter, pointing to the full data to send in a HTTP POST operation. You must make sure that the data is formatted the way you want the server to receive it.
Note: you can easily discover this by using the --libcurl option:
--libcurl FILE Dump libcurl equivalent code of this command line
i am very new to the java EE.
I have built some simple restful client for calling rest api's from java using jersey framework.
I have one curl request which i have to call from java
curl -i -X POST 'https://{url}' -H 'Content-Type: application/x-www-form-urlencoded' -H "Accept: application/vnd.newbay.dv-1.8+json" -H "X-HTTP-METHOD-OVERRIDE: DELETE" -H "X-Client-Platform: SERVICES" -H "X-Client-Identifier: IL" --insecure --data 'path=%2FMy+Samsung+GT-I9082%2F20130822_172409.jpg&path=%2FMy+Samsung+GT-I9082%2F20130905_085407.jpg'
Here the paths of file are passed as form post in curl using --data.
Can anyone tell me how to call this api from my java program using jersey framework? I am having difficulty because of this --data part? how exactly i will pass that in my restful java client that i cant understand.
Also can anyone tell me how to pass form post --data part while calling the same api from soapUI?
In your cURL command, --data provides post data in a classic html form format. You shoud alrealy be aware of that because the content-type is explicit : application/x-www-form-urlencoded
Jersey can handle this encoding for you, using MultivaluedMapImpl.
see Sending Name Value Pair in POST using Jersey Client
I'm currently working on a visual studio C++ project that needs to use libcurl to access some apis of my website which uses Django.
I'm using django-tastypie to provide api, and right now I can use this api by using this curl command:
curl --dump-header - -H "Content-Type: application/json" -X PATCH --data '{"body": "This actually is my last post."}' http://localhost:8000/api/v1/entry/4/
to update data on my server. As a test, it works, but what I really need is to write code in C/C++ with libcurl that does the same.
I'm just starting to look at libcurl and not quite sure how to do that. I assume that I should use curl_easy_setopt but I can't find the right options to set.
So my question is
Is it possible to use libcurl to achieve the same goal as that command? I'm not familiar with libcurl.
If so, how to set options like -x PATCH and content-Type? I looked at the document and didn't seem to find how.
Thanks for any help.
As #deltheil suggested, I tried --libcurl.
But notice the code generated isn't quite right in my case. The -H "Content-Type: application/json" part wasn't in the code.
You need to set headers like this
headers = curl_slist_append(headers, "Content-Type: application/json");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
But the option --libcurl is quite helpful.
This post about json requests in C helped me a lot.