I am writing an arduino library to post http request on web.
I am using the String class from http://arduino.cc/en/Tutorial/TextString
My code is behaving strangely when I am referring to my defined string objects after a function call.
Here actually I am trying to get the body of my GET request and removing the http headers from the http GET request's response.
Following is the description:
Method Call:
String body;
if(pinger.Get(host,path,&body))
{
Serial.println("Modified String Outside :");
Serial.println(body);
Serial.println();
Serial.println("Modified String Outside Address");
Serial.println((int)&body);
}
Output
Modified String Outside :
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: text/html
Content-Length: 113
Date: Wed, 13 Jan 2010 14:36:28 GMT
<html>
<head>
<title>Ashish Sharma
</title>
</head>
<body>
Wed Jan 13 20:06:28 IST 2010
</body>
</html>
Modified String Outside Address
2273
Method Description:
bool Pinger::Get(String host, String path, String *response) {
bool connection = false;
bool status = false;
String post1 = "GET ";
post1 = post1.append(path);
post1 = post1.append(" HTTP/1.1");
String host1 = "Host: ";
host1 = host1.append(host);
for (int i = 0; i < 10; i++) {
if (client.connect()) {
client.println(post1);
client.println(host1);
client.println();
connection = true;
break;
}
}
int nlCnt = 0;
while (connection) {
if (client.available()) {
int c = client.read();
response->append((char) c);
if (c == 0x000A && nlCnt == 0) {
nlCnt++;
if (response->contains("200")) {
status = true;
continue;
} else {
client.stop();
client.flush();
break;
}
}
}
if (!client.connected()) {
client.stop();
connection = false;
}
}
response = &response->substring(response->indexOf("\n\r\n"),response->length());
Serial.println("Modified String: ");
Serial.println(*response);
Serial.println();
Serial.print("Modified String Address: ");
Serial.println((int)&response);
return status;
}
Output:
Modified String:
Ø
<html>
<head>
<title>Ashish Sharma
</title>
</head>
<body>
Wed Jan 13 20:06:28 IST 2010
</body>
</html>
Modified String Address: 2259
As can be seen from the example the string reference object is giving me the correct string inside the Get method but the reference of the string contents change when the Get method returns.
If I understood correctly your code, you probably would want to do something like this:
*response = response->substring(response->indexOf("\n\r\n"),response->length());
instead of
response = &response->substring(response->indexOf("\n\r\n"),response->length());
Also there's probably no need to pass in a pointer ( reference would probably make the code look much nicer ).
First off, you are modyfying the address of the string, not the string itself. But the address of the string was passed to the function by value, and thus copied. Modifying it inside the function won’t modify it on the outside.
Secondly, this code here is bad:
response = &response->substring(response->indexOf("\n\r\n"),response->length());
Because it creates a pointer to a temporary object – which means: a dangling pointer because the temporary object will be destroyed after the expression has been evaluated.
What you really want is pass the object by reference (String& response) and modify it, not its pointer:
response = response->substring(response->indexOf("\n\r\n"),response->length());
This should work, provided the String class you use correctly overloads the behaviour of the assignment operator =.
The line
Serial.println((int)&response);
inside your function is wrong response is already a pointer (String * response), with &response you get the pointer to the pointer.
Change it to
Serial.println((int)response);
and you should get the same address as with
Serial.println((int)&body);
where body is a String and &body is the pointer to the string
Related
I want to get the size of an http:/.../file before I download it. The file can be a webpage, image, or a media file. Can this be done with HTTP headers? How do I download just the file HTTP header?
Yes, assuming the HTTP server you're talking to supports/allows this:
public long GetFileSize(string url)
{
long result = -1;
System.Net.WebRequest req = System.Net.WebRequest.Create(url);
req.Method = "HEAD";
using (System.Net.WebResponse resp = req.GetResponse())
{
if (long.TryParse(resp.Headers.Get("Content-Length"), out long ContentLength))
{
result = ContentLength;
}
}
return result;
}
If using the HEAD method is not allowed, or the Content-Length header is not present in the server reply, the only way to determine the size of the content on the server is to download it. Since this is not particularly reliable, most servers will include this information.
Can this be done with HTTP headers?
Yes, this is the way to go. If the information is provided, it's in the header as the Content-Length. Note, however, that this is not necessarily the case.
Downloading only the header can be done using a HEAD request instead of GET. Maybe the following code helps:
HttpWebRequest req = (HttpWebRequest)WebRequest.Create("http://example.com/");
req.Method = "HEAD";
long len;
using(HttpWebResponse resp = (HttpWebResponse)(req.GetResponse()))
{
len = resp.ContentLength;
}
Notice the property for the content length on the HttpWebResponse object – no need to parse the Content-Length header manually.
Note that not every server accepts HTTP HEAD requests. One alternative approach to get the file size is to make an HTTP GET call to the server requesting only a portion of the file to keep the response small and retrieve the file size from the metadata that is returned as part of the response content header.
The standard System.Net.Http.HttpClient can be used to accomplish this. The partial content is requested by setting a byte range on the request message header as:
request.Headers.Range = new RangeHeaderValue(startByte, endByte)
The server responds with a message containing the requested range as well as the entire file size. This information is returned in the response content header (response.Content.Header) with the key "Content-Range".
Here's an example of the content range in the response message content header:
{
"Key": "Content-Range",
"Value": [
"bytes 0-15/2328372"
]
}
In this example the header value implies the response contains bytes 0 to 15 (i.e., 16 bytes total) and the file is 2,328,372 bytes in its entirety.
Here's a sample implementation of this method:
public static class HttpClientExtensions
{
public static async Task<long> GetContentSizeAsync(this System.Net.Http.HttpClient client, string url)
{
using (var request = new System.Net.Http.HttpRequestMessage(System.Net.Http.HttpMethod.Get, url))
{
// In order to keep the response as small as possible, set the requested byte range to [0,0] (i.e., only the first byte)
request.Headers.Range = new System.Net.Http.Headers.RangeHeaderValue(from: 0, to: 0);
using (var response = await client.SendAsync(request))
{
response.EnsureSuccessStatusCode();
if (response.StatusCode != System.Net.HttpStatusCode.PartialContent)
throw new System.Net.WebException($"expected partial content response ({System.Net.HttpStatusCode.PartialContent}), instead received: {response.StatusCode}");
var contentRange = response.Content.Headers.GetValues(#"Content-Range").Single();
var lengthString = System.Text.RegularExpressions.Regex.Match(contentRange, #"(?<=^bytes\s[0-9]+\-[0-9]+/)[0-9]+$").Value;
return long.Parse(lengthString);
}
}
}
}
WebClient webClient = new WebClient();
webClient.OpenRead("http://stackoverflow.com/robots.txt");
long totalSizeBytes= Convert.ToInt64(webClient.ResponseHeaders["Content-Length"]);
Console.WriteLine((totalSizeBytes));
HttpClient client = new HttpClient(
new HttpClientHandler() {
Proxy = null, UseProxy = false
} // removes the delay getting a response from the server, if you not use Proxy
);
public async Task<long?> GetContentSizeAsync(string url) {
using (HttpResponseMessage responce = await client.GetAsync(url))
return responce.Content.Headers.ContentLength;
}
I'm trying to send invoices to amazon mws through _UPLOAD_VAT_INVOICE_ following the java example in this guide:
Link
pdf file is a simple invoice of 85 kb
The error is status code 414 that is "Uri too long"
Debugging original amazon class MarketplaceWebServiceClient I see this:
if( request instanceof SubmitFeedRequest ) {
// For SubmitFeed, HTTP body is reserved for the Feed Content and the function parameters
// are contained within the HTTP header
SubmitFeedRequest sfr = (SubmitFeedRequest)request;
method = new HttpPost( config.getServiceURL() + "?" + getSubmitFeedUrlParameters( parameters ) );
getSubmitFeedUrlParameters method takes every parameter and add it to querystring. One of these parameters is contentMD5 from:
String contentMD5 = Base64.encodeBase64String(pdfDocument);
So there is a very large string representing pdf file passed as parameter. This causes error 414
But that class is the original one taken from MaWSJavaClientLibrary-1.1.jar
Can anybody help me please?
Thanks
For the last 2 days I was working on the same problem,
I changed like this and it works now
InputStream contentStream = new ByteArrayInputStream(pdfDocument);
String contentMD5 =computeContentMD5Header(new ByteArrayInputStream(pdfDocument));
public static String computeContentMD5Header(InputStream inputStream) {
// Consume the stream to compute the MD5 as a side effect.
DigestInputStream s;
try {
s = new DigestInputStream(inputStream,
MessageDigest.getInstance("MD5"));
// drain the buffer, as the digest is computed as a side-effect
byte[] buffer = new byte[8192];
while (s.read(buffer) > 0)
;
return new String(
org.apache.commons.codec.binary.Base64.encodeBase64(s
.getMessageDigest().digest()), "UTF-8");
} catch (NoSuchAlgorithmException e) {
throw new RuntimeException(e);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
Title may be unclear, forgive me. I believe this is a client-side problem and I'm not really familiar enough with websockets. I can successfully initialize my web page with a message from my c++ server by sending concatenated string of following:
HTTP header
const char * http_header = "HTTP/1.1 200 OK\r\n"
"Content-length: %ld\r\n"
"Content-Type: text/html\r\n"
"\r\n"
"%s";
Page Source
const char * html_source = R"(<html lang='en'>
<head>
<meta charset='UTF-8'>
<meta name='viewport' content='width = device-width, initial-scale = 1.0'>
<meta http-equiv='X-UA-Compatible' content = 'ie=edge'>
...
MY PAGE CODE GOES HERE
...
</script>
</body>
</html>)";
Transmission loop
char browser_init[strlen(http_header)+strlen(html_source)];
sprintf(browser_init,http_header,strlen(html_source),html_source);
const char * test_message = "a";
while(true)
{
if(ping_count == 0){
connection_status = 3;
recv(client_socket,client_message,sizeof(client_message),0);
send(client_socket,browser_init,sizeof(browser_init),0);
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
}
if(ping_count>0){
if(send(client_socket,test_message,sizeof(test_message),0)){
connection_status = 2;
}
if(!send(client_socket,test_message,sizeof(test_message),0)){
connection_status = 1;
std::cout<<"Connection failed";
}
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
}
ping_count++;
}
This first message is successfully initializing my webpage on browser but I cannot receive following ones.
I tried adding the code below inside <script> , I no longer see my javascript content and Firefox status bar says : Transferring data from localhost (page is continuously loading)
var ws = require("nodejs-websocket");
var client = ws.createServer(function(conn){
console.log("New Connection");
conn.on("message",function(str){
console.log(str);
}).listen(12000,'127.0.0.1');
});
I'm literally sitting here and waiting for help at this point
I've found an C++ API there was called from Java JNI. The libcma.so contains the method Java_com_smule_android_network_core_NetworkUtils_makeDigest which will create an Digest - I Think it's MD5.
Here is the reversed source:
https://raw.githubusercontent.com/Bizarrus/Sing/master/cma.cpp
And here is the JNI definition:
https://github.com/Bizarrus/Sing/blob/7ef0e748bae710bde2d07111bd66f1e7fc0810b4/com/smule/android/network/core/NetworkUtils.java#L53
Can anyone tell me, which Algorithm will be used?
Edit
Following HTTP-Request will be made:
POST http://api-sing.smule.com/v2/login/guest?msgId=1776&appVersion=5.7.5&app=sing_google&appVariant=1&digest=179645edb702ce4a57197141522d848145f8861f HTTP/1.1
User-Agent: com.smule.singandroid/5.7.5 (6.0,F3311,de_DE)
Content-Type: application/json; charset=UTF-8
Content-Length: 501
Host: api-sing.smule.com
Connection: Keep-Alive
Accept-Encoding: gzip
{
"common": {
"advId": "e133b6d9-25b1-4651-b4b8-94d80fa25ed9",
"automaticLogin": true,
"device": {
"carrierCountry": "de",
"country": "DE",
"deviceId": "a:e133b6d9-25b1-4651-b4b8-94d80fa25ed9",
"deviceType": "AND",
"googlePlayServices": "12.6.85 (040306-197041431)",
"hasMail": true,
"lang": "de",
"limitAdTrack": false,
"locale": "de_DE",
"machine": "F3311",
"manufacturer": "Sony",
"os": "6.0",
"product": "F3311",
"screenSize": "normal",
"script": ""
},
"pulp": 17,
"tzOffset": 3600,
"vorgom": true
},
"forceNewPlayer": true,
"lookupAccount": true
}
with following Response:
HTTP/1.1 200 OK
Server: nginx/1.11.5
Date: Sun, 24 Jun 2018 16:27:31 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
X-Smule-Host: a155.sf.smle.co
X-Smule-Digest: 64dc15893bbf43240798c73ae652bfb80e848f57
Set-Cookie: L=N; Max-Age=172800; Expires=Tue, 26 Jun 2018 16:27:31 GMT; Path=/; Domain=.smule.com; Secure
Cache-Control: no-cache
Content-Encoding: gzip
{
"data": {
"loginResult": {
"elControl": {
"npt": false
},
"handleNew": false,
"handlePrefill": true,
"language": "de",
"loginCount": 1,
"playerId": 1762444898,
"playerNew": true,
"playerNewlyRegistered": false,
"playerStat": {
"installDate": 1529857651000
},
"policyUrl": "https://www.smule.com/privacy/embed/20180523",
"policyVersion": "PRIVACY_POLICY_20180523",
"serverTime": 1529857651,
"sessionToken": "g4_10_wma5HOX13kDeho2gvEuIQyf5EnUaAp0Uw3C24O5w9s9xUB1U0JOC0w==",
"sessionTtl": 86400,
"showEmailOpt": true,
"termUrl": "https://www.smule.com/termsofservice/embed/20180523"
},
"settings": {}
},
"status": {
"code": 0,
"message": "ok",
"version": 1
}
}
On the POST request, the Query-Argument **digestis theHashof the request. TheseDigestwill be created byJava_com_smule_android_network_core_NetworkUtils_makeDigest(i think). I don't know, which data will be used for theHashing-Algorithm- I've tried to reproduce theDigest` with following parameters:
Only all Query-Parameters, sorted alphabetically, exclude the digest-Argument (These method will be used on the Website under JavaScript`
Only the Content-Body, in JSON-String (tried as pretty-printed and comprimized)
Content-Body and Query-Parameters expected as above
I've tried some combinations of data to reproduce the digest, but i've found no solutions, which data reprensent the original digest.
From the Response on the X-Smule-Digest header will be hashed by MD5, that i had reproducted last week. Here (in PHP) a working sample to calculate the X-Smule-Digest:
$string = sprintf('%s=%s', 'secret', 'M=|ZUyMu^-qWb}VL^jJd}Mv)8y%bQWXf>IFBDcJ>%4zg2Ci|telj`dVZ#');
$string .= sprintf('&%s=%s', 'path', '/user/json/login'); // sample url-path
$string .= sprintf('&%s=%s', 'csrf', $this->csrf); // from Website, <meta content="([^"]+)" name="csrf-token" or from Header "x-csrf-token"
$string .= sprintf('&%s', http_build_query($this->data)); // POST-Body
$array = explode('&', $string);
sort($array); // Alphabetical sort
$x_smule_digest = md5(implode('&', $array));
1 . The digest you indicate is 40 characters long, and contains numbers from 0 to 9 and letters from a to f. So it is 40 hex values. So it is most probably created using SHA-1 digest function.
2 . The C++ function you indicate has the following signature :
int32_t Java_com_smule_android_network_core_NetworkUtils_makeDigest(int32_t * a1, int32_t a2, int32_t a3)
And the corresponding signature in the Java source code is :
private static native String makeDigest(String str, byte[] bArr);
The first argument a1 may be a pointer reserved by the JNI environment. So, input a1 input set aside, the function takes two arguments : a string and an array of byte, and return one string : the digest. The C++ code itself is too complex for me to dig in.
3 . This JNI function is used twice in the Java file you indicate. In functions m18107a and m18106a. Let's add some comments to the source code :
// Function that takes three arguments :
// - a list of pairs of string we will call the "json pairs"
// - a string we will call the "secret"
// - an other string we will call the "optional token"
// and return the returned string of makeDigest which may be the digest.
public static String m18107a(List<Pair<String, String>> list, String str, String str2)
{
// Sort the json pairs alphabetically according to the first string of each pairs.
// (Check the source code of C35442 to verify my statement, I may be wrong )
Collections.sort(list, new C35442());
// Create a single string, that concatenate all the json pairs.
// ( I guess { age : 21, name : "bob" } becomes "age21namebob" )
// We will call that the "digest input"
StringBuilder stringBuilder = new StringBuilder();
for (Pair pair : list) {
stringBuilder.append((String) pair.first).append((String) pair.second);
}
// Append the "digest input" just created with the "optional token" if it exists.
if (str2 != null) {
stringBuilder.append(str2);
}
// Send the "secret" string, and the "digest input" as BYTES (may be important)
// Return the digest computed as returned string of the function.
return makeDigest(str, stringBuilder.toString().getBytes());
}
// Function that takes four arguments :
// - a string str
// - a number j
// - a string str2
// - a string str3 (can be null, as seen later)
// and return the returned string of makeDigest which may be the digest.
public static String m18106a(String str, long j, String str2, String str3)
{
// Create a single string, that concatenate j and str2.
// We will call that the "digest input"
StringBuilder append = new StringBuilder().append("").append(j).append(str2);
// If str3 is null, make it empty.
if (str3 == null) {
str3 = "";
}
// Send the "secret" string, and the "digest input" concatenated with str3, as BYTES (may be important).
// Return the digest computed as returned string of the function.
return makeDigest(str, append.append(str3).toString().getBytes());
}
It will be hard to go further. Try to dig few things :
Where are these functions m18107a and m18106a called ? Can you at least find what are the inputs of these ? So you can play with the good inputs.
Can you put breakpoints around these functions ? To track inputs and outputs and try to reproduce ?
From all the suspected inputs, can you find a way to modify only one per one and check if the digest changes ? To eliminate suspected inputs.
I try to create simple file upload service in C++. I get all user request body as one big string. User can upload any type of data. I need to get only user file contents from request boby string.
so for example now I have next code working with my service API provider:
std::cout << "Request body: " << request->body << std::endl << "Request size: " << request->body.length() << std::endl;
and this would print as:
Request body: ------WebKitFormBoundaryAZlJcLinxYi6OCzX
Content-Disposition: form-data; name="datafile"; filename="crossdomain.xml"
Content-Type: text/xml
я╗┐<?xml version="1.0"?>
<!DOCTYPE cross-domain-policy SYSTEM "http://www.macromedia.com/xml/dtds/cross-
omain-policy.dtd">
<cross-domain-policy>
<allow-access-from domain="*" to-ports="*" />
</cross-domain-policy>
------WebKitFormBoundaryAZlJcLinxYi6OCzX--
Request size: 411
So I need to get from request->body (which is string) all data from first /r/n/r/n until last line -2 lines. How to do such thing with string in C++?
This isn't the most elegant approach, but one option would be to do something like this:
std::string contents = /* ... get the string ... */
/* Locate the start point. */
unsigned startPoint = contents.find("\r\n\r\n");
if (startPoint == string::npos) throw runtime_error("Malformed string.");
/* Locate the end point by finding the last newline, then backing up
* to the newline before that.
*/
unsigned endPoint = contents.rfind('\n');
if (endPoint == string::npos || endPoint == 0) throw runtime_error("Malformed string.");
endPoint = contents.rfind('\n', endPoint - 1);
if (endPoint == string::npos) throw runtime_error("Malformed string.");
/* Hand back that slice of the string. */
return std::string(contents.begin() + startPoint, contents.begin() + endPoint);
You can use regular expressions for that. This page has some nice c++ examples: http://www.math.utah.edu/docs/info/libg++_19.html