sf::Http::sendRequest never returns - c++

I've written a simple web service using pistache. I'm seding requests to it using sf::Http and sf::Http::Request classes. However, call of sf::Http::sendRequest never returns, even though I specified a 250 ms timeout. The thing happens only with requests to my website. If I send GET request to www.google.com the method returns correct response quite quickly.
Here's the client-side code sample:
sf::Http http("http://192.168.1.10", 8080);
sf::Http::Request request("/highscores", sf::Http::Request::Method::Get);
request.setHttpVersion(1, 1);
//the call below never returns
auto response = http.sendRequest(request, sf::seconds(0.25f));
std::cout << response.getBody();
The service response seems correct in browser and in curl:
$ curl -v 192.168.1.10:8080/highscores
* Trying 192.168.1.10...
* Connected to 192.168.1.10 (192.168.1.10) port 8080 (#0)
> GET /highscores HTTP/1.1
> Host: 192.168.1.10:8080
> User-Agent: curl/7.47.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Connection: Keep-Alive
< Content-Length: 2
<
* Connection #0 to host 192.168.1.10 left intact
[]%
Using strace on my application shows that it sends correct request and even at some point it receives the correct response:
$ strace -s 192 ./sfmlApplication
...
sendto(20, "GET /highscores HTTP/1.1\r\nconnection: close\r\ncontent-length: 0\r\ncontent-type: application/json\r\nfrom: user#sfml-dev.org\r\nhost: 192.168.1.10\r\nuser-agent: libsfml-network/2.x\r\n\r\n", 176, MSG_NOSIGNAL, NULL, 0) = 176
recvfrom(20, "HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nConnection: Keep-Alive\r\nContent-Length: 2\r\n\r\n[]", 1024, MSG_NOSIGNAL, NULL, NULL) = 96
recvfrom(20,
These are the last lines from strace output, after recvfrom(20, the program stops responding and has to be killed.
And the top of stack trace of blocked operation is:
recv() at 0x7ffff7bcd10f
sf::TcpSocket::receive() at 0x7ffff77b12c0
sf::Http::sendRequest() at 0x7ffff77ad5ed
SFML Version: 2.3.2
System: Fedora 4.8.4-200.fc24.x86_64
Any ideas why the sf::Http::sendRequest method call never returns?

Related

AWS Api Gateway cannot rewrite path - 400 Bad Request error

I'm trying to set up an AWS API Gateway for something which was previously handled by an nginx reverse proxy. My endpoints are EC2 instances inside a VPC. I've already set it up so the gateway can access these instances.
The previous nginx setup looked like this:
http {
server {
listen 80;
location /host1/ {
proxy_pass http://host1:8000/;
}
location /host2/ {
proxy_pass http://host2:8070/;
}
...
}
}
The Problem arises when I try to rewrite the request path. I've set up a test route in the Gateway: ANY /test/{proxy+}, which I passed to the corresponding EC2 instance. I've verified, that requests pass through, but they contain the complete paths of the requests:
# machine 1:
curl -v 'https://<endpoint>.amazonaws.com/test/hello_world/test/a'
< HTTP/2 404
< date: Sat, 18 Dec 2021 09:21:42 GMT
< content-type: text/html;charset=utf-8
< content-length: 469
< server: SimpleHTTP/0.6 Python/3.7.10
< apigw-requestid: Kic2FiLIFiAEN_g=
<
--- response ---
# server:
192.168.9.6 - - [18/Dec/2021 09:15:05] "GET /test/hello_world/test/a HTTP/1.1" 404 -
(the 404 is expected, the important part is the request hitting the server)
I then tried to rewrite the request path to remove the leading /test using a parameter mapping: I specified "all incoming requests", Parameter to modify: path, Modification type: overwrite, Value: $request.path.proxy (the catch-all field defined in the route).
Now I get a 400 error, and the requests don't hit my server anymore:
# machine 1:
curl -v 'https://<endpoint>.amazonaws.com/test/hello_world/test/a'
< HTTP/2 400
< date: Sat, 18 Dec 2021 09:19:53 GMT
< content-type: text/html
< content-length: 122
< server: awselb/2.0
< apigw-requestid: KiclDhxXFiAEMhg=
<
<html>
<head><title>400 Bad Request</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
</body>
</html>
# server:
-nothing-
When I map the $request.path.proxy to querystring.path, instead of path the requests hit the server:
# machine 1:
curl -v 'https://<endpoint>.amazonaws.com/test/hello_world/test/a'
< HTTP/2 404
< date: Sat, 18 Dec 2021 09:21:42 GMT
< content-type: text/html;charset=utf-8
< content-length: 469
< server: SimpleHTTP/0.6 Python/3.7.10
< apigw-requestid: Kic2FiLIFiAEN_g=
<
--- response ---
# server:
192.168.9.6 - - [18/Dec/2021 09:21:42] "GET /test/hello_world/test/a?path=hello_world%2Ftest%2Fa HTTP/1.1" 404 -
notice the value of the path query parameter is exactly the correct value which I would have wanted to replace the original requests path.
Is this a bug with AWS, or am I just missing some documentation, stating that you cannot rewrite path that way? Notably, when the {proxy+} path parameter is empty, requests get routed through correctly...
The problem was with the value of the path rewrite: It should have been /$request.path.proxy instead of $request.path.proxy.

Pre-Flight OPTIONS requests are canceled in Chrome (CORS already supported)

I'm working on a project that use Angular + Django(Django Rest Framework). During the development, the CORS support is done by using django-cors-headers, with CORS_ORIGIN_ALLOW_ALL = True and CORS_ALLOW_CREDENTIALS = True.
When I'm trying to send POST requests to create some resources in frontend (Angular), some pre-flight OPTIONS requests are sent by Chrome and responded successfully by backend server (python manage.py runserver), but others are not. These requests are canceled due to unknown reason, backend server logs indicate that requests are received and accepted by server, details are shown in fig below.
The headers of failed requests are shown below.
However, if a copy the content of the headers and try sending it with curl, it works as expected.
$ curl -v -X OPTIONS -H "Access-Control-Request-Headers: authorization,content-type" -H "Access-Control-Request-Method: POST" -H "DNS: 1" -H "Origin: http://localhost:4200" -H "Referer: http://localhost:4200" -H "User-Agent: Mozilla/5.0" http:/localhost:8000/api/user-permissions/
* Unwillingly accepted illegal URL using 1 slash!
* Trying ::1...
* TCP_NODELAY set
* Connection failed
* connect to ::1 port 8000 failed: Connection refused
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8000 (#0)
> OPTIONS /api/user-permissions/ HTTP/1.1
> Host: localhost:8000
> Accept: */*
> Access-Control-Request-Headers: authorization,content-type
> Access-Control-Request-Method: POST
> DNS: 1
> Origin: http://localhost:4200
> Referer: http://localhost:4200
> User-Agent: Mozilla/5.0
>
< HTTP/1.1 200 OK
< Date: Wed, 20 Feb 2019 02:47:39 GMT
< Server: WSGIServer/0.2 CPython/3.7.1
< Content-Type: text/html; charset=utf-8
< Content-Length: 0
< Vary: Origin
< Access-Control-Allow-Credentials: true
< Access-Control-Allow-Origin: http://localhost:4200
< Access-Control-Allow-Headers: accept, accept-encoding, authorization, content-type, dnt, origin, user-agent, x-csrftoken, x-requested-with
< Access-Control-Allow-Methods: DELETE, GET, OPTIONS, PATCH, POST, PUT
< Access-Control-Max-Age: 86400
<
* Connection #0 to host localhost left intact
Any ideas how this happen? Thx.
Sample Code:
// The method of the component that invokes the methods of PermissionService.
/** Update selected user's permissions. */
updatePermissions() {
const diff = this.diffPermissions();
const toBeCreated = diff[0];
const toBeDeleted = diff[1];
this.isLoading = true;
zip(
this.permissionService.createUserPermissions(toBeCreated),
this.permissionService.deleteUserPermissions(toBeDeleted),
).pipe(
map(() => true),
catchError((err: HttpErrorResponse) => {
alert(err.message);
return observableOf(false);
}),
).subscribe(succeed => {
this.isLoading = false;
});
}
// The methods of PermissionService that issue the HTTP requests.
createUserPermission(req: UserPermissionRequest) {
return this.http.post(`${environment.API_URL}/user-permissions/`, req);
}
createUserPermissions(reqs: UserPermissionRequest[]) {
// TODO(youchen): Evaluate the performance cost.
return forkJoin(reqs.map(req => this.createUserPermission(req)));
}
deleteUserPermission(permissionId: number) {
return this.http.delete(`${environment.API_URL}/user-permissions/${permissionId}/`);
}
deleteUserPermissions(permissionIds: number[]) {
// TODO(youchen): Evaluate the performance cost.
return forkJoin(permissionIds.map(id => this.deleteUserPermission(id)));
}
Found the cause: zip() with no parameters
In my case, I'm using zip to combine creations and deletions, see:
const createRequests = [c1, c2];
const deleteRequests = [d1, d2];
zip(
this.service.create(createRequests),
this.service.delete(deleteRequests),
)....
---
service.ts
create(reqs: CreateRequest[]) {
return zip(...reqs.map(req => this.createSingle(req));
}
delete(reqs: DeleteRequest[]) {
return zip(...reqs.map(req => this.deleteSingle(req));
}
But if one of the createRequests and deleteRequests is an empty list, this logic will go wrong. For example, if createRequests is empty while deleteRequests isn't, all HTTP requests fired by this.service.delete(deleteRequests) will be canceled due to an empty zip() is returned by this.service.create(createRequests).
Solution:
The solution to this problem is that we check the length of the reqs and return other observable instaned.
Fixed code:
create(reqs: CreateRequest[]) {
if (reqs.length === 0) return of([]);
return zip(...reqs.map(req => this.createSingle(req));
}
delete(reqs: DeleteRequest[]) {
if (reqs.length === 0) return of([]);
return zip(...reqs.map(req => this.deleteSingle(req));
}

webserver based on node.js couldn't parse the right data which sent by method 'POST' via CURL

I implemented a simple webserver based on node.js, which handle the data
sent by method 'POST' via libCURL.
Client: c++ code based on libCURL
Server: Node.js
Result:
Client
I implemented a simple webserver based on node.js, which handle the data
sent by method 'POST' via libCURL.
Client: c++ code based on libCURL
Server: Node.js
###Client####
>>>
* Found bundle for host 109.123.121.146: 0x1b13f00 [can pipeline]
* Re-using existing connection! (#0) with host 109.123.121.146
* Connected to 109.123.121.146 (109.123.121.146) port 8124 (#0)
> POST / HTTP/1.1
Host: 109.123.121.146:8124
Accept: */*
Content-Type: text/xml
Content-Length: 11
* upload completely sent off: 11 out of 11 bytes
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Content-Type: text/plain;charset=utf-8
Content-Type: text/plain;charset=utf-8
< Date: Thu, 05 Jan 2017 21:14:05 GMT
Date: Thu, 05 Jan 2017 21:14:05 GMT
< Connection: keep-alive
Connection: keep-alive
< Transfer-Encoding: chunked
Transfer-Encoding: chunked
<
* Connection #0 to host 109.123.121.146 left intact
###Server####
####### 'POST' ######
Partial body: B=~� size:11
Body: B=~� size:11 ---->> why the data is invalid char, size is right.
Source Code
Client.cpp
string strUrl = "http://109.123.121.146:8124/";
string strLog = "Hello,CURL!";
curl_easy_setopt(g_curl_handle, CURLOPT_URL, strUrl.c_str());
curl_easy_setopt(g_curl_handle, CURLOPT_POST, 1);
curl_easy_setopt(g_curl_handle, CURLOPT_POSTFIELDS, strLog.c_str());
Server.js
req.addListener('data', function (data) {
body += data;
console.log("Partial body: " + data + " size:"+data.toString().length);
});
req.addListener('end', function () {
console.log("Body: " + body + " size:"+body.toString().length);
});
I wonder why the request data is invalid char:"B=~�" , or not the right string "Hello,CURL!",
Anyone konws, help to explain to me, thanks ~~~

POST url encoded form to Amazon API Gateway

I'm creating a webhook to receive notifications from a 3rd-party service, they sent the data in the body of a POST with content type application/x-www-form-urlencoded.
But it generates the same error:
{"message": "Could not parse request body into json: Unrecognized token \'name\': was expecting \'null\', \'true\', \'false\' or NaN\n at [Source: [B#456fe137; line: 1, column: 6]"}
I could reproduce the error with the following curl call:
% curl -v -X POST -d 'name=Ignacio&city=Tehuixtla' https://rl9b6lh8gk.execute-api.us-east-1.amazonaws.com/prod/mandrillListener
* Trying 54.230.227.63...
* Connected to rl9b6lh8gk.execute-api.us-east-1.amazonaws.com (54.230.227.63) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: *.execute-api.us-east-1.amazonaws.com
* Server certificate: Symantec Class 3 Secure Server CA - G4
* Server certificate: VeriSign Class 3 Public Primary Certification Authority - G5
> POST /prod/mandrillListener HTTP/1.1
> Host: rl9b6lh8gk.execute-api.us-east-1.amazonaws.com
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Length: 27
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 27 out of 27 bytes
< HTTP/1.1 400 Bad Request
< Content-Type: application/json
< Content-Length: 180
< Connection: keep-alive
< Date: Thu, 28 Jan 2016 12:29:40 GMT
< x-amzn-RequestId: cd4d9232-c5ba-11e5-a158-b9b39f0b0599
< X-Cache: Error from cloudfront
< Via: 1.1 1915b8b49d2fbff532431a79650103eb.cloudfront.net (CloudFront)
< X-Amz-Cf-Id: cxU2_b5DzIw4M_n3hJBFXTu9AVRBL3GpbQqUId9IxgS004DfLYqYmg==
<
* Connection #0 to host rl9b6lh8gk.execute-api.us-east-1.amazonaws.com left intact
{"message": "Could not parse request body into json: Unrecognized token \'name\': was expecting \'null\', \'true\', \'false\' or NaN\n at [Source: [B#d92973b; line: 1, column: 6]"}
If I wrap the body with double-quotes it works fine:
% curl -v -X POST -d '"name=Ignacio&city=Tehuixtla"' https://rl9b6lh8gk.execute-api.us-east-1.amazonaws.com/prod/mandrillListener
* Trying 54.230.227.19...
* Connected to rl9b6lh8gk.execute-api.us-east-1.amazonaws.com (54.230.227.19) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: *.execute-api.us-east-1.amazonaws.com
* Server certificate: Symantec Class 3 Secure Server CA - G4
* Server certificate: VeriSign Class 3 Public Primary Certification Authority - G5
> POST /prod/mandrillListener HTTP/1.1
> Host: rl9b6lh8gk.execute-api.us-east-1.amazonaws.com
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Length: 29
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 29 out of 29 bytes
< HTTP/1.1 200 OK
< Content-Type: application/json
< Content-Length: 6
< Connection: keep-alive
< Date: Thu, 28 Jan 2016 12:33:20 GMT
< x-amzn-RequestId: 50610606-c5bb-11e5-b140-5d837ffe26ed
< X-Cache: Miss from cloudfront
< Via: 1.1 a670cda0e28541e40881b95b60c672b7.cloudfront.net (CloudFront)
< X-Amz-Cf-Id: mCLKL4eOnpUMd15IXQZw0RStJHw9Vdf3ivdCl37dcmno2JFOfxw0Vg==
<
* Connection #0 to host rl9b6lh8gk.execute-api.us-east-1.amazonaws.com left intact
"true"%
The lamba has only one line:
context.succeed('true');
How can I make the api gateway do not treat the body as json?
I tried the documentation about template mapping with no success, I even tried to convert it to a static template, with no variables at all! In all cases the error happens before getting to my code.
Try to set mapping template as following:
{
"body" : $input.json('$')
}
This would convert you string into json and pass to lambda.
From amazon docs:
$input.json(x) function evaluates a JSONPath expression and returns the results as a JSON string.
This is not entirely related, but if you are new to Amazon API Gateway, one additional step I did not know was required was to (re) deploy your API after adding the mapping template as others have suggested (in the case you had previously deployed your API). This cost me a bunch of debugging time as I did not understand why I was continuing to get this error even after making the suggestions posted here.
If using the AWS Console,
navigate to any pane within your API
Select Actions menu at the top
Select Deploy API from the menu, choose the relevant stage and confirm
The mapping template to make form data work is pretty complicated. Here is a gist: https://gist.github.com/ryanray/668022ad2432e38493df
Also, you can see this post I wrote that has an example of how to integrate with Slack(their hooks send a POST as form data to API Gateway): http://www.ryanray.me/serverless-slack-integrations
In the API Gateway, select the POST method for your resource, select Integration Request and create a new Mapping Template for application/x-www-form-urlencoded:
#set($body = $input.path('$'))
#set($jsonString = $util.urlencode($body))
#set($json = $util.parsejson($jsonString))
{
"body" : $json,
}
Alternatively, you can simply pass the url encoded string:
#set($body = $input.path('$'))
{
"body" : "$body",
}
and url decode and parse the JSON in your lambda.

HTTP/1.1 401 Unauthorized in Response Headers in Load runner for GET Requests

I am new to Load runner , Am facing am issue while play back of the script
LR 12.50
O.S Windows 7 SP2
Protocol is Mobile HTTP/HTML
Recording mode is Proxy
Let me explain my scenario
While executing following function:
web_custom_request("authenticate",
"URL=https://ws-xx.xxx.com/tcs/rest/authenticate?include=user,company",
"Method=POST",
"Resource=0",
"RecContentType=application/json",
"Referer=",
"Snapshot=t1.inf",
"Mode=HTTP",
"EncType=application/json",
"Body={\"password\":\"xxx\",\"username\":\"xxx\",\"version\":\"1.0.40\"}",
LAST);
For the above POST method , am getting response as below
HTTP/1.1 200 OK\r\n
Date: Tue, 13 Oct 2015 19:19:21 GMT\r\n
Server: Apache-Coyote/1.1\r\n
Content-Type: application/json\r\n
Set-Cookie: dtCookie=DBE9311E44E5C47902702DC762030583|TXlBcHB8MQ; Path=/;
Domain=.xxx.com\r\n
Connection: close\r\n
Transfer-Encoding: chunked\r\n
Which is fine ,Now the second custom request is shown below
web_custom_request("profiles",
"URL=https://ws-test.xxx.com/tcs/rest/profiles",
"Method=GET",
"Resource=1",
"RecContentType=application/json",
"Referer=",
"Snapshot=t2.inf",
LAST);
For the above GET requests in the replay logs am getting:
401 unauthorized error.
GET /tcs/rest/profiles HTTP/1.1\r\n
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT)\r\n
Accept: */*\r\n
Connection: Keep-Alive\r\n
Host: ws-test.xxx.com\r\n
Cookie: dtCookie=DBE9311E44E5C47902702DC762030583|TXlBcHB8MQ\r\n
\r\n
t=5921ms: 172-byte response headers for "https://ws-test.xxx.com/tcs/rest/profiles" (RelFrameId=1, Internal ID=2)
HTTP/1.1 401 Unauthorized\r\n
Date: Tue, 13 Oct 2015 19:19:22 GMT\r\n
Server: Apache-Coyote/1.1\r\n
Content-Type: application/json\r\n
Connection: close\r\n
Transfer-Encoding: chunked\r\n
\r\n
t=5922ms: 4-byte chunked response overhead for "https://ws-test.xxx.com/tcs/rest/profiles" (RelFrameId=1, Internal ID=2)
8b\r\n
t=5923ms: 139-byte chunked response body for "https://ws-test.xxx.com/tcs/rest/profiles" (RelFrameId=1, Internal ID=2)
{"errors":[{"message":"Authentication required to access endpoint","status":"401","code":"
NotAuthenticated","header":"Not Authenticated"}]}
I refereed this link.
My understanding from the above custom request , login is success but the next
subsequent requests are getting failed.
I have used web_cleanup_cookies() function but didn't solve the issue .
I tried to capture the Cookie ID using the below function
web_reg_save_param("COOKIE_ID",
"LR= Cookie: dtCookie=" ,
"RB= |TXlBcHB8MQ\r\n",
"Ord=All",
"RelFrameId=1",
"Search=All",
LAST);
web_add_header("Cookie",lr_eval_string("{COOKIE_ID}"));
Now question is where to place parameter "COOKIE_ID" in my script while there is
no value in script for COOKIE_ID?
How to handle this issue ? Can anybody please help me .
Please add below headers to the script
web_set_sockets_option("SSL_VERSION","TLS");
web_set_user("username", "password", "domain:portno" );
web_set_sockets_option("INITIAL_BASIC_AUTH","1");
In Vugen, Select snapshot view and compare both record and replay requests, suspecting there might be a missing of header in replay request.
If cookie is the only thing changing you can add it by using web_add_cookie function.