I'm trying to set up an AWS API Gateway for something which was previously handled by an nginx reverse proxy. My endpoints are EC2 instances inside a VPC. I've already set it up so the gateway can access these instances.
The previous nginx setup looked like this:
http {
server {
listen 80;
location /host1/ {
proxy_pass http://host1:8000/;
}
location /host2/ {
proxy_pass http://host2:8070/;
}
...
}
}
The Problem arises when I try to rewrite the request path. I've set up a test route in the Gateway: ANY /test/{proxy+}, which I passed to the corresponding EC2 instance. I've verified, that requests pass through, but they contain the complete paths of the requests:
# machine 1:
curl -v 'https://<endpoint>.amazonaws.com/test/hello_world/test/a'
< HTTP/2 404
< date: Sat, 18 Dec 2021 09:21:42 GMT
< content-type: text/html;charset=utf-8
< content-length: 469
< server: SimpleHTTP/0.6 Python/3.7.10
< apigw-requestid: Kic2FiLIFiAEN_g=
<
--- response ---
# server:
192.168.9.6 - - [18/Dec/2021 09:15:05] "GET /test/hello_world/test/a HTTP/1.1" 404 -
(the 404 is expected, the important part is the request hitting the server)
I then tried to rewrite the request path to remove the leading /test using a parameter mapping: I specified "all incoming requests", Parameter to modify: path, Modification type: overwrite, Value: $request.path.proxy (the catch-all field defined in the route).
Now I get a 400 error, and the requests don't hit my server anymore:
# machine 1:
curl -v 'https://<endpoint>.amazonaws.com/test/hello_world/test/a'
< HTTP/2 400
< date: Sat, 18 Dec 2021 09:19:53 GMT
< content-type: text/html
< content-length: 122
< server: awselb/2.0
< apigw-requestid: KiclDhxXFiAEMhg=
<
<html>
<head><title>400 Bad Request</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
</body>
</html>
# server:
-nothing-
When I map the $request.path.proxy to querystring.path, instead of path the requests hit the server:
# machine 1:
curl -v 'https://<endpoint>.amazonaws.com/test/hello_world/test/a'
< HTTP/2 404
< date: Sat, 18 Dec 2021 09:21:42 GMT
< content-type: text/html;charset=utf-8
< content-length: 469
< server: SimpleHTTP/0.6 Python/3.7.10
< apigw-requestid: Kic2FiLIFiAEN_g=
<
--- response ---
# server:
192.168.9.6 - - [18/Dec/2021 09:21:42] "GET /test/hello_world/test/a?path=hello_world%2Ftest%2Fa HTTP/1.1" 404 -
notice the value of the path query parameter is exactly the correct value which I would have wanted to replace the original requests path.
Is this a bug with AWS, or am I just missing some documentation, stating that you cannot rewrite path that way? Notably, when the {proxy+} path parameter is empty, requests get routed through correctly...
The problem was with the value of the path rewrite: It should have been /$request.path.proxy instead of $request.path.proxy.
Related
I have trouble with setting default file for subdirectory.
http://d2770ni5llmjzr.cloudfront.net/songinfo/index.html works properly, but http://d2770ni5llmjzr.cloudfront.net/songinfo ends up with downloading blank file.
I already set static domain of my bucket as origin of distribution.
Following is log of curl -v http://d2770ni5llmjzr.cloudfront.net/songinfo
* Trying 13.225.105.93...
* TCP_NODELAY set
* Connected to d2770ni5llmjzr.cloudfront.net (13.225.105.93) port 80 (#0)
> GET /songinfo HTTP/1.1
> Host: d2770ni5llmjzr.cloudfront.net
> User-Agent: curl/7.55.1
> Accept: */*
>
HTTP/1.1 302 Moved Temporarily
Content-Type: text/html; charset=utf-8
Content-Length: 313 < Connection: keep-alive
x-amz-error-code: Found
x-amz-error-message: Resource Found
Location: /songinfo/
Date: Wed, 31 Jul 2019 20:21:42 GMT
Server: AmazonS3
Age: 9262
X-Cache: Hit from cloudfront
Via: 1.1 2f061b9f7c9e6f3de68229cddaa32e46.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: ICN54-C1
X-Amz-Cf-Id: KgKM9xl5ggcwOWvoUng9rWiVE12UI0q0oPvCUJ2IvJxmyPlJtvmMRg==
<html> <head><title>302 Moved Temporarily</title></head> <body> <h1>302 Moved Temporarily</h1> <ul> <li>Code: Found</li> <li>Message: Resource Found</li> <li>RequestId: 454F29ADAC061274</li> <li>HostId: FsYd6DZeb9nuWA/wQazkp+doZmsriSoakDrcsyQy7jkFQFw/y8ngKW0lR1yT1sYprBJ6CtjK3KU=</li> </ul> <hr/> </body> </html>
* Connection #0 to host d2770ni5llmjzr.cloudfront.net left intact
Web files named index (ex. index.html, index.php) are the default files for their directory. The index.html file loaded automatically on safari, chrome, and firefox for me.
>HTTP/1.1 302 Moved Temporarily
>Location: /songinfo/
This is an HTTP redirect, and it is exactly what S3 is supposed to do when you ask for a "directory" that will serve an index document, but you don't supply the trailing slash in the request.
Your site is working correctly, but curl doesn't follow these redirects by default. You need to tell it to, with the --location (long form) or -L (short form) option.
curl -v -L https://...
You should then find that curl does what you expect.
I need to return a response from Django without returning a cookie.
I'm trying to implement a webhook client API that requires:
the use of https
response within 5 seconds
no body in the response
no cookies in the response headers
a 401 unauthorised status code for invalid hmac signatures
I'm working on Django 1.10 (soon to be upgraded to 2.x) where the rest of the app is protected by user validation via sessions.
Part of the endpoint view is as follows:
response200 = HttpResponse(status=200)
response401 = HttpResponse(status=401)
response401.close() # attempt not to set cookie
signature = request.META.get('HTTP_WEBHOOK_SIGNATURE')
if not request.method == 'POST':
return response401
if not signature:
return response401
and so on.
However my attempt to avoid setting the session using response401.close() doesn't work. I've also tried del response401['Set-Cookie']see Django docs
The cookie LocalTest... is still set in this curl session:
$ curl -d "param1=value1¶m2=value2" \
-H "webhook-signature: $SIGVAL" \
-H "Content-Type: application/x-www-form-urlencoded" \
-X POST http://127.0.0.1:8000/invoices/webhookendpoint \
-w "\n" -v
...
* Connected to 127.0.0.1 (127.0.0.1) port 8000 (#0)
> POST /invoices/webhookendpoint HTTP/1.1
> Host: 127.0.0.1:8000
> User-Agent: curl/7.52.1
> Accept: */*
> x-xero-signature: ZSlYlcsLbYmas53uHNrBFiVL0bLbIKetQI6x8JausfA=n
> Content-Type: application/x-www-form-urlencoded
> Content-Length: 27
>
* upload completely sent off: 27 out of 27 bytes
* HTTP 1.0, assume close after body
< HTTP/1.0 401 Unauthorized
< Date: Thu, 11 Apr 2019 08:32:50 GMT
< Server: WSGIServer/0.1 Python/2.7.13
< Vary: Cookie
< Content-Type: text/html; charset=utf-8
< Set-Cookie: LocalTest=gwx7jhsshy2qvtct1rmzv86h7xshe6ot; httponly; Path=/
<
* Curl_http_done: called premature == 0
* Closing connection 0
It appears that this works:
# ensure no cookie header is set
del request.session
response200 = HttpResponse(status=200)
response401 = HttpResponse(status=401)
...
as shown in the curl response:
< HTTP/1.0 200 OK
< Date: Thu, 11 Apr 2019 08:49:28 GMT
< Server: WSGIServer/0.1 Python/2.7.13
< Content-Type: text/html; charset=utf-8
<
Naturally, if you go to this endpoint as a logged in user, you will have to log in again.
I'm sending Cookie from Laravel application (http://backend.local) to Vue SPA (http://frontend.local:8080):
Laravel side (dummy route)
//api routes
Route::post('login', function () {
setcookie("name", 'value', time()+3600, "/", ".local");
return response('ok');
});
Vue side
axios.post('//backend.local/api/login')
.then(response => {
console.log(response)
}).catch(error => {
})
Response Headers:
HTTP/1.1 200 OK
Date: Mon, 21 May 2018 09:42:35 GMT
Server: Apache
Set-Cookie: name=value; expires=Mon, 21-May-2018 10:42:36 GMT; Max-Age=3600; path=/; domain=.local
Cache-Control: no-cache, private
Access-Control-Allow-Origin: http://frontend.local:8080
Vary: Origin
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 59
Content-Length: 2
Keep-Alive: timeout=10, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=UTF-8
But there is no Cookies in broswer storage (Application tab in Developers Console).
What's wrong?
UPDATE: I think the problem is that there is port number in http://frontend.local:8080. Can I remove port number from url?
Original Answer
The browser will automatically reject any domain which doesn't have two parts to it. So using .local is not considered a domain at all. You need to two levels at least. Consider the below flask app I created for demoing the same
from flask import Flask, request, make_response
app = Flask(__name__)
#app.route("/")
def index():
resp = make_response()
resp.set_cookie('value1', ".frontend.local", domain=".frontend.local")
resp.set_cookie('value2', "frontend.local", domain="frontend.local")
resp.set_cookie('value3', ".local", domain=".local")
return resp
app.run(debug=True)
Now testing this through curl
$ curl -v frontend.local:5000
* Rebuilt URL to: frontend.local:5000/
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to frontend.local (127.0.0.1) port 5000 (#0)
> GET / HTTP/1.1
> Host: frontend.local:5000
> User-Agent: curl/7.54.0
> Accept: */*
>
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Content-Type: text/html; charset=utf-8
< Set-Cookie: value1=.frontend.local; Domain=.frontend.local; Path=/
< Set-Cookie: value2=frontend.local; Domain=frontend.local; Path=/
< Set-Cookie: value3=.local; Domain=.local; Path=/
< Content-Length: 0
< Server: Werkzeug/0.14.1 Python/3.6.5
< Date: Tue, 29 May 2018 12:58:20 GMT
<
* Closing connection 0
When you navigate this in browser, you can see that the cookie with Domain=.local gets discarded but rest 2 remains
And later you can see that both the cookies have been sent to the reload of the same page
Update 1: 30th May 2018
Since you are using 2 different domains, you should use frontend.local.com and backend.local.com and the cookies should be returned with domain as .local.com, so that backend and frontend works as a subdomain of local.com. Then frontend.local.com gets the cookies set by backend.local.com through domain sharing as the cookie domain is set to .local.com
Since this is a CORS call, you will need to set setCredentials to true. This is a standard behavior: Standard CORS requests do not send or set any cookies by default.
The withCredentials property will include any cookies from the other domain in the request and also set any cookies from other domain.
So your Vue.js code will be:
axios.defaults.withCredentials = true
axios({
method: 'POST',
url: '//backend.local/api/login',
// THIS IS IMPORTANT
withCredentials: true
})
.then(response => {
console.log(response)
}).catch(error => {})
[Edit]: Read more about CORS at HTML Rocks.
Please let me know if it doesn't help.
I'm creating a webhook to receive notifications from a 3rd-party service, they sent the data in the body of a POST with content type application/x-www-form-urlencoded.
But it generates the same error:
{"message": "Could not parse request body into json: Unrecognized token \'name\': was expecting \'null\', \'true\', \'false\' or NaN\n at [Source: [B#456fe137; line: 1, column: 6]"}
I could reproduce the error with the following curl call:
% curl -v -X POST -d 'name=Ignacio&city=Tehuixtla' https://rl9b6lh8gk.execute-api.us-east-1.amazonaws.com/prod/mandrillListener
* Trying 54.230.227.63...
* Connected to rl9b6lh8gk.execute-api.us-east-1.amazonaws.com (54.230.227.63) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: *.execute-api.us-east-1.amazonaws.com
* Server certificate: Symantec Class 3 Secure Server CA - G4
* Server certificate: VeriSign Class 3 Public Primary Certification Authority - G5
> POST /prod/mandrillListener HTTP/1.1
> Host: rl9b6lh8gk.execute-api.us-east-1.amazonaws.com
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Length: 27
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 27 out of 27 bytes
< HTTP/1.1 400 Bad Request
< Content-Type: application/json
< Content-Length: 180
< Connection: keep-alive
< Date: Thu, 28 Jan 2016 12:29:40 GMT
< x-amzn-RequestId: cd4d9232-c5ba-11e5-a158-b9b39f0b0599
< X-Cache: Error from cloudfront
< Via: 1.1 1915b8b49d2fbff532431a79650103eb.cloudfront.net (CloudFront)
< X-Amz-Cf-Id: cxU2_b5DzIw4M_n3hJBFXTu9AVRBL3GpbQqUId9IxgS004DfLYqYmg==
<
* Connection #0 to host rl9b6lh8gk.execute-api.us-east-1.amazonaws.com left intact
{"message": "Could not parse request body into json: Unrecognized token \'name\': was expecting \'null\', \'true\', \'false\' or NaN\n at [Source: [B#d92973b; line: 1, column: 6]"}
If I wrap the body with double-quotes it works fine:
% curl -v -X POST -d '"name=Ignacio&city=Tehuixtla"' https://rl9b6lh8gk.execute-api.us-east-1.amazonaws.com/prod/mandrillListener
* Trying 54.230.227.19...
* Connected to rl9b6lh8gk.execute-api.us-east-1.amazonaws.com (54.230.227.19) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: *.execute-api.us-east-1.amazonaws.com
* Server certificate: Symantec Class 3 Secure Server CA - G4
* Server certificate: VeriSign Class 3 Public Primary Certification Authority - G5
> POST /prod/mandrillListener HTTP/1.1
> Host: rl9b6lh8gk.execute-api.us-east-1.amazonaws.com
> User-Agent: curl/7.43.0
> Accept: */*
> Content-Length: 29
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 29 out of 29 bytes
< HTTP/1.1 200 OK
< Content-Type: application/json
< Content-Length: 6
< Connection: keep-alive
< Date: Thu, 28 Jan 2016 12:33:20 GMT
< x-amzn-RequestId: 50610606-c5bb-11e5-b140-5d837ffe26ed
< X-Cache: Miss from cloudfront
< Via: 1.1 a670cda0e28541e40881b95b60c672b7.cloudfront.net (CloudFront)
< X-Amz-Cf-Id: mCLKL4eOnpUMd15IXQZw0RStJHw9Vdf3ivdCl37dcmno2JFOfxw0Vg==
<
* Connection #0 to host rl9b6lh8gk.execute-api.us-east-1.amazonaws.com left intact
"true"%
The lamba has only one line:
context.succeed('true');
How can I make the api gateway do not treat the body as json?
I tried the documentation about template mapping with no success, I even tried to convert it to a static template, with no variables at all! In all cases the error happens before getting to my code.
Try to set mapping template as following:
{
"body" : $input.json('$')
}
This would convert you string into json and pass to lambda.
From amazon docs:
$input.json(x) function evaluates a JSONPath expression and returns the results as a JSON string.
This is not entirely related, but if you are new to Amazon API Gateway, one additional step I did not know was required was to (re) deploy your API after adding the mapping template as others have suggested (in the case you had previously deployed your API). This cost me a bunch of debugging time as I did not understand why I was continuing to get this error even after making the suggestions posted here.
If using the AWS Console,
navigate to any pane within your API
Select Actions menu at the top
Select Deploy API from the menu, choose the relevant stage and confirm
The mapping template to make form data work is pretty complicated. Here is a gist: https://gist.github.com/ryanray/668022ad2432e38493df
Also, you can see this post I wrote that has an example of how to integrate with Slack(their hooks send a POST as form data to API Gateway): http://www.ryanray.me/serverless-slack-integrations
In the API Gateway, select the POST method for your resource, select Integration Request and create a new Mapping Template for application/x-www-form-urlencoded:
#set($body = $input.path('$'))
#set($jsonString = $util.urlencode($body))
#set($json = $util.parsejson($jsonString))
{
"body" : $json,
}
Alternatively, you can simply pass the url encoded string:
#set($body = $input.path('$'))
{
"body" : "$body",
}
and url decode and parse the JSON in your lambda.
I am new to Load runner , Am facing am issue while play back of the script
LR 12.50
O.S Windows 7 SP2
Protocol is Mobile HTTP/HTML
Recording mode is Proxy
Let me explain my scenario
While executing following function:
web_custom_request("authenticate",
"URL=https://ws-xx.xxx.com/tcs/rest/authenticate?include=user,company",
"Method=POST",
"Resource=0",
"RecContentType=application/json",
"Referer=",
"Snapshot=t1.inf",
"Mode=HTTP",
"EncType=application/json",
"Body={\"password\":\"xxx\",\"username\":\"xxx\",\"version\":\"1.0.40\"}",
LAST);
For the above POST method , am getting response as below
HTTP/1.1 200 OK\r\n
Date: Tue, 13 Oct 2015 19:19:21 GMT\r\n
Server: Apache-Coyote/1.1\r\n
Content-Type: application/json\r\n
Set-Cookie: dtCookie=DBE9311E44E5C47902702DC762030583|TXlBcHB8MQ; Path=/;
Domain=.xxx.com\r\n
Connection: close\r\n
Transfer-Encoding: chunked\r\n
Which is fine ,Now the second custom request is shown below
web_custom_request("profiles",
"URL=https://ws-test.xxx.com/tcs/rest/profiles",
"Method=GET",
"Resource=1",
"RecContentType=application/json",
"Referer=",
"Snapshot=t2.inf",
LAST);
For the above GET requests in the replay logs am getting:
401 unauthorized error.
GET /tcs/rest/profiles HTTP/1.1\r\n
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT)\r\n
Accept: */*\r\n
Connection: Keep-Alive\r\n
Host: ws-test.xxx.com\r\n
Cookie: dtCookie=DBE9311E44E5C47902702DC762030583|TXlBcHB8MQ\r\n
\r\n
t=5921ms: 172-byte response headers for "https://ws-test.xxx.com/tcs/rest/profiles" (RelFrameId=1, Internal ID=2)
HTTP/1.1 401 Unauthorized\r\n
Date: Tue, 13 Oct 2015 19:19:22 GMT\r\n
Server: Apache-Coyote/1.1\r\n
Content-Type: application/json\r\n
Connection: close\r\n
Transfer-Encoding: chunked\r\n
\r\n
t=5922ms: 4-byte chunked response overhead for "https://ws-test.xxx.com/tcs/rest/profiles" (RelFrameId=1, Internal ID=2)
8b\r\n
t=5923ms: 139-byte chunked response body for "https://ws-test.xxx.com/tcs/rest/profiles" (RelFrameId=1, Internal ID=2)
{"errors":[{"message":"Authentication required to access endpoint","status":"401","code":"
NotAuthenticated","header":"Not Authenticated"}]}
I refereed this link.
My understanding from the above custom request , login is success but the next
subsequent requests are getting failed.
I have used web_cleanup_cookies() function but didn't solve the issue .
I tried to capture the Cookie ID using the below function
web_reg_save_param("COOKIE_ID",
"LR= Cookie: dtCookie=" ,
"RB= |TXlBcHB8MQ\r\n",
"Ord=All",
"RelFrameId=1",
"Search=All",
LAST);
web_add_header("Cookie",lr_eval_string("{COOKIE_ID}"));
Now question is where to place parameter "COOKIE_ID" in my script while there is
no value in script for COOKIE_ID?
How to handle this issue ? Can anybody please help me .
Please add below headers to the script
web_set_sockets_option("SSL_VERSION","TLS");
web_set_user("username", "password", "domain:portno" );
web_set_sockets_option("INITIAL_BASIC_AUTH","1");
In Vugen, Select snapshot view and compare both record and replay requests, suspecting there might be a missing of header in replay request.
If cookie is the only thing changing you can add it by using web_add_cookie function.