I've discovered that if you send a request with an empty POST body, meaning no Content-Length header, the GCP Load Balancer (in this case from an Ingress controller through GKE) will reject your request with this error:
$ curl -L -X POST 'http://example.com/fund?amount=0'
<html><head>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>411 Length Required</title>
</head>
<body text=#000000 bgcolor=#ffffff>
<h1>Error: Length Required</h1>
<h2>POST requests require a <code>Content-length</code> header.</h2>
<h2></h2>
</body></html>
Assume I can't change the clients, is there some way to make the LB just accept empty bodies in POST requests?
The workarounds available as of the moment would be adding content headers content-length: 0 if you are to send HTTP POST requests with empty body. Per this RFC 2616 documentation:
If no response body is included, the response MUST include a Content-Length field with a field-value of "0"
Or to set the client to use http/1.1 as default:
For compatibility with HTTP/1.0 applications, HTTP/1.1 requests containing a message-body MUST include a valid Content-Length header field unless the server is known to be HTTP/1.1 compliant. If a request contains a message-body and a Content-Length is not given, the server SHOULD respond with 400 (bad request) if it cannot determine the length of the message, or with 411 (length required) if it wishes to insist on receiving a valid Content-Length.
Both options require intervention from the client side. Unfortunately, there are no available workarounds/adjustments that can be done from the GCP Load Balancer side at this time.
Related
CloudFront fails for an HTTP 1.0 request without the Host header (optional for HTTP 1.0):
POST / HTTP/1.0
Content-Type: application/ocsp-request
Content-Length: 75
HTTP/1.1 400 Bad Request
Server: CloudFront
Date: Wed, 16 Mar 2022 21:22:41 GMT
Content-Type: text/html
Content-Length: 915
Connection: close
X-Cache: Error from cloudfront
Via: 1.1 edd67566d372ed79fbaa7f9cc3d7815e.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: ICN51-C1
X-Amz-Cf-Id: KOUV_x5KqMc2f1CsGn1oXTrgaLFSSJn76dycoN97BqIdmgRYjySN3g==
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<TITLE>ERROR: The request could not be satisfied</TITLE>
</HEAD><BODY>
<H1>400 ERROR</H1>
<H2>The request could not be satisfied.</H2>
<HR noshade size="1px">
Bad request.
We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.
<BR clear="all">
If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation.
<BR clear="all">
<HR noshade size="1px">
<PRE>
Generated by cloudfront (CloudFront)
Request ID: KOUV_x5KqMc2f1CsGn1oXTrgaLFSSJn76dycoN97BqIdmgRYjySN3g==
</PRE>
<ADDRESS>
</ADDRESS>
</BODY></HTML>
Now, I'm aware that a CDN edge server using shared public IPs (the default when using CloudFront) wouldn't be capable of identifying the correct distribution if it isn't provided with the Host header, so I enabled dedicated IP address for my distribution (https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/cnames-https-dedicated-ip-or-sni.html#cnames-https-dedicated-ip) hoping that it could help, but CloudFront still fails.
Now, considering that (at least technically) using dedicated IP addresses would make CloudFront capable of identifying the correct distribution and send the requests to the expected origin even when the Host header is missing, is this something supported at all?.
PS: I don’t even see the previous request through Standard logging, so I guess it is lost even before reaching my distribution.
I am trying to hit this URL https://subdomain.example.com in JMeter and recorded using the Blazemeter Chrome extension has all the necessary config elements but get an error:
HTTP/1.1 429 Too Many Requests
Content-Type: text/html; charset=utf-8
Content-Length: 1031
Connection: keep-alive
Cache-Control: private, no-cache, no-store, must-revalidate
Date: Tue, 20 Aug 2019 01:21:35 GMT
Expires: 0
p3p: CP="This site does not specify a policy in the P3P header"
I have tried coping the Header Cookies from Browser Header Response which works for sometime but then start throwing an error
As per HTTP Status Code 429 Too Many Requests description:
The HTTP 429 Too Many Requests response status code indicates the user has sent too many requests in a given amount of time ("rate limiting").
A Retry-After header might be included to this response indicating how long to wait before making a new request.
So there are following options:
Your server is overloaded, in this case there is nothing you can do here apart from reporting the error as the bottleneck
Your script doesn't have proper correlation implemented, i.e. you're sending recorded hard-coded values instead of getting dynamic parameters
Your server doesn't allow such amount of requests from a single IP address within the given timeframe, you could try implementing IP Spoofing so your server would "think" that the requests are coming from the different machines.
Thanks for your reply. In the end I figured out that no limitation for number of calls implemented.
Now come to answer this is how I managed to work this:
Opened the page in chrome and from the header section copied all the header elements into the header manager hard coded.
First time it fails and returns p3p: CP="This site does not specify a policy in the P3P header" but also return the update variable value needed for next request which I extract and used in the next and subsequent Requests. The way I was able to find out which variable is changing by using the string comparison of 2 Response Headers
This was a difficult one but somehow worked with very minor change I also added the Header Manager to each request for safer side.
I published an api to AWS with Visual Studio and using the template for AWS Serverless application project, for now I am testing the methods with postman, but all get methods that require a body are returning an error that mentions cloudfront in the response, I do not know if the issue is related to cloudfront or if it is the AWS HTTP 1.1 specification implementation that does not allow get requests with body:
Note:Get requests with body were a requirement from our client
RFC 7231 HTTP/1.1 specification says the following:
A payload within a GET request message has no defined semantics;
sending a payload body on a GET request might cause some existing
implementations to reject the request.
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<TITLE>ERROR: The request could not be satisfied</TITLE>
</HEAD>
<BODY>
<H1>403 ERROR</H1>
<H2>The request could not be satisfied.</H2>
<HR noshade size="1px">
Bad request.
<BR clear="all">
<HR noshade size="1px">
<PRE>
Generated by cloudfront (CloudFront)
Request ID:
</PRE>
<ADDRESS></ADDRESS>
</BODY>
</HTML>
so my questions are:
are get request with body allowed in AWS?
How AWS deals with get request with body?
is there a way to make work get requests with body on AWS?
I saw almost the same question here:AWS GET request with body rejected by CloudFront
and they point to this document: https://docs.aws.amazon.com/apigateway/latest/developerguide/getting-started-lambda-non-proxy-integration.html that says if you send a get request with body it returns a 400 error, but the error I am getting is 403 error
so could you clarify a little bit more? or could you point to an amazon document that mentions the restrictions on get requests?
Many Thanks
GET request with Body is not allowed on CloudFront, You will get 403 if you send body, though RFC does not specifically say that you should reject GET with body but CloudFront doesn't allow that. The best option to pass body in GET request is by query string(Maximum length of a request, including headers and query strings 20,480 bytes).
https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/RequestAndResponseBehaviorCustomOrigin.html
GET Requests That Include a Body
If a viewer GET request includes a body, CloudFront returns an HTTP status code 403 (Forbidden) to the viewer.
I have a client that sends http requests with absolute request uris and does not define "host"
There for a web server like Jetty rejects the POST request saying:
org.eclipse.jetty.http.BadMessageException: 400: No Host
Is there any way to configure jetty or any other web server to accept post requests that don't have a host?
https://www.rfc-editor.org/rfc/rfc7230#section-5.4
A client MUST send a Host header for it to be compliant to HTTP/1.1
If you send an absolute URI in the target, then the Host header must be the same as what is found in that absolute URI target.
Note that the HTTP/1.1 spec even states ...
A server MUST respond with a 400 (Bad Request) status code to any
HTTP/1.1 request message that lacks a Host header field and to any
request message that contains more than one Host header field or a
Host header field with an invalid field-value.
This makes it a requirement to respond with error 400 in this situation, always.
If you use HTTP/2, then you can avoid this because the split between target URI and Host header was addressed with the :authority pseudo-header.
See https://httpwg.org/specs/rfc7540.html#rfc.section.8.1.2.3
I use Jetty 9.4.8 and i want limits the amount of data that can post to the server. For that i added to jetty.xml:
<Call name="setAttribute">
<Arg>org.eclipse.jetty.server.Request.maxFormContentSize</Arg>
<Arg>10000</Arg>
</Call>
I tested jetty like (request-xxlarge.xml - text file(20mb)):
curl -X POST --data #request-xxlarge.xml http://localhost:8080/test -v
As a result i got
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> POST /test HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.47.0
> Accept: */*
> Content-Length: 21232818
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
>
< HTTP/1.1 200 OK
< Content-Length: 4
< Connection: close
< Server: Jetty(9.4.8.v20171121)
<
* Closing connection 0
POST
server processed the request
EDITED
The server returns 413 when Content-Type: application / x-www-form-urlencoded.
But if I expect it to be a web service and i want to process Content-Type: application / soap + xml - then the maxFormContentSize parameter does not work.
How can I limit the size of the body for a web service?
There are 3 limits that Jetty can impose on request body content based on specific Content-Type values.
Content-Type: application/x-www-form-urlencoded
This is the standard html <form> submission, where the submitted form is accessed via the HttpServletRequest.getParameter(String key) API.
This kind of request body content can be filtered in 2 different ways.
org.eclipse.jetty.server.Request.maxFormContentSize - this is a limit on the overall size (in bytes) of the request body content.
org.eclipse.jetty.server.Request.maxFormKeys - this is a limit on the overall number of form keys of the form being submitted.
The above 2 keys expect integer values, and can be set in any of the following techniques ...
1) As a Server attribute that applies to all deployed webapps
Server.setAttribute("org.eclipse.jetty.server.Request.maxFormContentSize", 100000)
2) As a System Property that applies to all deployed webapps
$ java -Dorg.eclipse.jetty.server.Request.maxFormContentSize=100000 \
-jar /path/to/jetty-home/start.jar
or
System.setProperty("org.eclipse.jetty.server.Request.maxFormContentSize", "1000000")
3) As a Context parameter on the deployed WebAppContext
In your ${jetty.base}/webapps/<context>.xml you can set these values
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN"
"http://www.eclipse.org/jetty/configure_9_3.dtd">
<Configure class="org.eclipse.jetty.webapp.WebAppContext">
<Set name="contextPath">/mycontext</Set>
<Set name="war"><Property name="jetty.webapps"/>/mycontext.war</Set>
<Set name="maxFormKeys">100</Set>
<Set name="maxFormContentSize">202020</Set>
</Configure>
Content-Type: multipart/form-data
This is also a standard form submission as above, but the data can also be accessed via the HttpServletRequest.getPart(String name) APIs.
This kind of data is restricted via the configuration present for the destination Servlet's #MultipartConfig values (or WEB-INF/web.xml entries for the <servlet><multipart-config> elements)
To limit this kind of content type by entire request size, use the maxRequestSize configuration.
All other request Content-Type entries
For all other Content-Type values, the servlet spec, and Jetty are not involved in parsing or limiting the size.
If you are using a library to handle your requests (REST, SOAP, etc) then check with that library to see if it has a mechanism to reject requests based on size, or other reasons (eg: not well-formed, missing expected/required data, etc).
As always, if you want a feature, feel free to request it on the Eclipse Jetty issue tracker at
https://github.com/eclipse/jetty.project/issues
Not sure what your issue is, but I'll give it a shot.
###Theory###
The Expect: 100-continue header is specified in HTTP 1.1 and allows the server to acknowledge or reject a POST/PUT request immediately after the headers are send but before the client starts sending potentially large amounts of actual data (body of the request). Thus, a conforming client must then wait for a HTTP/1.1 100 Continue before sending the data. This scheme is advantageous for larger POST/PUT requests since in the case of rejection the client and server don't waste their time with superfluous network communication.
###Possible Solutions###
Disable Expect Logic
curl -H 'Expect:' -H 'Transfer-Encoding: ...' -H 'Content-Type: ...'
--data-binary '#mediumfile' example.com/path -v
Send next packet with the big file as expected after the first response packet.
More info here : https://www.rfc-editor.org/rfc/rfc7231#section-5.1.1