I'm getting a 400 Bad Request response from my WAF/CDN, with the reason for failure being given as ERR_HRS_SPACE_IN_HDR_NAME. What is causing this and how can I avoid it?
The HRS in the error reason is "HTTP Request Smuggling", which is a way of interfering with the sequences of HTTP requests that are received from one or more users by a web server. It's likely you have some unnecessary whitespace in your request headers (probably before the colon - e.g. Accept : application/json rather than Accept: application/json or preceding the header name - e.g. [space]Transfer-Encoding: chunked).
Checking your headers and removing the offending whitespace should prevent this issue.
Related
I am trying to hit this URL https://subdomain.example.com in JMeter and recorded using the Blazemeter Chrome extension has all the necessary config elements but get an error:
HTTP/1.1 429 Too Many Requests
Content-Type: text/html; charset=utf-8
Content-Length: 1031
Connection: keep-alive
Cache-Control: private, no-cache, no-store, must-revalidate
Date: Tue, 20 Aug 2019 01:21:35 GMT
Expires: 0
p3p: CP="This site does not specify a policy in the P3P header"
I have tried coping the Header Cookies from Browser Header Response which works for sometime but then start throwing an error
As per HTTP Status Code 429 Too Many Requests description:
The HTTP 429 Too Many Requests response status code indicates the user has sent too many requests in a given amount of time ("rate limiting").
A Retry-After header might be included to this response indicating how long to wait before making a new request.
So there are following options:
Your server is overloaded, in this case there is nothing you can do here apart from reporting the error as the bottleneck
Your script doesn't have proper correlation implemented, i.e. you're sending recorded hard-coded values instead of getting dynamic parameters
Your server doesn't allow such amount of requests from a single IP address within the given timeframe, you could try implementing IP Spoofing so your server would "think" that the requests are coming from the different machines.
Thanks for your reply. In the end I figured out that no limitation for number of calls implemented.
Now come to answer this is how I managed to work this:
Opened the page in chrome and from the header section copied all the header elements into the header manager hard coded.
First time it fails and returns p3p: CP="This site does not specify a policy in the P3P header" but also return the update variable value needed for next request which I extract and used in the next and subsequent Requests. The way I was able to find out which variable is changing by using the string comparison of 2 Response Headers
This was a difficult one but somehow worked with very minor change I also added the Header Manager to each request for safer side.
I'm testing this web service with resttesttest and getting the same error all the time:
Oh no! Javascript returned an HTTP 0 error. One common reason this
might happen is that you requested a cross-domain resource from a
server that did not include the appropriate CORS headers in the
response. Better open up your Firebug...
Is this something with the URL or it's just my ignorance?
How to get JSON data from there?
Update. Solution found.
There was 2 reasons why my request didn't work:
1) I've copied the url from a pdf file and because of that it had some problems with encoding, which resulted in requesting the url in Punycode.
2) For some reason resttesttest didn't want to work even with correct url so I've checked once again in Chrome's Advanced REST client and that worked for me.
You need to set what headers to allow, in order to get a Cross-origin resource sharing (wiki here). If you use Node.JS, check something like this:
res.setHeader('Access-Control-Allow-Origin', '*');
res.setHeader('Access-Control-Allow-Headers', 'Origin, Accept, Accept- Version, Content-Length, Content-MD5, Content-Type, Date, X-Api-Version, X-Response-Time, X-PINGOTHER, X-CSRF-Token,Authorization');
res.setHeader('Access-Control-Allow-Methods', 'GET, POST, PUT, DELETE, PATCH, OPTIONS');
Or use your language syntax to set response headers.
I am using jetty-9.2.2 with CometD-3.0.1. I am seeing below warning in my setup. It comes ~4,5 times in a day.:
2014-08-28 08:50:53.712:WARN:oejh.HttpParser:qtp607635164-15194: badMessage:
400 Illegal character for HttpChannelOverHttp#5946f125{r=1,a=IDLE,uri=-}
There is no details that can be debugged from the warning message. I have already logged a request https://bugs.eclipse.org/bugs/show_bug.cgi?id=443049 to provide detailed warning.
Meanwhile I want to know what is causing this warning? Can I ignore this or some messages are lost because of this?
Change https to http in the url.
I had the same error, then found out it's because my application did not support https, so jetty cannot recognize the https encrypted request.
Update May 2017
For Jetty 9.3+ users, you might see a log message that makes this response code more clear.
See Header parse error after upgrade to Jetty 9.3 for details.
Original Answer
The Bad Message: 400 Illegal Character can occur during parsing of a bad HTTP Request.
That is the HTTP error response that the client sees.
Some (not all) situations in which it can occur.
The EOL is not "\r\n" (CR + LF) (HTTP spec requirement)
The HTTP Method token is either not recognized or has invalid whitespace after it
The HTTP Version is not recognized or has invalid characters
HTTP Header name does not follow spec
HTTP Header value does not follow spec
This message is common on public (internet facing) servers.
You have bad HTTP requests coming in. Why?
A legitimate HTTP client has a bug
A legitimate HTTP client is not following the HTTP spec
A non HTTP client attempted to connect to your server (such as attempting to use non-encrypted HTTP on a SSL/TLS/HTTPS port, or even something as odd as an SMTP/IMAP email client attempting to talk to your HTTP port)
A malicious client is attempting to probe your system for weaknesses
This error can be caused, as it was for me, by a silly little mistake.
When testing on my localhost Jetty instance, I received a very similar 400 Illegal Character message. Then I realized why. I had simply assumed application address on my local Jetty was:
https://localhost:8080
whereas the correct address was unsecured:
http://localhost:8080
No problems after that.
Jetty is cautious about detailed error messages that include user sent data, as these can be part of an attack - even if echo'd just to a terminal.
However, we can do better and log some sanitised data. Acting on the bugzilla
Well, I met this problem because I mistook the "http://" as "https://"
I have a HTTP "rest" web service API that I am creating for accessing user data.
The web service integrates with a XACML policy decision point. The policy decision point determines if the request is authorized as a function of the user and the data being accessed. Generally, the response is "permit" (action is allowed) or "deny" (action is not allowed).
If permit, then the call continues and data is returned.
If deny, then the call is aborted and an HTTP 403 (forbidden) is returned.
However, certain policies have "obligations" to indicate that the action is conditionally allowed with further action. The analogy that I generally use is the "this credit card transaction may be allowed, but the clerk needs to view the customer's ID, and then make this call again asserting that the ID matches the credit card".
In my web service, I want to prompt the client to take action and include additional information in the request URL to indicate that the obligation was met. I am communicating this information with structured body response understood by the client application.
My question is what is the appropriate HTTP status code to use in the scenario.
"403" would not be appropriate (text from HTTP spec Authorization will not help and the request SHOULD NOT be repeated).
My best guess would "401" (unauthorized), but I am not sure if this status code is specifically around the use of the authorization header and username/password type of concerns.
Maybe you could use the HTTP status code 303 or 307 to point the user to a temporary redirect location that somehow encodes the extra "obligation"?
A 401 response seems reasonable to me. The HTTP authentication mechanism is extensible, so you could conceivably create a custom authentication scheme so that you are properly setting the WWW-Authenticate header as required by the RFC. Browsers won't know what to do with your custom scheme, but I assume your clients aren't browsers anyway, if they have to understand what an obligation is.
Example request sequence:
PUT /some/resource/that/has/obligations HTTP/1.1
Content-Type: application/json
Authorization: token my-oauth-token
HTTP/1.1 401 Unauthorized
WWW-Authenticate: obligation urn:my-app:my-obligation;param1;param2
PUT /some/resource/that/has/obligations HTTP/1.1
Content-Type: application/json
Authorization: token my-oauth-token
Authorization: obligation urn:my-app:my-obligation:result=ok
HTTP/1.1 201 Created
Location: /some/resource/that/has/obligations/1
Another option would be to return a 202 on the initial post or put, and then confirm it on a later post. This option would require a little more state management server side, since you'd first accept the operation, and then wait for the client to confirm it.
PUT /some/resource/that/has/obligations HTTP/1.1
Content-Type: application/json
Authorization: token my-oauth-token
HTTP/1.1 202 Accepted
Location: /some/resource/that/has/obligations/1
X-Obligation: urn:my-app:my-obligation;param1;param2
POST /some/resource/that/has/obligations/1 HTTP/1.1
Content-Type: application/json
Authorization: token my-oauth-token
X-Obligation: urn:my-app:my-obligation;result=ok
HTTP/1.1 200 OK
One thing to always keep in mind with obligations, though... they're always enforced client-side, so unless the obligation involves hitting another service that your service can double-check against, you never know if the client actually performed the obligation. If you don't control the client, then obligations are really just theatre.
I'm implementing an API. The API accepts/returns JSON content type.
Now, suppose that the data submitted by some POST request is not valid, like a missing attribute, or a duplication exists for the same data.
What is the standard HTML response code in that case?
The error lies on the client side, so you want to use a 4xx status code. I'd go with 400 - Bad Request:
The request could not be understood by
the server due to malformed syntax.
The client SHOULD NOT repeat the
request without modifications.
There are two answers:
If you have submitted a form, just return 200 - OK with HTML explaining why the object was not created.
If you have an API you should use the following
200 OK
When the request was OK and returned the proper data.
201 CREATED
The call was successful and the new object created.
400 BAD REQUEST
Invalid request URI
Invalid HTTP Header
Receiving an unsupported, nonstandard parameter
Receiving an invalid HTTP Message Body
401 UNAUTHORIZED
Authorization problems. E.g. wrong API key, etc.
403 FORBIDDEN
Properly authorized, but not allowed.
404 NOT FOUND
The resource does not exist (e.g. on Read or Update)
405 METHOD NOT ALLOWED
Use in situations that a given REST method is not allowed. E.g. a POST on a single resource, or a DELETE on the entire collection of resources.
409 CONFLICT
When an update fails, send "Conflict" to allow the client side to resolve the conflict themselves and retry.
500 INTERNAL SERVER ERROR
Internal error. This is the default code that is used for all unrecognized errors.
501 NOT IMPLEMENTED
Use for expected, but not yet implemented features.
The closest i can find would be 400 Bad Request.
As Ariejan said you should base your API in the HTTP codes already defined. If you want to send a error message the best way should be not use the HTTP message, but better include the message in the response body, JSON formatted.
422 Unprocessable Entity (see RFC 4918, Section 11.2)