I have a custom domain example.com that is redirecting to my API gateway api-example.com, but it doesn't seem to pass the user-agent field, all my user-agent values are AmazonAPIGateway_5rfp2g9h9b.
If I call directly the api-example.com then it works fine, but if I call example.com, doesn't work.
Any idea on how I could pass the correct user-agent HTTP Header?
Thanks
It’s not clear what you mean by redirect or the domains you have listed, so you have two custom domains ? And if so how did you do that, Cloudfront with a custom origin? And what type of integration request do you have? Is this a REST or HTTP API? Probably why you are getting down voted because you don’t have any detail and the domains don’t make sense.
Either way in your API make sure you have the user-angent field defined where it is applicable:
Request Part of your API, and make sure your integration request is forwarding this header
Likewise make sure Cloudfront forwards the ‘user-agent’ header, that it is also whitelisted if you are using Cloudfront
Note this header comes from your Web browser or SDK being used sometimes sets this too. So if you don’t set this header for whatever reason that could be a problem, I don’t know if for example when you say from this domain that means you are using a hosted website, and another means making a request from Postman, etc.
Short answer: Validate the contents of your header
Ref AWS user-agent redirect here.. as listed below.
Redirects and HTTP user-agents:
..Programs that use the Amazon S3 REST API should handle redirects either at the application layer or the HTTP layer. Many HTTP client libraries and user agents can be configured to correctly handle redirects automatically; however, many others have incorrect or incomplete redirect implementations.
Before you rely on a library to fulfill the redirect requirement, test the following cases:
Verify all HTTP request headers are correctly included in the redirected request (the second request after receiving a redirect) including HTTP standards such as Authorization and Date.
Verify non-GET redirects, such as PUT and DELETE, work correctly.
Verify large PUT requests follow redirects correctly.
Verify PUT requests follow redirects correctly if the 100-continue response takes a long time to arrive.
HTTP user-agents that strictly conform to RFC 2616 might require explicit confirmation before following a redirect when the HTTP request method is not GET or HEAD. It is generally safe to follow redirects generated by Amazon S3 automatically, as the system will issue redirects only to hosts within the amazonaws.com domain and the effect of the redirected request will be the same as that of the original request...
Optional/Additional help, I was trying to understand your description, if you're going across domains, thats CORS.
Please consider CORS which you seem to be missing, please see configuration
here.
Also very important you Enabling CORS support for a resource and its methods does not recursively enable it for child resources and their methods.
If you want to setup your custom header for
user-agent
Setup CORS in Console
How to setup from console under the resources enable the CORS.
Setup your Headers
As a last step you have to REdeploy to a stage, for the settings to take effect!
Related
Infra of system
Expected:
I want to block requests, which is not from Server FE (domain.com)
Ex: Users make request from another apps such as Postman -> it will response 403, message access denied.
I used the rules of ALB, it works but users can cheat on Postman
Also I use AWS WAF to detect request. But it's not work.
Is there any way to block request from Postman or another apps?
We can generate secret_key and check between Server FE and Server BE. But users can see it on Headers and simulator the headers on Postman and call API success.
Current Solution:
I use Rule of Application Load Balancer to check Host and Origin. But users can add these params on Postman and request success.
Rule ALB
When I add Origin matching value (set on ALB) -> We can request successful
Postman success
Postman denied
Users can cheat and call API success.
Thanks for reading. Please help me give any solution for this one. Thanks a lot.
No. HTTP servers have no way to know what client is being used to make any HTTP request. Any HTTP client (Browsers, PostMan, curl, whatever) is capable of making exactly the same requests as each other.
The user-agent header is a superficial way to do this, but it's easy enough for PostMan or any other HTTP client to spoof the user-agent header to one that makes the request look like it is coming from a web browser agent.
You can only make it more challenging to do so. Some examples to thwart this behavior includes using tools like Google captcha or CloudFlare browser integrity check, but they're not bulletproof and ultimately aren't 100% effective at stopping people from using tools/automation to access your site in unintended ways. At the end of the day, you're limited to what can be done with HTTP, and PostMan can do everything at the HTTP layer.
Note: My site is in production mode, not testing. It is pending verification due to me adding an icon. This issue persisted before the verification was started.
Whenever my browser makes a request to Google for the one-tap widget or the pill, both requests return 400 Bad Request with an empty HTML page and the console is sent a message stating "The given origin is not allowed for the given client ID." I've gone onto the Google Cloud Console and checked my origins. I have only one listed, and it's the exact site I'm sending requests from my browser. My site also has its traffic proxied through Cloudflare if that makes a difference. In addition, I am using JavaScript callbacks (which work when used in PI#1).
Potential issue #1: The URLs are typed in wrong
When I insert localhost (I add https and http because I test with a HTTPS webserver locally using a Cloudflare origin certificate), the requests go through perfectly. However, the moment the requests are from my browser when it's not localhost, the requests fail. I've copied and pasted straight from the URL bar just to make sure that there's no typos or anything but the same results return.
Potential issue #2: The widget is making bad requests
I do open the URLs in other tabs (Which yield the same results from PI#1) and insert bogus URLs like example.com and thisisnotaurl.com to ensure it's not just dropping every request. Those requests return 403 Forbidden instead of 400 Bad Request.
Potential issue #3: The issue is browser specific
I've checked this issue on both Firefox and Microsoft Edge, both on the stable branches and completely up to date. I've disabled my ad block (UBlock Origin & Firefox built-in protection) to ensure they aren't messing with requests but everything except the crucial requests fail with 400 Bad Request. I have yet to test other browsers as I do not have them installed but I assume the same results come from them.
An example of the code can be found here: https://gist.github.com/Coder-Tavi/772ea25b16f3fa0b6b0e04739a1689dd.
The origins shown below are the exact website I am accessing. In addition, I've verified the client IDs are exactly the same as the ones I have added
Referrer Policy is improperly configured
The HTTP header Referrer-Policy controls the exact amount of data sent to servers regarding the origin of the request. In most cases, this is set to same-origin which means that the Referer header will send the origin to servers with the same origin.
Consider you have a webserver at example.com and another at web.example.com with a Referrer-Policy of same-origin. When example.com sends a request to web.example.com, the Referer header will contain the origin of the request, since it is the same origin. However, if example.com sends a request to google.com, the Referer header will not send any origin data, as google.com and example.com are not the same origin.
If we look at the requests, this directive is what we see
As such, we need to update the directive to allow the browser to send the origin in the Referer header. This can be done by inserting the following into the HTML of the current page.
<meta name="referrer" content="origin">
This meta tag will allow the browser to send the origin only to other webservers, and as such, Google will see the origin.
Consider the example above again. This time, when example.com sends a request to google.com, the request will contain a Referer header with the origin, as the directive allows for sharing of the origin. However, with this current policy, only the origin is sent, not the query parameters and other parts of the URL. With the following URL: https://example.com/test/123, google.com will only see https://example.com. The MDN Web Docs contain all the possible values and their effects.
I was trying to use my api and I have a function on ember app to login but when the login action is trigerred I receive a message below. What is the reason I am receiving this error?
login:1 Access to fetch at 'https://app-dev.some-url.com.au/api/login' from origin
'http://localhost:4099' has been blocked by CORS policy: Response to preflight request doesn't
pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested
resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch
the resource with CORS disabled.
Your API server isn’t configured to accept requests that come from a web application on another origin. Some options:
Configure your API to serve the Access-Control-Allow-Origin header
on the OPTIONS request that the browser makes to test this. It
could be Access-Control-Allow-Origin: * if you’re lazy or can’t
know in advance what origins people will be coming from. It’s hard
to be more specific about this without knowing details of your API.
Use Ember CLI’s API
proxying
feature to bypass the need for CORS. This is for development mode
only, though. You’d need a similar solution in production where
Ember CLI’s development server isn’t present.
I need to call a REST-ful webservice in using a GET method with some parameters and save the output of the same.
My first approach was to make some requests in JavaScript and log the output using console.log(), but the server doesn't allow CORS. So I can't make it that way.
I am pretty sure this might be a common thing but I can't seem to find a simple way to do it. What would be the simplest way to do it? Is there any software that would allow me to make an array, let's say with 100 parameters, save 100 calls or what would be a better way to do it? PHP script?
p.s. I can't activate CORS in the server, nor can I place code in the same domain. So far I have an example I can call in the browser and have the XML return.
As far as CORS is concerned, that has to do with the API you're running on not allowing requests to be made from a different domain. This could be fixed by allowing CORS in the API you are developing on.
CORS Link: https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS
The other option would be to have your website on the same domain as the API.
Once you have done that, you can simply make multiple AJAX requests to the API.
AJAX HTTP GET example: HTTP GET request in JavaScript?
EDIT:
You might also have to enable CORS in the HTTP header of your request. This would have to be added to an AJAX request:
Access-Control-Request-Headers: x-requested-with
Here is a helpful link for jquery in particular: How to get a cross-origin resource sharing (CORS) post request working
I've been looking on the web regarding CORS, and I wanted to confirm if whatever I made of it is, what it actually is.
Mentioned below is a totally fictional scenario.
I'll take an example of a normal website. Say my html page has a form that takes a text field name. On submitting it, it sends the form data to myPage.php. Now, what happens internally is that, the server sends the request to www.mydomain.com/mydirectory/myPage.php along with the text fields. Now, the server sees that the request was fired off from the same domain/port/protocol
(Question 1. How does server know about all these details. Where does it extract all these details froms?)
Nonetheless, since the request is originated from same domain, it server the php script and returns whatever is required off it.
Now, for the sake of argument, let's say I don't want to manually fill the data in text field, but instead I want to do it programmatically. What I do is, I create a html page with javascript and fire off a POST request along with the parameters (i.e. values of textField). Now since my request is not from any domain as such, the server disregards the service to my request. and I get cross domain error?
Similarly, I could have written a Java program also, that makes use of HTTPClient/Post request and do the same thing.
Question 2 : Is this what the problem is?
Now, what CORS provide us is, that the server will say that 'anyone can access myPage.php'.
From enable cors.org it says that
For simple CORS requests, the server only needs to add the following header to its response:
Access-Control-Allow-Origin: *
Now, what exactly is the client going to do with this header. As in, the client anyway wanted to make call to the resources on server right? It should be upto server to just configure itself with whether it wants to accept or not, and act accordingly.
Question 3 : What's the use of sending a header back to client (who has already made a request to the server)?
And finally, what I don't get is that, say I am building some RESTful services for my android app. Now, say I have one POST service www.mydomain.com/rest/services/myPost. I've got my Tomcat server hosting these services on my local machine.
In my android app, I just call this service, and get the result back (if any). Where exactly did I use CORS in this case. Does this fall under a different category of server calls? If yes, then how exactly.
Furthermore, I checked Enable Cors for Tomcat and it says that I can add a filter in my web.xml of my dynamic web project, and then it will start accepting it.
Question 4 : Is that what is enabling the calls from my android device to my webservices?
Thanks
First of all, the cross domain check is performed by the browser, not the server. When the JavaScript makes an XmlHttpRequest to a server other than its origin, if the browser supports CORS it will initialize a CORS process. Or else, the request will result in an error (unless user has deliberately reduced browser security)
When the server encounters Origin HTTP header, server will decide if it is in the list of allowed domains. If it is not in the list, the request will fail (i.e. server will send an error response).
For number 3 and 4, I think you should ask separate questions. Otherwise this question will become too broad. And I think it will quickly get close if you do not remove it.
For an explanation of CORS, please see this answer from programmers: https://softwareengineering.stackexchange.com/a/253043/139479
NOTE: CORS is more of a convention. It does not guarantee security. You can write a malicious browser that disregards the same domain policy. And it will execute JavaScript fetched from any site. You can also create HTTP headers with arbitrary Origin headers, and get information from any third party server that implements CORS. CORS only works if you trust your browser.
For question 3, you need to understand the relationship between the two sites and the client's browser. As Krumia alluded to in their answer, it's more of a convention between the three participants in the request.
I recently posted an article which goes into a bit more detail about how CORS handshakes are designed to work.
Well I am not a security expert but I hope, I can answer this question in one line.
If CORS is enabled then server will just ask browser if you are calling the request from [xyz.com]? If browser say yes it will show the result and if browser says no it is from [abc.com] it will throw error.
So CORS is dependent on browser. And that's why browsers send a preflight request before actual request.
In my case I just added
.authorizeRequests().antMatchers(HttpMethod.OPTIONS, "/**").permitAll()
to my WebSecurityConfiguration file issue is resolved