I need to call a REST-ful webservice in using a GET method with some parameters and save the output of the same.
My first approach was to make some requests in JavaScript and log the output using console.log(), but the server doesn't allow CORS. So I can't make it that way.
I am pretty sure this might be a common thing but I can't seem to find a simple way to do it. What would be the simplest way to do it? Is there any software that would allow me to make an array, let's say with 100 parameters, save 100 calls or what would be a better way to do it? PHP script?
p.s. I can't activate CORS in the server, nor can I place code in the same domain. So far I have an example I can call in the browser and have the XML return.
As far as CORS is concerned, that has to do with the API you're running on not allowing requests to be made from a different domain. This could be fixed by allowing CORS in the API you are developing on.
CORS Link: https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS
The other option would be to have your website on the same domain as the API.
Once you have done that, you can simply make multiple AJAX requests to the API.
AJAX HTTP GET example: HTTP GET request in JavaScript?
EDIT:
You might also have to enable CORS in the HTTP header of your request. This would have to be added to an AJAX request:
Access-Control-Request-Headers: x-requested-with
Here is a helpful link for jquery in particular: How to get a cross-origin resource sharing (CORS) post request working
Related
Infra of system
Expected:
I want to block requests, which is not from Server FE (domain.com)
Ex: Users make request from another apps such as Postman -> it will response 403, message access denied.
I used the rules of ALB, it works but users can cheat on Postman
Also I use AWS WAF to detect request. But it's not work.
Is there any way to block request from Postman or another apps?
We can generate secret_key and check between Server FE and Server BE. But users can see it on Headers and simulator the headers on Postman and call API success.
Current Solution:
I use Rule of Application Load Balancer to check Host and Origin. But users can add these params on Postman and request success.
Rule ALB
When I add Origin matching value (set on ALB) -> We can request successful
Postman success
Postman denied
Users can cheat and call API success.
Thanks for reading. Please help me give any solution for this one. Thanks a lot.
No. HTTP servers have no way to know what client is being used to make any HTTP request. Any HTTP client (Browsers, PostMan, curl, whatever) is capable of making exactly the same requests as each other.
The user-agent header is a superficial way to do this, but it's easy enough for PostMan or any other HTTP client to spoof the user-agent header to one that makes the request look like it is coming from a web browser agent.
You can only make it more challenging to do so. Some examples to thwart this behavior includes using tools like Google captcha or CloudFlare browser integrity check, but they're not bulletproof and ultimately aren't 100% effective at stopping people from using tools/automation to access your site in unintended ways. At the end of the day, you're limited to what can be done with HTTP, and PostMan can do everything at the HTTP layer.
I have a custom domain example.com that is redirecting to my API gateway api-example.com, but it doesn't seem to pass the user-agent field, all my user-agent values are AmazonAPIGateway_5rfp2g9h9b.
If I call directly the api-example.com then it works fine, but if I call example.com, doesn't work.
Any idea on how I could pass the correct user-agent HTTP Header?
Thanks
It’s not clear what you mean by redirect or the domains you have listed, so you have two custom domains ? And if so how did you do that, Cloudfront with a custom origin? And what type of integration request do you have? Is this a REST or HTTP API? Probably why you are getting down voted because you don’t have any detail and the domains don’t make sense.
Either way in your API make sure you have the user-angent field defined where it is applicable:
Request Part of your API, and make sure your integration request is forwarding this header
Likewise make sure Cloudfront forwards the ‘user-agent’ header, that it is also whitelisted if you are using Cloudfront
Note this header comes from your Web browser or SDK being used sometimes sets this too. So if you don’t set this header for whatever reason that could be a problem, I don’t know if for example when you say from this domain that means you are using a hosted website, and another means making a request from Postman, etc.
Short answer: Validate the contents of your header
Ref AWS user-agent redirect here.. as listed below.
Redirects and HTTP user-agents:
..Programs that use the Amazon S3 REST API should handle redirects either at the application layer or the HTTP layer. Many HTTP client libraries and user agents can be configured to correctly handle redirects automatically; however, many others have incorrect or incomplete redirect implementations.
Before you rely on a library to fulfill the redirect requirement, test the following cases:
Verify all HTTP request headers are correctly included in the redirected request (the second request after receiving a redirect) including HTTP standards such as Authorization and Date.
Verify non-GET redirects, such as PUT and DELETE, work correctly.
Verify large PUT requests follow redirects correctly.
Verify PUT requests follow redirects correctly if the 100-continue response takes a long time to arrive.
HTTP user-agents that strictly conform to RFC 2616 might require explicit confirmation before following a redirect when the HTTP request method is not GET or HEAD. It is generally safe to follow redirects generated by Amazon S3 automatically, as the system will issue redirects only to hosts within the amazonaws.com domain and the effect of the redirected request will be the same as that of the original request...
Optional/Additional help, I was trying to understand your description, if you're going across domains, thats CORS.
Please consider CORS which you seem to be missing, please see configuration
here.
Also very important you Enabling CORS support for a resource and its methods does not recursively enable it for child resources and their methods.
If you want to setup your custom header for
user-agent
Setup CORS in Console
How to setup from console under the resources enable the CORS.
Setup your Headers
As a last step you have to REdeploy to a stage, for the settings to take effect!
I'm trying to learn more about XSRF and thus about SOP, I'm reading a lot and I have a more or less clear idea about what should happen and what should not, but I can't figure out why i'm obtaining different result while performing the same thing (I assume) in two different ways.
Same basic scenario of may similar questions, goodsite.com and badsite.com; badsite would like to make a GET request on goodsite.com using user's cookies.
I was able to make this work in a few different ways, a form with an onload send, a button with with an onclick function, they are all working correctly, the query is issued (and ofc i cannot see the response due to SOP).
However i can't figure out why this is not working
xhReq = new XMLHttpRequest();
xhReq.open("GET", "https://goodsite.com/doStuff");
xhReq.withCredentials=true;
xhReq.send();
This code, with the same settings/parameters of all the other working solutions is blocked by my browser with this error.
Access to XMLHttpRequest at 'https://goodsite.com/doStuff' from origin 'https://badguy.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
What am I missing? Thanks.
Is the javascript CORS policy. Servers send a header to tell browsers "Hey, i don't want anyone outside my white list to send me requests using javascript".
Is a security mechanism to avoid things like read the request cookie headers etc.
You can avoid it using a server. There are some online like https://cors-anywhere.herokuapp.com/
Just add the url to this link if you are feching some content cross origen.
Extended: https://portswigger.net/web-security/cors
My app is an Ember.js front end with a Go API on the server. I created the Ember app using the FixtureAdapter. Now that I have the Go API back end I converted it to RESTAdapter.
When I hit my API directly with the browser, I seem to get the appropriate CORS headers back:
Access-Control-Allow-Credentials:true
Access-Control-Allow-Headers:Origin,x-requested-with
Access-Control-Allow-Methods:PUT,PATCH,GET,POST
Access-Control-Allow-Origin:*
Access-Control-Expose-Headers:Content-Length
However, when my Ember.js app hits the API, I get XMLHttpRequest cannot load https://192.168.59.103:8001/notifications. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://192.168.59.103:9000' is therefore not allowed access..
I don't know how else to see what's going on between Ember and the API. I've read this CORS tutorial and everything seems to be okay on the server end.
I included the bit about x-requested-with because of another Stack Overflow question suggesting that jQuery requests need something different than plain old JavaScript requests.
Your endpoints also need to respond with CORS headers to OPTIONS requests- those will execute before the actual request to make sure that the request is allowed first.
Do you happen to be using Nginx as your reverse proxy for your API? If so, we experienced this same issue. In our case, the problem was that Nginx returns the correct CORS headers just fine for HTTP 200 OK responses, but for any error response, it will fail to add the COR headers and therefore the actual server response gets obscured by the browser complaining that it doesn't have the appropriate CORS headers to render a response.
See https://serverfault.com/a/431580/199943.
Even if you're not using Nginx, try calling your API directly (e.g. using Postman to avoid the CORS restrictions to see what it's returning. If your API is returning anything other than an HTTP 200 OK, that may be why the CORS headers aren't getting added.
I've been looking on the web regarding CORS, and I wanted to confirm if whatever I made of it is, what it actually is.
Mentioned below is a totally fictional scenario.
I'll take an example of a normal website. Say my html page has a form that takes a text field name. On submitting it, it sends the form data to myPage.php. Now, what happens internally is that, the server sends the request to www.mydomain.com/mydirectory/myPage.php along with the text fields. Now, the server sees that the request was fired off from the same domain/port/protocol
(Question 1. How does server know about all these details. Where does it extract all these details froms?)
Nonetheless, since the request is originated from same domain, it server the php script and returns whatever is required off it.
Now, for the sake of argument, let's say I don't want to manually fill the data in text field, but instead I want to do it programmatically. What I do is, I create a html page with javascript and fire off a POST request along with the parameters (i.e. values of textField). Now since my request is not from any domain as such, the server disregards the service to my request. and I get cross domain error?
Similarly, I could have written a Java program also, that makes use of HTTPClient/Post request and do the same thing.
Question 2 : Is this what the problem is?
Now, what CORS provide us is, that the server will say that 'anyone can access myPage.php'.
From enable cors.org it says that
For simple CORS requests, the server only needs to add the following header to its response:
Access-Control-Allow-Origin: *
Now, what exactly is the client going to do with this header. As in, the client anyway wanted to make call to the resources on server right? It should be upto server to just configure itself with whether it wants to accept or not, and act accordingly.
Question 3 : What's the use of sending a header back to client (who has already made a request to the server)?
And finally, what I don't get is that, say I am building some RESTful services for my android app. Now, say I have one POST service www.mydomain.com/rest/services/myPost. I've got my Tomcat server hosting these services on my local machine.
In my android app, I just call this service, and get the result back (if any). Where exactly did I use CORS in this case. Does this fall under a different category of server calls? If yes, then how exactly.
Furthermore, I checked Enable Cors for Tomcat and it says that I can add a filter in my web.xml of my dynamic web project, and then it will start accepting it.
Question 4 : Is that what is enabling the calls from my android device to my webservices?
Thanks
First of all, the cross domain check is performed by the browser, not the server. When the JavaScript makes an XmlHttpRequest to a server other than its origin, if the browser supports CORS it will initialize a CORS process. Or else, the request will result in an error (unless user has deliberately reduced browser security)
When the server encounters Origin HTTP header, server will decide if it is in the list of allowed domains. If it is not in the list, the request will fail (i.e. server will send an error response).
For number 3 and 4, I think you should ask separate questions. Otherwise this question will become too broad. And I think it will quickly get close if you do not remove it.
For an explanation of CORS, please see this answer from programmers: https://softwareengineering.stackexchange.com/a/253043/139479
NOTE: CORS is more of a convention. It does not guarantee security. You can write a malicious browser that disregards the same domain policy. And it will execute JavaScript fetched from any site. You can also create HTTP headers with arbitrary Origin headers, and get information from any third party server that implements CORS. CORS only works if you trust your browser.
For question 3, you need to understand the relationship between the two sites and the client's browser. As Krumia alluded to in their answer, it's more of a convention between the three participants in the request.
I recently posted an article which goes into a bit more detail about how CORS handshakes are designed to work.
Well I am not a security expert but I hope, I can answer this question in one line.
If CORS is enabled then server will just ask browser if you are calling the request from [xyz.com]? If browser say yes it will show the result and if browser says no it is from [abc.com] it will throw error.
So CORS is dependent on browser. And that's why browsers send a preflight request before actual request.
In my case I just added
.authorizeRequests().antMatchers(HttpMethod.OPTIONS, "/**").permitAll()
to my WebSecurityConfiguration file issue is resolved