We have a REST endpoint that provides some back end services to our publicly available Web site. The web site does not require any user authentication to access its content. Anyone can access it anonymously.
Given this scenario, we would still like to protect the back-end REST api to be somewhat secured in the sense that only users using our Web site can call it.
We dont want a malicious user to run a script outside the browser bombarding it for example.
We dont even want him to run a script automating the UI to access the endpoint.
I understand that a fully public endpoint without user authentication is somewhat impossible to secure. But can we restrict usage to valid scenarios?
Some ideas:
Use TLS/SSL for the communication - this protects the channel only.
Use some Api key (that periodically expires) that the client/browser needs to pass to the server. (a malicious user can still use the key)
Use the key to throttle the number of requests.
Use it with conjunction of a CSRF token??
Use CAPTCHA on the web site to ensure human user ( adds an element of annoyance to the final user).
Use IP whitelisting.
Use load balancing and scaling of server to handle loads.
I suppose this should be a scenario occurring in the wild.
What security steps are prevalent?
Is it possible to restrict usage via only the website and not via a script?
If its not possible to secure, what kind of mitigations are used with such public rest endpoints?
Related
Let me explain in detail
I've 2 servers hosted and one of them for Back-end and other is for Front-end.
Back-end server : 127.0.0.1:8000 (just for explaining)
Front-end server : 127.1.1.1:9000
User requests UI from Front-end server and if he wants to create an account on my website he needs to send POST request to 127.0.0.1:8000/create-account/ and this works fine
but if I open console of other website or make use of Postman, I'm able to achieve the same results.
So I want to prevent this thing and only allow anyone to create account from my website only.
Methods which I've tried
I've used windows.location() and sent it to server and then verify if domain name matches. But in this method everyone can just pass it simply via fetch()
I've used allow only IP address, But if I push my website in production Other visitors get 403 error.
I develop back-end with help of Django and rustlang
It isn't possible. You can make it hard for entry-level programmers to reverse-engineer your solution, but there isn't any way to prevent access to your API if you are going to allow access to it from some public UI.
IP address-based restriction will not work here because your backend will receive the IP of the user. In this case, you will be blocking access to users, not to any UI. Even the host header verification doesn't work here as anyone can use a proxy server, i.e. NGINX, to override the headers and can fake the request to originate from an intended website.
IP address-based authorization can work only if API calls originate from a server and your API server receives the same IP address for each API call. But for your use case, it isn't applicable.
The older techniques like CSRF are useless too as anyone can easily retrieve the token and can send it. In short, if you make something public, it can be reverse-engineered. If you are accepting public registrations, there shouldn't be anything to worry about the registration source. You should think about solutions like email verification etc. to reduce the spam if that's the concern.
You could have your frontend solve a recaptcha and send the solution to the backend. Verify the solution before accepting the request. It is still possible to bypass although a bit harder.
We have gone through the authentication overview but it doesn't answer our question so reaching out to the experts here.
Our web application once loaded on the browser for a user makes requests to our service on Cloud Run. We would like to secure our backend service so only our Domain can access this and not some user using Postman, or a shell script to access / update data.
end-user-access auth
Because the user doesn't have to be always signed in, we would like Cloud Run to respond to the requests, even if there's no auth token present. This eliminates the end-user access to service option for us.
service-to-service auth:
Because it is the browser, and not another service in our GCP platform that's making the request, even this option doesn't work for us.
The user data in our backend is secure, because those end points require Auth tokens, however, pages that don't need auth token are insecure, and could be vulnerable to DDOS or bot/scraping attacks.
How can we secure our Cloud Run service so only calls from our Web Application are responded to and not others?
You can't be sure that the web application is the sender of the request. Indeed, in the web environment, you send your code on the client side. From here, the users (good or bad) can go into your code and understand how it works, what are the header of the request, how these headers are built,...
Complex, and inefficient things can be imagined... Maybe, the best way is to plug an API Gateway and to perform rate limit for the users. Like this, even if the user uses an automated thing, they will be limited. I wrote an article on the Cloud Endpoint Rate limit. Not sure that is the best product for this, maybe Apigee can fit better your requirements.
EDIT
What I would like to say in the first paragraph is "it's not possible with Cloud Run and with any other web technology".
Is it possible to block connections to a web service (server) from outside its domain?
For example consider a web app that fetches data from Twitter's API using Twitter's "application only auth". The web app's client uses AJAX to call it's own server, which in turn calls Twitter's API with Twitter's token.
While the token is never exposed to the client side code is there anything to stop an outside server side app from calling the web app's server using the URLs used by the client and for example exhausting the Twitter tokens rate limits?
Is it possible to block connections to a web service (server) from outside its domain?
Certainly. Set your web server's access control lists to drop connections from outside of your IP range. Alternately, install a firewall. That's very straightforward, but I suspect you mean something else by "outside its domain?"
From your description, you seem to be really asking whether you verify that you're only talking to your own client application. As a general rule, no. You can authenticate users. That's easy. If the user isn't logged in and authorized to use your service, you don't forward requests to Twitter. But you can't authenticate applications.
If you're going to accept any user who shows up, you can't stop them from using whatever client they want. There is no way to ensure that it is your unmodified client if you've allowed it to be run on their machine. They can always modify it, and they can always send you arbitrary traffic from other programs and you can't tell the difference. On the network, bytes are bytes.
It's not all hopeless; there are things you can do. See https://stackoverflow.com/a/9183066/97337 for another version of this question, and links to several other versions of the question. (They're not exactly duplicates in how they're asked, but they all wind up being basically the same answer.)
You should secure your web service with user and password security or certificate security. The basic idea is that the web service client must authenticate in order to call your web service.
Here are some technics (there are others or variations):
1) HTTP basic authentication and HTTPS
2) Mutual SSL authentication - Also called two-way authentication, is a process in which both entities authenticate with each other. The server presents a certificate to the client and the client present a certificate to the server.
3) With SOAP web services you can use WS-Security standard.
4) OAuth framework
5) With Rest services you can use options 1), 2), 4). Or implement one by your own. This are good recomendations.
As you can see, there are a lot of ways to secure a web service.
I have a public facing API that returns some data, internally using the Google Maps API Service. This API is mostly for interal purposes right now, invoked through the webapplication.
However, I wish to restrict the usage of this API i.e. it should only be invoked from my Web Application ( or mobile app) when a user (non-registered) browses it. An http request directly to this API should not be authorized.
I cannot use API keys since the webapp flow should work for non-registered users as well.
If you're not using HTTPS, any security mechanism is flawed, because it can be replicated. IMHO, you could add a HTTP header (e.g. "Request-source: YourApp") and check for its existance in your API.
Of course, once it's documented somehow, anyone can mimic this header. But if you use HTTPS and create a header that's unknown for other people, you prevent this from happening.
This is a similar situation to the one raised in this question:
Javascript Calling a Rest API with App Name and App Password - How Can i Secure it
Here is the architecture overview:
The site is Html5/jquerymobile
It contacts what I call a "Wrapper" service.... This is a REST API I wrote in C#, to contact another 3rd party REST API. I do this because there are credentials in the Header and the API uses Basic Authentication. Credentials are therefore not publicized as they are only known server-side.
My "Wrapper" service does not currently implement any additional security. It is currently accessible from anywhere. The easiest and quickest way to lock it down is to restrict by IP, so no other IP anywhere except the server can actually contact my wrapper service.
The questions:
Is the locking by IP the only way to ensure that the API won't get hammered if it was otherwise accessible from anywhere?
If I convert this using Phonegap (which I have... and deployed successfully on Android), obviously the native app won't work if the web service is restricted.
Is there a way around this so I can allow traffic only from the mobile app, and not from any other source? I'm thinking along the lines of MD5 hash or something that could be sent to the wrapper API.. but unfortunately I'm thinking that info can easily be "sniffed".
Is my only viable option here to release the app as a web app, forcing browser use, thereby removing any concerns about allowing my web service to be hammered??
I believe the answer to this is a combination of a user token and encrypting the message through SSL.
The server can issue a valid user a token so we can identify him in future requests.
Encrypting it via SSL will ensure that this token cannot be sniffed.
https://security.stackexchange.com/questions/12531/ssl-with-get-and-post