I have a REST API and some of the endpoints take a significant time to generate the responses, so I want to add some response caching and etag support. I have looked at the conditional response implementation in Django and at both response caching and conditional response in DRF extensions package. The problem I am having is that my data changes very frequently on one side, but is also heavily segregated, so if something changes in the response to user A calling endpoint X, nothing might change for users B, C and D calling the same endpoint. Since my data changes often, if I invalidate all responses on every change, I will never hit cache. The endpoints in question all generate lists of JSON objects, so the question is how can I only invalidate cached responses that contain a changed object rather than invalidating all of them?
Related
I have an issue with throttling user requests to a certain endpoint. I set the limit for them to make only one request/hour and it works okay locally. However on production, the request may take a bit to return a response and in that case, if the users make any new requests during the processing time, it will pass, until the first request returns a response. Any help or pointers?
I have created an API and enable the response cache from run time configuration but still gateway sending request to backend API on Version 3.0
The response cache key includes all the headers and request body, hashed. Therefore, if 2 of your requests have at least one different header, it will be a cache miss.
In the case of Postman, it always sends a random value in Postman-Token header. Hense the cache miss always.
This is more of a theory question, so I'm not going to post any code.
On the frontend, the user types in a search command. On the backend (Django in my case), it hits an API, the results of the search are saved into a Django View in views.py. On the frontend, the user interacts with this returned data and sends another request. On the backend, is the data from the first Django View still available for use? How do you access it?
(The data is also in the frontend and I can send it with the second request. But if it's still stored on the backend then I wouldn't need to.)
HTTP by it's own nature is a stateless protocol. It does mean that protocol doesn't know what or when should happen any request. Request comes and your API just reacts to this request by your implemented logic.
If you want to persist/save any state/data on your API side, you can do it by persisting them to database or saving to any local/global variable. Then you can access this saved state/data while recieving other requests to your back-end and implement the logic to use of previous state with the new incoming data.
I'm designing a web service that parses a large document (150-200k) and returns some analytical data. The contents of the document are sensitive, and currently not persisted by the backend.
With a stateless REST web service, where all requests are idempotent, this would require every request to include the large document payload, which seems less than ideal.
Would a stateful alternative be a more appropriate design for this scenario, where a session is established after the initial document is POSTed? The client could then make further requests to endpoints which would provide differing analytical results, using the document in memory?
You can think of it as a REST interface tacked onto a document storage service.
The document is stored temporarily. Perhaps it stays for 10 minutes or until released by the owner. The doc storage service returns a token allowing access to the document. But the token expires with the document timeout.
Then you only need REST services to ask questions about the document. Each call needs to include the token but can be repeated indefinitely and still get the same response.
You may want to cache certain information about each document. That's a performance issue.
You might want to consider how to encrypt the token in such a way that it can't be copied off the "wire" and used by a "bad guy(TM)".
I have a RESTful API which has annotations like #Consumes(MediaType.JSON) - in that case, would the CSRF attack still be possible on such a service? I've been tinkering with securing my services with CSRFGuard on server side or having a double submit from client side. However when I tried to POST requests using FORM with enctype="text/plain", it didn't work. The technique is explained here This works if I have MediaType.APPLICATION_FORM_URLENCODED in my consumes annotation. The content negotiation is useful when I'm using POST/PUT/DELETE verbs but GET is still accessible which might need looking into.
Any suggestions or inputs would be great, also please let me know if you need more info.
Cheers
JAX-RS is designed to create REST API which is supposed to be stateless.
The Cross Site Request Forgery is NOT a problem with stateless applications.
The way Cross Site Request Forgery works is someone may trick you to click on a link or open a link in your browser which will direct you to a site in which you are logged in, for example some online forum. Since you are already logged in on that forum the attacker can construct a url, say something like this: someforum.com/deletethread?id=23454
That forum program, being badly designed will recognize you based on the session cookie and will confirm that you have the capability to delete the thread and will in fact delete that thread.
All because the program authenticated you based on the session cookie (on even based on "remember me" cookie)
With RESTful API there is no cookie, no state is maintaned between requests, so there is no need to protect against session hijacking.
The way you usually authenticate with RESTFul api is be sending some additional headers. If someone tricks you into clicking on a url that points to restful API the browser is not going to send that extra headers, so there is no risk.
In short - if REST API is designed the way it supposed to be - stateless, then there is no risk of cross site forgery and no need to CSRF protection.
Adding another answer as Dmitri’s answer mixes serverside state and cookies.
An application is not stateless if your server stores user information in the memory over multiple requests. This decreases horizontal scalability as you need to find the "correct" server for every request.
Cookies are just a special kind of HTTP header. They are often used to identify a users session but not every cookie means server side state. The server could also use the information from the cookie without starting a session. On the other hand using other HTTP headers does not necessarily mean that your application is automatically stateless. If you store user data in your server’s memory it’s not.
The difference between cookies and other headers is the way they are handled by the browser. Most important for us is that the browser will resend them on every subsequent request. This is problematic if someone tricks a user to make a request he doesn’t want to make.
Is this a problem for an API which consumes JSON? Yes, in two cases:
The attacker makes the user submit a form with enctype=text/plain: Url encoded content is not a problem because the result can’t be valid JSON. text/plain is a problem if your server interprets the content not as plain text but as JSON. If your resource is annotated with #Consumes(MediaType.JSON) you should not have a problem because it won’t accept text/plain and should return a status 415. (Note that JSON may become a valid enctype one day and this won’t be valid any more).
The attacker makes the user submit an AJAX request: The Same Origin Policy prevents AJAX requests to other domains so you are safe as long as you don’t disable this protection by using CORS-headers like e.g. Access-Control-Allow-Origin: *.