I have started to write an API using the Django REST Framework. I am struggling to think about what tests I should be writing.
My ideas so far are...
Authentication: making sure users are logged in
Authorisation: checking users have the correct permissions
Response body: making sure all the desired fields are present
Allowed HTTP methods: making sure that users can't perform unintended actions.
Since the Django REST framework uses Django's underlying permissions system, is it really necessary to test permissions at both the model level and the API level? In this regard, it seems like some of my tests are testing for the same thing.
One of the more important things to test in an API is that it handles requests correctly. Good requests and Bad ones.
Related
Intro
I've built a web application with multiple services:
frontend (react)
backend (API and admin panel) (Django Rest Framework + simple jwt auth)
Redis, DB, Nginx and etc
Kubernetes cluster
The app isn't small like 60k+ lines of code. It's a startup. I mentioned it to let you know that probably I wouldn't have that much attention from hackers or traffic at all. Hence I have a space to improve gradually.
The auth is done with DRF simple jwt library. Expiring Access + Refresh token.
Problem statement
I did a security audit and found imperfections from the security architecture perspective. I don't know how those issues are crucial, how should I fix them, or what issues can be fixed later. So I'm seeking solutions and advice. I would prefer an optimal proportion between the speed and quality rather than quality only (If I miss about that let me know) hence if something is "nice to have" rather than "important" I would put it to the backlog of next releases.
The actual list of issues
Let's refer by its number if you want to.
#1 Authentication methods
My current setup:
REST_FRAMEWORK = {
'DEFAULT_PERMISSION_CLASSES': (
'rest_framework.permissions.IsAuthenticated',
),
'DEFAULT_AUTHENTICATION_CLASSES': (
'rest_framework.authentication.SessionAuthentication',
'rest_framework.authentication.BasicAuthentication',
'rest_framework_simplejwt.authentication.JWTAuthentication',
),
.....
}
As you see I have 3 methods. JWT is ok but BasicAuthentication and SessionAuthentication seem to be not ok. What I want to achieve is to have the real JWT auth and as the only way of auth for the API views (and I did really believe that I had it until I find out the opposite).
As I understood (might be wrong) I don't need SessionAuthentication and BasicAuthentication in the production setting but I do for the dev because it allows me to log in to the DRF API UI with the login form which is cool for testing. Am I right about that?
#2 Sessions
When I got to the Chrome Dev Tools and inspected the cookies I was discouraged. At this moment I deprecated the SessionAuthentication and BasicAuthentication as for the test.
As I understood I have the session id cookie because of the SessionMiddleware. And it's ok to have it because it is used only for the admin panel authentication and ignored by the DRF API views so the only way to auth is JWT but is it? Maybe it can have more impact and exploits. Hence, should I completely drop the SessionMiddleware, especially in order to achieve the goal to have JWT auth as the only auth type?
*I understand that it would abandon the ability to use the Admin Panel feature and I will address this point later.
#3 I store Access and Refresh tokens in the local storage
Yeah, seems to be I wrong for that. I admit it. It was the lack of experience in the beginning. The frontend app and tests (I use Cypress) heavily depend on the tokens to be in the local storage but it's feasible to migrate. On another hand, I am just afraid of the new bugs that can appear afterward. Also, I suspect expect migration can be a little painful. The question is how crucial is that and hence should I migrate the token store to the cookies now or I can do it later?
#4.1 Separation of Admin Panel from the API
Django Admin panel is awesome we all know but it is tightly coupled to the app. But. Issue #2 has brought me to the idea to separate API and Admin. So since I use Kubernetes the idea is to run the two services. One is API, I imagine it as the same codebase but the settings are different (disabled SessionMiddlware and admin panel). And another service where the admin panel feature is fully enabled. Does it make sense?
*I feel that it smells a little bit of over-engineering to me. So please stop if I am wrong about this.
** Seems to heavily depend on the #2 because if there are no issues and exploits with SessionMiddleware hence there are no strong reasons to do so.
#4.2 Admin Panel production setup
I just wonder what are the best security practices for setting up Admin Panel access in prod. I have absolutely bare setup. No captcha. No VPN. Fully vanilla. And the question is what is the most feasible but efficient access setup? I feel that it should be secured somehow. At least /admin to be not a public endpoint (VPN?) but I have no clue how to achieve this. I am on Google Cloud Platform so maybe I can use one of its solutions?
Outro
What else security checks do you do before going into production as an engineer? I mean of course, the best way is to hire the security team but I can't do that.
What I've done:
made sure that there are no data leaks possibilities with (tests and propper backend and frontend app architecture)
did an extensive acceptance and e2e testing on the auth
ran through the serializer and made sure that there are no exploits (at least visible) (of course a lot of unit and e2e tests)
Thanks,
Artem
#1 Yes, remove both other auth methods and only leave the jwt auth, this is only for the REST framework, so should not be a problem for the /admin.
#2 First points will already solve this problem. But remember you can also change cookies paths from django. So you can set that the /admin in your django app uses other cookies. This will allow you to complete separate the /admin from other urls of your app. You will find this interesting:
SESSION_COOKIE_PATH = '/admin'
CSRF_COOKIE_PATH = '/admin'
LANGUAGE_COOKIE_PATH = '/admin'
SESSION_COOKIE_NAME = 'backend-sessionid'
CSRF_COOKIE_NAME = 'backend-csrftoken'
#3 As far as I know... that's the only way we can save tokens on the frontend app. So the security part there is done by having the "expiration time" of the tokens, in case someone get to obtain tokens from a client, he will only have access for a short period of time. This depends a lot on the logic of the business and how you want to manage this expirations
#4.1 I will not recommend to run your app in two instances, you should be able to setup the app correctly and avoid this kind of solutions.
#4.2 As you mentioned, /admin should have a restricted access. I have done that by whitelisting IPs (can be done on nginx), but you will have to know the IP's beforehand. You can also do it by the http auth of nginx, so you can have a user and password (that you can share without having to know IPs) asked before even connecting to the django app.
Recommendations:
Run an scan on your app, like: https://observatory.mozilla.org/ This will let you know lot of other security things you will need in your app. With this you should be ready to feel safe. And also, there are sometime you cannot avoid to fail some of the tests, just try to do the better to mitigate the problems.
I am currently at the planning stage of an app that will consist of standard
Django part for supervisors that can perform all CRUD operations on employee users mostly add, delete and view statistics - viewed in browser (no frontend framework just using Djangos server side rendering), two step email authentication on each login, session based auth
DRF part for employees - API connected to mobile app, authentication based on device ID. (no username, or password)
DRF part for clients to contact the supervisors if employees do something wrong - Token or JWT based authentication using passcode delivered by mail.
I am not used to splitting Django projects into multiple sub-projects (or using same database for different projects) but it feels like every part of the project should be a standalone app due to different authentication type and the fact of simultaniousily using DRF with standard Django
Can anyone who had similar problem or has some experience, advise what should I do considering different authentications and overall different user types in this project? What would be pros and cons of going into solo or multiple projects? Thanks in advance!
You're asking for opinions, so don't be surprised if the question gets closed, but I'll answer with facts:
A split over different projects using the same database has the following issue: shared migrations. They all use built-in users, so require some standard apps to be enabled that have migrations and they won't run on the 2nd and 3rd project.
You're going to need a custom user model to support the device id authentication method: You need information that is not on the standard user model to be available at authentication time - the number 1 reason to create a custom user model. Ties into migrations and also a synchronization hell code-wise.
Django's Authentication Backends system allows for different authentication methods to exist at the same time, so there is no need to split anything. If you're worried about security, you can always use different hostnames and the Sites framework to add an extra layer of protection, but they would still use the same code.
DRF started as an addition to Django's view-based approach, not a replacement to make part of a project's code available as an API. While current usage is more "DRF or templates" this is a result of people increasingly becoming binary ("this" or "that") and wanting to be in the cool club, but has nothing to do with technical reasons. They can and always will be able to co-exist as they solve different problems. In fact, DRF's generic views make use of Django's CBV's and the built-in browsable API makes use of templates. Also, the admin is template/view based and it's convenient to develop the app or manage data with the built-in admin.
I am relatively new to DRF, but found viewsets an amazing abstraction technique for writing RESTful API. I am having a hard time correlating Viewsets with REST methods though. Let's say I have a viewset for Userprofiles and somebody new creates a profile on client.
Should this send a PUT or a POST ?
Which url should this request go to, http://user or http://user/new_id ?
If I want this profile only accessible to the user or admin(all CRUD operations), then where should I handle the code for making it inaccessible to others ?
Should I create a new permission ? If yes, should I handle rest methods in has_permission/has_object_permission ?
I have gone through the tutorial, and know how permissions/mixins works, but I am not able to connect these dots here.
1/ In general, POST is for creating new, PUT is for updating. See the docs on the SimpleRouter to show how the various types of Http methods align with various actions in your Django backend.
2/ You'll find that different situations call for different routing methods. If yours is standard, you may want to use a SimpleRouter like the example above. In that case, creating a new user would be a POST request to /user/ and updating a user would be a PUT request to /user/{{user_id}}/.
3/ To limit access to various API methods, you want to use Permissions. It's possible that you could use one of DRF's Custom Permissions. I've found that in general, if you want only the user to access his/her own profile, it's easier to either use conditional logic within a view (i.e., return a DRF PermissionDenied exception in the view logic if the request.user.pk is not the pk of that REST url. More often than not, I've used the UserPassesTestMixin from Django Braces, that works pretty well to filter user permissions.
4/ I guess the Django Braces mixin above answers this question. You should include a test_func method if you're using Django Braces that returns True if you want to grant the user access, and False otherwise.
Hope this helps! I agree that these are difficult parts of DRF and perhaps some of these could more effectively get incorporated into the source. Good luck!
I'm trying to select a REST framework for Django that will allow me to easily use ETags for optimistic locking. I'm planning on examining Django-pistons and the Django Rest Framework libraries, but I'm open to any non-GPL solution (corporate licensing requirements prevent my using those).
My application is vending data from a SQLAlchemy model (not a Django model) in JSON/YAML form, and modulo the ETag issue is working just fine with the Django Rest Framework. However, I can't see an easy way to apply the ETag headers on my views.
For my views, I want to do this:
Given a response, easily add an ETag to the response headers I'm sending out on success. This must be calculated by me, since it will be model dependent; it's not enough to hash the response value or anything like that.
On POST/PUT, ensure that the ETag I'm receiving matches the one I sent out, or reject the request.
It's step 1 that is giving me a bit of trouble; I'm not sure which REST framework will make this easiest, nor am I sure what the best way to accomplish it is.
Django supports ETags through decorators (#etag, #last_modified, #condition) or middleware - you can check the docs. If you want to use those decorators with Django Rest Framework, you can use django-rest-framework-condition.
Choice of framework does not matter. In any case, you have to maintain an etag in cache(memcache or redis) representing the state of the resource. You can use Generational Caching Algorithm (https://signalvnoise.com/posts/3113-how-key-based-cache-expiration-works). Then you can easily write a mixin that extracts the etag of every resource and sends it. In my personal experience, Django Rest Framework would be ideal for it due to its heavy flexibility and well-written code.
A web app written in Python is planned, Django is a leading contender as framework.
One requirement is CAC access, wihout the need to hand enter username and password. From what I can tell, CAC access is not part of the "batteries" included with Django.
As a monolithic framework (not necessarily a bad attribute) Django has a rep for being high-maintenance once you modify the core. Can I easily add CAC access to a Django site? Can it be easily maintained thereafter?
Or maybe we should consider a different Python framework?
FYI.. interesting presentation on CAC access link
You don't need to modify the core to enable this. Django supports third-party authentication backends and they're fairly easy to write - you just need to support two methods, get_user and authenticate. So your implementation just needs to perform these operations using your CAC interface, and all will work as usual.
See the documentation for details.
Edited after other answers I don't know why people are saying this is difficult in Django. Yes, many parts of Django are difficult to customise. But this is one particular part that is made very easy. I've written several authentication backends in Django and they are not only really simple, but they "just work" with the rest of the framework, including the admin. There isn't any need to modify anything else to get this to work.
I just did this today by subclassing django.contrib.auth.middleware.RemoteUserMiddleware and changed the header property to the one I had set in my apache conf. I just added the django.contrib.auth.backends.RemoteUserBackend and my middleware to the settings and it works perfectly.
Extending contrib.auth is a pain in the neck. It's the single worst thing in django. If you need highly customized auth backend, i would suggest using a different framework.