Why is Selenium causing a CSRF 403? - django

I'm trying to create a simple login test using Django and Selenium, but getting a 403 due to a CSRF failure. I'm expecting the middleware to add the cookie on the GET request and then parse it back out on the POST.
Here's what I've checked so far:
1. Is the cookie being set on the GET request to /accounts/login/?
Yes, the cookie is being set in the process_response method
2. Is the cookie available on the Selenium driver?
Yes
ipdb> self.selenium.get_cookies()
[{u'domain': u'localhost', u'name': u'csrftoken', u'value': u'DzNbEn9kZw0WZQ4OsRLouriFN5MOIQos', u'expiry': 1470691410, u'path': u'/', u'httpOnly': False, u'secure': True}]
3. Is the cookie found during the POST request?
No, this try/except from django.middleware.CsrfViewMiddleware.process_view fails:
source
try:
csrf_token = _sanitize_token(
request.COOKIES[settings.CSRF_COOKIE_NAME])
# Use same token next time
request.META['CSRF_COOKIE'] = csrf_token
except KeyError:
csrf_token = None
# Generate token and store it in the request, so it's
# available to the view.
request.META["CSRF_COOKIE"] = _get_new_csrf_key()
Code
class TestLogin(StaticLiveServerTestCase):
#classmethod
def setUpClass(cls):
cls.selenium = getattr(webdriver, settings.SELENIUM_WEBDRIVER)()
cls.selenium.maximize_window()
cls.selenium.implicitly_wait(5)
super(TestLogin, cls).setUpClass()
#classmethod
def tearDownClass(cls):
cls.selenium.quit()
super(TestLogin, cls).tearDownClass()
def test_login(self):
self.selenium.get('{}{}'.format(self.live_server_url, '/accounts/login/?next=/'))
assert "Django" in self.selenium.title
un_el = self.selenium.find_element_by_id('id_username').send_keys('the_un')
pw_el = self.selenium.find_element_by_id('id_password')
pw_el.send_keys('the_pw')
pw_el.send_keys(Keys.RETURN)
try:
WebDriverWait(self.selenium, 5).until(EC.title_contains("New Title"))
except TimeoutException as e:
msg = "Could not find 'New Title' in title. Current title: {}".format(self.selenium.title)
raise TimeoutException(msg)
finally:
self.selenium.quit()
Question
What can I try next to debug this?

Oldish question, but after getting stuck with this for a few hours the answer was simple.
From the docs:
If a browser connects initially via HTTP, which is the default for
most browsers, it is possible for existing cookies to be leaked. For
this reason, you should set your SESSION_COOKIE_SECURE and
CSRF_COOKIE_SECURE settings to True. This instructs the browser to
only send these cookies over HTTPS connections. Note that this will
mean that sessions will not work over HTTP, and the CSRF protection
will prevent any POST data being accepted over HTTP (which will be
fine if you are redirecting all HTTP traffic to HTTPS).
Like me, you are probably using django_extensions + Werkzeug for the majority of your work, and are by default running all of your local work over SSL.
If you're using unittest or Djangos version of it, I'd recommend that you modify these settings at test runtime, like so:
...
from django.conf import settings
class ProfilePagetest(LiveServerTestCase):
def setUp(self):
settings.CSRF_COOKIE_SECURE = False
settings.SESSION_COOKIE_SECURE = False
self.url = reverse('clientpage:profile')
self.username = 'name#names.com'
self.password = 'strange decisions...'
get_user_model().objects.create_user(self.username, self.username, self.password)
self.browser = webdriver.Firefox()
This should stop the CSRF validation issues.

Related

Unable to create an integration test for Django's reset password flow

I'm trying to implement an integration test for the password reset flow, but I'm stuck at the "password_reset_confirm" view. I already tested the flow manually, and it works fine. Unfortunately, the Django unit test client seems unable to follow correctly the redirects required in this view.
urls config
from django.contrib.auth import views as auth_views
url(r"^accounts/password_change/$",
auth_views.PasswordChangeView.as_view(),
name="password_change"),
url(r"^accounts/password_change/done/$",
auth_views.PasswordChangeDoneView.as_view(),
name="password_change_done"),
url(r"^accounts/password_reset/$",
auth_views.PasswordResetView.as_view(email_template_name="app/email/accounts/password_reset_email.html",
success_url=reverse_lazy("app:password_reset_done"),
subject_template_name="app/email/accounts/password_reset_subject.html"),
name="password_reset"),
url(r"^accounts/password_reset/done/$",
auth_views.PasswordResetDoneView.as_view(),
name="password_reset_done"),
url(r"^accounts/reset/(?P<uidb64>[0-9A-Za-z_\-]+)/(?P<token>[0-9A-Za-z]{1,13}-[0-9A-Za-z]{1,20})/$",
auth_views.PasswordResetConfirmView.as_view(
success_url=reverse_lazy("app:password_reset_complete"),
form_class=CustomSetPasswordForm),
name="password_reset_confirm"),
url(r"^accounts/reset/complete/$",
auth_views.PasswordResetCompleteView.as_view(),
name="password_reset_complete"),
Test code
import re
from django.urls import reverse, NoReverseMatch
from django.test import TestCase, Client
from django.core import mail
from django.test.utils import override_settings
from django.contrib.auth import authenticate
VALID_USER_NAME = "username"
USER_OLD_PSW = "oldpassword"
USER_NEW_PSW = "newpassword"
PASSWORD_RESET_URL = reverse("app:password_reset")
def PASSWORD_RESET_CONFIRM_URL(uidb64, token):
try:
return reverse("app:password_reset_confirm", args=(uidb64, token))
except NoReverseMatch:
return f"/accounts/reset/invaliduidb64/invalid-token/"
def utils_extract_reset_tokens(full_url):
return re.findall(r"/([\w\-]+)",
re.search(r"^http\://.+$", full_url, flags=re.MULTILINE)[0])[3:5]
#override_settings(EMAIL_BACKEND="anymail.backends.test.EmailBackend")
class PasswordResetTestCase(TestCase):
#classmethod
def setUpClass(cls):
super().setUpClass()
cls.myclient = Client()
def test_password_reset_ok(self):
# ask for password reset
response = self.myclient.post(PASSWORD_RESET_URL,
{"email": VALID_USER_NAME},
follow=True)
# extract reset token from email
self.assertEqual(len(mail.outbox), 1)
msg = mail.outbox[0]
uidb64, token = utils_extract_reset_tokens(msg.body)
# change the password
response = self.myclient.post(PASSWORD_RESET_CONFIRM_URL(uidb64, token),
{"new_password1": USER_NEW_PSW,
"new_password2": USER_NEW_PSW},
follow=True)
self.assertIsNone(authenticate(username=VALID_USER_NAME,password=USER_OLD_PSW))
Now, the assert fails: the user is authenticated with the old password. From the log I'm able to detect that the change password is not executed.
A few extra useful information:
post returns a successful HTTP 200;
the response.redirect_chain is [('/accounts/reset/token_removed/set-password/', 302)] and I think this is wrong, as it should have another loop (in the manual case I see another call to the dispatch method);
I'm executing the test with the Django unit test tools.
Any idea on how to properly test this scenario? I need this to make sure emails and logging are properly executed (and never removed).
Many thanks!
EDIT: solution
As well explained by the accepted solution, here the working code for the test case:
def test_password_reset_ok(self):
# ask for password reset
response = self.myclient.post(PASSWORD_RESET_URL,
{"email": VALID_USER_NAME},
follow=True)
# extract reset token from email
self.assertEqual(len(mail.outbox), 1)
msg = mail.outbox[0]
uidb64, token = utils_extract_reset_tokens(msg.body)
# change the password
self.myclient.get(PASSWORD_RESET_CONFIRM_URL(uidb64, token), follow=True)
response = self.myclient.post(PASSWORD_RESET_CONFIRM_URL(uidb64, "set-password"),
{"new_password1": USER_NEW_PSW,
"new_password2": USER_NEW_PSW},
follow=True)
self.assertIsNone(authenticate(username=VALID_USER_NAME,password=USER_OLD_PSW))
This is very interesting; so it looks like Django has implemented a security feature in the password reset page to prevent the token from being leaked in the HTTP Referrer header. Read more about Referrer Header Leaks here.
TL;DR
Django is basically taking the sensitive token from the URL and placing it in Session and performing a internal redirect (same domain) to prevent you from clicking away to a different site and leaking the token via the Referer header.
Here's how:
When you hit /accounts/reset/uidb64/token/ (you should be doing a GET here, however you are doing a POST in your test case) the first time, Django pulls the token from the URL and sets it in session and redirects you to /accounts/reset/uidb64/set-password/.
This now loads the /accounts/reset/uidb64/set-password/ page, where you can set the passwords and perform a POST
When you POST from this page, the same View handles your POST request since the token URL param can handle both the token and the string set-password.
This time though, the view sees that you have accessed it with set-password and not a token, so it expects to pull your actual token from session, and then change the password.
Here's the flow as a chart:
GET /reset/uidb64/token/ --> Set token in session --> 302 Redirect to /reset/uidb64/set-token/ --> POST Password --> Get token from Session --> Token Valid? --> Reset password
Here's the code!
INTERNAL_RESET_URL_TOKEN = 'set-password'
INTERNAL_RESET_SESSION_TOKEN = '_password_reset_token'
#method_decorator(sensitive_post_parameters())
#method_decorator(never_cache)
def dispatch(self, *args, **kwargs):
assert 'uidb64' in kwargs and 'token' in kwargs
self.validlink = False
self.user = self.get_user(kwargs['uidb64'])
if self.user is not None:
token = kwargs['token']
if token == INTERNAL_RESET_URL_TOKEN:
session_token = self.request.session.get(INTERNAL_RESET_SESSION_TOKEN)
if self.token_generator.check_token(self.user, session_token):
# If the token is valid, display the password reset form.
self.validlink = True
return super().dispatch(*args, **kwargs)
else:
if self.token_generator.check_token(self.user, token):
# Store the token in the session and redirect to the
# password reset form at a URL without the token. That
# avoids the possibility of leaking the token in the
# HTTP Referer header.
self.request.session[INTERNAL_RESET_SESSION_TOKEN] = token
redirect_url = self.request.path.replace(token, INTERNAL_RESET_URL_TOKEN)
return HttpResponseRedirect(redirect_url)
# Display the "Password reset unsuccessful" page.
return self.render_to_response(self.get_context_data())
Notice the comment in the code where this magic happens:
Store the token in the session and redirect to the
password reset form at a URL without the token. That
avoids the possibility of leaking the token in the
HTTP Referer header.
I think this makes it clear how you can fix your unit test; do a GET on the PASSWORD_RESET_URL which will give you the redirect URL, you can then POST to this redirect_url and perform password resets!

Django Rest Framework test fails: Expected view to be called with a URL keyword argument

I am running tests on UserDetail view written using Django Rest framework.
urls.py
url(r'^api/users/', include('calorie_counter.users.urls', namespace='users')),
users/urls.py
url(r'^(?P<pk>[0-9]+)$', views.UserDetail.as_view(), name='user-detail'),
test_api.py
BaseAPITestCase(APITestCase):
def setUp(self):
self.superuser = User.objects.create_superuser('admin', 'admin#test.com', 'johnpassword')
self.client.login(username='john', password='johnpassword')
self.user1 = User.objects.create(username="user1", password="pass", email="user1#test.com")
class ReadUserTest(BaseAPITestCase):
# check read permissions
def test_user_can_read_self_detail(self):
url = '/api/users/'+str(self.user1.id)
factory = APIRequestFactory()
request = factory.get(url)
force_authenticate(request, self.user1)
response = (UserDetail.as_view())(request)
self.assertEqual(response.status_code, status.HTTP_200_OK)
However, running this test, gives me following error. The 'pk' agrument is not getting passed to the UserDetail view.
AssertionError: Expected view UserDetail to be called with a URL keyword argument named "pk". Fix your URL conf, or set the .lookup_field attribute on the view correctly.
How do I test views with URL arguments?
UPDATE:
Now using APIClient instead of factory..
def test_user_can_read_self_detail(self):
client = APIClient()
client.login(username='user1', password='pass')
# response = self.client.get('/api/users/', {'pk': self.user.id})
response = client.get('/api/users/' + str(self.user1.id))
self.assertEqual(response.status_code, status.HTTP_200_OK)
Now I am getting following error:
AttributeError: 'AnonymousUser' object has no attribute 'is_manager'
where is manager is an attribute of my custom user model. I guess there is some problem with client authentication. I have session authentication enabled. Still getting this error.
UPDATE: My login wasn't working for APICLient because I was creating user using User.objects.create instead of User.objects.create_user. Changing that fixed the problem. :)
I don't think you need all of this set up you're doing, it's unusual to need to instantiate view classes yourself – perhaps you'll have more success leveraging the test client with something like:
def test_user_can_read_self_detail(self):
url = reverse('api:user-detail', kwargs={'pk': self.user.id})
response = self.client.get(url)
self.assertEqual(response.status_code, status.HTTP_200_OK)
If you're having issues with authentication, which I suspect may have been what lead you here, you may want to try:
self.client.login(username='example', password='changeme')
or
import base64
self.client.defaults['HTTP_AUTHORIZATION'] = 'Basic ' + base64.b64encode('example:changeme')
I've used both in the past to test authenticated API endpoints.

Django SSLMiddleware issue on Webfaction

I've been using the following SSLMiddleware on Linode for a while, and my SSL worked perfectly on that, now I've changed my server to Webfaction, and all of sudden, my HTTPS pages are not working in a way as it's redirected to https page correctly, but all my css files, images within the css files(no absolute url), javascript have all become non secure sources(referring to http:// instead of https://), I'm really puzzled right now as I don't know if it's got to do with SSLMiddleware or something else, I haven't changed anything in settings.py either apart from database parameter value.. Please help. Thanks in advance.
__license__ = "Python"
__copyright__ = "Copyright (C) 2007, Stephen Zabel"
__author__ = "Stephen Zabel - sjzabel#gmail.com"
__contributors__ = "Jay Parlar - parlar#gmail.com"
from django.conf import settings
from django.http import HttpResponseRedirect, HttpResponsePermanentRedirect, get_host
SSL = 'SSL'
class SSLRedirect:
def process_view(self, request, view_func, view_args, view_kwargs):
if SSL in view_kwargs:
secure = view_kwargs[SSL]
del view_kwargs[SSL]
else:
secure = False
if settings.ENABLE_SSL:
if not secure == self._is_secure(request):
return self._redirect(request, secure)
else:
return
def _is_secure(self, request):
if request.is_secure():
return True
#Handle the Webfaction case until this gets resolved in the request.is_secure()
if 'HTTP_X_FORWARDED_SSL' in request.META:
return request.META['HTTP_X_FORWARDED_SSL'] == 'on'
return False
def _redirect(self, request, secure):
protocol = secure and "https" or "http"
newurl = "%s://%s%s" % (protocol,get_host(request),request.get_full_path())
if settings.DEBUG and request.method == 'POST':
raise RuntimeError, \
"""Django can't perform a SSL redirect while maintaining POST data.
Please structure your views so that redirects only occur during GETs."""
return HttpResponsePermanentRedirect(newurl)
I recently implemented SSL on WebFaction without any custom Middleware tinkering and it was a very straightforward process.
Have a look here: http://community.webfaction.com/questions/512/how-do-i-set-up-a-https-ssl-django-site
If that doesn't help, open up a ticket with them. They're usually very good about resolving issues very quickly.

Django URL Detail

I have to assign to work on one Django project. I need to know about the URL say, http://....
Since with ‘urls.py’ we indeed have ‘raw’ information. How I come to know about the complete URL name; mean with
http+domain+parameters
Amit.
Look at this snippet :
http://djangosnippets.org/snippets/1197/
I modified it like this :
from django.contrib.sites.models import RequestSite
from django.contrib.sites.models import Site
def site_info(request):
site_info = {'protocol': request.is_secure() and 'https' or 'http'}
if Site._meta.installed:
site_info['domain'] = Site.objects.get_current().domain
site_info['name'] = Site.objects.get_current().name
else:
site_info['domain'] = RequestSite(request).domain
site_info['name'] = RequestSite(request).name
site_info['root'] = site_info['protocol'] + '://' + site_info['domain']
return {'site_info':site_info}
The if/else is because of different versions of Django Site API
This snippet is actually a context processor, so you have to paste it in a file called context_processors.py in your application, then add to your settings :
TEMPLATE_CONTEXT_PROCESSORS = DEFAULT_SETTINGS.TEMPLATE_CONTEXT_PROCESSORS + (
'name-of-your-app.context_processors.site_info',
)
The + is here to take care that we d'ont override the possible default context processor set up by django, now or in the future, we just add this one to the tuple.
Finally, make sure that you use RequestContext in your views when returning the response, and not just Context. This explained here in the docs.
It's just a matter of using :
def some_view(request):
# ...
return render_to_response('my_template.html',
my_data_dictionary,
context_instance=RequestContext(request))
HTTPS status would be handled differently by different web servers.
For my Nginx reverse proxy to Apache+WSGI setup, I explicitly set a header that apache (django) can check to see if the connection is secure.
This info would not be available in the URL but in your view request object.
django uses request.is_secure() to determine if the connection is secure. How it does so depends on the backend.
http://docs.djangoproject.com/en/dev/ref/request-response/#django.http.HttpRequest.is_secure
For example, for mod_python, it's the following code:
def is_secure(self):
try:
return self._req.is_https()
except AttributeError:
# mod_python < 3.2.10 doesn't have req.is_https().
return self._req.subprocess_env.get('HTTPS', '').lower() in ('on', '1')
If you are using a proxy, you will probably find it useful that HTTP Headers are available in HttpRequest.META
http://docs.djangoproject.com/en/dev/ref/request-response/#django.http.HttpRequest.META
Update: if you want to log every secure request, use the above example with a middleware
class LogHttpsMiddleware(object):
def process_request(self, request):
if request.is_secure():
protocol = 'https'
else:
protocol = 'http'
print "%s://www.mydomain.com%s" % (protocol, request.path)
Add LogHttpsMiddleware to your settings.py MIDDLEWARE_CLASSES

Django: test failing on a view with #login_required

I'm trying to build a test for a view that's decorated with
#login_required, since I failed to make it work, I did a simple test
and still can't make it pass.
Here is the code for the simple test and the view:
def test_login(self):
user = self._create_new_user()
self.assertTrue(user.is_active)
login = self.client.login(username=user.username,
password=self.data['password1'])
self.failUnless(login, 'Could not log in')
response = self.client.get('/accounts/testlogin/')
self.assertEqual(response.status_code, 200)
#login_required
def testlogin(request):
print 'testlogin !! '
return HttpResponse('OK')
_create_new_user() is saving the user and there is a test inside that
method to see that is working.
The test fails in the response.status_code, returning 302 and the
response instance is of a HttpResponseRedirect, is redirecting it as
if not logged in.
Any clue? I'm missing something?
Regards
Esteban
This testcase works for me:
from django.contrib.auth.models import User
from django.core.urlresolvers import reverse
from django.test.client import Client
import unittest
class LoginTestCase(unittest.TestCase):
def setUp(self):
self.client = Client()
self.user = User.objects.create_user('john', 'lennon#thebeatles.com', 'johnpassword')
def testLogin(self):
self.client.login(username='john', password='johnpassword')
response = self.client.get(reverse('testlogin-view'))
self.assertEqual(response.status_code, 200)
I suggest you (if you don't use them already) to use the reverse() function and name your URLs. This way you are sure that you get always the right URL.
Here is the answer:
Python 2.6.5 made a change to the way
cookies are stored which is subtly
incompatible with the test client.
This problem has been fixed in the
1.1.X and trunk branches, but the fix hasn't yet made it into a formal
release.
If you are using 1.1.X and Python
2.6.5, you're going to have problems with any test activity involving
cookies. You either need to downgrade
Python, or use the 1.1.X branch rather
than the 1.1.1 release.
A 1.1.2 release (that will include the
fix for the problem you describe) will
be made at the same time that we
release 1.2 - hopefully, very very
soon.
Yours, Russ Magee %-)
http://groups.google.com/group/django-users/browse_frm/thread/617457f5d62366ae/05f0c01fff0b9e6d?hl=en&lnk=gst&q=2.6.5#05f0c01fff0b9e6d
OK I was to facing same problem #resto solved my problem.
creating user this way below, lets the test client get the user logged in and get the response other than redirect (302)
self.user = User.objects.create_user('john', 'lennon#thebeatles.com', 'johnpassword')