I'm trying to write a functional test that uses Selenium to test a Django view. When the user comes to a page ("page2"), the view that renders that page expects to find a session variable "uid" (user ID). I've read a half dozen articles on how this is supposed to be done but none of them have worked for me. The code below shows how the Django documentation says it should be done but it doesn't work for me either. When I run the test, the view never completes executing and I get a "server error occurred" message. Could someone please tell me what I'm doing wrong? Thank you.
views.py:
from django.shortcuts import render_to_response
def page2(request):
uid = request.session['uid']
return render_to_response('session_tests/page2.html', {'uid': uid})
test.py:
from django.test import LiveServerTestCase
from selenium import webdriver
from django.test.client import Client
class SessionTest(LiveServerTestCase):
def setUp(self):
self.browser = webdriver.Firefox()
self.browser.implicitly_wait(3)
self.client = Client()
self.session = self.client.session
self.session['uid'] = 1
def tearDown(self):
self.browser.implicitly_wait(3)
self.browser.quit()
def test_session(self):
self.browser.get(self.live_server_url + '/session_tests/page2/')
body = self.browser.find_element_by_tag_name('body')
self.assertIn('Page 2', body.text)
Here's how to solve this problem. James Aylett hinted at the solution when he mentioned the session ID above. jscn showed how to set up a session but he didn't mention the importance of the session key to a solution and he also didn't discuss how to link the session state to Selenium's browser object.
First, you have to understand that when you create a session key/value pair (e.g. 'uid'=1), Django's middleware will create a session key/data/expiration date record in your backend of choice (database, file, etc.). The response object will then send that session key in a cookie back to the client's browser. When the browser sends a subsequent request, it will send a cookie back that contains that key which is then used by the middleware to lookup the user's session items.
Thus, the solution required 1.) finding a way to obtain the session key that is generated when you create a session item and then; 2.) finding a way to pass that key back in a cookie via Selenium's Firefox webdriver browser object. Here's the code that does that:
selenium_test.py:
-----------------
from django.conf import settings
from django.test import LiveServerTestCase
from selenium import webdriver
from django.test.client import Client
import pdb
def create_session_store():
""" Creates a session storage object. """
from django.utils.importlib import import_module
engine = import_module(settings.SESSION_ENGINE)
# Implement a database session store object that will contain the session key.
store = engine.SessionStore()
store.save()
return store
class SeleniumTestCase(LiveServerTestCase):
def setUp(self):
self.browser = webdriver.Firefox()
self.browser.implicitly_wait(3)
self.client = Client()
def tearDown(self):
self.browser.implicitly_wait(3)
self.browser.quit()
def test_welcome_page(self):
#pdb.set_trace()
# Create a session storage object.
session_store = create_session_store()
# In pdb, you can do 'session_store.session_key' to view the session key just created.
# Create a session object from the session store object.
session_items = session_store
# Add a session key/value pair.
session_items['uid'] = 1
session_items.save()
# Go to the correct domain.
self.browser.get(self.live_server_url)
# Add the session key to the cookie that will be sent back to the server.
self.browser.add_cookie({'name': settings.SESSION_COOKIE_NAME, 'value': session_store.session_key})
# In pdb, do 'self.browser.get_cookies() to verify that it's there.'
# The client sends a request to the view that's expecting the session item.
self.browser.get(self.live_server_url + '/signup/')
body = self.browser.find_element_by_tag_name('body')
self.assertIn('Welcome', body.text)
There are a couple of tickets in Django's bug tracker around this kind of problem, the main one seems to be: https://code.djangoproject.com/ticket/10899 which hasn't had any movement on it for a few months. Basically, you need to do some extra set up to get the session to work properly. Here's what worked for me (may not work as is with your particular set up, as I wasn't using Selenium):
def setUp(self):
from django.conf import settings
engine = import_module(settings.SESSION_ENGINE)
store = engine.SessionStore()
store.save()
self.client.cookies[settings.SESSION_COOKIE_NAME] = store.session_key
Now you should be able to access self.client.session and it should remember any changes you make to it.
Here is my solution for django==2.2.
from importlib import import_module
from django.conf import settings
from django.contrib.auth import get_user_model
# create the session database instance
engine = import_module(settings.SESSION_ENGINE)
session = engine.SessionStore()
# create the user and instantly login
User = get_user_model()
temp_user = User.objects.create(username='admin')
temp_user.set_password('password')
self.client.login(username='admin', password='password')
# get session object and insert data
session = self.client.session
session[key] = value
session.save()
# update selenium instance with sessionID
selenium.add_cookie({'name': 'sessionid', 'value': session._SessionBase__session_key,
'secure': False, 'path': '/'})
Related
I have a Django 2.2 based project that uses a custom user model, the built-in Auth app and Django-All-Auth for user management. Almost every page on the site is behind a login and we use varying levels of permissions to determine what can be accessed a user.
So far, so good, but now I'm being asked to designate everything behind a specific part of the site as "sensitive", requiring a second login prompt using the same login credentials. What this means is that the client wants to see a login appear when they try to access anything under /top-secret/ the first time in a set time, say 30 mins, regardless of whether they're already logged in or not.
I've dug around on the internet for ideas on how to do this, but so far I've been unable to find a good solution. Has anyone here had any experience with something similar, and if so, could they point me in the right direction?
Figured out how to make this work in my situation. I debated deleting this question, but on the off chance that there's someone else out there that wants to do something similar, here's how I did it.
Create a new middleware with the following:
from allauth.account.adapter import get_adapter
from datetime import datetime
from dateutil.relativedelta import relativedelta
from django.conf import settings
from django.shortcuts import redirect
from django.urls import reverse
from django.utils import timezone
class ExtraLoginPromptMiddleware:
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
if request.user.is_authenticated:
for url in settings.EXTRA_LOGIN_URLS:
if request.path.startswith(url):
last_seen = request.session.get(settings.EXTRA_LOGIN_SESSION_VARIABLE)
if not last_seen:
last_seen = request.user.last_login
else:
last_seen = datetime.fromisoformat(last_seen)
if last_seen + relativedelta(seconds=settings.EXTRA_LOGIN_EXPIRY) < timezone.now():
# NB, this uses allauth, if you want to use django.auth, just
# call django.contrib.auth's logout instead
get_adapter(request).logout(request)
return redirect('{}?next={}'.format(reverse('account_login'), url))
else:
request.session[settings.EXTRA_LOGIN_SESSION_VARIABLE] = timezone.now().isoformat()
return self.get_response(request)
To make use of this, you'll want to add this middleware after SessionMiddleware and AuthenticationMiddleware
You'll also want to define a few variables in your settings:
EXTRA_LOGIN_URLS = ['/top-secret/', 'another-secret']
EXTRA_LOGIN_EXPIRY = 120 # How long before the "extra security" session expires
EXTRA_LOGIN_SESSION_VARIABLE = 'extra-login-last-seen'
I'm trying to create a Flask app that can browse to the user's Google Drive to select a file to convert to CSV format.
I found online a code to add Google Login (see below), but I don't understand how to print all files from the logged user. I figured out I need to add https://www.googleapis.com/auth/drive.file to the scope but I can't understand how to list all files from an authorized user (current_user.paying==True) in index. I know that this could be done with the following code, but I'm not sure how to defined the credentials.
In callback an access token is already generated, so I tried to access the Google Drive API to just search for the user's Google sheets:
from httplib2 import Http
from apiclient import discovery
from oauth2client import file
with open("./credentials.json", 'w') as outfile:
json.dump(token_response.json(), outfile)
store = file.Storage("./credentials.json")
credentials = store.get()
drive = discovery.build("drive", "v3", http=credentials.authorize(Http()))
files = drive.files().list(q="mimeType='application/vnd.google-apps.spreadsheet'").execute()
But this gives me an "KeyError: '_module'" error that's not very informative.
This is the entire Flask code with Google Login
import os
import requests
import json
from httplib2 import Http
from apiclient import discovery
from oauthlib.oauth2 import WebApplicationClient
from flask import Flask, redirect, request, url_for
from flask_login import LoginManager, current_user, login_required, login_user, logout_user
from flask_login import UserMixin
authorized_users = ["myuser#gmail.com"]
class User(UserMixin):
def __init__(self, id_, paying):
self.id = id_
self.paying = paying
#staticmethod
def get(user_email):
if user_email not in authorized_users:
user = User(user_email, False)
else:
user = User(user_email, True)
return user
# Configuration
GOOGLE_CLIENT_ID = os.environ.get("GOOGLE_CLIENT_ID", None)
GOOGLE_CLIENT_SECRET = os.environ.get("GOOGLE_CLIENT_SECRET", None)
GOOGLE_DISCOVERY_URL = ("https://accounts.google.com/.well-known/openid-configuration")
# Flask app setup
app = Flask(__name__)
app.secret_key = os.environ.get("SECRET_KEY") or os.urandom(24)
# User session management setup
login_manager = LoginManager()
login_manager.init_app(app)
# OAuth 2 client setup
client = WebApplicationClient(GOOGLE_CLIENT_ID)
# Flask-Login helper to retrieve a user from our db
#login_manager.user_loader
def load_user(user_email):
return User.get(user_email)
def get_google_provider_cfg():
return requests.get(GOOGLE_DISCOVERY_URL).json()
google_provider_cfg = get_google_provider_cfg()
token_endpoint = google_provider_cfg["token_endpoint"]
print("Google provider cfg", google_provider_cfg)
ACCESS_TOKEN_URI = 'https://www.googleapis.com/oauth2/v4/token'
#app.route("/")
def index():
if current_user.is_authenticated:
if current_user.paying:
return f"<p>Hello, {current_user.id}! You're logged in!</p><br/><a class='button' href='/logout'>Logout</a>"
else:
return "Not authorized!<br/><a class='button' href='/logout'>Logout</a>"
else:
return f"<a class='button' href='/login'>Google Login</a>"
#app.route("/login")
def login():
# Find out what URL to hit for Google login
google_provider_cfg = get_google_provider_cfg()
authorization_endpoint = google_provider_cfg["authorization_endpoint"]
# Use library to construct the request for Google login and provide
# scopes that let you retrieve user's profile from Google
request_uri = client.prepare_request_uri(
authorization_endpoint,
redirect_uri=request.base_url + "/callback",
scope=["openid", "email", "profile", "https://www.googleapis.com/auth/drive.file"],
)
return redirect(request_uri)
#app.route("/login/callback")
def callback():
# Get authorization code Google sent back to you
code = request.args.get("code")
# Prepare and send a request to get tokens! Yay tokens!
token_url, headers, body = client.prepare_token_request(
token_endpoint,
authorization_response=request.url,
redirect_url=request.base_url,
code=code
)
token_response = requests.post(
token_url,
headers=headers,
data=body,
auth=(GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET),
)
# Parse the tokens!
client.parse_request_body_response(json.dumps(token_response.json()))
# Now that you have tokens (yay) let's find and hit the URL
# from Google that gives you the user's profile information,
# including their Google profile image and email
userinfo_endpoint = google_provider_cfg["userinfo_endpoint"]
uri, headers, body = client.add_token(userinfo_endpoint)
userinfo_response = requests.get(uri, headers=headers, data=body)
if userinfo_response.json().get("email_verified"):
# unique_id = userinfo_response.json()["sub"]
users_email = userinfo_response.json()["email"]
picture = userinfo_response.json()["picture"]
users_name = userinfo_response.json()["given_name"]
else:
return "User email not available or not verified by Google.", 400
# Create a user in your db with the information provided
# by Google
user = User(users_email, False)
#
# # Doesn't exist? Add it to the database.
# if not User.get(unique_id):
# User.create(unique_id, users_name, users_email, picture)
print("Logging", users_name, users_email, picture)
# Begin user session by logging the user in
login_user(user)
# Send user back to homepage
return redirect(url_for("index"))
#app.route("/logout")
#login_required
def logout():
logout_user()
print("Logging out")
return redirect(url_for("index"))
if __name__ == "__main__":
app.run(ssl_context="adhoc", debug=True)
An easy way to obtain the credentials is
Visit the Google Drive API Quickstart
Click on Enable Drive
Either note down the Client Id and Client Secret or click on DOWNLOAD CLIENT CONFIGURATION and open the generated and downloaded credentials.json file
Alternatively, you can
visit the Google Cloud Console
Choose APIs & Services -> Credentials
Click on +CREATE CREDENTIALS
UPDATE
KeyError: '_module' is an error originating from retrieving a json file with an incorrect structure.
Possible reasons:
Using the API Client Library that is expecting a client_secrets.json file - this is different from the credentials.json file you obtain from the Google Drive API Quickstart
Storing not valid credentials in the json file
Storing the json file in another folder than the py file
Using invalid scopes
I understand that you use the WebApplicationClient and need to create respective client credentials for a Web Server.
The json file should have the content of type
{"web":{"client_id":"XXXX.apps.googleusercontent.com","project_id":"XXXX","auth_uri":"https://accounts.google.com/o/oauth2/auth","token_uri":"https://oauth2.googleapis.com/token","auth_provider_x509_cert_url":"https://www.googleapis.com/oauth2/v1/certs","client_secret":"XXX","redirect_uris":["https://script.google.com/oauthcallback"]}}
ALSO:
The scope https://www.googleapis.com/auth/drive.file is not sufficient to list all files on a user's drive - it only gives you access to the files that were created / opened with your app. More information here.
I recommend you for debugging purposes to give your App in the first generous scopes. Once you solve the credentials issue and your App works - see how far you can restrict the scopes without impacting the functionality of your App.
How to write custom authentication of user, that connects to chat over ws:// protocol? This user is on the other side of Django app, he is the mobile user, connecting websocket via ws:// from mobile app. I tried to test websocket with chrome extenshion, it couldn't connect to my websocket. I think it was because of authentication.
In Django channels docs it says:
If you have a custom authentication scheme, you can write a custom middleware to parse the details and put a user object (or whatever other object you need) into your scope.Middleware is written as a callable that takes an ASGI application and wraps it to return another ASGI application. Most authentication can just be done on the scope, so all you need to do is override the initial constructor that takes a scope, rather than the event-running coroutine.
Here’s a simple example of a middleware that just takes a user ID out of the query string and uses that:
The same principles can be applied to authenticate over non-HTTP protocols; for example, you might want to use someone’s chat username from a chat protocol to turn it into a user.
from django.db import close_old_connections
class QueryAuthMiddleware:
def __init__(self, inner):
# Store the ASGI application we were passed
self.inner = inner
def __call__(self, scope):
# Look up user from query string (you should also do things like
# check it's a valid user ID, or if scope["user"] is already populated)
user = User.objects.get(id=int(scope["query_string"]))
close_old_connections()
# Return the inner application directly and let it run everything else
return self.inner(dict(scope, user=user))
What query do I need to do? I don't know anything about that user, in fact its anonymous.
Help me, please.
In this example code, you probably have to open the websocket connection with:
ws://SERVER:PORT/PATH?1
Everything after the ? is the query string. In your example code, your query_string has to be a user id, so for example 1.
You could change the code to use different query strings. For example you could use:
from urllib.parse import parse_qs
from django.db import close_old_connections
class QueryAuthMiddleware:
def __init__(self, inner):
# Store the ASGI application we were passed
self.inner = inner
def __call__(self, scope):
# Look up user from query string (you should also do things like
# check it's a valid user ID, or if scope["user"] is already populated)
query_string = parse_qs(self.scope['query_string'])
if b'user_id' in query_string:
user = User.objects.get(id=int(query_string[b'user_id'][0]))
close_old_connections()
else:
user = AnonymousUser
# Return the inner application directly and let it run everything else
return self.inner(dict(scope, user=user))
Now, you can use this uri:
ws://SERVER:PORT/PATH?user_id=1
You sill have to make sure, that the user with the ID exists in the database. You also have to write actual auth code. Every user can connect to this application with a arbitrary user id. There is no password or auth-token required.
I had the same issue as well, did research a bit came across solution with the code snippet below:
let's say you have User class defined. You want authenticate user through query send query while establishing connection with ws.
I will pass channels installation and configuration let's say you have successfully installed channels and configured.
QueryAuthMiddleware class as shown below:
from channels.auth import AuthMiddlewareStack
from django.conf import LazySettings
from urllib import parse
from rest_socket.models import User
from urllib.parse import parse_qs
from urllib.parse import unquote, urlparse
from channels.auth import AuthMiddlewareStack
from django.contrib.auth.models import AnonymousUser
class QueryAuthMiddleware:
"""
QueryAuthMiddleware authorization
"""
def __init__(self, inner):
self.inner = inner
def __call__(self, scope):
query_string = parse_qs(scope['query_string']) #Used for query string token url auth
headers = dict(scope['headers']) #Used for headers token url auth
print("query", query_string)
if b'user' in query_string:
try:
user = query_string[b'user'][0].decode()
print("user", user)
existing = User.objects.filter(id=user).last()
print(existing)
if existing:
print("existinguser")
scope['user'] = existing
else:
scope["user"] ="no user found"
except User.DoesNotExist:
pass
return self.inner(scope)
QueryAuthMiddlewareStack = lambda inner: QueryAuthMiddleware(AuthMiddlewareStack(inner))
Your routing.py should be like this:
from channels.security.websocket import AllowedHostsOriginValidator
from channels.auth import AuthMiddlewareStack
from channels.routing import ProtocolTypeRouter, URLRouter
from channels.security.websocket import OriginValidator
from django.urls import path
import your_app.routing
from project.utils.auth import QueryAuthMiddlewareStack
application = ProtocolTypeRouter({
# (http->django views is added by default)
'websocket': QueryAuthMiddlewareStack(
URLRouter(
your_app.routing.websocket_urlpatterns
)
),
})
Routing inside application:
websocket_urlpatterns = [
path("tasks/", consumers.Task)
]
and your ws connection to django-channels:
<script>
var notification;
if (location.protocol === 'https:') {
notification = new WebSocket('wss://' + "window.location.host" + "/tasks"+ "/?user=id");
console.log("with htpps")
}
else {
notification = new WebSocket('ws://' + window.location.host + "/tasks"+ "/?userr=id");
console.log("htpp")
}
notification.onopen = function open() {
console.log('notification connection created for NotificationWebsocket.');
};
notification.onmessage = function message(event) {
var data = JSON.parse(event.data);
console.log("Socket response from NotificationWebsocket => ", data);
};
if (notification.readyState === WebSocket.OPEN) {
notification.onopen();
}
</script>
I am creating Selenium tests for my App.
I can create a new user, but I can't seem to figure out how to have it deleted from the database.
After the tests run successfully the first time, subsequent tests fail because the username already exists.
Why am I not able to query the newly created record in the debugger despite being able to see the new record on the page?
How do I delete a record from the database in a test?
This is what I have been doing:
from selenium import webdriver
from django.utils import unittest
from forum.models import Question, Answer, User
class TestOSQAAuthentication(unittest.TestCase):
scheme = 'http'
host = 'localhost'
port = '4444'
def setUp(self):
self._driver = webdriver.Firefox()
self._driver.implicitly_wait(25)
def test_anon_can_create_new_account_manually(self):
self._driver.get('http://localhost:8000/account/local/register/')
self._driver.find_element_by_id('id_username').send_keys('MrManual')
self._driver.find_element_by_id('id_email').send_keys('test#gmail.com')
self._driver.find_element_by_id('id_password1').send_keys('test')
self._driver.find_element_by_id('id_password2').send_keys('test')
self._driver.find_element_by_id('bnewaccount').click()
# verify MrManual was created
self._driver.get('http://localhost:8000/users/')
self._driver.find_element_by_link_text('MrManual')
# MrManual seems to be created, but I don't see MrManual in the database during debugging with:
# import ipdb; ipdb.set_trace()
#ipdb> User.objects.all()
#[<User: Bryan>, <User: Kallie>, <User: Stalin>]
# here I am trying to delete the user from the database directly.
User.objects.filter(username="MrManual").delete()
"""For some reason I can't delete the record from the database from the test.
Selenium can find the new user in the browser, but I can't query the database to find it."""
If you use Selenium, instead of django.utils.unittest.TestCase please use django.test.TransactionalTestCase or even better, LiveServerTestCase.
I would like to read some session variables from a test (Django TestCase)
How to do that in a clean way ?
def test_add_docs(self):
"""
Test add docs
"""
# I would access to the session here:
self.request.session['documents_to_share_ids'] = [1]
response = self.client.get(reverse(self.document_add_view_id, args=[1]), follow=True)
self.assertEquals(response.status_code, 200)
As of Django 1.7+ this is much easier. Make sure you set the session as a variable instead of referencing it directly.
def test_something(self):
session = self.client.session
session['somekey'] = 'test'
session.save()
andreaspelme's workaround is only needed in older versions of django. See docs
Unfortunately, this is not a easy as you would hope for at the moment. As you might have noticed, just using self.client.session directly will not work if you have not called other views that has set up the sessions with appropriate session cookies for you. The session store/cookie must then be set up manually, or via other views.
There is an open ticket to make it easier to mock sessions with the test client: https://code.djangoproject.com/ticket/10899
In addition to the workaround in the ticket, there is a trick that can be used if you are using django.contrib.auth. The test clients login() method sets up a session store/cookie that can be used later in the test.
If you have any other views that sets sessions, requesting them will do the trick too (you probably have another view that sets sessions, otherwise your view that reads the sessions will be pretty unusable).
from django.test import TestCase
from django.contrib.auth.models import User
class YourTest(TestCase):
def test_add_docs(self):
# If you already have another user, you might want to use it instead
User.objects.create_superuser('admin', 'foo#foo.com', 'admin')
# self.client.login sets up self.client.session to be usable
self.client.login(username='admin', password='admin')
session = self.client.session
session['documents_to_share_ids'] = [1]
session.save()
response = self.client.get('/') # request.session['documents_to_share_ids'] will be available
If you need to initialize a session for the request in tests to manipulate it directly:
from django.contrib.sessions.middleware import SessionMiddleware
from django.http import HttpRequest
request = HttpRequest()
middleware = SessionMiddleware()
middleware.process_request(request)
request.session.save()
If testing my django project with pytest, I can not see any modifications to the session that are made in the view. (That is because the Sessions middleware doesn't get called.)
I found the following approach to be useful:
from unittest.mock import patch
from django.test import Client
from django.contrib.sessions.backends import db
def test_client_with_session():
client = Client()
session = {} # the session object that will persist throughout the test
with patch.object(db, "SessionStore", return_value=session):
client.post('/url-that-sets-session-key/')
assert session['some_key_set_by_the_view']
client.get('/url-that-reads-session-key/')
This approach has the benefit of not requiring database access.
You should be able to access a Client's session variales through its session property, so I guess self.client.session['documents_to_share_ids'] = [1] should be what you are looking for!