I am following the TddjangoTutorial. My question is that when I visit
localhost:8000/admin/polls/poll to create a new poll, it complains that the poll table is not there. This is fixed by running the syncdb command, which creates the table. But when I ran selenium test before running the command, it worked fine. The test opens localhost:8081/admin/polls/poll. It showed the page to add a new poll. Does the functional test automatically create this table?
Code for functional test:
from django.test import LiveServerTestCase
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from userFactory import UserFactory
class PollsTest(LiveServerTestCase):
def setUp(self):
self.browser = webdriver.Firefox()
self.browser.implicitly_wait(3)
self.user = UserFactory.create()
def tearDown(self):
self.browser.quit()
def test_can_create_new_poll_via_admin_site(self):
self.browser.get(self.live_server_url+'/admin/')
body = self.browser.find_element_by_tag_name('body')
self.assertIn('Django administration', body.text)
username_field = self.browser.find_element_by_name('username')
username_field.send_keys(self.user.username)
password_field = self.browser.find_element_by_name('password')
password_field.send_keys('adm1n')
password_field.send_keys(Keys.ENTER)
body = self.browser.find_element_by_tag_name('body')
self.assertIn('Site administration', body.text)
polls_links = self.browser.find_elements_by_link_text('Polls')
self.assertEqual(len(polls_links), 2)
polls_links[1].click()
#the user is taken to the polls listing page, which shows no polls yet
body = self.browser.find_element_by_tag_name('body')
self.assertIn('0 polls', body.text)
# the user clicks on add to add a new poll
new_poll_link = self.browser.find_element_by_link_text('Add poll')
new_poll_link.click()
body = self.browser.find_element_by_tag_name('body')
self.assertIn('Question:', body.text)
self.assertIn('Date published:', body.text)
As you can read in the original doc: https://docs.djangoproject.com/en/dev/topics/testing/overview/#the-test-database
Tests that require a database (namely, model tests) will not use your “real” (production) database. Separate, blank databases are created for the tests.
Regardless of whether the tests pass or fail, the test databases are destroyed when all the tests have been executed.
Related
I'm using Django and Pytest specifically to run the test suite and am trying to test that a specific form shows up with expected data when a user hits the site (integration test).
This particular view uses a stored procedure, which I am mocking since the test would never have access to that.
My test code looks like this:
#test_integrations.py
from my_app.tests.data_setup import setup_data, setup_sb7_data
from unittest.mock import patch
...
# Setup to use a non-headless browser so we can see whats happening for debugging
#pytest.mark.usefixtures("standard_browser")
class SeniorPageTestCase(StaticLiveServerTestCase):
"""
These tests surround the senior form
"""
#classmethod
def setUpClass(cls):
cls.host = socket.gethostbyname(socket.gethostname())
super(SeniorPageTestCase, cls).setUpClass()
def setUp(self):
# setup the dummy data - this works fine
basic_setup(self)
# setup the 'results'
self.sb7_mock_data = setup_sb7_data(self)
#patch("my_app.utils.get_employee_sb7_data")
def test_senior_form_displays(self, mock_sb7_get):
# login the dummy user we created
login_user(self, "futureuser")
# setup the results
mock_sb7_get.return_value = self.sb7_mock_data
# hit the page for the form
self.browser.get(self.live_server_url + "/my_app/senior")
form_id = "SeniorForm"
# assert that the form displays on the page
self.assertTrue(self.browser.find_element_by_id(form_id))
# utils.py
from django.conf import settings
from django.db import connections
def get_employee_sb7_data(db_name, user_number, window):
"""
Executes the stored procedure for getting employee data
Args:
user_number: Takes the user_number
db (db connection): Takes a string of the DB to connect to
Returns:
"""
cursor = connections[db_name].cursor()
cursor.execute(
'exec sp_sb7 %s, "%s"' % (user_number, window.senior_close)
)
columns = [col[0] for col in cursor.description]
results = [dict(zip(columns, row)) for row in cursor.fetchall()]
return results
# views.py
from myapp.utils import (
get_employee_sb7_data,
)
...
###### Senior ######
#login_required
#group_required("user_senior")
def senior(request):
# Additional Logic / Getting Other Models here
# Execute stored procedure to get data for user
user_number = request.user.user_no
results = get_employee_sb7_data("production_db", user_number, window)
if not results:
return render(request, "users/senior_not_required.html")
# Additional view stuff
return render(
request,
"users/senior.html",
{
"data": data,
"form": form,
"results": results,
},
)
If I run this test itself with:
pytest my_app/tests/test_integrations.py::SeniorPageTestCase
The tests pass without issue. The browser shows up - the form shows up with the dummy data as we would expect and it all works.
However, if I run:
pytest my_app
All other tests run and pass - but all the tests in this class fail because it's not patching the function.
It tries to call the actual stored procedure (which fails because it's not on the production server yet) and it fails.
Why would it patch correctly when I call that TestCase specifically - but not patch correctly when I just run pytest on the app or project level?
I'm at a loss and not sure how to debug this very well. Any help is appreciated
So what's happening is that your views are imported before you're patching.
Let's first see the working case:
pytest imports the test_integrations file
the test is executed and patch decorator's inner function is run
there is no import of the utils yet and so patch imports and replaces the function
test body is executed, which passes a url to the test client
the test client imports the resolver and in turn it imports the views, which imports the utils.
Since the utils are already patched, everything works fine
If another test case runs first, that also imports the same views, then that import wins and patch cannot replace the import.
Your solution is to reference the same symbol. So in test_integrations.py:
#patch("myapp.views.get_employee_sb7_data")
I am running some django integration tests of code that passes some functions to Django-Q for processing. This code essentially updates some fields on a model object.
The app uses signals.py to listen for a post_save change of the status field of the Foo object. Assuming it's not a newly created object and it has the correct status, the Foo.prepare_foo() function is called. This is handled by services.py which hands it off to q-cluster. Assuming this executes without error hooks.py changes the status field of Foo to published, or keeps it at prepare if it fails. If it passes then it also sets prepared to True. (I know this sounds convoluted and with overlapping variables - part of the desire to get tests running is to be able to refactor).
The code runs correctly in all environments, but when I try to run tests to assert that the fields have been updated, they fail.
(If I tweak the code to bypass the cluster and have the functions run in-memory then the tests pass.)
Given this, I'm assuming the issue lies in how I've written the tests, but I cannot diagnose the issue.
tests_prepare_foo.py
import time
from django.test import TestCase, override_settings, TransactionTestCase
from app.models import Foo
class CreateCourseTest(TransactionTestCase):
reset_sequences = True
#classmethod
def setUp(cls):
cls.foo = Foo(status='draft',
prepared=False,
)
cls.foo.save()
def test_foo_prepared(self):
self.foo.status = 'prepare'
self.foo.save()
time.sleep(15) # to allow the cluster to finish processing the request
self.assertEquals(self.foo.prepared, True)
models.py
import uuid
from django.db import models
from model_utils.fields import StatusField
from model_utils import Choices
class Foo(models.Model):
ref = models.UUIDField(default=uuid.uuid4,
editable=False,
)
STATUS = Choices('draft', 'prepare', 'published', 'archived')
status = StatusField(null=True,
blank=True,
)
prepared = models.BooleanField(default=False)
def prepare_foo(self):
""""
...
do stuff
"""
signals.py
from django.dispatch import receiver
from django.db.models.signals import post_save
from django_q.tasks import async_task
from app.models import Foo
#receiver(post_save, sender=Foo)
def make_foo(sender, instance, **kwargs):
if not kwargs.get('created', False) and instance.status == 'prepare' and not instance.prepared:
async_task('app.services.prepare_foo',
instance,
hook='app.hooks.check_foo_prepared',
)
services.py
def prepare_foo(foo):
foo.prepare_foo()
hooks.py
import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
def check_foo_prepared(task):
foo = task.args[0]
if task.success:
logger.info("Foo Prepared: Successful")
foo.status = 'published'
foo.prepared = True
foo.save()
logger.info("Foo Status: %s", foo.status)
logger.info("Foo Prepared: %s", foo.prepared)
else:
logger.info("Foo Prepared: Unsuccessful")
foo.status = 'draft'
foo.save()
Finally - logs when the test is run with the cluster on:
q-cluster server
INFO:app.hooks:Foo Prepared: Successful
INFO:app.hooks: Foo Status: published
INFO:app.hooks: Foo Prepared: True
django server
h:m:s [Q] INFO Enqueued 1
F
======
FAIL
self.assertEquals(self.foo.prepared, True)
AssertionError: False != True
I think I'm either missing something obvious in my tests, or something really subtle, but I can't work out which. I've tried setting the cluster to run synchronously (sync=True), and in my test reloading Foo just before the assertion:
self.foo.save()
time.sleep(15)
test_foo = Foo.objects.get(pk=1)
self.assertEquals(test_foo.prepared, True)
But this also fails with
self.assertEquals(test_foo.prepared, True)
AssertionError: False != True
Which leads me to believe that the cluster is not updating the object under test (unlikely), or the assertion is being checked before the cluster has updated the object (more likely).
This is the first time I've written tests that require hand-offs to a cluster, so any pointers, suggestions gratefully received!
I have a Django project that pulls data from legacy database (read only connection) into its own database, and when I run integration tests, it tries to read from test_account on legacy connection.
(1049, "Unknown database 'test_account'")
Is there a way to tell Django to leave the legacy connection alone for reading from the test database?
I actually wrote something that lets you create integration test in djenga (available on pypi) if you want to take a look at how to create a separate integration test framework.
Here is the test runner I use when using the django unit test framework:
from django.test.runner import DiscoverRunner
from django.apps import apps
import sys
class UnManagedModelTestRunner(DiscoverRunner):
"""
Test runner that uses a legacy database connection for the duration of the test run.
Many thanks to the Caktus Group: https://www.caktusgroup.com/blog/2013/10/02/skipping-test-db-creation/
"""
def __init__(self, *args, **kwargs):
super(UnManagedModelTestRunner, self).__init__(*args, **kwargs)
self.unmanaged_models = None
self.test_connection = None
self.live_connection = None
self.old_names = None
def setup_databases(self, **kwargs):
# override keepdb so that we don't accidentally overwrite our existing legacy database
self.keepdb = True
# set the Test DB name to the current DB name, which makes this more of an
# integration test, but HEY, at least it's a start
DATABASES['legacy']['TEST'] = { 'NAME': DATABASES['legacy']['NAME'] }
result = super(UnManagedModelTestRunner, self).setup_databases(**kwargs)
return result
# Set Django's test runner to the custom class defined above
TEST_RUNNER = 'config.settings.test_settings.UnManagedModelTestRunner'
TEST_NON_SERIALIZED_APPS = [ 'legacy_app' ]
from django.test import TestCase, override_settings
#override_settings(LOGIN_URL='/other/login/')
class LoginTestCase(TestCase):
def test_login(self):
response = self.client.get('/sekrit/')
self.assertRedirects(response, '/other/login/?next=/sekrit/')
https://docs.djangoproject.com/en/1.10/topics/testing/tools/
You should theoretically be able to use the override settings here and switch to a dif
How can I persist data to DB when using pytest/pytest-django in a test-run of a Django application?
I run pytest with py.test --nomigrations --reuse-db -s and the Postgres DB test_<configued_db_name> is created as expected, however nothing seems to be persisted to DB between tests and at the end of the test run the DB is empty.
import pytest
from django.contrib.auth.models import User
#pytest.mark.django_db(transaction=False)
def test_insert_user():
user = User.objects.create_user(username="test_user", email="test_user#test.com", password="test")
users = User.objects.all()
assert len(users) > 0
#pytest.mark.django_db(transaction=False)
def test_check_user():
users = User.objects.all()
assert len(users) > 0
The first test passes, the second does not making me wonder if anything is persisted to DB at all. According to the pystest-django documentation #pytest.mark.django_db(transaction=False) will not rollback whatever has been affected by the decorated test.
Thank you,
/David
Another way of prefilling the database with data for each function is like that:
import pytest
from django.contrib.auth.models import User
#pytest.fixture(scope='module')
def django_db_setup(django_db_setup, django_db_blocker):
print('setup')
with django_db_blocker.unblock():
User.objects.create(username='a')
assert set(u.username for u in User.objects.all()) == {'a'}
#pytest.mark.django_db
def test1():
print('test1')
User.objects.create(username='b')
assert set(u.username for u in User.objects.all()) == {'a', 'b'}
#pytest.mark.django_db
def test2():
print('test2')
User.objects.create(username='c')
assert set(u.username for u in User.objects.all()) == {'a', 'c'}
The good thing about this method is that the setup function is only called once:
plugins: django-3.1.2
collected 2 items
mytest.py setup
test1
.test2
.
=================== 2 passed in 1.38 seconds ====================
The bad thing is that 1.38 seconds is a bit too much for such a simple test. --reuse-db is a faster way to do it.
I have solved this problem -- prefill the DB for every function -- by defining a fixture with scope function (i.e. model and session will not work).
Here is the code for testing the views in Django.
# This is used to fill the database more easily
from mixer.backend.django import mixer
import pytest
from django.test import RequestFactory
from inventory import views
from inventory import services
pytestmark = pytest.mark.django_db
#pytest.fixture(scope="function")
def fill_db():
""" Just filling the DB with my data """
for elem in services.Search().get_lookup_data():
mixer.blend('inventory.Enumeration', **elem)
def test_grid_anonymous(fill_db):
request = RequestFactory().get('/grid/')
response = views.items_grid(request)
assert response.status_code == 200, \
"Should be callable by anyone"
def test_list_anonymous(fill_db):
request = RequestFactory().get('/')
response = views.items_list(request)
assert response.status_code == 200, \
"Should be callable by anyone"
I'm trying out using Flask-KVSession as an alternative session implementation for a Flask web site. I've created a test website (see Code 1 below). When I run this, I can use the browser to store values into the session by navigating between the various resources in my web browser. This works correctly. Also, when I look at the sessions table in the resulting SQLite database, I see a single record that was being used to store this session the entire time.
Then I try to add Flask-Security to this (see Code 2 below). After running this site (making sure to first delete the existing test.db sqlite file), I am brought to the login prompt and I log in. Then I proceed to do the same thing of jumping back and forth between the resources. I get the same results.
The problem is that when I look in the sqlitebrowser sessions table, there are 8 records. It turns out a new session record was created for EACH request that was made.
Why does a new session record get created for each request when using Flask-Security? Why isn't the existing session updated like it was before?
Code 1 (KVSession without Flask-Security)
import os
from flask import Flask, session
app = Flask(__name__)
app.secret_key = os.urandom(64)
#############
# SQLAlchemy
#############
from flask.ext.sqlalchemy import SQLAlchemy
db = SQLAlchemy(app)
DB_DIR = os.path.dirname(os.path.abspath(__file__))
DB_URI = 'sqlite:////{0}/test.db'.format(DB_DIR)
app.config['SQLALCHEMY_DATABASE_URI'] = DB_URI
#app.before_first_request
def create_user():
db.create_all()
############
# KVSession
############
from simplekv.db.sql import SQLAlchemyStore
from flask.ext.kvsession import KVSessionExtension
store = SQLAlchemyStore(db.engine, db.metadata, 'sessions')
kvsession = KVSessionExtension(store, app)
#app.route('/a')
def a():
session['last'] = 'b'
return 'Thank you for visiting A!'
#app.route('/b')
def b():
session['last'] = 'b'
return 'Thank you for visiting B!'
#app.route('/c')
def c():
return 'You last visited "{0}"'.format(session['last'])
app.run(debug=True)
Code 2 (KVSession WITH Flask-Security)
import os
from flask import Flask, session
app = Flask(__name__)
app.secret_key = os.urandom(64)
#############
# SQLAlchemy
#############
from flask.ext.sqlalchemy import SQLAlchemy
db = SQLAlchemy(app)
DB_DIR = os.path.dirname(os.path.abspath(__file__))
DB_URI = 'sqlite:////{0}/test.db'.format(DB_DIR)
app.config['SQLALCHEMY_DATABASE_URI'] = DB_URI
###########
# Security
###########
# This import needs to happen after SQLAlchemy db is created above
from flask.ext.security import (
Security, SQLAlchemyUserDatastore, current_user,
UserMixin, RoleMixin, login_required
)
# Define models
roles_users = db.Table('roles_users',
db.Column('user_id', db.Integer(), db.ForeignKey('user.id')),
db.Column('role_id', db.Integer(), db.ForeignKey('role.id')))
class Role(db.Model, RoleMixin):
id = db.Column(db.Integer(), primary_key=True)
name = db.Column(db.String(80), unique=True)
description = db.Column(db.String(255))
class User(db.Model, UserMixin):
id = db.Column(db.Integer, primary_key=True)
email = db.Column(db.String(255), unique=True)
password = db.Column(db.String(255))
active = db.Column(db.Boolean())
confirmed_at = db.Column(db.DateTime())
roles = db.relationship('Role', secondary=roles_users,
backref=db.backref('users', lazy='dynamic'))
user_datastore = SQLAlchemyUserDatastore(db, User, Role)
security = Security(app, user_datastore)
#app.before_first_request
def create_user():
db.create_all()
user_datastore.create_user(email='test#example.com', password='password')
db.session.commit()
############
# KVSession
############
from simplekv.db.sql import SQLAlchemyStore
from flask.ext.kvsession import KVSessionExtension
store = SQLAlchemyStore(db.engine, db.metadata, 'sessions')
kvsession = KVSessionExtension(store, app)
#app.route('/a')
#login_required
def a():
session['last'] = 'b'
return 'Thank you for visiting A!'
#app.route('/b')
#login_required
def b():
session['last'] = 'b'
return 'Thank you for visiting B!'
#app.route('/c')
#login_required
def c():
return 'You last visited "{0}"'.format(session['last'])
app.run(debug=True)
Version Info
Python 2.7.3
Flask==0.9
Flask==0.9
Flask-KVSession==0.3.2
Flask-Login==0.1.3
Flask-Mail==0.8.2
Flask-Principal==0.3.5
Flask-SQLAlchemy==0.16
Flask-Security==1.6.3
SQLAlchemy==0.8.1
Turns out this is related to a known problem with flask-login (which flask-security uses) when flask-login is used with a session storage library like KVSession.
Basically, KVSession needs to update the database with the new session information whenever data in the session is created or modified. And in the sample above, this happens correctly: the first time I hit a page, the session is created. After that, the existing session is updated.
However, in the background the browser sends a cookie-less request to my web server looking for my favicon. Therefore, flask is handling a request to /favicon.ico. This request (or any other request that would 404) is still handled by flask. This means that flask-login will look at the request and try to do its magic.
It so happens that flask-login doesn't TRY to put anything into the session, but it still LOOKS like the session has been modified as far as KVSession is concerned. Because it LOOKS like the session is modified, KVSession updates the database. The following is code from flask-login:
def _update_remember_cookie(self, response):
operation = session.pop("remember", None)
...
The _update_remember_cookie method is called during the request lifecycle. Although session.pop will not change the session if the session doesn't have the "remember" key (which in this case it doesn't), KVSession still sees a pop and assumes that the session changes.
The issue for flask-login provides the simple bug fix, but it has not been pushed into flask-login. It appears that the maintainer is looking for a complete rewrite, and will implement it there.