I am using pytest with pytest-django and pytest-bdd to test a simple django application.
The settings file defines a test sqlite3 database, which is created when the tests are run.
My first function creates a model object:
#given("A message exists")
def given_a_message_exists(transactional_db, message_text):
message = message_models.Message(text=message_text)
message.save()
But although I can open and inspect the database using sql-browser, the model is never persisted to the database. I can, in my next function, obtain the message from the django ORM, but the message isn't persisted in the database, and a call to a url route to delete the message returns a 404 with 'Message not found matching the query'.
#when("User visits the delete message page", target_fixture="page")
def user_visits_delete_message_page(transactional_db, browser, message_text):
message = message_models.Message.objects.get(text=message_text)
url = f"{browser.domain}/delete_message/{message.id}/{message.slug}/"
browser.visit(url)
return browser
When I run the site normally, everything works as expected.
Here are the fixtures from my conftest.py...
MESSAGE_TEXT = "Ipsum Lorum Dolum Est"
CREATE_MESSAGE_URL = reverse("django_messages:message_create")
LIST_MESSAGE_URL = reverse("django_messages:message_list")
LINKS_DICT = {
"create_message": f"a[href='{CREATE_MESSAGE_URL}']",
"list_message": f"a[href='{LIST_MESSAGE_URL}']",
}
PAGES_DICT = {
"create_message": CREATE_MESSAGE_URL,
"list_message": LIST_MESSAGE_URL,
}
#pytest.fixture()
def message_text():
return MESSAGE_TEXT
#pytest.fixture()
def browser(sb, live_server, settings):
staging_server = os.environ.get("STAGING_SERVER")
if staging_server:
sb.visit(staging_server)
else:
sb.visit(live_server)
sb.domain = sb.get_domain_url(sb.get_current_url())
settings.EMAIL_PAGE_DOMAIN = sb.domain
sb.pages = PAGES_DICT
sb.links = LINKS_DICT
return sb
Why is the model not persisted to the database when I call message.save()?
btw, I have tried, transactional_db, and db, along with a whole host of other permutations...
It turns out that using the browser fixture causes a problem that I don't fully understand yet. If I pass the browser into a step, and then return it as a target_fixture for later functions to consume then everything works as expected.
[EDIT]
I discovered that if I define the browser in a fixture, that I must have referenced that browser fixture before or at the same time as the django object using it, or the browser refers to a different domain.
So, in the below, even though the message is created in a fixture (test_message), I must refer to it having referenced the browser. If the function message_exists is not passed 'browser', then the message is not listed on the message_list page.
def message_exists(browser, test_message):
test_message.save()
return test_message
#when("User visits the message list page", target_fixture="page")
def user_visits_messages_page(db, browser):
browser.visit(browser.domain + browser.pages["list_message"])
return browser
#then("The message is listed")
def message_is_listed(message, page):
page.assert_element(f"a[href='/{message.id}/{message.slug}/']")
Related
I have a Django application that executes a full-text-search on a database. The view that executes this query is my search_view (I'm ommiting some parts for the sake of simplicity). It just retrieve the results of the search on my Post model and send to the template:
def search_view(request):
posts = m.Post.objects.all()
query = request.GET.get('q')
search_query = SearchQuery(query, config='english')
qs = Post.objects.annotate(
rank=SearchRank(F('vector_column'), search_query) + TrigramSimilarity('post_title', query)
).filter(rank__gte=0.15).order_by('-rank'), 15
)
context = {
results = qs
}
return render(request, 'core/search.html', context)
The application is working just fine. The problem is with a test I created. Here is my tests.py:
class SearchViewTests(TestCase):
def test_search_without_results(self):
"""
If the user's query did not retrieve anything
show him a message informing that
"""
response = self.client.get(reverse('core:search') + '?q=eksjeispowjskdjies')
self.assertEqual(response.status_code, 200)
self.assertContains(response.content, "We didn\'t find anything on our database. We\'re sorry")
This test raises an ProgrammingError exception:
django.db.utils.ProgrammingError: function similarity(character varying, unknown) does not exist
LINE 1: ...plainto_tsquery('english'::regconfig, 'eksjeispowjskdjies')) + SIMILARITY...
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
I understand very well this exception, 'cause I got it sometimes. The SIMILARITY function in Postgres accepts two arguments, and both need to be of type TEXT. The exception is raising because the second argument (my query term) is of type UNKNOWN, therefore the function won't work and Django raises the exception. And I don't understand why, because the actual search is working! Even in the shell it works perfectly:
In [1]: from django.test import Client
In [2]: c = Client()
In [3]: response = c.get(reverse('core:search') + '?page=1&q=eksjeispowjskdjies')
In [4]: response
Out[4]: <HttpResponse status_code=200, "text/html; charset=utf-8">
Any ideas about why test doesn't work, but the actual execution of the app works and console test works too?
I had the same problem and this how I solved it in my case:
First of all, the problem was that when Django creates the test database that it is going to use for tests it does not actually run all of your migrations. It simply creates the tables based on your models.
This means that migrations that create some extension in your database, like pg_trgm do not run when creating the test database.
One way to overcome this is to use a fixture in your conftest.py file which will make create said extensions before any tests run.
So, in your conftest.py file add the following:
# the following fixture is used to add the pg_trgm extension to the test database
#pytest.fixture(scope="session", autouse=True)
def django_db_setup(django_db_setup, django_db_blocker):
"""Test session DB setup."""
with django_db_blocker.unblock():
with connection.cursor() as cursor:
cursor.execute("CREATE EXTENSION IF NOT EXISTS pg_trgm;")
You can of course replace pg_trgm with any other extension you require.
PS: You must make sure the extension you are trying to use works for the test database you have chosen. In order to change the database used by Django you can change the value of
DATABASES = {'default': env.db('your_database_connection_uri')}
in your application's settings.py.
I have a simple form for registering new user. I wrote a test case for it. It looks as follows:
class AccountTestCase(LiveServerTestCase):
def setUp(self):
self.selenium = webdriver.Firefox()
super(AccountTestCase, self).setUp()
def tearDown(self):
self.selenium.quit()
super(AccountTestCase, self).tearDown()
def test_register(self):
selenium = self.selenium
#Opening the link we want to test
selenium.get('http://localhost:8000/register/')
#find the form element
first_name = selenium.find_element_by_id('id_first_name')
last_name = selenium.find_element_by_id('id_last_name')
username = selenium.find_element_by_id('id_username')
email = selenium.find_element_by_id('id_email')
password1 = selenium.find_element_by_id('id_password1')
password2 = selenium.find_element_by_id('id_password2')
submit = selenium.find_element_by_id('btn_signup')
#Fill the form with data
first_name.send_keys('abc')
last_name.send_keys('abc')
username.send_keys('abc')
email.send_keys('abc#gmail.com')
password1.send_keys('abcabcabc')
password2.send_keys('abcabcabc')
#submitting the form
submit.send_keys(Keys.RETURN)
#check the returned result
self.assertTrue('Success!' in selenium.page_source)
When I ran the test case for the first time it passed with flying colors, but on the second run it failed.
After little investigation I realized a user with credentials from test cases is already created in my database. Thus, when I ran test case for second time it failed to create a new user, as user with these details is already present. (I can see user from Django admin).
I believe this is not the expected behaviour of LiveServerTestCase(or any type of test case). In setup, a temporary database is created, test case is ran on it, and destroyed in tearDown phase.
I want to know if, this is the intended behavior? if not why this is happening ? How can I avoid doing it ?
I have not made any changes in settings.py which are related to selenium or testing (is there a flag or something that needs to be set?). Also, I need to keep the server running for this to work (is this normal?) .
As pointed by #Paulo Almeida :
I was using a wrong url. The URL that I should be using is self.live_server_url . Since I was using http://localhost:8000/register/ It was expecting server to be running and creating records there.
Thanks.
I am having an issue with automated testing in web py framework.
I am going through the last exercise of learn python the hard way. In this exercise we make a web application "engine" that runs a map of rooms.
I want to be able to automate test every single room, but there is one problem, is that the engine depends on the previous room to decide which room to go to next (and user input).
if web.config.get("_session") is None:
store = web.session.DiskStore("sessions")
session = web.session.Session(app, store, initializer={"room":None})
web.config._session = session
else:
session = web.config._session
This class handles GET request sent to /
class Index(object):
def GET(self):
session.room = map.START
web.seeother("/game")
This class handles GET and POST requests to /game
class GameEngine(object):
def GET(self):
if session.room:
return render.show_room(room=session.room)
else:
return render.you_died()
def POST(self):
form = web.input(action=None)
if session.room and form.action:
session.room = session.room.go(form.action)
web.seeother("/game")
In my automated testing I use two things: first I use the app.request API:
app.request(localpart='/', method='GET',data=None,
host='0.0.0.0:8080', headers=None, https=False)
create a response object, something like:
resp = app.request("/game", method = "GET")
Second I pass the resp object to this function to check for certain things:
from nose.tools import *
import re
def assert_response(resp, contains=None, matches=None, headers=None,
status="200"):
assert status in resp.status, "Expected response %r not in %r" %
(status, resp.status)
if status == "200":
assert resp.data, "Response data is empty"
if contains:
assert contains in resp.data, "Response does not contain %r" %
contains
if matches:
reg = re.compile(matches)
assert reg.matces(resp.data), "Response does not match %r" %
matches
if headers:
assert_equal(resp.headers, headers)
We can pass variables as a dictionary to the keyword argument data in the API app.request to modify the web.input().
my question is: in my automated test module how do we "pass" a value that overwrite the room value in the initializer dictionary in our session:
session = web.session.Session(app, store, initializer={"room":None})
In the app module its done by setting
session.room = map.START
and then session.room updates using:
if session.room and form.action:
session.room = session.room.go(form.action)
Thanks for taking the time to read this, and any insights would be appreciated!
Alright I finally found it! The main issue here was that every time I make a http request through app.request it gives me a new session ID.
The trick that I found thanks to this post:
How to initialize session data in automated test? (python 2.7, webpy, nosetests)
is to record the session ID of the request to reuse that ID in my automated tests by passing it to the headers keyword argument in the request!
record the session ID using this function (which I placed as suggested in the post in tests/tools.py):
def get_session_id(resp):
cookies_str = resp.headers['Set-Cookie']
if cookies_str:
for kv in cookies_str.split(';'):
if 'webpy_session_id=' in kv:
return kv
then in the automated tests something like:
def test_session():
resp = app.request('/')
session_id = get_session_id(resp)
resp1 = app.request('/game', headers={'Cookie':session_id})
assert_response(resp1, status='200', contains='Central Corridor')
I hope that helps in the future for programmers who get stuck on the same issue!
I am writing a simple web application in flask(python) which will ask students to fill experimental data in a form and that data will be submitted on a server.
My user class looks like:
class User(db.Model):
id=db.Column(db.Integer,primary_key=True)
email=db.Column(db.String,unique=True)
name=db.Column(db.String)
password=db.Column(db.String)
data = db.relationship('ExpData', backref='student', lazy='dynamic')
and ExpData class, which will store data looks like:
class ExpData(db.Model):
id=db.Column(db.Integer,primary_key=True)
exp_1_data=db.Column(db.String)
submission_date=db.Column(db.DateTime)
user_id=db.Column(db.Integer, db.ForeignKey('user.id'))
Now User model works perfectly, whide ExpData models does not put anything in database. I used the following lines to achieve it:
u=models.User(email=usern,name=nameu,password=userp,rollno=rollno)
db.session.add(u)
db.session.commit()
Then in required function:
user=g.user
dat=models.ExpData(exp_1_data='dataStr',submit=1,submission_date=datetime.now(),student=user)
db.session.add(dat)
db.session.commit()
in Python shell it works perfectly. But for some reason through web interface its not working.
Other details:
Users are logged in via a login_user() function provided by Flask-Login
Then user is assigned to g.user through another decorator
#app.before_request
def before_request():
g.user = current_user
Then Database entry is done whenever following function is executed upon visiting the url
#app.route('/exp1echo', methods=['GET','POST'])
#login_required
def exp1echo():
exp1_data={}
exp1_data["ans"]=float(request.args.get("ans")) #getting some data through ajax 'get' call
user=g.user
filename = user.name+" simplePrint.pdf" # save file with user name in file name
pdf=render_template('exp1_post.html',exp1_data=exp1_data)
pisa.CreatePDF(pdf.encode("ISO-8859-1"), file(filename, "wb"))
pdf.close() # Just fyi this call does return an error
#"No handlers could be found for logger "xhtml2pdf"" but all works fine
abc={'a':1,'b':2,'c':3}
dataStr=str(abc)
dat=ExpData(exp_1_data='dataStr',submit=1,submission_date=datetime.now(),student=g.user)
db.session.add(dat)
db.session.commit()
return jsonify({'base_vl':1}) # return some value to ajax call
User loader is defined as:
lm = LoginManager()
lm.init_app(app)
#lm.user_loader
def load_user(id):
return models.User.query.get(int(id))
Users are registered through following URL:
#app.route('/register', methods=['GET','POST'])
def register():
if request.method == 'POST':
user_pass=request.form["password"]
user_name=request.form["username"]
name_=request.form["name"]
u=models.User(email=user_name,name=name_,password=user_pass)
db.session.add(u)
db.session.commit()
return redirect(url_for('login'))
return render_template('register_form.html')
Well this is embarrassing! Found out the error. For some reason pdf.close() function in exp1echo() gives an error (as I mentioned earlier). That error does not show any message during debugging nor it exits the programme. Hence I ignored it. But it terminates the execution of that function. So it was not sending any call to data base. So I commented it out and voila! It worked.
I have a small web app with AngularJS front-end and Django ReST in the back. There's a strange hitch going on when I make POST request to the web service: the browser console clearly shows 3 parameters being sent, but the backend logging reports only 2 params received. The result is that the server throws a code 500 error due to a bad database lookup.
Here's the code:
Client
var b = newQuesForm.username.value;
$http.post('/myapp/questions/new', {username:b,title:q.title,description:q.description}).
success(function(data, status, headers, config) {
$http.get('/myapp/questions').success(function(data){
$scope.questions = data;
q = null;
$scope.newQuesForm.$setPristine();
}).error(function(data, status, headers, config) {
console.log(headers+data);
});
}).
error(function(data, status, headers, config) {
console.log(headers+data);
});
Both my manual logging and the dev console show a string like:
{"username":"admin","description":"What's your name?","title":"question 1"}
Server
class CreateQuestionSerializer(serializers.Serializer):
author = UserSerializer(required=False)
title = serializers.CharField(max_length=150)
description = serializers.CharField(max_length=350)
def create(self, data):
q= Question()
d = data
q.title = d.get('title')
q.description = d.get("description")
q.author = User.objects.get(username=d.get('username'))
q.save()
return q
Server-side logging shows the username parameter never succeeds in making the trip, and thus I end up with code 500 and error message:
User matching query does not exist. (No user with id=none)
What's causing some of the data to get lost?
So it turns out the problem was really with the serialization of fields, as #nikhiln began to point out. I followed his lead to refactor the code, moving the create() method to api.py, rather than serializers.py, and stopped relying altogether on the client-side data for the user's identity, something that was a bit silly in the first place (passing User to a hidden input in the view, and then harvesting the username from there and passing it back to the server in the AJAX params). Here's the new code, that works perfectly:
class QuestionCreate(generics.CreateAPIView):
model = Question
serializer_class = CreateQuestionSerializer
def create(self, request,*args,**kwargs):
q= Question()
d = request.data
q.title = d.get('title')
q.description = d.get("description")
q.author = request.user
q.save()
if q.pk:
return Response({'id':q.pk,'author':q.author.username}, status=status.HTTP_201_CREATED)
return Response({'error':'record not created'}, status=status.HTTP_400_BAD_REQUEST)
So here, I do it the right way: pull the User from the request param directly in the backend.