Django test script to pre-populate DB - django

I'm trying to pre-populate the database with some test data for my Django project. Is there some easy way to do this with a script that's "outside" of Django?
Let's say I want to do this very simple task, creating 5 test users using the following code,
N = 10
i = 0
while i < N:
c = 'user' + str(i) + '#gmail.com'
u = lancer.models.CustomUser.objects.create_user(email=c, password="12345")
i = i + 1
The questions are,
WHERE do I put this test script file?
WHAT IMPORTS / COMMANDS do I need to put at the beginning of the file so it has access to all the Django environment & resources as if I were writing this inside the app?
I'm thinking you'd have to import and set up the settings file, and import the app's models, etc... but all my attempts have failed one way or another, so would appreciate some help =)
Thanks!
Providing another answer
The respondes below are excellent answers. I fiddled around and found an alternative way. I added the following to the top of the test data script,
from django.core.management import setup_environ
from project_lancer import settings
setup_environ(settings)
import lancer.models
Now my code above works.

I recommend you to use fixtures for these purposes:
https://docs.djangoproject.com/en/dev/howto/initial-data/
If you still want to use this initial code then read:
If you use south you can create migration and put this code there:
python manage.py schemamigration --empty my_data_migration
class Migration(SchemaMigration):
no_dry_run = False
def forwards(self, orm):
# more pythonic, you can also use bulk_insert here
for i in xrange(10):
email = "user{}#gmail.com".format(i)
u = orm.CustomUser.objects.create_user(email=email, password='12345)
You can put it to setUp method of your TestCase:
class MyTestCase(TestCase):
def setUp(self):
# more pythonic, you can also use bulk_insert here
for i in xrange(10):
email = "user{}#gmail.com".format(i)
u = lancer.models.CustomUser.objects.create_user(email=email,
password='12345')
def test_foo(self):
pass
Also you can define your BaseTestCase in which you override setUp method then you create your TestCase classes that inherit from BaseTestCase:
class BaseTestCase(TestCase):
def setUp(self):
'your initial logic here'
class MyFirstTestCase(BaseTestCase):
pase
class MySecondTestCase(BaseTestCase):
pase
But I think that fixtures is the best way:
class BaseTestCase(TestCase):
fixtures = ['users_for_test.json']
class MyFirstTestCase(BaseTestCase):
pase
class MySecondTestCase(BaseTestCase):
fixtures = ['special_users_for_only_this_test_case.json']
Updated:
python manage.py shell
from django.contrib.auth.hashers import make_password
make_password('12312312')
'pbkdf2_sha256$10000$9KQ15rVsxZ0t$xMEKUicxtRjfxHobZ7I9Lh56B6Pkw7K8cO0ow2qCKdc='

You can also use something like this or this to auto-populate your models for testing purposes.

Related

how to keep data created in the ready method? Django production vs test database

As you know django give you clear database in testing, but I have a ready() method that create some data for me and I need to query these data in my tests.
class YourAppConfig(AppConfig):
default_auto_field = 'django.db.models.AutoField'
name = 'Functions.MyAppsConfig'
def ready(self):
from django.contrib.auth.models import Permission
from django import apps
from django.contrib.contenttypes.models import ContentType
try:
Permission.objects.get_or_create(....)
MyOtherModel.objects.get_or_create(....)
except:
pass
class TestRules(APITestCase):
def test_my_model(self):
....
x = MyOtherModel.objects.filter(....).first()
# x = None # <=========== problem is here ========= I need to see the data that I created in the ready method
....
You can use the fixtures for that, in each Test case you can fixtures to it as stated documentation example is
class Test(TransactionTestCase):
fixtures = ['user-data.json']
def setUp():
…
Django will load the fixtures before under test case

Why does mock patch only work when running specific test and not whole test suite?

I'm using Django and Pytest specifically to run the test suite and am trying to test that a specific form shows up with expected data when a user hits the site (integration test).
This particular view uses a stored procedure, which I am mocking since the test would never have access to that.
My test code looks like this:
#test_integrations.py
from my_app.tests.data_setup import setup_data, setup_sb7_data
from unittest.mock import patch
...
# Setup to use a non-headless browser so we can see whats happening for debugging
#pytest.mark.usefixtures("standard_browser")
class SeniorPageTestCase(StaticLiveServerTestCase):
"""
These tests surround the senior form
"""
#classmethod
def setUpClass(cls):
cls.host = socket.gethostbyname(socket.gethostname())
super(SeniorPageTestCase, cls).setUpClass()
def setUp(self):
# setup the dummy data - this works fine
basic_setup(self)
# setup the 'results'
self.sb7_mock_data = setup_sb7_data(self)
#patch("my_app.utils.get_employee_sb7_data")
def test_senior_form_displays(self, mock_sb7_get):
# login the dummy user we created
login_user(self, "futureuser")
# setup the results
mock_sb7_get.return_value = self.sb7_mock_data
# hit the page for the form
self.browser.get(self.live_server_url + "/my_app/senior")
form_id = "SeniorForm"
# assert that the form displays on the page
self.assertTrue(self.browser.find_element_by_id(form_id))
# utils.py
from django.conf import settings
from django.db import connections
def get_employee_sb7_data(db_name, user_number, window):
"""
Executes the stored procedure for getting employee data
Args:
user_number: Takes the user_number
db (db connection): Takes a string of the DB to connect to
Returns:
"""
cursor = connections[db_name].cursor()
cursor.execute(
'exec sp_sb7 %s, "%s"' % (user_number, window.senior_close)
)
columns = [col[0] for col in cursor.description]
results = [dict(zip(columns, row)) for row in cursor.fetchall()]
return results
# views.py
from myapp.utils import (
get_employee_sb7_data,
)
...
###### Senior ######
#login_required
#group_required("user_senior")
def senior(request):
# Additional Logic / Getting Other Models here
# Execute stored procedure to get data for user
user_number = request.user.user_no
results = get_employee_sb7_data("production_db", user_number, window)
if not results:
return render(request, "users/senior_not_required.html")
# Additional view stuff
return render(
request,
"users/senior.html",
{
"data": data,
"form": form,
"results": results,
},
)
If I run this test itself with:
pytest my_app/tests/test_integrations.py::SeniorPageTestCase
The tests pass without issue. The browser shows up - the form shows up with the dummy data as we would expect and it all works.
However, if I run:
pytest my_app
All other tests run and pass - but all the tests in this class fail because it's not patching the function.
It tries to call the actual stored procedure (which fails because it's not on the production server yet) and it fails.
Why would it patch correctly when I call that TestCase specifically - but not patch correctly when I just run pytest on the app or project level?
I'm at a loss and not sure how to debug this very well. Any help is appreciated
So what's happening is that your views are imported before you're patching.
Let's first see the working case:
pytest imports the test_integrations file
the test is executed and patch decorator's inner function is run
there is no import of the utils yet and so patch imports and replaces the function
test body is executed, which passes a url to the test client
the test client imports the resolver and in turn it imports the views, which imports the utils.
Since the utils are already patched, everything works fine
If another test case runs first, that also imports the same views, then that import wins and patch cannot replace the import.
Your solution is to reference the same symbol. So in test_integrations.py:
#patch("myapp.views.get_employee_sb7_data")

How to test unmanaged models using pytest-django

In my django project, I have 5 applications, with total 15 models,and all of them are unmanaged. I've written some tests in pytest-django, and when I run them, they fail due to not being able to find tables.
How can I create database entries for all these models so that the tests don't fail?
I was trying to get this to work on Django==4.0.4, pytest-django==4.5.2 and none of the results I could find out there worked for me. This is what I could figure out:
# add this to conftest.py
#pytest.hookimpl(tryfirst=True)
def pytest_runtestloop():
from django.apps import apps
unmanaged_models = []
for app in apps.get_app_configs():
unmanaged_models += [m for m in app.get_models()
if not m._meta.managed]
for m in unmanaged_models:
m._meta.managed = True
It seems intuitive to think that we can achieve what we need by overriding django_db_setup, and that is the solution provided in other answers on SO. However I was not able to get this to work, that might have something to do with changes to the order of execution of these fixtures over the years, I am not sure.
Excerpt from the current pytest-django docs:
pytest-django calls django.setup() automatically. If you want to do anything before this, you have to create a pytest plugin and use the pytest_load_initial_conftests() hook
Read more
You can override the django_db_setup fixture in conftest.py file:
#pytest.fixture(scope="session")
def django_db_setup(django_db_blocker):
with django_db_blocker.unblock():
from django.apps import apps
models_list = apps.get_models()
for model in models_list:
with connection.schema_editor() as schema_editor:
schema_editor.create_model(model)
if model._meta.db_table not in connection.introspection.table_names():
raise ValueError(
"Table `{table_name}` is missing in test database.".format(
table_name=model._meta.db_table
)
)
yield
for model in models_list:
with connection.schema_editor() as schema_editor:
schema_editor.delete_model(model)
This will create the tables for unmanaged models before running tests, and delete those tables after test.

Start and Stop a periodically background Task with Django

I would like to make a bitcoin notification with Django. If managed to have a working Telegram bot that send the bitcoin stat when I ask him to do so. Now I would like him to send me a message if bitcoin reaches a specific value. There are some tutorials with running python script on server but not with Django. I read some answers and descriptions about django channels but couldn't adapt them to my project.
I would like to send, by telegram, a command about the amount and duration. Django would then start a process with these values and values of the channel I'm sending from in the background. If now, within the duration, the amount is reached, Django sends a message back to my channel. This should also be possible for more than one person.
Is these possible to do with Django out of the box, maybe with decorators, or do I need django-channels or something else?
Edit 2018-08-10:
Maybe my code explains a little bit better what I want to do.
import requests
import json
from datetime import datetime
from django.shortcuts import render
from django.http import HttpResponse
from django.conf import settings
from django.views.generic import TemplateView
from django.views.decorators.csrf
import csrf_exempt
class AboutView(TemplateView):
template_name = 'telapi/about.html'
bot_token = settings.BOT_TOKEN
def get_url(method):
return 'https://api.telegram.org/bot{}/{}'.format(bot_token, method)
def process_message(update):
data = {}
data['chat_id'] = update['message']['from']['id']
data['text'] = "I can hear you!"
r = requests.post(get_url('sendMessage'), data=data)
#csrf_exempt
def process_update(request, r_bot_token):
''' Method that is called from telegram-bot'''
if request.method == 'POST' and r_bot_token == bot_token:
update = json.loads(request.body.decode('utf-8'))
if 'message' in update:
if update['message']['text'] == 'give me news':
new_bitcoin_price(update)
else:
process_message(update)
return HttpResponse(status=200)
bitconin_api_uri = 'https://api.coinmarketcap.com/v2/ticker/1/?convert=EUR'
# response = requests.get(bitconin_api_uri)
def get_latest_bitcoin_price():
response = requests.get(bitconin_api_uri)
response_json = response.json()
euro_price = float(response_json['data']['quotes']['EUR']['price'])
timestamp = int(response_json['metadata']['timestamp'])
date = datetime.fromtimestamp(timestamp).strftime('%Y-%m-%d %H:%M:%S')
return euro_price, date
def new_bitcoin_price(update):
data = {}
data['chat_id'] = update['message']['from']['id']
euro_price, date = get_latest_bitcoin_price()
data['text'] = "Aktuel ({}) beträgt der Preis {:.2f}€".format(
date, euro_price)
r = requests.post(get_url('sendMessage'), data=data)
Edit 2018-08-13:
I think the solution would be celery-beat and channels. Does anyone know a good tutorial?
One of my teammates uses django-celery-beat, that is available at https://github.com/celery/django-celery-beat to do this and he gave me some excellent feedback from it. You can schedule the celery tasks using the crontab syntax.
I had same issue, there are several typical approaches: Celery, Django-Channels, etc.
But you can avoid them all with simple approach: https://docs.djangoproject.com/en/2.1/howto/custom-management-commands/
I have used django commands in my project to run periodically tasks to rebuild users statistics:
Implement yourself application command, for example your application name is myapp and you have placed my_periodic_task.py in myapp/management/commands folder, so you can run your task once by typing python manage.py my_periodic_task
place beside manage.py file new file for example background.py with same code:
-
import os
from subprocess import call
BASE = os.path.dirname(__file__)
MANAGE_BASE = os.path.join(BASE, 'manage.py')
while True:
sleep(YOUR_TIMEOUT)
call(['python', MANAGE_BASE , 'my_periodic_task'])
Run your server for example: python background.py & python manage.py runserver 0.0.0.0:8000

Avoiding circular imports in Django Models (Config class)

I've created a Configuration model in django so that the site admin can change some settings on the fly, however some of the models are reliant on these configurations. I'm using Django 2.0.2 and Python 3.6.4.
I created a config.py file in the same directory as models.py.
Let me paracode (paraphase the code? Real Enum has many more options):
# models.py
from .config import *
class Configuration(models.Model):
starting_money = models.IntegerField(default=1000)
class Person(models.Model):
funds = models.IntegarField(default=getConfig(ConfigData.STARTING_MONEY))
# config.py
from .models import Configuration
class ConfigData(Enum):
STARTING_MONEY = 1
def getConfig(data):
if not isinstance(data, ConfigData):
raise TypeError(f"{data} is not a valid configuration type")
try:
config = Configuration.objects.get_or_create()
except Configuration.MultipleObjectsReturned:
# Cleans database in case multiple configurations exist.
Configuration.objects.exclude(Configuration.objects.first()).delete()
return getConfig(data)
if data is ConfigData.MAXIMUM_STAKE:
return config.max_stake
How can I do this without an import error? I've tried absolute imports
You can postpone loading the models.py by loading it in the getConfig(data) function, as a result we no longer need models.py at the time we load config.py:
# config.py (no import in the head)
class ConfigData(Enum):
STARTING_MONEY = 1
def getConfig(data):
from .models import Configuration
if not isinstance(data, ConfigData):
raise TypeError(f"{data} is not a valid configuration type")
try:
config = Configuration.objects.get_or_create()
except Configuration.MultipleObjectsReturned:
# Cleans database in case multiple configurations exist.
Configuration.objects.exclude(Configuration.objects.first()).delete()
return getConfig(data)
if data is ConfigData.MAXIMUM_STAKE:
return config.max_stake
We thus do not load models.py in the config.py. We only check if it is loaded (and load it if not) when we actually execute the getConfig function, which is later in the process.
Willem Van Onsem's solution is a good one. I have a different approach which I have used for circular model dependencies using django's Applications registry. I post it here as an alternate solution, in part because I'd like feedback from more experienced python coders as to whether or not there are problems with this approach.
In a utility module, define the following method:
from django.apps import apps as django_apps
def model_by_name(app_name, model_name):
return django_apps.get_app_config(app_name).get_model(model_name)
Then in your getConfig, omit the import and replace the line
config = Configuration.objects.get_or_create()
with the following:
config_class = model_by_name(APP_NAME, 'Configuration')
config = config_class.objects.get_or_create()