I have a app that is deployed to Heroku, and I'd like to be able to run the test suite post-deployment on the target environment. I am using the Heroku Postgres add-on, which means that I have access to a single database only. I have no rights to create new databases, which in turn means that the standard Django test command fails, as it can't create the test_* database.
$ heroku run python manage.py test
Running `python manage.py test` attached to terminal... up, run.9362
Creating test database for alias 'default'...
Got an error creating the test database: permission denied to create database
Is there any way around this?
Turns out I was in the wrong. I was not testing what I thought was being tested... Since Heroku's Routing Mesh was sending requests to different servers, the LiveServerTestCase was starting a web server on one machine and Selenium was connecting to other machines altogether.
By updating the Heroku Procfile to:
web: python src/manage.py test --liveserver=0.0.0.0:$PORT
overriding the DATABASES setting to point to the test database, and customizing the test suite runner linked to below (the same idea still holds: override setup_databases so that it only drops/re-creates tables, not the entire database), I was able to run remote tests. But this is even more hacky/painful/inelegant. Still looking for something better! Sorry about the confusion.
(updated answer below)
Here are the steps that worked for me:
Create an additional, free Postgres database using the Heroku toolbelt
heroku addons:add heroku-postgresql:dev
Use the HerokuTestSuiteRunner class which you'll find here.
This custom test runner requires that you define a TEST_DATABASES setting which follows the typical DATABASES format. For instance:
TEST_DATABASES = {
'default': dj_database_url.config(env='TEST_DATABASE_URL')
}
Then, have the TEST_RUNNER setting be a Python path to wherever HerokuTestSuiteRunner can be found.
You should now be able to run Django tests on Heroku using the given database. This is very much a quick hack... Let me know how it could be improved / made less hackish. Enjoy!
(original answer below)
A few relevant solutions have been discussed here. As you can read in the Django docs, "[w]hen using the SQLite database engine, the tests will by default use an in-memory database".
Although this doesn't thoroughly test the database engine you're using on Heroku (I'm still on the lookout for a solution that does that), setting the database engine to SQLite will at least allow you to run your tests.
See the above-linked StackOverflow question for some pointers. There are at least two ways out: testing if 'test' in sys.argv before forcing SQLite as the database engine, or having a dedicated settings file used in testing, which you can then pass to django manage.py test using the --settings option.
Starting with version 1.8, Django now has an option called keepdb, which allows for the same database to be reused during tests.
The --keepdb option can be used to preserve the test database between test runs.
This has the advantage of skipping both the create and destroy actions which can greatly decrease the time to run tests, especially those in a large test suite.
If the test database does not exist, it will be created on the first run and then preserved for each subsequent run.
Any unapplied migrations will also be applied to the test database before running the test suite.
Since it also allows for the test database to exist prior to running tests, you can simply add a new Postgres Heroku instance to your dyno and configure the tests to use that particular database.
Bonus : you can also use the failfast option, which exits as soon as your first test crashes, so that you don't have to wait for all tests to complete.
However, if you are deploying things to Heroku and you are using Heroku Pipelines, an even better option is available : Heroku CI.
Related
I am developing Django Wagtail application on my local machine connected to a local postgres server.
I have a test server and a production server.
However when I develop locally and try upload it there is always some issue with makemigration and migrate e.g. KeyError etc.
What are the best practices of ensuring I do not get run into these issues? What files do I need to port across?
so ill tell you what i do and what most of the companies that i worked as a django developer did and i can tell you by experience that worked pretty well.
First containerize your application, this will make your life much more easy and you will remove external influence in your code, also will get you an easy way to reproduce your environment.
Your Dockerfile should be from some python image and should do 3 basically things:
Install your requirements dependencies
Run the python manage.py migrate --noinput command
Run a http server such as gunicorn with gunicorn -c /gunicorn.py wsgi:application
You ill do the makemigration in your local machine and make sure that everything is working before commit then to the repo.
In your gunicorn.py you ill put your settings to run the app such as the number of CPU, the binding port, the folder that your app is, something like:
import os
import multiprocessing
# Chdir to specified directory before apps loading.
# https://docs.gunicorn.org/en/stable/settings.html#chdir
chdir = '/app/'
# Bind the application on localhost both on ipv6 and ipv4 interfaces.
# https://docs.gunicorn.org/en/stable/settings.html#bind
bind = '0.0.0.0:8000'
Second containerize your other stuff, for example the postgres database, the redis (for cache), a connection pooler for the database depending on the size of your application.
Its highly recommend that you have a step in the pipeline to do tests, they need to run before everything, maybe just after lint
Ok what now? now you need a way to deploy that stuff, the best for that scenario is: pull your image to github registry, and you can add a tag to that for example:
IMAGE_ID=ghcr.io/${{ github.repository_owner }}/$IMAGE_NAME
# Change all uppercase to lowercase
IMAGE_ID=$(echo $IMAGE_ID | tr '[A-Z]' '[a-z]')
docker tag $IMAGE_NAME $IMAGE_ID:staging
docker push $IMAGE_ID:staging
This can be add in a github action in the build step for example.
After having your new code in a new image inside github you just need to update the current one, this can be done by creaaing a script to do it in the server and running that script from github action, is something like:
docker pull ghcr.io/${{ github.repository_owner }}/$IMAGE_NAME
echo 'Restarting Application...'
docker stop {YOUR_CONTAINER} && docker up -d
sudo systemctl restart nginx
echo 'Cleaning old images'
sudo docker system prune -af
You can see that i create the image with a staging tag, you can create a rule in github actions for example to trigger that action when you create a new release for example, and create another action to be trigger in every new commit and build/deploy for a dev tag.
For the migration problem, the first thing is, when your application go live squash every migration to the first one (you can drop the database and all the migration then create the database and run the makemigration command again to reach this), so you can have a clean migration in the server. Never creates unnecessary relation between the tables, prefer always doing cached properties instead of add new columns, use UUID for unique ids, and try to not do breaking changes in the database, its hard but if you plan the database before is not so difficult to do.
Hit me if you have any questions. A lot of the stuff that i said can be done in a lot of other platforms such as gitlab, travis, circle ci, but i use the github action in the example because i think is more simple to picture.
EDIT:
I forgot to tell you to have a cron in your server doing backups of your databases, the migrate command ill apply the changes only after the verification but if something else break the database this can save your life.
I'm a hobby web developer looking to build my first product with the django web framework which I will host on Heroku. Cypress looks like a great modern tool for UI testing which I certainly want to use. In local development I am using it within django testing so that the django staticliveservertestcase runs first - which creates a separate development server process, and a test database, if one hasn't been created already. Within this test I then call a bash script which executes a cypress UI test. See the example below.
Bash Script -
#!/bin/bash
echo $1 # the 1st command line argument
echo $2 # the 2nd command line argument
./node_modules/.bin/cypress run --env baseUrl=$1 --spec $2
Django Test Code -
class ExampleTest(StaticLiveServerTestCase):
def test_view_url(self):
from subprocess import run
exit_code = run([
'./cypress/run.sh',
f'http://{self.server_thread.host}:{self.server_thread.port}',
'cypress/integration/sample_spec.js'
])
self.assertEqual(0, exit_code.returncode)
From my understanding this is the opposite to the standard cypress approach. Normally cypress runs outside of the app independently and simply runs through all the tests which interact with the app. Here the cypress tests are called by the django process. Unless I am mistaken there are obvious advantages to this approach. I can test the UI and whether objects where created in the database after the UI test has run.
Is it possible, and if this how, to do this within Heroku? So Heroku would simply call the test script -
python manage.py test
And I get UI testing and unit testing all done in one go. It would also be nice to see the tests running in interactive mode. Which may be a challenge because Heroku obviously runs everything in a linux container - I don't plan on using Docker incidentally because I think that limits a lot of the features Heroku offers.
I have a django app running on heroku. I would like to run my South migrations before any code that depends on them goes live. After a quick review of current recommended practices I have found two suggested migration procedures.
Recommendation 1
Commit and push all changes
Run heroku run python manage.py migrate <APP_NAME> for each app
This suffers from having a period in between steps 1 and 2 where my code is assuming the latest schema is in place, but the db hasn't yet been updated.
Recommendation 2
commit and push all database changes.
Migrate.
Push all code changes.
This solves the previous problem, but adds a lot more complexity to the deployment process, and some day I will mess this up.
Potential Solution?
It seems that I can avoid the problem in Recommendation 1 and keep my deployment to a single step by utilising a custom post_compile script
that calls python $MANAGE_FILE migrate <APP_NAME> for each of my apps (in dependency order).
I have not seen this recommended anywhere, so my question is twofold. Can you see any potential problem with this approach, and do you have a better method?
If your application can afford some downtime, the easiest way seems to me to
Pause your app using $ heroku maintenance:on
Migrate all the apps at once with heroku run python manage.py migrate
Restart your app: $ heroku maintenance:off
Is it enough or do you have more complex needs ?
I am trying to create a development server from a production server from which I can test out new ideas.
I created a duplicate of my production server's database by dumping it using Postgres' db_dump and then imported the dump into a new database.
I then copied my production django directory and altered all .py files to refer to server_debug. rather than server in my import statements.
Using the admin interface to alter some data works in that only the development server has its data altered.
However, when I then try adding a new field in my models.py in my development server, manage.py syncdb fails to create it.
Is there something I am neglecting that could cause manage.py to refer to my production server rather than my development server?
syncdb doesn't touch tables that already exist. You need to either reset the app (easiest if you don't care about the data), modify the table manually (more of a quick hack) or use a migration app and version your models — South, for example.
Here is my current setup:
GitHub repository, a branch for dev.
myappdev.appspot.com (not real url)
myapp.appspot.com (not real url)
App written on GAE Python 2.7, using django-nonrel
Development is performed on a local dev server. When I'm ready to release to dev, I increment the version, commit, and run "manage.py upload" to the myappdev.appspot.com
Once testing is satisfactory, I merge the changes from dev to main repo. I then run "manage.py upload" to upload the main repo code to the myapp.appspot.com domain.
Is this setup good? Here are a few issues I've run into.
1) I'm new to git, so sometimes I forget to add files, and the commit doesn't notify me. So I deploy code to dev that works, but does not match what is in the dev branch. (This is bad practice).
2) The datastore file in the git repo causes issues. Merging binary files? Is it ok to migrate this file between local machines, or will it get messed up?
3) Should I be using "manage.py upload" for each release to the dev or prod environment, or is there a better way to do this? Heroku looks like it can pull right from GitHub. The way I'm doing it now seems like there is too much room for human error.
Any overall suggestions on how to improve my setup?
Thanks!
I'm on a pretty similar setup, though I'm still runing on py2.5, django-nonrel.
1) I usually use 'git status' or 'git gui' to see if I forgot to check in files.
2) I personally don't check in my datastore. Are you familiar with .gitignore? It's a text file in which you list files for git to ignore when you run 'git status' and other functions. I put in .gaedata as well as .pyc and backup files.
To manage the database I use "python manage.py dumpdata > file" which dumps the database to a json encoded file. Then I can reload it using "python manage.py loaddata".
3) I don't know of any deploy from git. You can probably write a little python script to check whether git is up to date before you deploy. Personally though, I deploy stuff to test to make sure it's working, before I check it in.