I'm writing some django apps and I have this setup:
local machine (laptop) that I use for development, with local dev
virtualenv
remote machine VPS (with public address) used for test. I
need to have some end-users testing my app before moving to prod
with test virtualenv
remote machine VPS (with public address, same as
above) used for production with production virtualenv
I use git for versioning.
The idea that I have so far (after reading various tutorials) to manage everything is:
develop on local machine new branch
push branch to git
deploy branch into test virtualenv
test it
test passed, push branch to master and deploy into production
virtualenv
And I have lot of questions about this:
is this a recommended approach?
how can I get the new branch to test virtualenv and not to production? Do I need to have two separate app folders, one for prod and one for test?
How can I then move code from test to prod?
Thanks in advance, I'm a django/git novice so I'm trying to approach it in the best way from start.
It seems almost right to me (but there are many strategies),
I'd make a testing-branch, so you could continue pushing to develop-branch while others are testing the test-branch. Then when it passes the test merge to Master.
(Also, if you want to make your live easier, use fab files to 'pull' on the remote machine.)
Related
Although my specific example involves Django, Docker, and Heroku, I believe these are pretty general testing/QA questions.
I have a dockerized Django app tested in dev with Selenium confirming that my static files are being served correctly from my local folder (EXPECTED_ROOT = '/staticfiles/'). This app is deployed to Heroku and I can see (visually and in the dev tools) that the static files are being pulled in from CloudFront correctly as well. I want to formalize this with the same test I'm using in dev. My first question is related to if/how environment variables are used for tests:
Do I add for example EXPECTED_ROOT = 'https://<somehash>.cloudfront.net/' as an env var to Heroku and use it in the Selenium test?
Also, to run this test in staging I would need to install Firefox in my Docker image like I do in dev. Perhaps this is ok in staging, but in prod I believe I should be aiming for the the smallest image possible. So the question is about differences between staging and prod:
Do I keep Firefox in my staging image, run the tests, and then send
to production a replica of that Dockerfile, but now without firefox?
Any help is appreciated.
The idea of Config Var is to setup configuration variables that differ from environment to environment. Having said that you are in control of the environment and can define what you need.
I personally would use a different approach: create a test that is independent of the environment (for example instead of testing the expected root I would confirm a given DIV ID is found, or some other element).
This would be enough to confirm the test is successful and the functionality works as expected.
The production Dockerfile indeed does not need Selenium and can be different from the one from staging.
I have successfully built several web sites hosted on an Nginx server using Django, uWSGI and virtualenv.
I had never used version control but now I am starting to use Git.
I understand how to create, commit and push branches.
My question is: how to make different Git branches visible at the web address of the site I'm working on?
Do I change the Nginx config file to point somewhere different?
I just updated the dev branch of my project, and of course the site does not reflect the changes.
How can I tell the server to serve the dev branch or the master branch of the project?
I would prefer to avoid a complicated system with different subdomains for different branches — I really just want the simplest thing that will work.
[update] I have found lots of pages that explain complex ways to set up staging servers etc., but I really just want to understand what is going on... there's a giant conceptual hole in my understanding about how the server interacts with a local Git project.
Right now, the Nginx config and the uWSGI config point to a folder like:
/var/www/sitefiles
That is the Django folder (inside it is sitefiles/settings.py etc.).
It is in that folder that I did git init, some commits, branching & pushes.
Does using Git mean that the Nginx and uWSGI config's should point elsewhere?
Its pretty simple goto the path of the project where git is installed and checkout to required banch and touch the wsgi file
git checkout dev
touch project/wsgi.py
or to roll back to master branch
git checkout master
touch project/wsgi.py
I am quite new to programming, and all of my development has been on my local runserver using textmate and terminal. I have written a small app with a few hundred and I'd like to push it to an EC2 server. My only knowledge in terms of 'developing tools' is Django's local runserver, TextMate and Terminal.
What tools or programs should I look into learning to be able to have an effective workflow?Should I be using some sort of IDE over TextMate for this? My main concerns are being able to develop on my local runserver and then painlessly push that to my production server.
As #isbadawi said, use Fabric. It's better than just using the terminal because you can automate things. As far as deployments go, you can simplify it down to: fab -H your.host.com deploy. Inside the file you write commands, a simple one might go:
Cause the server to download the most recent code from SCM
Update the database (syncdb / migrations / what have you)
Cause apache or whatever you're using to reload the configuration
As far as some more general tips go:
If you're using WSGI, put it under source control
Same goes with local settings files, have your deploy script rename them to local_settings.py as part of the build
If you're really going for painless, look into Django hosting services like Gondor or Ep.io. Those will have clients that you can just deploy to pseudo-painlessly, although you will have to change some settings on your side to match theirs better, as there are many many ways to deploy a Django app.
Update: Ep.io is no longer in business as a hosting service. My new go-to is Heroku.
Update 2: I used to link local_settings.py in deployments, but now I'm leaning towards using the DJANGO_SETTINGS_MODULE config variable. See rdegge's "django-skel" settings for a good way to do this.
A DVCS such as git or Mercurial will allow you to develop and test locally, and then push the changes to a remote system for staging and production.
We are starting a web project with Django (and are pretty new to this), and I was wondering what would been the best way to setup a productive development environment.
Here is some info:
We are 2 developers working on the project
We have a development/production server hosted on Webfactional (using an apache server)
The DB (MySQL) is hosted on this server
We are both using Eclipse with Pydev
The project is hosted on a Github repository
So far, we didn't setup any local development server, we are syncing modifications to the server through Github. But it is not really convenient...
We were thinking of setting up apache servers locally, that uses the remote database, and only sync once in a while.
Do you think it would be better?
Do you have other ideas/additional tips?
Thanks!
Don't try and share servers and databases while you're developing. You'll only get confused.
Each developer should have a local copy of MySQL on their own machine, and run the development server as mipadi recommends. Manage db schema changes via South, and keep data in fixtures or use South's data migrations. Each developer should commit to their local version of the git repository, but only push changes to Github when the particular task is complete and working (or, better, use a remote branch so that changes are synched, but only merge back to master when complete).
A good idea is to have a continuous integration server like Hudson/Jenkins that, after each commit to master, runs the tests and - if they pass - builds an up-to-date version of the site, which you can use for functional/user testing.
Edit to add There's no relationship between the development server and any particular database backend. As I recommend above, it is quite simple to install MySQL or PostgreSQL on your local machine, and use the development server against that. I have been working this way for several years, after initially encountering some of the issues you were worried about when switching between sqlite3 and the production MySQL db.
Django has its own development server that you can use for local testing.
I'm new to django and my very first project is my blog. I wonder how django developers who use pydev normally synchronize with remote hosting server, updating their sites?
I also would like to know, how do you combine usage of git with a django project? Should I just make a repository for the entire project?
At my company we've got an entire git repository for each project, including the Django sources that are put in the PYTHONPATH for each project, making Django versions project dependant. The folder structure is something like:
/.git
/projectname/app1
/projectname/app2
/projectname/manage.py
/django-lib/django/...
As django-lib is not a Python module, we include both / and /django-lib in the PYTHONPATH. If your project is becoming large, you might want to consider using git submodules on your apps.
We've also setup several servers to support the developers. There's a testing server running a central testing database and a setup including Apache with WSGI to make testing on a real server possible, which sometimes is a bit different then the local manage.py the developers use before committing their changes.
The testing server is updated with the master branch of our git repository. We've made several scripts to allow all developers to do this without letting them login to the server via SSH, but that is just during pre-release. After release, that server will become our staging server, and we'll remove all scripts from it to make it just like our production server.
Every developer has setup their local project to make sure that it communicates with the central testing database, containing several test data. I myself push my changes from the commandline, but you could also use EGit for this.
When we've got a release, we put it in a separate branch, called 'release' (obviously) and the production server will pull only from that branch. This is done via SSH, but I don't really know how your server setup looks like, so I guess that that last step is entirely up to you.
I hope that this has helped you a bit. I won't say that this is the best workflow possible, but it works for us and you should figure out what works for you.
Most experienced Django developers use pip(or distribute) and virtualenv deal with all the python packages you might need for your Django projects (including Django itself).
Personally, all I keep in my projects git repository is a bunch of segregated requirements lists generated by pip :
. ~/Dev/environs/$PROJECT_NAME/bin/activate
pip freeze > ./docs/requirements/main.list
I'm fairly sure most django developers would be familiar with Fabric, which I use for :
streamlining local interaction with git and,
pushing to our central repository,
pulling from our production or test server
touching the wsgi on the relevant server
and pretty much any other kind of task you might find yourself using ssh terminal session for.
For those cases where I need to make changes to someone elses django application in order to make it work or suit our purposes, I :
fork it on github,
clone from my forked repo
make the changes
push it up to my own repo
and provide merge requests to the original repo owner
This way, i have a repo where i can use pip requirement lists to keep pulling from until the original application owner gets their own repo updated.