I'm new to Django coming from Rails, and I am finding the Django fixtures (most commonly JSON from what I gather), to be somewhat awkward and unweildy, at least in comparison to the rails fixtures that I am familiar with. I liked being able to embed some ruby code, e.g <%= Time.now %>, or reference other fixtures by name when relating things with foreign keys (so as to avoid having to keep track of ids).
So the question, how do you more experience Django developers build your fixtures. Do you sit down and write the JSON/XML/YAML by hand, or are there other tools available to help similar to what rails gives me? Or is it just easiest to create some data in ./manage.py shell and dump that data to a fixture, instead of writing the fixtures by hand? Or even still, do you find yourself putting a lot of data in your setUp(self) method in your test classes. Maybe writing these things out by hand is less tedious once you have a little more experience?
Django's admin site is great for quickly entering dummy data or initial data. You can then dump that to a json file (or whatever format).
http://docs.djangoproject.com/en/dev/ref/django-admin/#dumpdata-appname-appname-appname-model
django-admin.py dumpdata | pbcopy will dump all the data in json format to your clipboard.
Be careful with dumping contenttypes and auth tables as that may cause problems when loading the fixture back in to a database.
Check out django-dilla. It generates random data for your models, even images. Useful for testing without the tedium of manually entering data into admin.
Simple data dump of all data in project to a Json fixture
python manage.py dumpdata --format=json myapp > /path/to/myapp/fixtures/initial_data.json
Then in tests.py add this to include fixtures:
class ViewTests(TestCase):
# load fixtures
fixtures = ['data2.json']
def setUp(self):
# continue remainder of test code
Related
I spin up a django project. Afterwards, i didn't write models.py but instead I created a database from MySQL command line(independent from django) and created three tables with required columns. Finally i connected my django app with that database successfully. I applied migrations. But now i am confused do i need to write models.py with every field name as in column?
I remember implementing a basic project in which i did write models.py and created database using "python manage.py shell" and then put values using
"from polls.models import Choice, Question"? How do i put data now initially and then using python on some action from UI?
Do i need models.py even for ready made mysql databases?
You do not need to construct models. Some (small) webservers are even completely stateless, and thus do not use a database. But a large part of how Django can help you is based on models.
You can write your own queries, forms, etc. But often by using a ModelForm, Django can for example remove a large amount of boilerplate code. It will make it furthermore less likely that there are mistakes in your code. So although not strictly necessary, the models are usually a keystone in how Django can help you.
You can use the inspectdb [Django-doc] command to inspect the database, and let Django "sketch" the models for you. Usually you will have still some work. Since Django can, for example, not derive that a field is an EmailField, since both a CharField and EmailField look exactly the same at the database side.
You do not need to use inspectdb however. You can construct your own models. If you create your own models, but these exist already at the database side, you might want to set managed = False [Django-doc] in the Meta of your model, to prevent Django from constructing migrations.
EDIT: The issue described below was caused not by bad workflow but by an apparent bug when loading fixtures. One of my apps had the fixture initial_data.json. The testing framework loads the fixture before performing the necessary migrations. (FWIW, I'm using Django 1.7 + python3.4) This issue is described here. (My workaround: rename the fixture to data.json.)
I'll leave the rest of the post intact in case this helps someone else in the future.
I'm trying to use Django's built-in tests to rapidly test my Django models during development. Unfortunately, when I try this, I get the error:
psycopg2.ProgrammingError: relation "app_relation" does not exist
The workflow I was imagining was
Define a few model fields (possibly across apps)
Test logic using Django tests
Fix logic errors, modify fields, and iterate this process.
This way, I can build my models incrementally without creating a large number of migrations. Migrations create headaches for me because I frequently add, remove, and rename fields or models as I'm validating my logic.
For example, my model has demographic fields, and I'm not sure whether I should keep an male_under_18 field or split it up into male_under_5, male_5_to_9, male_10_to_15, and male_16_to_18 granularity.
It sure would be nice to verify the decision using tests.py before making my migrations.
My understanding was that Django's manage.py test myapp created a database independent of the development database, and so doesn't require that my development database match the current schema defined by my models.
If the workflow above is impossible (or downright silly), I'm open to other ways to solving my problem.
Related question: django unit tests without a db. (Doesn't work because I want to test the DB!)
I am building a web interface for a database at my school. The database will hold our school's versions of academic standards.
When you build a site using django, does it create a clean database? For example, wysiwyg website builders like dreamweaver create ugly html and css code on the backend. I would hate to see a similar degree of auto-generated cruft in my database.
Should I create the database myself and then build a django site to access the database, or go ahead and let django create the database?
Under any simple to moderately complex application, Django will do a fine job creating the database for you. I've yet to run into any issues with what it's made.
I would suggest that you use South to handle your table migrations. And use virtualenv and pip to set up and maintain your Django environment.
You can use the sqlall predicate of manage.py to see the exact SQL that will be executed in order to generate the database.
Obviously django needs database tables for its basic functionality (contrib.apps).
Sure, you don't have to use them, but generally you want to use a least contrib.auth and some other bundled apps:
Each of these applications makes use
of at least one database table,
though, so we need to create the
tables in the database before we can
use them.
I any case you can't and shouldn't compare it to ugly html code generated by dreamweaver or word.
On a more abstract level:
One of key concepts of a web framework (following the mvc pattern) is that you define models which are "translated" (mapped) by the framework into database tables.
A model is the single, definitive
source of data about your data. It
contains the essential fields and
behaviors of the data you’re storing.
Generally, each model maps to a single
database table.
If you want to create the whole database scheme by hand you totally missed the point of using a web framework. In most cases you simply don't need to write sql manually. You define your classes and then you can query your objects using the builtin orm.
I need to populate sample data for Django Image/File fields in automated python code.
I want to initialize Django ImageFields and FileFields with some sample images and data files that are stored in my "site_media" directory of my Django project.
What would the code look like? This is not for testing, I want to autopopulate sample data into my Django website user's accounts (including this sample Imgae/File media.)
This should be done in python code without using fixtures.
If I understand you correctly you want to use fixtures, basically a JSON formatted file that holds data that will be put into the project database. They can be executed through the django-admin command like :
django-admin.py loaddata mydata.json
see http://docs.djangoproject.com/en/1.2/ref/django-admin/#loaddata-fixture-fixture
What would be the best way to port an existing Drupal site to a Django application?
I have around 500 pages (mostly books module) and around 50 blog posts. I'm not using any 3rd party modules.
I would like to keep the current URLS (for SEO purposes) and migrate database to Django. I will create a simple blog application, so migrating blog posts should be ok. What would be the best way to serve 500+ pages with Django? I would like to use Admin to edit/add new pages.
All Django development is similar, and yours will fit the pattern.
Define the Django model for your books and blog posts.
Unit test that model using Django's built-in testing capabilities.
Write some small utilities to load your legacy data into Django. At this point, you'll realize that your Django model isn't perfect. Good. Fix it. Fix the tests. Redo the loads.
Configure the default admin interface to your model. At this point, you'll spend time tweaking the admin interface. You'll realize your data model is wrong. Which is a good thing. Fix your model. Fix your tests. Fix your loads.
Now that your data is correct, you can create templates from your legacy pages.
Create URL mappings and view functions to populate the templates from the data model.
Take the time to get the data model right. It really matters, because everything else is very simple if your data model is solid.
It may be possible to write Django models which work with the legacy database (I've done this in the past; see docs on manage.py inspectdb).
However, I'd follow advice above and design a clean database using Django conventions, and then migrate the data over. I usually write migration scripts which write to the new database through Django and read the old one using the raw Python DB APIs (while it is possible to tie Django to multiple databases simultaneously, too).
I also suggest taking a look at the available blogging apps for Django. If the one included in Pinax suits your need, go ahead and use Pinax as a starting point.
S.Lott answer is still valid after years, I try to complete the analysis with the tools and format to do the job.
There are many Drupal export tools out of there by now but with the very same request I go for Views Datasource choosing JSON as format. This module is very solid and available for the last version of Drupal. The JSON format is very fast in both parsing and encoding and it's easy to read and very Python-friendly (import json).
Using Views Datasource you can create a node view sorted by node id (nid), show a limited number of elements per page, configure a view path, add to it a filter identifier and pass to it the nid to read all elements until you get an empty JSON response.
When importing in Django you have a wide set of tools as well, starting from loaddata to load fixtures. Views Datasource exported JSON but it's not formatted as Django expects fixtures: you can write a custom admin command to do the import, where you can have the full control of the import flow.
You can start your command passing a nid=0 as argument and then let the procedure read, import and then fetch data from the next page passing simply the last nid read in the previous HTTP request. You can even restrict access to the path on view but you need additional configuration on the import side.
Regarding performance, just for example I parsed and imported 15.000+ nodes in less than 10 minutes via a Django 1.8 custom admin command on an 8 core / 8 GB Linux virtual machine and PostgreSQL as DBMS, logging success and error information into a custom model for each node.
These are the basics for import/export between these two platform, for detailed information I described all the major steps for export from Drupal and then import to Django in this guide.