I am trying to add some default/fixed values to postgres database and came across fixtures.
My model is like this
app/models.py
class Category(models.Model):
category = models.CharField(max_length=255)
def __str__(self):
return self.category
app/fixtures/category.json
[
{
"model": "core.category",
"pk": 1,
"fields": {
"category": "foo"
}
]
However, I am getting the following error when I run manage.py dumpdata
[CommandError: Unable to serialize database: cursor "_django_curs_139823939079496_sync_1" does not exist
It looks like you have json file already (serialize, means that your django models data has been translated already to json format) you just need to to migrate your data into a new database(you say postgres), try to run these few lines of codes NOTE: I used python3 because I am using macbook pro:-
python3 manage.py migrate
OR
python3 manage.py migrate --run -syncdb
Then, enter your django shell and delete all contenttypes:
>>> from django.contrib.contenttypes.models import ContentType
>>> c = ContentType.objects.all()
>>> c.delete()
(55, {'auth.Permission': 44, 'contenttypes.ContentType': 11})
>>> exit()
python3 manage.py loaddata datadump.json
After being notified that all contenttypes has been deleted, You need to ensure that those data will be displayed in your templates. Simply by loading them from from your json file as shown above.*
Related
I currently have a database using Heroku and want to migrate it to AWS, where both use PostgresSQL. So, after some digging on how to get it done I followed the steps as on this youtube video.
I initially ran python manage.py dumpdata > dumpdata.json with my Heroku database credentials in Django.
Afterwards, I changed my database credentials in settings.py to the AWS database credentials, and ran python manage.py migrate --run-syncdb which worked successfully.
And then I ran the code python manage.py loaddata dumpdata.json, when where I was thrown an error.
The following error came up:
django.db.utils.IntegrityError: Problem installing fixture 'C:\Users\Acer\Desktop\Web Development\Proffesional\eblossom\eblossom\eblossom\dumpdata.json': Could not load contenttypes.ContentType(pk=9): duplicate key value violates unique constraint "django_content_type_app_label_model_76bd3d3b_uniq"
DETAIL: Key (app_label, model)=(user, profile) already exists.
I don't understand what has gone wrong over here, the site is working perfectly fine all this time with no database compilation error, but now when I try to migrate it to AWS, I am thrown with this problem.
Just in case my models.py:
class Profile (models.Model):
user = models.OneToOneField(User, on_delete=models.CASCADE)
mobile_number = models.CharField(max_length=12, null=True)
guest = models.BooleanField(default=False)
def __str__(self):
return f"{self.user}"
I do this all the time between databases, local and remote, postgres or sqlite.
In order to achieve that, do the following steps in order :
# Start the same as you did on local database
python manage.py dumpdata > db.json
# push this file to your production/server location
# and delete or start with a fresh new database
python manage.py migrate
python manage.py shell
# Enter the following in the shell
from django.contrib.contenttypes.models import ContentType
ContentType.objects.all().delete()
# Exit shell and run following command
python manage.py loaddata db.json
I am trying to put values to my django database.
My app_name/models.py looks like:
from django.db import models
# Create your models here.
class Language(models.Model):
id = models.AutoField(primary_key=True)
code = models.TextField(max_length=14)
I ran this commands:
python3 manage.py makemigrations app_name
python3 manage.py migrate app_name
After that I starting importing values to my database. So I run this command:
python3 manage.py shell
and put this code to the shell:
from app_name.models import *
with open('lang.txt', 'r') as file:
for i in file:
a = Language.objects.create(code=i.strip())
File lang.txt contains 309 lines with language codes. The file looks like:
en
fr
af
de
...
When I run this code in the manage.py shell Django created 309 Language objects. But when I try type random id in the shell -> Language(id=1).code it return only empty string - ''.
I tried use save() method too but still same problem.
from definitions.models import *
with open('lang.txt', 'r') as file:
for i in file:
a = Language(code=i.strip())
a.save()
Django version 1.10.1
So my question is where can be a problem?
This is happening because you are not fetching Language objects properly.
When you do Language(id=1).code, You are instantiating Language object with no value in given to code field.
In django, to fetch a object, you do Model.objects.get(field=value). So your code becomes:
Language.objects.get(id=1).code
So I'm following the DOCS and just want to make sure I'm understanding correctly.
https://docs.djangoproject.com/en/1.10/ref/contrib/gis/install/postgis/
Do I just create a file called migrations.py with:
from django.contrib.postgres.operations import CreateExtension
from django.db import migrations
class Migration(migrations.Migration):
operations = [
CreateExtension('postgis'),
...
]
and drop it in my project directory? And then run python manage.py makemigrations ?
Still the better way ist to create extension directly by making a sql query:
CREATE EXTENSION postgis;
After that you just have to navigate to your project-root (there is a manage.py file inside) and run python manage.py migrate (since django 1.9 - before v.1.9 first run python manage.py makemigrations and after that python manage.py migrate)
But if you want to use your code, you have to add it to "models.py".
This is the file called by "python manage.py migrate"
So your models.py looks like:
from django.contrib.gis.db import models
from django.contrib.postgres.operations import CreateExtension
from django.db import migrations
class Migration(migrations.Migration):
operations = [
CreateExtension('postgis'),
]
class model1(models.Model):
geom = models.GeometryField(srid=4326,blank=True,null=True)
name = models.TextField(null=True)
I tried Googling this problem, but I mostly saw answers that were from versions of Django where South was needed for migration and/or syncdb was still a thing. The ones that were relevant matched the symptom, but not the cause.
I started with the default project/app that Django creates when you do django-admin startproject project-name and django-admin startapp products.
models.py
from django.db import models
from django.contrib.auth.models import User
from django.core.exceptions import ObjectDoesNotExist
# Create your models here.
class Artist(models.Model):
user = models.OneToOneField(User, null=True, blank=True)
name = models.CharField(max_length=50)
def __str__(self): return "<Artist: %s>"%self.name
class Category(models.Model):
artists = models.ManyToManyField(Artist)
name = models.CharField(max_length=50)
desc = models.CharField(max_length=200)
My process:
python manage.py flush to clear the database
rm -rf products\migrations to delete prior migrations
python manage.py migrate
Output:
Operations to perform:
Apply all migrations: sessions, contenttypes, auth, admin
Running migrations:
No migrations to apply.
python manage.py makemigrations products
Output:
Migrations for 'products':
0001_initial.py:
- Create model Artist
- Create model Category
python manage.py migrate
Output:
Operations to perform:
Apply all migrations: contenttypes, admin, sessions, products, auth
Running migrations:
No migrations to apply.
At this point, I'm confused. Aren't there migrations to be applied, and yet there aren't any? I press on nonetheless.
python manage.py shell
In this shell, I enter these commands:
>>> from products.models import *
>>> artist = Artist(name="foo")
>>> artist.save()
>>> categ = Category(name="bar", desc="baz")
>>> categ.save()
>>> categ.artists.add(artist)
At this point, I get a huge error traceback. The problem seems to be this:
django.db.utils.OperationalError: no such table: products_category_artists
I see the same error if I try to use the built-in admin site.
My guess is that the problem is that migrations aren't actually being applied or that a particular table isn't being created, but I don't know how to fix either of these problems.
There is a table generated by django called django_migrations which keeps track of what migrations have been applied. If you delete your migrations, re-generate them and try to migrate without deleting the records from the table than django will think it already applied them. You should never delete your migrations, it will cause django to get confused.
If you are actively developing and want to skip the entire migrations system you can, but once you start using migrations, never delete them. Here is what I use while developing a new project:
dropdb mydb && createdb mydb && python manage.py migrate --run-syncdb && python manage.py loaddata initial
First, it drops the database and all data. Then it creates an empty one. The --run-syncdb generates the schema and the loaddata loads data from a fixtures file.
So, if you are still developing and can delete all your data and move what you care about to a fixtures file than you can delete all your migration folders and run the command above each time you change your model.
I want to load a fixture for my selenium testing. Using fixtures was successful in my initial tests, so I know I am capable of loading the fixtures in my test setup and using them in my tests. I have attempted several approaches.
First, I generated fixtures specific to the models I was testing using dumpdata. An example is below:
python manage.py dumpdata protocols.Step --indent=2 > functional_tests/fixtures/step.json
When used in my test as so:
class SignInTest(FunctionalTest):
fixtures = ['admin_user.json', 'protocol.json', 'step.json',
'protocol_element.json']
def test_login_and_view_user_data(self):
...
I get error:
django.db.utils.IntegrityError: Problem installing fixtures: The row in table 'protocols_protocolelement' with primary key '37' has an invalid foreign key: protocols_protocolelement.element_content_type_id contains a value '41' that does not have a corresponding value in django_content_type.id.
Second attempt involved using all the test data in my tables, but excluding contenttypes:
python manage.py dumpdata --indent=2 -e contenttypes > functional_tests/fixtures/initial_data.json
class SignInTest(FunctionalTest):
fixtures = ['initial_data.json']
...
Getting the error:
django.db.utils.OperationalError: Problem installing fixture '.../mike/mike/functional_tests/fixtures/initial_data.json': Could not load auth.Permission(pk=103): no such table: auth_permission
Next, I tried using natural to show the natural keys:
python manage.py dumpdata --natural -e contenttypes -e auth.Permission --indent=2 > functional_tests/fixtures/initial_data2.json
Only to get the error:
django.db.utils.OperationalError: Problem installing fixture '.../mike/mike/functional_tests/fixtures/initial_data.json': Could not load auth.User(pk=1): no such table: auth_user
Noticing natural was depreciated I tried --natural-foreign and wanted to include user and permission models (I need contenttypes for my models anyway):
python manage.py dumpdata --natural-foreign --indent=2 > functional_tests/fixtures/initial_data3.json
Only to get the error:
django.db.utils.IntegrityError: Problem installing fixture '.../mike/mike/functional_tests/fixtures/initial_data3.json': Could not load contenttypes.ContentType(pk=35): UNIQUE constraint failed: django_content_type.app_label, django_content_type.model
So, any ideas on how to load the fixture so I can run my tests? Is there something simple I'm missing? Thanks!
After some more reading about how Django maintains its own models and such, it is my understanding that Django caches the contenttype, auth.Permission, etc and uses them in testing frameworks (I was using StaticLiveServerTestCase). This means that when I was loading my fixture, it was clashing with the data Django had stored for its own uses causing the integrity error. This is what worked for me:
python manage.py dumpdata -e contenttypes -e admin -e auth.Permission --natural-foreign --indent=2 > functional_tests/fixtures/initial_data4.json
This post has some additional helpful information to help me solve the problem:
Problems with contenttypes when loading a fixture in Django.
All this exercises around dumpdata didn't bring me any result.
But I found this advise:
https://stackoverflow.com/a/3855764/12191031
For some reason reset doesn't work in my code, but it was enough load fixture before every test and flush DB after each one.
So, my solution is this:
from django.core import management
from django.contrib.staticfiles.testing import StaticLiveServerTestCase
from selenium import webdriver
class login_page_test(StaticLiveServerTestCase):
#classmethod
def setUpClass(cls):
cls.selenium = webdriver.Firefox()
cls.selenium.implicitly_wait(5)
super(login_page_test, cls).setUpClass()
#classmethod
def tearDownClass(cls):
cls.selenium.quit()
super(login_page_test, cls).tearDownClass()
def setUp(self):
# for every test I load fixture
management.call_command("loaddata", "user-data.yaml", verbosity=0)
super(login_page_test, self).setUp()
def tearDown(self):
# after each test I flush all DB
management.call_command("flush", verbosity=0, interactive=False)
super(login_page_test, self).setUp()
I'm happy )))