Can't insert data, Django primary key doesn't have a default - django

Question

In Django 3.0, I have a set of Models consisting of an abstract BaseData, a Model Data that extends it, and a Model of underlying data FKData that Data foreign-keys to:
# models.py
from django.db import models
class BaseData(models.Model):
class Meta:
abstract = True
class Data(BaseData):
data = models.ForeignKey(
FKData,
on_delete=models.CASCADE,
blank=True,
null=True,
default=None
)
class FKData(models.Model):
text = models.CharField(
help_text='underlying data'
)
The error occurs when I try to INSERT into Data. For example,
mysql> INSERT INTO appname_data (data_id) SELECT data_id FROM appname_othertable
ERROR 1364 (HY000): Field 'basedata_ptr_id' doesn't have a default value
basedata_ptr_id is a Django-generated primary key field that Data has due to being a subclass of BaseData:
mysql> DESCRIBE appname_data;
+-----------------+---------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+---------+------+-----+---------+-------+
| basedata_ptr_id | int(11) | NO | PRI | NULL | |
| data_id | int(11) | YES | MUL | NULL | |
+-----------------+---------+------+-----+---------+-------+
2 rows in set (0.00 sec)
The error sounds like it isn't being set to AUTO INCREMENT the way normal primary keys are?
If I try filling the Data table using the Django shell, I get the same error but with a bigger stack trace that might be useful:
Traceback (most recent call last):
File "/home/username/ProjectName/env/lib/python3.6/site-packages/django/db/backends/utils.py", line 86, in _execute
return self.cursor.execute(sql, params)
File "/home/username/ProjectName/env/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 74, in execute
return self.cursor.execute(query, args)
File "/home/username/ProjectName/env/lib/python3.6/site-packages/MySQLdb/cursors.py", line 209, in execute
res = self._query(query)
File "/home/username/ProjectName/env/lib/python3.6/site-packages/MySQLdb/cursors.py", line 315, in _query
db.query(q)
File "/home/username/ProjectName/env/lib/python3.6/site-packages/MySQLdb/connections.py", line 239, in query
_mysql.connection.query(self, query)
MySQLdb._exceptions.IntegrityError: (1364, "Field 'basedata_ptr_id' doesn't have a default value")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<console>", line 6, in <module>
File "/home/username/ProjectName/env/lib/python3.6/site-packages/django/db/models/base.py", line 746, in save
force_update=force_update, update_fields=update_fields)
File "/home/username/ProjectName/env/lib/python3.6/site-packages/django/db/models/base.py", line 784, in save_base
force_update, using, update_fields,
File "/home/username/ProjectName/env/lib/python3.6/site-packages/django/db/models/base.py", line 887, in _save_table
results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)
File "/home/username/ProjectName/env/lib/python3.6/site-packages/django/db/models/base.py", line 926, in _do_insert
using=using, raw=raw,
File "/home/username/ProjectName/env/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/home/username/ProjectName/env/lib/python3.6/site-packages/django/db/models/query.py", line 1204, in _insert
return query.get_compiler(using=using).execute_sql(returning_fields)
File "/home/username/ProjectName/env/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1391, in execute_sql
cursor.execute(sql, params)
File "/home/username/ProjectName/env/lib/python3.6/site-packages/django/db/backends/utils.py", line 100, in execute
return super().execute(sql, params)
File "/home/username/ProjectName/env/lib/python3.6/site-packages/django/db/backends/utils.py", line 68, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/home/username/ProjectName/env/lib/python3.6/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/home/username/ProjectName/env/lib/python3.6/site-packages/django/db/backends/utils.py", line 86, in _execute
return self.cursor.execute(sql, params)
File "/home/username/ProjectName/env/lib/python3.6/site-packages/django/db/utils.py", line 90, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/home/username/ProjectName/env/lib/python3.6/site-packages/django/db/backends/utils.py", line 86, in _execute
return self.cursor.execute(sql, params)
File "/home/username/ProjectName/env/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 74, in execute
return self.cursor.execute(query, args)
File "/home/username/ProjectName/env/lib/python3.6/site-packages/MySQLdb/cursors.py", line 209, in execute
res = self._query(query)
File "/home/username/ProjectName/env/lib/python3.6/site-packages/MySQLdb/cursors.py", line 315, in _query
db.query(q)
File "/home/username/ProjectName/env/lib/python3.6/site-packages/MySQLdb/connections.py", line 239, in query
_mysql.connection.query(self, query)
django.db.utils.IntegrityError: (1364, "Field 'basedata_ptr_id' doesn't have a default value")
What went wrong, and how do I fix it?
Edit: I've decided to wipe my database and bring all development to a halt while I spend a couple of weeks rebuilding it, all because I overlooked a single line, because I'm using "The framework for perfectionists with deadlines."
Nevertheless, this question deserves a better answer. Based on the comments, I suspect the best way to approach the answer is the following:
Consider two scenarios.
Scenario A
I define BaseData as an abstract class
class BaseData(models.Model):
class Meta:
abstract = True
I run makemigrations to create a migration file
Scenario B
I define BaseData as a non-abstract class
class BaseData(models.Model):
class Meta:
pass
I run makemigrations to create a migration file.
I alter BaseData to be an abstract class
class BaseData(models.Model):
class Meta:
abstract = True
I run makemigrations to create a second migration file.
In principle, the combination of the two migrations from Scenario B should exactly equal the migration from Scenario A. In practice, something is different. The first step in answering this question is determining what's different about the migration directives between the two scenarios. Then, we can formulate a process of manually editing the migration files to solve the problem without wiping the database, for people in the future who have the same problem.