django migrate from URLField to foreign key without south - django

I have a model that have a URLField field and I need to make a migration that turns this field into a foreign key, where the string is a unique field of the other object, and, if the object does not exist create it.
for example, turn this:
class Event_UserVisit(Event_Base):
dest_url = models.URLField(max_length=1000)
into this:
class Event_UserVisit(Event_Base):
dest_url = models.ForeignKey(Page)
I've never done a manual migration like this and didn't find any tutorial or instructions to do something like this.
obviously doing a naive migration return errors like this:
django.db.utils.ProgrammingError: column "source_url_id" cannot be cast automatically to type integer
what's the best approach to do it?
note: I need to do this on a production db with lots of data, so I can't have long down time and can't lose any data.
thanks! :)

I don't think you can do this at the ORM level in one go (unless someone corrects me) You would need create a new FK maybe dest_url2 run migrations, then write a script to migrate the data. Next,delete dest_url again run migrations. Then rename dest_url2 to dest_url Django will detect the name change here.
However, I don't understand why you are linking an FK ID on page to a field called dest_url. A PK in Page should not be a 1000 max URL! It has no order and would make indexing hard and slow down your app. It would make more sense do have...
class Event_UserVisit(Event_Base):
page = models.ForeignKey(Page, related_name='eventvisits')
The I assume Page looks something like this...
class Page(models.Model):
dest_url = models.URLField(max_length=1000)

Related

unique_together does not replace primary key

In my Django app, I want to insert a record with a composite primary key. Apparently this should be possible by making use of "unique_together". I'm quite sure this code was working in the past, but for some reason it does not seem to be working now. This code used to run on a Linux VM, and now I'm hosting it in Google App Engine. However I don't see how this can be the cause for this error.
class TermsAndConditionsDocument(models.Model):
organization = models.ForeignKey(Organization, on_delete=models.CASCADE, verbose_name=_("Organization"))
language = models.CharField(_('Language'),choices=LANGUAGE_CHOICES, max_length=5, help_text=_("The language of the content."))
content = models.TextField()
class Meta:
unique_together = ('organization', 'language')
The error:
IntegrityError at /transactions/settings/terms_and_conditions
null value in column "id" violates not-null constraint
DETAIL: Failing row contains (null, nl-BE, <p>B</p>, 10).
According to what I've read, using "unique_together" should cause Django to not need or include an ID as primary key. I checked the database, and the ID field DOES exist. I do not understand where the database constraint and the ID field are still coming from?
Apparently, as pointed out in the comments, a primary key "id" field is always added, even if you don't need it. It's supposed to get out of your way, so you don't even notice its existence. In my case, it required me to give it a value when I created a new record, which is not how things are supposed to work.
A while back I migrated this database from one Postgres database to another Postgres database. I used an SQL dump and load method for this. Some sequences seem to have been lost during that migration.
Because there are no sequences, some fields now lacked autoincrement capabilities, explaining the IntegrityError on insertion.
In order to fix this, I did the following:
1) Export the current data:
manage.py dumpdata > data.json
2) Drop your database and create a new empty one.
3) Run database migrations:
manage.py migrate
4) Load the data again, excluding some default data already recreated by Django.
manage.py loaddata --exclude auth.permission --exclude contenttypes data.json
This procedure seems to have recreated the sequences while also keeping the data.
The unique_together only creates a DB constraint (https://docs.djangoproject.com/en/2.2/ref/models/options/#unique-together).
You could create a custom primary key with the option primary_key https://docs.djangoproject.com/en/2.2/ref/models/fields/#django.db.models.Field.primary_key but you could only do that for one field.
But I suggest to just keep the auto increment id field, this works better with Django.
For the error are you saving a model? or doing a raw import?

Error database query using ManyToManyField on model

I'm fairly new to django web development. And I got an error whereby I try to change a 'post' under admin url - so localhost:8080/admin. I'm able to create it successfully but when I try to click the post that I had just added. I'm getting this error:
Exception Type: DatabaseError Exception Value: This query is not
supported by the database.
And this is the code that I know is 'messing' with this query:
#Post is an abstract class
class BlogPost(Post):
...
translators = models.ManyToManyField(Staff, related_name='translators')
photographers = models.ManyToManyField(Staff, related_name='photographers')
authors = models.ManyToManyField(Staff, related_name='authors')
...
To explain what is going on with this blog post - it can have multiple 'owners'/people that contributed to this post and thus the decision using ManyToManyField. And vice-versa with the 'Staff' member - the type of 'member' can have multiple ownership on multiple posts (Let me know if this logic doesn't make any sense because it does to me).
I'm using mongodb for the database, django 1.5.11 and I have installed djangotoolbox. I've tried the following solutions with adding a relationship to BlogPost as shown below:
Class Staff(Member):
...
staff_posts = models.ManyToManyField(BlogPost, related_name="staff_posts")
...
But I'm getting an error on 'cannot import BlogPost'. I tried figuring out the reason of this error and I don't think that I have a circular dependance - after checking all of the files, there's no circular dependance.
MongoDB (or mongoengine, which I'm guessing you're using) doesn't support joins, so the typical way to model many-to-many relations in a relational database has to be implemented some other way.
One way is to use a ReferenceField inside a ListField. It might look like this (not tested):
class BlogPost(Post):
authors = models.ListField(models.ReferenceField(Staff))
...
Also see these answers:
https://stackoverflow.com/a/18747306/98057
https://stackoverflow.com/a/25568877/98057
Just to put it out there, I'm not real familiar with MongoDB.
However, I don't believe you need to define a ManyToManyField on your Staff class. You already have a ManyToMany defined in your BlogPost, having it defined in one class file is all that is required. (At least for MySQL).

Data not modified during South datamigration

I am doing a migration that adds a referral code to my custom user model. The field I've added looks like this:
refer_code = models.CharField(max_length=10, default=UUID_10)
In my datamigration I've added the following:
for u in orm['app.User'].objects.all():
u.refer_code = UUID_10()
print u.name + ': ' + u.refer_code
u.save()
However, when I perform the migrations, all of my users have the same code. Different codes for each user are spat out from the print command, so it's not a problem with my function.
Weirdly, if I roll the datamigration back (it has no backwards method) as well as the previous migration that added the field, and re-apply them, not only does every model have the same data in, it exactly has the same data in as before.
There are other parts of the datamigration (adding objects to the database with get_or_create) that are working fine.
What on earth is going on?
I think your function was called rather than new field was created. So try to do two migrations. First add new field with default value. And second fill that field.

Before syncdb, delete field from standard Django model

This is a follow-up question on Delete field from standard Django model. In short: a field can be dynamically deleted from a model that is already created, in this case the field User.email . So field email would be deleted from User without changing the code for User. See code below for example.
I can dynamically delete a a field from a model(1), but that happens when the server starts and is undone when it exists. Since syncdb doesn't require the server to be running, and generally seems to ignore the deletion code (somehow), this approach doesn't prevent the field from appearing in the database(2).
Is there a way to do delete the field from the model (without changing the file it's in, as it's a Django model), in a way that also makes it not appear in the database?
Thanks in advance!
Mark
EDIT: I problem is not that I am deleting the text "m = models.IntegerField()" from the model file and want the field removed from the database. The problem is that I am using the code below to remove a field from a model that has already been declared in another file. I do not think that creating a migration with South for every time I run syncdb is a solution(3).
Additional information:
1) Currently, the code is in models.py, but I suppose Where to put Django startup code? works.
2) I can delete it on post_syncdb signal with custom query, but I hope for something more elegant... Or elegant at all, more accurately.
3) If it even works at all, because obviously syncdb is still seeing the 'removed' field), so I think South would to as it's still somehow there.
This is the code (models.py):
class A(models.Model):
m = models.IntegerField()
for i, f in enumerate(A._meta.fields):
if f.name == 'm':
del A._meta.fields[i]
break
class B(A):
n = models.IntegerField()
for i, f in enumerate(B._meta.fields):
if f.name == 'm':
del B._meta.fields[i]
break
EDIT: I checked (with print) and the deletion code is executed on syncdb. It is executed before tables are created
django does a lot of meta class magic and i would guess that the meta class is responsible for defining the database table to back your model. Subsequently just deleting the field is not enough to alter the generated table.
as several people have pointed out, south is the way to deal with these problems.

Extending django-registration fields

I am trying to extend the fields in django-reg (with First name, last name and contact number). To this end, I have written an app (name=regfields), with the following contents:
http://dpaste.com/596163/
When I run syncdb, I see all the extra fields I have set up on the database, but, when I try and create an account, the extra fields I have given are not stored in my database.
What could be wrong?
Sorry, I am a django 101 user!
The indentation of the save() function in newforms.py looks off. Assuming that is not an error introduced by copying it into dpaste, that method is not a part of RegistrationFormZ.