Django model migrations with South: how-to migrate ForeignKey relation to ManyToMany

Yesterday I had an non-trivial for newbie South user issue: I need to migrate a big database table with existing FK relations to M2M relations and keep in safety all of existing data during migration.

So, I hadn't figure out in documentation how I can do it without handmade migration code in forward method. Below I described how I've fixed it:

1. Here is my initial model (sample), my application called app1

class TestData(models.Model):
    field1 = models.CharField(max_length=200)
    field2 = models.CharField(max_length=200)


class Knight(models.Model):
    name = models.CharField(max_length=100)
    additional_field_new_name = models.CharField(\
        max_length=155, default='')
    data = models.ForeignKey(TestData)

2. I've added new column data_new to Knight model with M2M relation to TestData:

data_new = models.ManyToManyField(TestData, \
        related_name='testdata_info')

3. Start schema migration:

python manage.py schemamigration app1 --auto

4. Change forward method in our new migration, code after creation of new column/proxy model:

for obj in orm.Knight.objects.all():
    obj.data_new.add(obj.data)
    obj.save()

5. Make migration:

python manage.py migrate app1

6. Remove column data from model and make migration; rename m2m column data_new to data and start schema migration

# remove column "data" from model
python manage.py schemamigration app1 --auto
python manage.py migrate app1

# rename m2m column from "data_new" to "data"
python manage.py schemamigration app1 --auto

7. At this point I need to keep all existing data in proxy table but South won't do that for me. So, I need to change forward method of migration (actually I need to replace generated code to following):

db.delete_unique('app1_knight_data_new', ['knight_id', 'testdata_id'])

db.rename_table('app1_knight_data_new', 'app1_knight_data')
db.create_unique('app1_knight_data', ['knight_id', 'testdata_id'])

Update: for Postgres (and I assume for MySQL with InnoDB db storage too) order of above commands is important: you have to delete old unique constraint before renaming of the proxy table.

8. After that just make migration:

python manage.py migrate app1

So, we've done and everything in safety.