Thursday, December 23, 2010

Django model migrations with South: how-to migrate ForeignKey relation to ManyToMany

Yesterday I had an non-trivial for newbie South user issue: I need to migrate a big database table with existing FK relations to M2M relations and keep in safety all of existing data during migration.

So, I hadn't figure out in documentation how I can do it without handmade migration code in forward method. Below I described how I've fixed it:

1. Here is my initial model (sample), my application called app1
class TestData(models.Model):
    field1 = models.CharField(max_length=200)
    field2 = models.CharField(max_length=200)

class Knight(models.Model):
    name = models.CharField(max_length=100)
    additional_field_new_name = models.CharField(\
        max_length=155, default='')
    data = models.ForeignKey(TestData)

2. I've added new column data_new to Knight model with M2M relation to TestData:
data_new = models.ManyToManyField(TestData, \

3. Start schema migration:
python schemamigration app1 --auto

4. Change forward method in our new migration, code after creation of new column/proxy model:
for obj in orm.Knight.objects.all():

5. Make migration:
python migrate app1

6. Remove column data from model and make migration; rename m2m column data_new to data and start schema migration
# remove column "data" from model
python schemamigration app1 --auto
python migrate app1

# rename m2m column from "data_new" to "data"
python schemamigration app1 --auto

7. At this point I need to keep all existing data in proxy table but South won't do that for me. So, I need to change forward method of migration (actually I need to replace generated code to following):
db.delete_unique('app1_knight_data_new', ['knight_id', 'testdata_id'])

db.rename_table('app1_knight_data_new', 'app1_knight_data')
db.create_unique('app1_knight_data', ['knight_id', 'testdata_id'])

Update: for Postgres (and I assume for MySQL with InnoDB db storage too) order of above commands is important: you have to delete old unique constraint before renaming of the proxy table.

8. After that just make migration:
python migrate app1

So, we've done and everything in safety.


  1. Great! Exactly what I was looking for!

    There's a typo in step 5, the command should be migrate and not schemamigration.

  2. i ran into problems with step 5. I needed to add no_dry_run=True just after the class Migration line.

  3. Thanks for sharing this!