aka, How we use Django Migrations at TopOPPS
Please do not read this as advice. Although we’ve spent a while thinking about it, I’m not convinced that we’ve arrived at the best solution. It seems to work oh-kay, but it’s a bit of a pain. Rather, I’m hoping that someone will tell me I’m wrong, and demonstrate a better way. After all, “the best way to get the right answer on the Internet is not to ask a question, it’s to post the wrong answer.” In the meantime, this is the best way I know of to manage Django migrations under version control.
Let’s say the latest migration in the production branch is
0014_user_phone_num.py. Sometimes, two different feature branches will add migrations numbered 15:
0015_opp_splits.py. First, the branch with
0015_create_taskcustom is merged to the
dev server. The migration is run, and it all goes swimmingly. But what happens when
0015_opp_splits is merged?
Django expects a single migration to be the ‘latest’ migration. When there are two ‘latest’ migrations, it will refuse to run until you make a merge migration;
./manage.py makemigrations --merge will do the job for you. It makes
0016_merge.py. It doesn’t do any work, just lists both of
opp_splits as dependencies. Dev is the only branch where this file exists. It’s not in either feature branch, and it won’t ever make it to production.
While those two are being tested, a new feature branch is cut from prod. It also adds a migration:
The branch with
0015_create_taskcustom is deemed worthy, and advances to production. The product family branch merges in the latest from production, which includes the new migration. That means there are now two 0015 migrations in that branch, so we make a merge migration:
0016_merge is different from the one that lives in the dev branch. It had dependencies for
product_family, while the dev one knows about
Danger ahead, friends. Now we merge the product family branch to dev for testing. Both branches have a file called
0016_merge, but they have different contents. We get a merge conflict that looks something like:
dependencies = [ ('topopps', '0015_create_taskcustom'), <<<<<<< HEAD ('topopps', '0015_opp_splits'), ======= ('topopps', '0015_product_family'), >>>>>>> feature/product-family ]
What should we do here? It seems like a sensible choice would be to include all three lines as dependencies. However, that way lies madness. Doing that, then pushing to the dev server will not run the migrations. The
django_migrations table the dev server’s database already includes an entry for
topopps, 0016_merge, so it sees that and thinks there’s nothing to do. That means the
0015_product_family migration won’t run.
The (sorta) solution
The solution we’ve come up with is to never allow a merge migration to be named
####_merge. We always add a couple of random words to change it to something like
0018_seashell_queenbee_merge.py. That way, instead of a merge conflict as above, we make an extra merge migration. Let’s consider what happens with this rule in place:
0015_create_taskcustomis merged to dev.
0015_opp_splitsis merged to dev.
- The branch containing
- The branch with
0015_create_taskcustomis merged to production.
- The product family branch merges in the latest from production, including
0015_create_taskcustom. This necessitates creating
- When the product family branch is merged to dev, both
0016_chilly_spider_mergeare present. This means we need to create
0017_humiliating_deer_merge, which depends on both of those two.
- Everything is good (I think?).
This is admittedly a bit of work. It’s not the prettiest solution. We end up creating merge migrations for merge migrations. I don’t have a better idea.
We’ve been doing this at TopOPPS since March 29 2016 with
0011_fancy_turtle_merge, all the way up to March 1, 2017 with
0052_unhappy_honeycreeper_merge (the latest as of this writing). We have a script that runs as a pre-commit and post-merge hook to checks if a merge migration needs to be created, and nettles you into choosing a unique-ish name (available at bgschiller/pre-commit-hooks).