Using dumpdata to create unit test fixtures causes duplicate foreign keys for auth permissions. Excluding auth causes other referenced auth models to be missing.

[ellisd23@…]

This seems to be a pretty fundamental issue, since models will frequently reference User. There are many instances of users having frustrations with contrib apps that are auto-generated upon migration/sync (e.g. ​contenttypes).

For contenttypes, this issue can be mitigated by excluding the app and using --natural. But for contrib.auth, if the app is excluded, any referenced User objects in fixture data will be missing. If the app is not excluded, duplicate permissions will be created, since the post-migration signal will have already created them. Since there is a unique key restraint, the importing of the fixture data will fail. It's a catch 22.

Developers can manually remove the permissions from the fixture data, or override fixture loading in their tests to remove the data, but it seems to me that since this is such a basic piece of functionality, there should be a simple interface to do such a thing. It'd be nice to have something simpler for contenttypes as well, rather than "--exclude contenttypes --natural".

[Anssi Kääriäinen]

I agree that fixture loading + contenttypes / permissions is a problem. I am marking this as accepted, although I don't have any idea what could be done here.

[Russell Keith-Magee]

I'm confused by this report. This is the exact problem that led to the introduction of natural foreign keys in the first place, and I was under the impression that it was fixed. Permissions have natural keys defined, so they should be able to be dumped with --natural.

Has something regressed in the 3-4 years since we committed natural keys, or have I missed an important detail about this report?

[ellisd23@…]

It's been a while since I made the ticket, but IIRC the issue is that even with --natural passed, the problem is that an attempt is made to create permissions twice: once during the loading of the fixtures, and once automatically. It's that second time that causes duplicate key errors.

[powderflask]

In case anyone finds this ticket, searching for an answer, this seems to be fixed in django1.9
I was having the problem described here, but it was resolved when I did my dump with:

dumpdata --natural-foreign --natural-primary

[powderflask]

Here is a KEY insight that seems obvious now, but busted my * for a couple hours:

if your models contain an M2M relation (e.g., auth.Group, auth.Permission), do NOT dump the M2M table (e.g., auth.group_permissions) into your test fixture! Otherwise, the M2M table will be populated once when the related objects are created, and then AGAIN when the M2M data is loaded from the fixture, thus causing an integrity error like the one reported here.

This mistake is SO easy to make and SUCH a bugger to track down, it might be worth a word of warning in the docs about creating test fixtures?