Opened 2 weeks ago

Last modified 10 days ago

#32674 new New feature

DEFAULT_AUTO_FIELD changes should be detected for PKs of auto-created intermediate M2M models.

Reported by: Cristiano Coelho Owned by: nobody
Component: Migrations Version: 4.0
Severity: Normal Keywords: DEFAULT_AUTO_FIELD AutoField BigAutoField ManyToManyField
Cc: ccoelho@…, Tom Forbes, Simon Charette, Andrew Godwin, Shai Berger, Markus Holtermann Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Django 3.2 introduced DEFAULT_AUTO_FIELD = 'django.db.models.AutoField'. However, if we set it to BigAutoField, I would expect that the intermediary table created for ManyToMany relationships also honor this value.

Right now, we end up with tables that have FKs as bigint, but their own PK as integer. This may not be a direct bug, but will cause issues for sure.

Ideally, ManyToManyField should have an easy way to configure its PK without having to implement the through model.

Change History (7)

comment:1 Changed 2 weeks ago by Mariusz Felisiak

Cc: Tom Forbes added
Component: UncategorizedMigrations
Easy pickings: unset
Severity: NormalRelease blocker
Summary: DEFAULT_AUTO_FIELD should also be used for ManyToMany table primary keysDEFAULT_AUTO_FIELD changes should be detected for PKs of auto-created intermediate M2M models.
Triage Stage: UnreviewedAccepted
Type: UncategorizedBug

Thanks for this report. I agree we should detect this change for PKs of auto-created intermediate models.

comment:2 Changed 10 days ago by Mariusz Felisiak

Cc: Simon Charette Andrew Godwin Shai Berger Markus Holtermann added
Severity: Release blockerNormal
Type: BugNew feature
Version: 3.24.0

I tried, I really really tried, but after few days of using different approaches I need to conclude that this is massive, extremely complicate, and maybe even unfeasible. As far as I'm aware we've never took into account changes in auto-created many-to-many tables except for those directly related with a definition of ManyToManyFields. A project state used by migrations autodetector doesn't include auto-created model, so the only fix that I can imagine would require including through_pk_type in the ManyToManyField.deconstruct() but it's also tricky and error-prone.

We're going to document this caveat and treat this ticket as a new feature.

comment:4 Changed 10 days ago by Cristiano Coelho

Can we at least provide more detailed migration steps? Right now, the only alternative is to go table by table and do the migration more or less manually. This is very time consuming for large projects.

comment:5 Changed 10 days ago by Carlton Gibson <carlton.gibson@…>

In 907d3a7:

Refs #32674 -- Noted that auto-created through table PKs cannot be automatically migrated.

comment:6 Changed 10 days ago by Carlton Gibson <carlton.gibson@…>

In bac41697:

[3.2.x] Refs #32674 -- Noted that auto-created through table PKs cannot be automatically migrated.

Backport of 907d3a7ff4e12ad4ccc86af26a728007fe4d6fa2 from main

comment:7 Changed 10 days ago by Simon Charette

A project state used by migrations autodetector doesn't include auto-created model, so the only fix that I can imagine would require including through_pk_type in the ManyToManyField.deconstruct() but it's also tricky and error-prone.

That seems like the only solution to me as well.

Having a through_pk: Field(primary_key=True) option instead of through_pk_type seems more flexible though as it would allow easier customization e.g. though_pk=UUIDField(default=uuid.uuid4, primary_key=True). I guess it could also serve as a way to define a composite primary key on the from_field and to_field if we ever manage to add support for that e.g. through_pk=CompositePrimaryKey('author', 'book').

Note: See TracTickets for help on using tickets.
Back to Top