Opened 2 years ago

Closed 2 years ago

#33471 closed Cleanup/optimization (fixed)

AlterField operation should be noop when adding/changing choices on SQLite.

Reported by: David Szotten Owned by: Sarah Boyce
Component: Migrations Version: 4.0
Severity: Normal Keywords:
Cc: Adam Johnson, Sarah Boyce Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: no

Description

while writing a test case for #33470 i found that for sqlite, even a seemingly db-transparent change like adding choices still generates sql (new table + insert + drop + rename) even though this shouldn't be needed. on e.g. postgres the same migration generates no sql

Change History (15)

comment:1 by Mariusz Felisiak, 2 years ago

Easy pickings: set
Summary: sqlite no-op migrationsAlterField operation should be noop when adding/changing choices.
Triage Stage: UnreviewedAccepted

It was missed in #25253 (see 9159d173c3822312c653db7ff5b9a94b14af1dca). Adding choices to the non_database_attrs should fix it:

  • django/db/backends/base/schema.py

    diff --git a/django/db/backends/base/schema.py b/django/db/backends/base/schema.py
    index 4cd4567cbc..822da656d3 100644
    a b class BaseDatabaseSchemaEditor:  
    11301130        # - adding only a db_column and the column name is not changed
    11311131        non_database_attrs = [
    11321132            'blank',
     1133            'choices',
    11331134            'db_column',
    11341135            'editable',
    11351136            'error_messages',

Would you like to prepare a patch? (a regression test is required).

comment:2 by Mariusz Felisiak, 2 years ago

Type: BugCleanup/optimization

comment:3 by David Szotten, 2 years ago

oh i didn't realise that this was done globally by attr name. (though how come pg does the right thing?)

could marking choices as a non_database_attrs adversely impact third party fields that eg use a pg enum type to model choices values?

comment:4 by Douglas Mumme, 2 years ago

Owner: changed from nobody to Douglas Mumme
Status: newassigned

Hey there, I'm new to this and trying to get into contributing to Django, but I would like to pick this ticket up and prepare a patch for it if no one is working on it right now. Please let me know if there is a problem with this, thanks!

comment:5 by Douglas Mumme, 2 years ago

Just made a pull request with the patch to fix this https://github.com/django/django/pull/15404

comment:6 by Douglas Mumme, 2 years ago

Has patch: set

in reply to:  3 comment:7 by Mariusz Felisiak, 2 years ago

Cc: Adam Johnson added
Patch needs improvement: set
Summary: AlterField operation should be noop when adding/changing choices.AlterField operation should be noop when adding/changing choices on SQLite.

Replying to David Szotten:

could marking choices as a non_database_attrs adversely impact third party fields that eg use a pg enum type to model choices values?

True, this would cause a regression for-party enum fields, e.g. django-mysql's EnumField a similar field may exist for PostgreSQL or Oracle. It seems we can safely overwrite _field_should_be_altered() only on SQLite.

comment:8 by Adam Johnson, 2 years ago

Indeed it would break Django-MySQL’s EnumField - docs / source.

If the "ignorable attrs" were extended on a per-field-class basis, that could work. It would be a bigger patch but it could allow many custom field classes to avoid unnecessary database changes.

comment:9 by Mariusz Felisiak, 2 years ago

Has patch: unset
Patch needs improvement: unset

comment:10 by David Szotten, 2 years ago

do you know off the top of your head what mechanism makes it a no-op on pg already?

in reply to:  10 comment:11 by Mariusz Felisiak, 2 years ago

Replying to David Szotten:

do you know off the top of your head what mechanism makes it a no-op on pg already?

There is nothing specific to PostgreSQL here, it's also a noop on MySQL and Oracle. It's caused by remaking tables on SQLite that is necessary in most of cases (see related ticket #32502). Overwriting _field_should_be_altered() in the SQLite backend should do the trick.

comment:12 by Sarah Boyce, 2 years ago

Cc: Sarah Boyce added
Has patch: set

I have added a PR which hopefully addresses the above comments: https://github.com/django/django/pull/15561

comment:13 by Mariusz Felisiak, 2 years ago

Owner: changed from Douglas Mumme to Sarah Boyce

comment:14 by Mariusz Felisiak, 2 years ago

Triage Stage: AcceptedReady for checkin

comment:15 by Mariusz Felisiak <felisiak.mariusz@…>, 2 years ago

Resolution: fixed
Status: assignedclosed

In 65effbdb:

Fixed #33471 -- Made AlterField operation a noop when changing "choices".

This also allows customizing attributes of fields that don't affect
a column definition.

Note: See TracTickets for help on using tickets.
Back to Top