Opened 5 years ago

Closed 5 years ago

Last modified 5 years ago

#23738 closed Bug (fixed)

Migrations broken with django 1.7.1 when changing null from true to false.

Reported by: Andrey Antukh Owned by: Markus Holtermann
Component: Migrations Version: 1.7
Severity: Release blocker Keywords:
Cc: info+coding@… Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Hi!

I start having strange exceptions when I have migrated a project from django 1.7 to 1.7.1

(taiga)[3/5.0.7]{1}niwi@niwi:~/taiga/taiga-back> python manage.py sqlmigrate projects 0006
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/niwi/.virtualenvs/taiga/lib/python3.4/site-packages/Django-1.7.2.dev20141030224451-py3.4.egg/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/home/niwi/.virtualenvs/taiga/lib/python3.4/site-packages/Django-1.7.2.dev20141030224451-py3.4.egg/django/core/management/__init__.py", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/niwi/.virtualenvs/taiga/lib/python3.4/site-packages/Django-1.7.2.dev20141030224451-py3.4.egg/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/niwi/.virtualenvs/taiga/lib/python3.4/site-packages/Django-1.7.2.dev20141030224451-py3.4.egg/django/core/management/commands/sqlmigrate.py", line 30, in execute
    return super(Command, self).execute(*args, **options)
  File "/home/niwi/.virtualenvs/taiga/lib/python3.4/site-packages/Django-1.7.2.dev20141030224451-py3.4.egg/django/core/management/base.py", line 338, in execute
    output = self.handle(*args, **options)
  File "/home/niwi/.virtualenvs/taiga/lib/python3.4/site-packages/Django-1.7.2.dev20141030224451-py3.4.egg/django/core/management/commands/sqlmigrate.py", line 61, in handle
    sql_statements = executor.collect_sql(plan)
  File "/home/niwi/.virtualenvs/taiga/lib/python3.4/site-packages/Django-1.7.2.dev20141030224451-py3.4.egg/django/db/migrations/executor.py", line 77, in collect_sql
    migration.apply(project_state, schema_editor, collect_sql=True)
  File "/home/niwi/.virtualenvs/taiga/lib/python3.4/site-packages/Django-1.7.2.dev20141030224451-py3.4.egg/django/db/migrations/migration.py", line 108, in apply
    operation.database_forwards(self.app_label, schema_editor, project_state, new_state)
  File "/home/niwi/.virtualenvs/taiga/lib/python3.4/site-packages/Django-1.7.2.dev20141030224451-py3.4.egg/django/db/migrations/operations/fields.py", line 139, in database_forwards
    schema_editor.alter_field(from_model, from_field, to_field)
  File "/home/niwi/.virtualenvs/taiga/lib/python3.4/site-packages/Django-1.7.2.dev20141030224451-py3.4.egg/django/db/backends/schema.py", line 473, in alter_field
    self._alter_field(model, old_field, new_field, old_type, new_type, old_db_params, new_db_params, strict)
  File "/home/niwi/.virtualenvs/taiga/lib/python3.4/site-packages/Django-1.7.2.dev20141030224451-py3.4.egg/django/db/backends/schema.py", line 617, in _alter_field
    sql, params = tuple(zip(*actions))
ValueError: need more than 0 values to unpack

I start research.

Firstly I have surprised that the last created migration does not include null=False change: https://github.com/taigaio/taiga-back/blob/improvements-migrations/taiga/projects/migrations/0006_auto_20141029_1040.py but this, more surprisingly with django 1.7 emits correct sql migration without having null=False in a migration file:

(taiga)[3/5.0.7]niwi@niwi:~/taiga/taiga-back> python manage.py sqlmigrate projects 0006
BEGIN;
--
-- MIGRATION NOW PERFORMS OPERATION THAT CANNOT BE WRITTEN AS SQL:
-- Raw Python operation
--
ALTER TABLE "projects_project" ALTER COLUMN "total_milestones" SET NOT NULL;
ALTER TABLE "projects_project" ALTER COLUMN "total_milestones" DROP DEFAULT;

COMMIT;

This is a proper behavior?

But with django 1.7.1 it seems not detects correctly the null attribute change, that causes that actions variable on django/db/backends/schema.py:617 to be empty, which in turn, causes that exception.

I have tried set manually the null=False parameter on the migrations file, and it no such effect (works with django 1.7 and raises exception with django 1.7.1)

Change History (12)

comment:1 Changed 5 years ago by Tim Graham

Did you generate the migration with Django 1.7 or 1.7.1? If the former, could you try regenerating it with 1.7.1? I think #23609 (fixed in 1.7.1) could be related.

comment:2 Changed 5 years ago by Andrey Antukh

The migration is generated by django 1.7 but I have tried generate it with django 1.7.1, but it no such effect. The same exception is raised and the same behavior on creating the migration file (missing null=True and blank=True on alterfield action).

Also, I have tried with 1.7.2dev (stable/1.7.x branch) and also I have found the same exception.

comment:3 Changed 5 years ago by Tim Graham

I've having trouble reproducing the issue. Could you outline the minimal steps (ideally one that doesn't involve a big project like the example you linked)? Not sure it matters, but which database backend are you using?

comment:4 Changed 5 years ago by Andrey Antukh

We are using the psycopg2 backend.

Tomorrow I'll try to make a small project for make easy to reproduce it.

But the process is very simple:

  • havind django 1.7, I have created a migration to one model that already have one integer field with null=True, blank=True and default=0 on first migration, changing null and blank parameters both to False.
  • update to django 1.7.1
  • try apply the created migration

comment:5 Changed 5 years ago by Markus Holtermann

Cc: info+coding@… added

I can confirm the problem

comment:6 Changed 5 years ago by Markus Holtermann

Owner: changed from nobody to Markus Holtermann
Status: newassigned

comment:7 Changed 5 years ago by Tim Graham

Severity: NormalRelease blocker
Triage Stage: UnreviewedAccepted

comment:8 Changed 5 years ago by Markus Holtermann

I opened a pull-request to fix this ticket: https://github.com/django/django/pull/3452

comment:9 Changed 5 years ago by Markus Holtermann

As long as you don't have a high traffic site, you should get it working with the following workaround:

class Migration(migrations.Migration):
    dependencies = [
        # your dependencies here
    ]

    operations = [
        migrations.AlterField(
            model_name='mymodel',
            name='myfield',
            field=models.IntegerField(null=True, blank=True, default=None),
        ),
        migrations.AlterField(
            model_name='mymodel',
            name='myfield',
            field=models.IntegerField(default=0),
        ),
    ]

What happened: Unfortunately I didn't took into account in the other patch that the default sticks between a NULL and NOT NULL change :-/. Thus changing the default from e.g. 0 to NULL while sticking to the NULL constraint and changing it back to 0 together with adding the NOT NULL constraint should work.

comment:10 Changed 5 years ago by Andrey Antukh

Thanks for this workaround. In our case that is not very problematic and we are reverted temporally to django 1.7.

Thanks for the quick fix.

comment:11 Changed 5 years ago by Tim Graham <timograham@…>

Resolution: fixed
Status: assignedclosed

In 715ccfde24f9f2b7f6710429370a1eff3c78fc2a:

Fixed #23738 -- Allowed migrating from NULL to NOT NULL with the same default value

Thanks to Andrey Antukh for the report.

comment:12 Changed 5 years ago by Tim Graham <timograham@…>

In 40ad022d5e366f56424dcdb885607d13c00a550e:

[1.7.x] Fixed #23738 -- Allowed migrating from NULL to NOT NULL with the same default value

Thanks to Andrey Antukh for the report.

Backport of 715ccfde24 from master

Note: See TracTickets for help on using tickets.
Back to Top