#21290 closed Cleanup/optimization (fixed)

Add migration docs for (or avoid) "Cannot serialize" errors

Reported by: timo Owned by: andrewgodwin
Component: Migrations Version: master
Severity: Release blocker Keywords:
Cc: loic@… Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

The migration writer raises a ValueError: Cannot serialize: <foo> for models that have defaults like:

  1. models.DateField(default=datetime.today)
  2. models.DecimalField(default=Decimal('1.000')

Suggested workarounds by loic84 in IRC:

  1. datetime.date.today is whitelisted, datetime.datetime.today isn't.
  2. def default_decimal(): return Decimal('1.000') then default=default_decimal.

This is confusing for users coming from south where things just worked. We should either add docs about this problem or figure out a way to avoid these errors.

Change History (6)

comment:1 Changed 22 months ago by loic84

  • Cc loic@… added

This should take care of date/datetime issues and @classmethods in general: https://github.com/django/django/pull/1775.

comment:2 Changed 22 months ago by Tim Graham <timograham@…>

In 8d6953d55c3aba04bbaf0f268499d6e405c653ff:

Added support for serializing class methods. - Refs #21290.

The new handling allows us to do away with the whitelisting that was
required to support date and datetime objects.

comment:3 Changed 22 months ago by loic84

I'd personally like to delay as much as possible the writing of the docs while things are still in motion, so we remain flexible with regards to the API. That said this is definitely a 1.7 release blocker.

The general concept is that things need to be either of basic types (str, int, etc.) or be "importable", with their __name__ attribute telling us how.

This means that functions, and classes (actual type, not instances), work out of the box (as long as they are not trapped in a closure).

Anything else can't be "guessed" and need to supply enough information so they can be reconstructed, to do so, they can expose a deconstruct attribute, which is a function that returns a triple (import_path <string>, args <tuple>, kwargs <dict>).

A few remarks:

  • One of the most common troublemakers is lamba, luckily it's easy to fix, just use a normal function instead.
  • My patch for #21275 provides a simple class decorator @deconstructible, which should work to reconstruct most class instances.
  • For advance usage, it's possible for an object's deconstruct method to point to an unrelated object, models.SET in d59f1993f150f83524051d96b52df08da4dcf011 takes advantage of that.
  • Right now, if the serializer encounters something it can't serialize, it just gives up. Ideally the interactive prompt would give a few options to choose from. For instance: 1. Cancel, 2. Set to NULL, 3. Continue with a marked error that needs to be hand edited (like for a merge conflict).

comment:4 Changed 22 months ago by loic84

https://github.com/loic/django/compare/MigrationWriter.interactive

This branch enables the serialization process to continue when it encounters things it can't serialize. It replaces the unserializable object by an instance of SerializationError which is an exception that raises itself in its __init__.

The idea is that the user can edit the faulty migration the same way one fixes a merge with conflicts in a VCS. Since SerializationError raises itself if the migration is run, users would get valuable feedback with the traceback.

I'm planning to add interactive feedback as mentioned in the previous post, but I'd like to discuss the details before having a go at it.

comment:5 Changed 19 months ago by andrewgodwin

  • Owner set to andrewgodwin
  • Severity changed from Normal to Release blocker
  • Status changed from new to assigned

Marking this as release blocker.

comment:6 Changed 19 months ago by Andrew Godwin <andrew@…>

  • Resolution set to fixed
  • Status changed from assigned to closed

In 6bbb82001455efb39a57c8ccfe09ff027633eced:

Fixed #21290: Documented migration serializing and improved error

Note: See TracTickets for help on using tickets.
Back to Top