Code

Opened 7 years ago

Closed 7 years ago

#3982 closed (fixed)

Allow Field subclasses to coerce arbitrary Python types

Reported by: Marty Alchin <gulopine@…> Owned by: nobody
Component: Database layer (models, ORM) Version: master
Severity: Keywords: coerce coercion
Cc: jm.bugtracking@…, elsdoerfer@… Triage Stage: Unreviewed
Has patch: yes Needs documentation: yes
Needs tests: yes Patch needs improvement: yes
Easy pickings: UI/UX:

Description

In working with a recent DurationField patch (#2443), I noticed that Django seems to rely on the backend database driver to coerce data into native Python types. However, when adding new fields with non-standard mappings (such as FloatField for storage, but timedelta as a native type, as in DurationField), to_python needs to be called explicitly when populating an object from the database in order to be usable. This simple patch does just that, using to_python to coerce native Python types, after whatever coercion might be done by the database.

Attachments (3)

to_python.diff (810 bytes) - added by Marty Alchin <gulopine@…> 7 years ago.
Path to force to_python data type coercion during data retrieval
coerce.diff (782 bytes) - added by Marty Alchin <gulopine@…> 7 years ago.
Adds support for a coerce Field method to handle coercion
lazy_attribute.diff (2.1 KB) - added by Marty Alchin <gulopine@…> 7 years ago.
An attempt at generic lazily-instantiated attributes

Download all attachments as: .zip

Change History (11)

Changed 7 years ago by Marty Alchin <gulopine@…>

Path to force to_python data type coercion during data retrieval

Changed 7 years ago by Marty Alchin <gulopine@…>

Adds support for a coerce Field method to handle coercion

comment:1 Changed 7 years ago by Marty Alchin <gulopine@…>

  • Keywords coerce coercion added; to_python removed
  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Summary changed from [patch] Use fields' to_python to coerce Python data types to Allow Field subclasses to coerce arbitrary Python types

The attached patch allows Field subclasses to define a coerce method that takes a single value and returns any Python type, that will then be attached to the model instance as the appropriate attribute. This happens after retrieval from the database, manual instantiation of objects, and deserialization.

The one pitfall is that, due to the way bits of Django work, the value sent to coerce might be:

  • a string (if the model came from the DB)
  • a native Python type (if it came from a serializer, having already gone through to_python)
  • possibly some other situation I haven't run into yet

So the coerce method should take these different situations into account.

Here's an example, from DurationField, which will be added to #2443 soon.

def coerce(self, value):
    return datetime.timedelta(seconds=float(value))

comment:2 Changed 7 years ago by Marty Alchin <gulopine@…>

  • Needs documentation set
  • Needs tests set
  • Patch needs improvement set

I forgot to add all the checkboxes, since there's much more needed to call this done.

comment:3 Changed 7 years ago by anonymous

  • Cc jm.bugtracking@… added

Changed 7 years ago by Marty Alchin <gulopine@…>

An attempt at generic lazily-instantiated attributes

comment:4 Changed 7 years ago by Marty Alchin <gulopine@…>

In speaking with Robert Coup and seeing his thoughts on #4322, I've come up with a way for Field subclasses to more easily implement lazy instantiation. I'll be demonstrating this method in #2443 as well as working with Robert on a new patch for #4322.

comment:5 Changed 7 years ago by mtredinnick

  • Owner changed from adrian to mtredinnick

There's some issue drift going on here. The last patch has close to nothing to do with the original ticket, which is essentially about helping create Field sub-classes. Lazy instantion is not a requirement there and is an orthogonal issue. Let's try to keep one issue per ticket, please.

I'm going to get around to tidying up the various Field sub-classing ticket this week and I'll close this when that happens. You should open another ticket for the lazy portion after that (probably best to wait until then so that you can write a patch against the code that exists in trunk).

comment:6 Changed 7 years ago by Marty Alchin <gulopine@…>

Yeah, I admit there is some drift here, but it's mainly because lazy instantiation seems to be a better approach to solve the original problem this ticket was opened for. I'll wait and see the results of your work this week, and if this one needs to just close as an invalid, I'm fine with that.

comment:7 Changed 7 years ago by miracle2k <elsdoerfer@…>

  • Cc elsdoerfer@… added

Would be nice if this could be fixd at some point. Field subclassing is severely limited currently.

comment:8 Changed 7 years ago by mtredinnick

  • Resolution set to fixed
  • Status changed from new to closed

Looks like this was fixed with the addition of SubFieldBase and the whole field subclassing stuff.

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.