Opened 7 years ago

Closed 7 years ago

#28456 closed Cleanup/optimization (fixed)

Defining __getstate__ in class derived from Model doesn't affect model pickling

Reported by: Nerl~ Owned by: Nerl~
Component: Database layer (models, ORM) Version: 1.11
Severity: Normal Keywords: Pickle, Models
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

Current implementation of Model.__reduce__ does not allow to simply override what attributes should or shouldn't be serialized in classes derived from Model class. Pickle documentation says that for that purpose developer can define __getstate__ method in class and return dictionary with needed content. That case isn't represented in Model.__reduce__ by now, current implementation just using __dict___ attribute without copying it so it's difficult to change what attributes must be pickled in derived classes.

For example, some model derived from Model class and adds attribute to model instance that cannot be pickled or just shouldn't be:

class MyModel(Model):
    def __init__(self, *args, **kwargs):
        self.service_obj = UnpickleableService()  # some service that shouldn't be pickled, just for example
        super().__init__(*args, **kwargs)

Now it's hard to define that attribute service_obj mustn't be pickled - what you need to do is define __reduce__ method in derived class and copy third attribute from result of the super method and then delete service_obj key from that, otherwise you will delete service_obj from original __dict__ attribute:

class MyModel(Model):
    def __init__(self, *args, **kwargs):
        self.service_obj = UnpickleableService() 
        super().__init__(*args, **kwargs)
    
    def __reduce__(self):
         callable, args, data = super().__reduce__()
         data = data.copy()
         del data['service_obj']
         return data

Pickle documentation propose easier way for doing so with defining __getstate__ method that returns object with content that must be pickled. If that method is absent then __dict__ attribute will be used. With this implementation, Django can define simple __getstate__ method in Model class that will return copy of __dict__ attribute. Developer can define __getstate__ method in derived classes and specify what should or shouldn't be serialized by pickle in that particular model.

That approach helps to write mixins for models or model hierarchy that will add some attributes to model instance and exclude them from pickling results. Also it corresponds to the Pickle documentation.

Change History (5)

comment:1 by Nerl~, 7 years ago

Owner: changed from nobody to Nerl~
Status: newassigned
Type: New featureCleanup/optimization

comment:2 by Tim Graham, 7 years ago

Triage Stage: UnreviewedAccepted

comment:3 by Chris Adams, 7 years ago

Ran the tests and all worked for me.

comment:4 by Tim Graham, 7 years ago

Patch needs improvement: set

comment:5 by Tim Graham <timograham@…>, 7 years ago

Resolution: fixed
Status: assignedclosed

In 97cb3bd1:

Fixed #28456 -- Allowed customizing Model pickling by overriding getstate().

Note: See TracTickets for help on using tickets.
Back to Top