Opened 2 years ago

Last modified 20 months ago

#27202 new Bug

Investigate RenameMethodsBase effect on yaml serialization

Reported by: Ilya Semenov Owned by: nobody
Component: Utilities Version: 1.8
Severity: Normal Keywords: yaml
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Django 1.7.10, PyYAML 3.11 (Python 3.4):

>>> import yaml
>>> from django import forms
>>> yaml.dump(forms.ChoiceField)
"!!python/name:django.forms.fields.ChoiceField ''\n"

Django 1.8.14, PyYAML 3.11 (Python 3.5):

>>> import yaml
>>> from django import forms
>>> yaml.dump(forms.ChoiceField)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/semenov/work/test/var/venv/lib/python3.5/site-packages/yaml/__init__.py", line 200, in dump
    return dump_all([data], stream, Dumper=Dumper, **kwds)
  File "/Users/semenov/work/test/var/venv/lib/python3.5/site-packages/yaml/__init__.py", line 188, in dump_all
    dumper.represent(data)
  File "/Users/semenov/work/test/var/venv/lib/python3.5/site-packages/yaml/representer.py", line 26, in represent
    node = self.represent_data(data)
  File "/Users/semenov/work/test/var/venv/lib/python3.5/site-packages/yaml/representer.py", line 51, in represent_data
    node = self.yaml_multi_representers[data_type](self, data)
  File "/Users/semenov/work/test/var/venv/lib/python3.5/site-packages/yaml/representer.py", line 313, in represent_object
    reduce = data.__reduce_ex__(2)
  File "/Users/semenov/work/test/var/venv/lib/python3.5/copyreg.py", line 65, in _reduce_ex
    raise TypeError("can't pickle %s objects" % base.__name__)
TypeError: can't pickle int objects

Anything we can do to have Django class types YAML-compatible again?

Change History (2)

comment:1 Changed 2 years ago by Tim Graham

Component: UncategorizedUtilities
Summary: yaml dump for django.forms.Field crashes with "TypeError: can't pickle int objects"Investigate RenameMethodsBase effect on yaml serialization
Triage Stage: UnreviewedAccepted
Type: UncategorizedBug

Fixed in Django 1.10 by a3fe4addcb9063fc327c8609c7ebba4d531da4ce. Unfortunately, Django 1.8 and 1.9 are only receiving security and data loss fixes by now. Perhaps it's worth investigating the cause of this (RenameMethodsBase, possibly) to try to prevent future regressions.

comment:2 Changed 20 months ago by Ryan Castner

Ok did a bit of investigating, RenameMethodsBase inherits from type. I may be crazy but that sounds a lot like https://docs.python.org/2/library/pickle.html#pickling-and-unpickling-extension-types. Specifically:

When the Pickler encounters an object of a type it knows nothing about — such as an extension type — it looks in two places for a hint of how to pickle it.

Those hints are the __reduce__ and __reduce_ex__ methods. The __reduce_ex__ method states method will be called with a single integer argument, the protocol version.

And that call seems to fail here https://github.com/python/cpython/blob/c30098c8c6014f3340a369a31df9c74bdbacc269/Objects/typeobject.c#L4169

Note: See TracTickets for help on using tickets.
Back to Top