Opened 5 years ago

Closed 5 years ago

Last modified 5 years ago

#21335 closed Bug (fixed)

Bundled importlib some times won't work properly with python3.3

Reported by: Andrei Antoukh Owned by: nobody
Component: Python 3 Version: master
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


In one of my project some times (the most common case, run runserver, change code, runsever reloads) and in some requests is raised strange KeyError:

Traceback (most recent call last):
  File "/home/niwi/.envs/greenmine/lib/python3.3/site-packages/django/core/handlers/", line 114, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/niwi/.envs/greenmine/lib/python3.3/site-packages/rest_framework/", line 78, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/niwi/.envs/greenmine/lib/python3.3/site-packages/django/views/decorators/", line 57, in wrapped_view
    return view_func(*args, **kwargs)
  File "/home/niwi/.envs/greenmine/lib/python3.3/site-packages/rest_framework/", line 399, in dispatch
    response = self.handle_exception(exc)
  File "/home/niwi/devel/greenmine-back/greenmine/base/", line 23, in handle_exception
    return super(APIView, self).handle_exception(exc)
  File "/home/niwi/.envs/greenmine/lib/python3.3/site-packages/rest_framework/", line 396, in dispatch
    response = handler(request, *args, **kwargs)
  File "/home/niwi/.envs/greenmine/lib/python3.3/site-packages/rest_framework/", line 139, in partial_update
    return self.update(request, *args, **kwargs)
  File "/home/niwi/.envs/greenmine/lib/python3.3/site-packages/django/db/", line 338, in inner
    return func(*args, **kwargs)
  File "/home/niwi/devel/greenmine-back/greenmine/base/", line 28, in update
    return super().update(*args, **kwargs)
  File "/home/niwi/.envs/greenmine/lib/python3.3/site-packages/rest_framework/", line 132, in update
    self.post_save(self.object, created=created)
  File "/home/niwi/devel/greenmine-back/greenmine/projects/userstories/", line 90, in post_save
    super().post_save(obj, created)
  File "/home/niwi/devel/greenmine-back/greenmine/base/notifications/", line 25, in post_save
    context["changed_fields_dict"] = obj.get_changed_fields_dict(self.request.DATA)
  File "/home/niwi/devel/greenmine-back/greenmine/base/notifications/", line 77, in get_changed_fields_dict
    field_dict.update(self._get_changed_field(field_name, data_value))
  File "/home/niwi/devel/greenmine-back/greenmine/base/notifications/", line 126, in _get_changed_field
    old_value = self._get_changed_field_old_value(field_name, data_value)
  File "/home/niwi/devel/greenmine-back/greenmine/base/notifications/", line 119, in _get_changed_field_old_value
    return (self.last_version and self.last_version.field_dict.get(field_name, data_value) or None)
  File "/home/niwi/.envs/greenmine/lib/python3.3/site-packages/reversion/", line 177, in field_dict
    object_version = self.object_version
  File "/home/niwi/.envs/greenmine/lib/python3.3/site-packages/reversion/", line 164, in object_version
    return list(serializers.deserialize(self.format, data, ignorenonexistent=True))[0]
  File "/home/niwi/.envs/greenmine/lib/python3.3/site-packages/django/core/serializers/", line 132, in deserialize
    d = get_deserializer(format)
  File "/home/niwi/.envs/greenmine/lib/python3.3/site-packages/django/core/serializers/", line 111, in get_deserializer
  File "/home/niwi/.envs/greenmine/lib/python3.3/site-packages/django/core/serializers/", line 144, in _load_serializers
    register_serializer(format, BUILTIN_SERIALIZERS[format], serializers)
  File "/home/niwi/.envs/greenmine/lib/python3.3/site-packages/django/core/serializers/", line 69, in register_serializer
    module = importlib.import_module(serializer_module)
  File "/home/niwi/.envs/greenmine/lib/python3.3/site-packages/django/utils/", line 36, in import_module
    return sys.modules[name]
KeyError: 'django.core.serializers.pyyaml'

Saying code of bundled importlib...

def import_module(name, package=None):
    """Import a module.

    The 'package' argument is required when performing a relative import. It
    specifies the package to use as the anchor point from which to resolve the
    relative import to an absolute import.

    if name.startswith('.'):
        if not package:
            raise TypeError("relative imports require the 'package' argument")
        level = 0
        for character in name:
            if character != '.':
            level += 1
        name = _resolve_name(name[level:], package, level)
    return sys.modules[name]

... seems __import__ is called with "django.core.serializers.pyyaml" but it does nothing (not raises ImportError, but it should do it) and sys.modules, obviously not contains "django.core.serializers.pyyaml".

(Note: I don't have pyyaml installed and I not use it on my project...)

I don't have deterministic way to reproduce this, because it hapens sometimes using python3.3 and django1.6(alpha->rc1).

I think that it happens because a python2.7 importlib isn't work properly with python3 and a simple solution is use python3 importlib if you running django with python3:

I know that importlib is now deprecated on master, but I think it should fixed also in 1.5, 1.6 and master.


Change History (7)

comment:1 Changed 5 years ago by Claude Paroz

Triage Stage: UnreviewedAccepted
Type: UncategorizedBug

I think this is a safe change and I'm ready to do the backporting job. However, I'd like to get fellow core devs approval before going forward.

comment:2 Changed 5 years ago by Aymeric Augustin

I'm OK with that.

comment:3 Changed 5 years ago by Claude Paroz <claude@…>

Resolution: fixed
Status: newclosed

In 3351e94ffa37d3824a49b687125ee4a430b69bbe:

Fixed #21335 -- Use importlib from python3 when using python3

comment:4 Changed 5 years ago by Claude Paroz <claude@…>

In e052ada0f6f21208d0f215d2f9184c4069ac1419:

[1.6.x] Fixed #21335 -- Use importlib from python3 when using python3

Backport of 3351e94ffa from master.

comment:5 Changed 5 years ago by Claude Paroz <claude@…>

In 1ef737416ab56a6becee36876769909a37541465:

[1.5.x] Fixed #21335 -- Use importlib from python3 when using python3

Backport of 3351e94ffa from master.

comment:6 Changed 5 years ago by Claude Paroz <claude@…>

In ec0a3f5085219cd647d3e93841018987ce5384bf:

[1.6.x] Removed relative usage of import_module

Python 3 version of importlib doesn't support this syntax.
Partial backport of fdd7a355bf. Refs #21335.

comment:7 Changed 5 years ago by Claude Paroz <claude@…>

In 3b4b376bd1dde3cbe64cf8f1a4fb1756904af710:

Revert "[1.5.x] Fixed #21335 -- Use importlib from python3 when using python3"

This reverts commit 1ef737416ab56a6becee36876769909a37541465.
This would require changing some import_module calls (relative
syntax, see ec0a3f50852), too much for a stable release.

Note: See TracTickets for help on using tickets.
Back to Top