Opened 7 years ago

Closed 3 years ago

Last modified 3 years ago

#25624 closed Bug (fixed)

Autoreload fails if jinja2.ModuleLoader used

Reported by: SvartalF Owned by: Tom Forbes
Component: Utilities Version: 1.8
Severity: Normal Keywords: autoreload, jinja2, ModuleLoader
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


It happens because jinja2.ModuleLoader injects a weakref to a fake module into a sys.modules:

So, because weakref is not a hashable object, autoreload fails.

Traceback (most recent call last):
  File "bug/", line 10, in <module>
  File "/home/svartalf/projects/dj-module-loader-bug/.env/local/lib/python2.7/site-packages/django/core/management/", line 351, in execute_from_command_line
  File "/home/svartalf/projects/dj-module-loader-bug/.env/local/lib/python2.7/site-packages/django/core/management/", line 343, in execute
  File "/home/svartalf/projects/dj-module-loader-bug/.env/local/lib/python2.7/site-packages/django/core/management/", line 394, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/svartalf/projects/dj-module-loader-bug/.env/local/lib/python2.7/site-packages/django/core/management/commands/", line 49, in execute
    super(Command, self).execute(*args, **options)
  File "/home/svartalf/projects/dj-module-loader-bug/.env/local/lib/python2.7/site-packages/django/core/management/", line 445, in execute
    output = self.handle(*args, **options)
  File "/home/svartalf/projects/dj-module-loader-bug/.env/local/lib/python2.7/site-packages/django/core/management/commands/", line 88, in handle**options)
  File "/home/svartalf/projects/dj-module-loader-bug/.env/local/lib/python2.7/site-packages/django/core/management/commands/", line 97, in run
    autoreload.main(self.inner_run, None, options)
  File "/home/svartalf/projects/dj-module-loader-bug/.env/local/lib/python2.7/site-packages/django/utils/", line 336, in main
    reloader(wrapped_main_func, args, kwargs)
  File "/home/svartalf/projects/dj-module-loader-bug/.env/local/lib/python2.7/site-packages/django/utils/", line 302, in python_reloader
  File "/home/svartalf/projects/dj-module-loader-bug/.env/local/lib/python2.7/site-packages/django/utils/", line 278, in reloader_thread
    change = fn()
  File "/home/svartalf/projects/dj-module-loader-bug/.env/local/lib/python2.7/site-packages/django/utils/", line 207, in code_changed
    for filename in gen_filenames():
  File "/home/svartalf/projects/dj-module-loader-bug/.env/local/lib/python2.7/site-packages/django/utils/", line 94, in gen_filenames
    module_values = set(sys.modules.values())
TypeError: unhashable type: 'weakproxy'

I had reproduced this bug with a clean project and following environment:

Python 2.7.6

$ pip freeze

Template settings:

template_loader = jinja2.ModuleLoader('compiled_templates')
        'BACKEND': 'django.template.backends.jinja2.Jinja2',
        'DIRS': [os.path.join(BASE_DIR, 'templates')],
        'APP_DIRS': True,
        'OPTIONS': {
            'loader': template_loader,

P.S. Not sure if it is an "Utilities" or a "Template system" component.

Attachments (1)

25624.diff (1.2 KB) - added by Paul 7 years ago.

Download all attachments as: .zip

Change History (11)

comment:1 Changed 7 years ago by Tim Graham

Triage Stage: UnreviewedAccepted

Changed 7 years ago by Paul

Attachment: 25624.diff added

comment:2 Changed 7 years ago by Paul

This patch fixes the issue by ignoring modules without a __file__ attribute altogether. There are probably cases where this is not quite desired behaviour, but I have no idea how to test this.

The other two options that I can think of to fix this, checking for weakref.ProxyType or the _jinja2_module_templates_ module name prefix, don't really seem like the right solution either...

comment:3 Changed 7 years ago by Carl Meyer

@hackerdd That fix looks fine to me. I don't think it should actually change the behavior, since modules without __file__ were excluded later on anyway. There's nothing we can do in the autoreloader with a module that doesn't have a __file__.

I think this should be testable without too much difficulty, just by inserting a problematic entry into sys.modules (in the test - and make sure it gets removed after the test, using a finally block) and then calling the gen_filenames function.

comment:4 Changed 7 years ago by Aymeric Augustin

Indeed there are a few tests for gen_filenames() in tests/utils_tests/ Could you try to add one as Carl says?

comment:5 Changed 6 years ago by Matthew Scouten

What version of django will the officially be fixed in? I ask because I have 1.8.9 and it still has this problem, and there is no 'target version' or 'fix version' on this site.

(I apologize if this is not an appropriate place to ask this question, but it is the only place I can find any information at all about this problem. )

comment:6 Changed 6 years ago by Tim Graham

There isn't a targeted version for the fix. It will be fixed when we have a patch (with a test) to review.

comment:7 Changed 3 years ago by Tom Forbes

Owner: changed from nobody to Tom Forbes
Status: newassigned

comment:8 Changed 3 years ago by Tom Forbes

Has patch: set

Edit: Wrong ticket, apologizes.

Version 1, edited 3 years ago by Tom Forbes (previous) (next) (diff)

comment:9 Changed 3 years ago by Tim Graham <timograham@…>

Resolution: fixed
Status: assignedclosed

In 1e92407:

Fixed #25624 -- Fixed autoreload crash with jinja2.ModuleLoader.

comment:10 Changed 3 years ago by Tim Graham <timograham@…>

In 728358c5:

[2.2.x] Fixed #25624 -- Fixed autoreload crash with jinja2.ModuleLoader.

Backport of 1e92407f83ed35be35f876777935b983ab9587be from master.

Note: See TracTickets for help on using tickets.
Back to Top