Autoreload: dictionary changed size during iteration.

[minusf]

Found this issue recently, not 100% reproducable. Not a huge project, actually a quite small one. macOS catalina, 10.15.3, python from brew (/usr/local/bin/python3) 3.7.7, Django 2.2.11, could not reproduce with 3.0.4 in light testing, but maybe it's useful to have this stacktrace here. (can't move to 3 now because of dependencies btw).

2020-03-17 14:48:33,118 INFO: autoreload.py:597: Watching for file changes with WatchmanReloader
Performing system checks...

Traceback (most recent call last):
  File "manage.py", line 22, in <module>
    main()
  File "manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/Users/user/src/venv/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/Users/user/src/venv/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/user/src/venv/lib/python3.7/site-packages/django/core/management/base.py", line 323, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/user/src/venv/lib/python3.7/site-packages/django/core/management/commands/runserver.py", line 60, in execute
    super().execute(*args, **options)
  File "/Users/user/src/venv/lib/python3.7/site-packages/django/core/management/base.py", line 364, in execute
    output = self.handle(*args, **options)
  File "/Users/user/src/venv/lib/python3.7/site-packages/django/core/management/commands/runserver.py", line 95, in handle
    self.run(**options)
  File "/Users/user/src/venv/lib/python3.7/site-packages/django/core/management/commands/runserver.py", line 102, in run
    autoreload.run_with_reloader(self.inner_run, **options)
  File "/Users/user/src/venv/lib/python3.7/site-packages/django/utils/autoreload.py", line 598, in run_with_reloader
    start_django(reloader, main_func, *args, **kwargs)
  File "/Users/user/src/venv/lib/python3.7/site-packages/django/utils/autoreload.py", line 583, in start_django
    reloader.run(django_main_thread)
  File "/Users/user/src/venv/lib/python3.7/site-packages/django/utils/autoreload.py", line 301, in run
    self.run_loop()
  File "/Users/user/src/venv/lib/python3.7/site-packages/django/utils/autoreload.py", line 307, in run_loop
    next(ticker)
  File "/Users/user/src/venv/lib/python3.7/site-packages/django/utils/autoreload.py", line 520, in tick
    self.update_watches()
  File "/Users/user/src/venv/lib/python3.7/site-packages/django/utils/autoreload.py", line 493, in update_watches
    self._update_watches()
  File "/Users/user/src/venv/lib/python3.7/site-packages/django/utils/autoreload.py", line 476, in _update_watches
    watched_files = list(self.watched_files(include_globs=False))
  File "/Users/user/src/venv/lib/python3.7/site-packages/django/utils/autoreload.py", line 262, in watched_files
    yield from iter_all_python_module_files()
  File "/Users/user/src/venv/lib/python3.7/site-packages/django/utils/autoreload.py", line 101, in iter_all_python_module_files
    modules_view = sorted(list(sys.modules.items()), key=lambda i: i[0])
RuntimeError: dictionary changed size during iteration

[minusf]

forgot to add these:

pywatchman           1.4.1

$ brew info watchman
watchman: stable 4.9.0 (bottled), HEAD

[Simon Charette]

Pretty sure this was inadvertently fixed in 3.0 by 1835563ab8a1da949a35316d16989236ac585c6c since sys.module.items isn't used anymore and we only perform __getitem__ on a fixed collection of modules.

Since 2.2 is currently only in maintenance mode I don't think this commit will be backported there.

[minusf]

would you be so kind and explained the core issue? what was touching sys.modules while beeing looped over it?

[Mariusz Felisiak]

It's hard to say why dictionary changed size during iteration, but I agree with Simon that it's not an issue in Django 3.0. Unfortunately Django 2.2 is in extended support so it's not receiving such bugfixes/cleanups anymore.