Autoreload fails to restart server when exception is SyntaxError.

[Peter J. Farrell]

Django: 2.2.3 using pywatchman with Watchman in a Docker container.

We are seeing this when there is a code syntax error in serializers for Django REST Framework. This causes the runserver to never reload even if the syntax error has been fixed and the file saved. The only way to resolve this is to restart the web container.

web_1      | Exception in thread django-main-thread:
web_1      | Traceback (most recent call last):
web_1      |   File "/usr/local/lib/python3.5/dist-packages/django/urls/resolvers.py", line 581, in url_patterns
web_1      |     iter(patterns)
web_1      | TypeError: 'module' object is not iterable
web_1      | 
web_1      | During handling of the above exception, another exception occurred:
web_1      | 
web_1      | Traceback (most recent call last):
web_1      |   File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
web_1      |     self.run()
web_1      |   File "/usr/lib/python3.5/threading.py", line 862, in run
web_1      |     self._target(*self._args, **self._kwargs)
web_1      |   File "/usr/local/lib/python3.5/dist-packages/django/utils/autoreload.py", line 54, in wrapper
web_1      |     fn(*args, **kwargs)
web_1      |   File "/usr/local/lib/python3.5/dist-packages/django/core/management/commands/runserver.py", line 117, in inner_run
web_1      |     self.check(display_num_errors=True)
web_1      |   File "/usr/local/lib/python3.5/dist-packages/django/core/management/base.py", line 390, in check
web_1      |     include_deployment_checks=include_deployment_checks,
web_1      |   File "/usr/local/lib/python3.5/dist-packages/django/core/management/base.py", line 377, in _run_checks
web_1      |     return checks.run_checks(**kwargs)
web_1      |   File "/usr/local/lib/python3.5/dist-packages/django/core/checks/registry.py", line 72, in run_checks
web_1      |     new_errors = check(app_configs=app_configs)
web_1      |   File "/usr/local/lib/python3.5/dist-packages/django/core/checks/urls.py", line 40, in check_url_namespaces_unique
web_1      |     all_namespaces = _load_all_namespaces(resolver)
web_1      |   File "/usr/local/lib/python3.5/dist-packages/django/core/checks/urls.py", line 57, in _load_all_namespaces
web_1      |     url_patterns = getattr(resolver, 'url_patterns', [])
web_1      |   File "/usr/local/lib/python3.5/dist-packages/django/utils/functional.py", line 80, in __get__
web_1      |     res = instance.__dict__[self.name] = self.func(instance)
web_1      |   File "/usr/local/lib/python3.5/dist-packages/django/urls/resolvers.py", line 588, in url_patterns
web_1      |     raise ImproperlyConfigured(msg.format(name=self.urlconf_name))
web_1      | django.core.exceptions.ImproperlyConfigured: The included URLconf 'XXXX.urls' does not appear to have any patterns in it. If you see valid patterns in the file then the issue is probably caused by a circular import.
web_1      | 

Whereas the same syntax in our Celery process outputs the right exception. When the syntax error is fixed -- the process restarts (we are using the autoreload process wrapped around our celery command in a custom management command for development):

celery_1   | Traceback (most recent call last):
celery_1   |   File "manage.py", line 10, in <module>
celery_1   |     execute_from_command_line(sys.argv)
celery_1   |   File "/usr/local/lib/python3.5/dist-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
celery_1   |     utility.execute()
celery_1   |   File "/usr/local/lib/python3.5/dist-packages/django/core/management/__init__.py", line 375, in execute
celery_1   |     self.fetch_command(subcommand).run_from_argv(self.argv)
celery_1   |   File "/usr/local/lib/python3.5/dist-packages/django/core/management/base.py", line 323, in run_from_argv
celery_1   |     self.execute(*args, **cmd_options)
celery_1   |   File "/usr/local/lib/python3.5/dist-packages/django/core/management/base.py", line 361, in execute
celery_1   |     self.check()
celery_1   |   File "/usr/local/lib/python3.5/dist-packages/django/core/management/base.py", line 390, in check
celery_1   |     include_deployment_checks=include_deployment_checks,
celery_1   |   File "/usr/local/lib/python3.5/dist-packages/django/core/management/base.py", line 377, in _run_checks
celery_1   |     return checks.run_checks(**kwargs)
celery_1   |   File "/usr/local/lib/python3.5/dist-packages/django/core/checks/registry.py", line 72, in run_checks
celery_1   |     new_errors = check(app_configs=app_configs)
celery_1   |   File "/usr/local/lib/python3.5/dist-packages/django/core/checks/urls.py", line 40, in check_url_namespaces_unique
celery_1   |     all_namespaces = _load_all_namespaces(resolver)
celery_1   |   File "/usr/local/lib/python3.5/dist-packages/django/core/checks/urls.py", line 57, in _load_all_namespaces
celery_1   |     url_patterns = getattr(resolver, 'url_patterns', [])
celery_1   |   File "/usr/local/lib/python3.5/dist-packages/django/utils/functional.py", line 80, in __get__
celery_1   |     res = instance.__dict__[self.name] = self.func(instance)
celery_1   |   File "/usr/local/lib/python3.5/dist-packages/django/urls/resolvers.py", line 579, in url_patterns
celery_1   |     patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
celery_1   |   File "/usr/local/lib/python3.5/dist-packages/django/utils/functional.py", line 80, in __get__
celery_1   |     res = instance.__dict__[self.name] = self.func(instance)
celery_1   |   File "/usr/local/lib/python3.5/dist-packages/django/urls/resolvers.py", line 572, in urlconf_module
celery_1   |     return import_module(self.urlconf_name)
celery_1   |   File "/usr/lib/python3.5/importlib/__init__.py", line 126, in import_module
celery_1   |     return _bootstrap._gcd_import(name[level:], package, level)
celery_1   |   File "<frozen importlib._bootstrap>", line 986, in _gcd_import
celery_1   |   File "<frozen importlib._bootstrap>", line 969, in _find_and_load
celery_1   |   File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
celery_1   |   File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
celery_1   |   File "<frozen importlib._bootstrap_external>", line 665, in exec_module
celery_1   |   File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
celery_1   |   File "/code/XXXXX/urls.py", line 22, in <module>
celery_1   |     path('', include('api.urls', namespace='api')),
celery_1   |   File "/usr/local/lib/python3.5/dist-packages/django/urls/conf.py", line 34, in include
celery_1   |     urlconf_module = import_module(urlconf_module)
celery_1   |   File "/usr/lib/python3.5/importlib/__init__.py", line 126, in import_module
celery_1   |     return _bootstrap._gcd_import(name[level:], package, level)
celery_1   |   File "<frozen importlib._bootstrap>", line 986, in _gcd_import
celery_1   |   File "<frozen importlib._bootstrap>", line 969, in _find_and_load
celery_1   |   File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
celery_1   |   File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
celery_1   |   File "<frozen importlib._bootstrap_external>", line 665, in exec_module
celery_1   |   File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
celery_1   |   File "/code/api/urls.py", line 5, in <module>
celery_1   |     from .versioned.v1.router import api_urlpatterns as api_v1
celery_1   |   File "/code/api/versioned/v1/router.py", line 3, in <module>
celery_1   |     from .viewsets import YYYSet, XXXViewSet
celery_1   |   File "/code/api/versioned/v1/viewsets.py", line 1, in <module>
celery_1   |     from ...base.viewsets import *
celery_1   |   File "/code/api/base/viewsets.py", line 3, in <module>
celery_1   |     from .serializers import YYYSerializer, XXXReportSerializer
celery_1   |   File "/code/api/base/serializers.py", line 201
celery_1   |     'url',,,,
celery_1   |           ^
celery_1   | SyntaxError: invalid syntax

[Keryn Knight]

This looks like the same exception reported in #30500. I'd guess knowing the point release of python3.5 being used might be of relevance, if so.

[Mariusz Felisiak]

Duplicate of #30500. It would be great if you could provide the point release of Python 3.5 in the original ticket.

[Peter J. Farrell]

Just to follow up, the version of Python we were running is Python 3.5.2 (default, Nov 12 2018, 13:43:14) [GCC 5.4.0 20160609] on linux