Django 2.2 check fails under sourceless project

[Sascha P.]

Hi,
We are shipping python in docker containers in bytecode for legal purposes and obfuscation. With the new Django 2.2 this stopped working, due to an encoding issue. To reproduce this error we receive, we created a minimal dockerfile:

FROM python:alpine3.6
 
RUN apk upgrade --no-cache && \
    pip install Django==2.2.0 && \
    django-admin startproject mysite && \
    python3 mysite/manage.py makemigrations && \
    python3 -m compileall -b -f mysite && \
    find mysite -type f -name '*.py' | xargs rm && \
    mv mysite/manage.pyc mysite/manage.py && \
    python3 mysite/manage.py makemigrations
 
 
CMD /bin/sh

You will get this error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/django/template/utils.py", line 66, in __getitem__
    return self._engines[alias]
KeyError: 'django'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "mysite/manage.py", line 21, in <module>
  File "mysite/manage.py", line 17, in main
  File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 323, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 361, in execute
    self.check()
  File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 390, in check
    include_deployment_checks=include_deployment_checks,
  File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 377, in _run_checks
    return checks.run_checks(**kwargs)
  File "/usr/local/lib/python3.6/site-packages/django/core/checks/registry.py", line 72, in run_checks
    new_errors = check(app_configs=app_configs)
  File "/usr/local/lib/python3.6/site-packages/django/contrib/admin/checks.py", line 80, in check_dependencies
    for engine in engines.all():
  File "/usr/local/lib/python3.6/site-packages/django/template/utils.py", line 90, in all
    return [self[alias] for alias in self]
  File "/usr/local/lib/python3.6/site-packages/django/template/utils.py", line 90, in <listcomp>
    return [self[alias] for alias in self]
  File "/usr/local/lib/python3.6/site-packages/django/template/utils.py", line 81, in __getitem__
    engine = engine_cls(params)
  File "/usr/local/lib/python3.6/site-packages/django/template/backends/django.py", line 23, in __init__
    options.setdefault('file_charset', settings.FILE_CHARSET)
  File "/usr/local/lib/python3.6/site-packages/django/conf/__init__.py", line 134, in FILE_CHARSET
    stack = traceback.extract_stack()
  File "/usr/local/lib/python3.6/traceback.py", line 207, in extract_stack
    stack = StackSummary.extract(walk_stack(f), limit=limit)
  File "/usr/local/lib/python3.6/traceback.py", line 358, in extract
    f.line
  File "/usr/local/lib/python3.6/traceback.py", line 282, in line
    self._line = linecache.getline(self.filename, self.lineno).strip()
  File "/usr/local/lib/python3.6/linecache.py", line 16, in getline
    lines = getlines(filename, module_globals)
  File "/usr/local/lib/python3.6/linecache.py", line 47, in getlines
    return updatecache(filename, module_globals)
  File "/usr/local/lib/python3.6/linecache.py", line 137, in updatecache
    lines = fp.readlines()
  File "/usr/local/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8d in position 4: invalid start byte

If you use Django==2.1.8 instead you will get the expected "No changes detected" and everything works fine. There has been some work around "FILE_CHARSET" with Django 2.2.0, so maybe thats where this comes from.

We normally always use the newest Django version, yet with this one we had to pin the version to 2.1.8.

If this is not a bug caused by Django, then Im sorry for reporting this, yet I hope that someone could point me to the right direction.

Best regards
Sascha

[Florian Apolloner]

Renaming a .pyc file to .py is not anything that is supposed to work.

[Tim Bell]

The deprecation check for settings.FILE_CHARSET added in #29817 uses traceback.extract_stack() to find out where the setting is being accessed from. When a project's .py files have been deleted, leaving only the .pyc files, extract_stack() throws an exception as it tries to read the .pyc file.

Running manage.py check will tickle this "bug" when it checks the template engines.

[Florian Apolloner]

That is not exactly true, at least not in this case. In this case the issue is caused by the fact that they rename the pyc file to py which extract_stack then fails on because a py file is supposed to be readable…

[Sascha P.]

Thanks at everyone. At least I know now into which direction to investigate.