#27308 closed Cleanup/optimization (fixed)
BytesWarning exception raised when running with python 3 -bb option
Reported by: | Gustavo J. A. M. Carneiro | Owned by: | nobody |
---|---|---|---|
Component: | Core (Other) | Version: | dev |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Ready for checkin | |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
When your template file does not exist, naturally Django gives an exception. But if TEMPLATE_DEBUG is True, it also tries to generate a pretty html traceback, and this is where the bug is, if you try to run with the python 3 -bb option, which complains when you try to do str() on a bytes value.
I have Django version 1.10.1.
$ python3 -bb Python 3.5.2 (default, Sep 10 2016, 08:21:44) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>> str(b'') Traceback (most recent call last): File "<stdin>", line 1, in <module> BytesWarning: str() on a bytes instance >>> repr(b'') "b''" >>>
Traceback:
2016-10-03 17:47:29,582 ERROR production.py:138 GET /login -> TemplateDoesNotExist(b'loginxxx.html',) Traceback (most recent call last): File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 39, in inner response = get_response(request) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/utils/decorators.py", line 149, in _wrapped_view response = view_func(request, *args, **kwargs) File "/home/gjc/projects/molly/mollybet/apps/home/views.py", line 124, in login new_molly_url=settings.NEW_MOLLY_URL) File "/home/gjc/projects/molly/mollybet/webutils.py", line 167, in render t = loader.get_template((name + '.html').encode("ascii")) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/template/loader.py", line 25, in get_template raise TemplateDoesNotExist(template_name, chain=chain) django.template.exceptions.TemplateDoesNotExist: <unprintable TemplateDoesNotExist object> Internal Server Error: /login Traceback (most recent call last): File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 39, in inner response = get_response(request) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/utils/decorators.py", line 149, in _wrapped_view response = view_func(request, *args, **kwargs) File "/home/gjc/projects/molly/mollybet/apps/home/views.py", line 124, in login new_molly_url=settings.NEW_MOLLY_URL) File "/home/gjc/projects/molly/mollybet/webutils.py", line 167, in render t = loader.get_template((name + '.html').encode("ascii")) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/template/loader.py", line 25, in get_template raise TemplateDoesNotExist(template_name, chain=chain) django.template.exceptions.TemplateDoesNotExist: <unprintable TemplateDoesNotExist object> 2016-10-03 17:47:29,583 ERROR exception.py:124 Internal Server Error: /login Traceback (most recent call last): File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 39, in inner response = get_response(request) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/utils/decorators.py", line 149, in _wrapped_view response = view_func(request, *args, **kwargs) File "/home/gjc/projects/molly/mollybet/apps/home/views.py", line 124, in login new_molly_url=settings.NEW_MOLLY_URL) File "/home/gjc/projects/molly/mollybet/webutils.py", line 167, in render t = loader.get_template((name + '.html').encode("ascii")) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/template/loader.py", line 25, in get_template raise TemplateDoesNotExist(template_name, chain=chain) django.template.exceptions.TemplateDoesNotExist: <unprintable TemplateDoesNotExist object> 2016-10-03 17:47:29,599 ERROR production.py:138 GET /login -> BytesWarning('str() on a bytes instance',) Traceback (most recent call last): File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 39, in inner response = get_response(request) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/utils/decorators.py", line 149, in _wrapped_view response = view_func(request, *args, **kwargs) File "/home/gjc/projects/molly/mollybet/apps/home/views.py", line 124, in login new_molly_url=settings.NEW_MOLLY_URL) File "/home/gjc/projects/molly/mollybet/webutils.py", line 167, in render t = loader.get_template((name + '.html').encode("ascii")) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/template/loader.py", line 25, in get_template raise TemplateDoesNotExist(template_name, chain=chain) django.template.exceptions.TemplateDoesNotExist: <unprintable TemplateDoesNotExist object> During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 39, in inner response = get_response(request) File "/home/gjc/projects/molly/mollybet/apps/home/geoipblocking.py", line 17, in middleware return get_response(request) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 41, in inner response = response_for_exception(request, exc) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 86, in response_for_exception response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info()) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 128, in handle_uncaught_exception return debug.technical_500_response(request, *exc_info) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/views/debug.py", line 84, in technical_500_response html = reporter.get_traceback_html() File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/views/debug.py", line 316, in get_traceback_html c = Context(self.get_traceback_data(), use_l10n=False) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/views/debug.py", line 308, in get_traceback_data c['exception_value'] = smart_text(self.exc_value, errors='replace') File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/utils/encoding.py", line 41, in smart_text return force_text(s, encoding, strings_only, errors) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/utils/encoding.py", line 76, in force_text s = six.text_type(s) BytesWarning: str() on a bytes instance Internal Server Error: /login Traceback (most recent call last): File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 39, in inner response = get_response(request) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/utils/decorators.py", line 149, in _wrapped_view response = view_func(request, *args, **kwargs) File "/home/gjc/projects/molly/mollybet/apps/home/views.py", line 124, in login new_molly_url=settings.NEW_MOLLY_URL) File "/home/gjc/projects/molly/mollybet/webutils.py", line 167, in render t = loader.get_template((name + '.html').encode("ascii")) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/template/loader.py", line 25, in get_template raise TemplateDoesNotExist(template_name, chain=chain) django.template.exceptions.TemplateDoesNotExist: <unprintable TemplateDoesNotExist object> During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 39, in inner response = get_response(request) File "/home/gjc/projects/molly/mollybet/apps/home/geoipblocking.py", line 17, in middleware return get_response(request) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 41, in inner response = response_for_exception(request, exc) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 86, in response_for_exception response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info()) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 128, in handle_uncaught_exception return debug.technical_500_response(request, *exc_info) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/views/debug.py", line 84, in technical_500_response html = reporter.get_traceback_html() File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/views/debug.py", line 316, in get_traceback_html c = Context(self.get_traceback_data(), use_l10n=False) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/views/debug.py", line 308, in get_traceback_data c['exception_value'] = smart_text(self.exc_value, errors='replace') File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/utils/encoding.py", line 41, in smart_text return force_text(s, encoding, strings_only, errors) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/utils/encoding.py", line 76, in force_text s = six.text_type(s) BytesWarning: str() on a bytes instance 2016-10-03 17:47:29,601 ERROR exception.py:124 Internal Server Error: /login Traceback (most recent call last): File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 39, in inner response = get_response(request) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/utils/decorators.py", line 149, in _wrapped_view response = view_func(request, *args, **kwargs) File "/home/gjc/projects/molly/mollybet/apps/home/views.py", line 124, in login new_molly_url=settings.NEW_MOLLY_URL) File "/home/gjc/projects/molly/mollybet/webutils.py", line 167, in render t = loader.get_template((name + '.html').encode("ascii")) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/template/loader.py", line 25, in get_template raise TemplateDoesNotExist(template_name, chain=chain) django.template.exceptions.TemplateDoesNotExist: <unprintable TemplateDoesNotExist object> During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 39, in inner response = get_response(request) File "/home/gjc/projects/molly/mollybet/apps/home/geoipblocking.py", line 17, in middleware return get_response(request) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 41, in inner response = response_for_exception(request, exc) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 86, in response_for_exception response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info()) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 128, in handle_uncaught_exception return debug.technical_500_response(request, *exc_info) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/views/debug.py", line 84, in technical_500_response html = reporter.get_traceback_html() File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/views/debug.py", line 316, in get_traceback_html c = Context(self.get_traceback_data(), use_l10n=False) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/views/debug.py", line 308, in get_traceback_data c['exception_value'] = smart_text(self.exc_value, errors='replace') File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/utils/encoding.py", line 41, in smart_text return force_text(s, encoding, strings_only, errors) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/utils/encoding.py", line 76, in force_text s = six.text_type(s) BytesWarning: str() on a bytes instance 2016-10-03 17:47:29,687 ERROR production.py:138 GET /login -> BytesWarning('str() on a bytes instance',) Traceback (most recent call last): File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 39, in inner response = get_response(request) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/utils/decorators.py", line 149, in _wrapped_view response = view_func(request, *args, **kwargs) File "/home/gjc/projects/molly/mollybet/apps/home/views.py", line 124, in login new_molly_url=settings.NEW_MOLLY_URL) File "/home/gjc/projects/molly/mollybet/webutils.py", line 167, in render t = loader.get_template((name + '.html').encode("ascii")) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/template/loader.py", line 25, in get_template raise TemplateDoesNotExist(template_name, chain=chain) django.template.exceptions.TemplateDoesNotExist: <unprintable TemplateDoesNotExist object> During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 39, in inner response = get_response(request) File "/home/gjc/projects/molly/mollybet/apps/home/geoipblocking.py", line 17, in middleware return get_response(request) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 41, in inner response = response_for_exception(request, exc) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 86, in response_for_exception response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info()) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 128, in handle_uncaught_exception return debug.technical_500_response(request, *exc_info) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/views/debug.py", line 84, in technical_500_response html = reporter.get_traceback_html() File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/views/debug.py", line 316, in get_traceback_html c = Context(self.get_traceback_data(), use_l10n=False) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/views/debug.py", line 308, in get_traceback_data c['exception_value'] = smart_text(self.exc_value, errors='replace') File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/utils/encoding.py", line 41, in smart_text return force_text(s, encoding, strings_only, errors) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/utils/encoding.py", line 76, in force_text s = six.text_type(s) BytesWarning: str() on a bytes instance During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 39, in inner response = get_response(request) File "/home/gjc/projects/molly/mollybet/apps/home/auth.py", line 113, in middleware return get_response(request) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 41, in inner response = response_for_exception(request, exc) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 86, in response_for_exception response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info()) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/core/handlers/exception.py", line 128, in handle_uncaught_exception return debug.technical_500_response(request, *exc_info) File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/views/debug.py", line 84, in technical_500_response html = reporter.get_traceback_html() File "/home/gjc/projects/molly/env/lib/python3.4/site-packages/django/views/debug.py", line 317, in get_traceback_html return t.render(c) [...]
Change History (8)
comment:1 by , 8 years ago
comment:2 by , 8 years ago
-b Issue warnings about str(bytes_instance), str(bytear‐ ray_instance) and comparing bytes/bytearray with str. (-bb: issue errors)
The main point is to catch bugs due to mixing bytes vs str. Also, how many times have you done in Python 3 this type of bug:
>>> def create_redis_key(param1): ... return "foo:bar:" + str(param1) ... >>> create_redis_key("zbr") 'foo:bar:zbr'
But if param1 comes from e.g. a network socket and you forget to decode the bytes:
>>> create_redis_key(b"zbr") "foo:bar:b'zbr'"
So I like running my app with -bb (just one -b just gives a warning, but warnings often pass silently). So it is useful if Django works with -bb. I am certainly running a production Django app with -bb, for a few weeks, and this is the first problem that I see caused by it.
comment:3 by , 8 years ago
Component: | Uncategorized → Core (Other) |
---|---|
Triage Stage: | Unreviewed → Accepted |
Type: | Uncategorized → Cleanup/optimization |
The goal of this ticket could be to get Django's test suite passing with -bb
, but I'm not sure if it's feasible due to errors coming from dependencies like Pillow.
comment:4 by , 8 years ago
In the reported issue, I think the warning and the problem come from that line in the user's project where the template name is unexpectedly converted to bytes:
loader.get_template((name + '.html').encode("ascii"))
.
I see we are giving some bytestrings to some debug view tests in the Django test suite. It's probably fine to test that as long as we are supporting Python 2, but after dropping that support, we could simply assume exception messages are strings, not bytes. Unless Django is itself creating exception with bytes content on Python 3, which would be something to fix, I don't think we should do anything more now on Django's side.
comment:5 by , 8 years ago
Has patch: | set |
---|
PR to fix BytesWarnings
identified by Django's test suite.
comment:6 by , 8 years ago
Triage Stage: | Accepted → Ready for checkin |
---|---|
Version: | 1.10 → master |
I'm not familiar with the
-bb
flag. Can a bug happen as a result of this warning?Should we be trying to get Django's test suite to pass with it enabled? Currently there are 2 failures and 14 errors with a few exceptions coming from
PIL
rather than Django itself.