Attempting to access a manage.py runserver instance over HTTPS raises UnicodeDecodeError

[Robert Rollins]

I realize that the runserver command is not designed to function over HTTPS, but it would be much more useful for users who don't realize that (like the 1-hour-ago version of me) if the exception that Django throws from HTTPS access attempts was not an apparently random UnicodeDecodeError from the logging system.

Right now, with Django 1.6.5, attempting to access any URL on a runserver instance over HTTPS will result in this traceback being shown multiple times in the server console:

Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/SocketServer.py", line 560, in process_request_thread
    self.finish_request(request, client_address)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/SocketServer.py", line 322, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/srv/ads/django/tmtpds/ve/lib/python2.6/site-packages/django/core/servers/basehttp.py", line 126, in __init__
    super(WSGIRequestHandler, self).__init__(*args, **kwargs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/SocketServer.py", line 617, in __init__
    self.handle()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/wsgiref/simple_server.py", line 131, in handle
    if not self.parse_request(): # An error code has been sent, just exit
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/BaseHTTPServer.py", line 289, in parse_request
    self.send_error(400, "Bad request syntax (%r)" % requestline)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/BaseHTTPServer.py", line 357, in send_error
    self.send_response(code, message)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/BaseHTTPServer.py", line 374, in send_response
    self.log_request(code)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/BaseHTTPServer.py", line 411, in log_request
    self.requestline, str(code), str(size))
  File "/srv/ads/django/tmtpds/ve/lib/python2.6/site-packages/django/core/servers/basehttp.py", line 138, in log_message
    msg = "[%s] %s\n" % (self.log_date_time_string(), format % args)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xcd in position 12: ordinal not in range(128)

It's not always exactly the same byte having problems, but it's always line 138 in django/core/servers/basehttp.py. The server is attempting to log an "HTTP 400: Bad request syntax" error, but it's failing to decode the URL from what it seems to think is unicode to ascii. From my googling, it appears that this is happening because the browser encrypts the URL, and runserver is getting that encrypted URL and having no idea what to do with it.

It would be really helpful for uniformed users like my past self if this error were reported as "You can't use HTTPS with runserver" rather than what we're currently getting.

[Aymeric Augustin]

Yes, I recently saw that nginx does something smart like this, it's a nice touch.

[Aymeric Augustin]

In fact there are two fairly independant parts to this ticket.

1. A bug: UnicodeDecodeError is raised. This is most likely a regression introduced when we ported Django to Python 3.

2. A feature request: providing a helpful message when the dev server receives a HTTPS request.

Here's the fix for 1 and a quick proposal for 2.

diff --git a/django/core/servers/basehttp.py b/django/core/servers/basehttp.py
index 010b5e0..c522d1b 100644
--- a/django/core/servers/basehttp.py
+++ b/django/core/servers/basehttp.py
@@ -135,7 +136,7 @@ class WSGIRequestHandler(simple_server.WSGIRequestHandler, object):
                 or self.path == '/favicon.ico'):
             return
 
-        msg = "[%s] %s\n" % (self.log_date_time_string(), format % args)
+        msg = str("[%s] %s\n") % (self.log_date_time_string(), format % args)
 
         # Utilize terminal colors, if available
         if args[1][0] == '2':
@@ -150,6 +151,10 @@ class WSGIRequestHandler(simple_server.WSGIRequestHandler, object):
         elif args[1] == '404':
             msg = self.style.HTTP_NOT_FOUND(msg)
         elif args[1][0] == '4':
+            # 0x16 = Handshake, 0x03 = SSL 3.0 or TLS 1.x
+            if args[0].startswith(str('\x16\x03')):
+                msg = ("You're accessing the developement server over HTTPS, "
+                       "but it only supports HTTP.\n")
             msg = self.style.HTTP_BAD_REQUEST(msg)
         else:
             # Any 5XX, or any other response

[Flavio Curella]

should we split this ticket into two? (unicode and https?)

[Flavio Curella]

initial PR up at ​https://github.com/django/django/pull/3167

[Flavio Curella]

not sure how or even where to look for existing tests that could be extended. Any pointer is appreciated

[Flavio Curella]

unchecked 'needs tests' as this might be too hard to test for, or not worth it

[Collin Anderson]

sorry i merge-confliced your change :)

[Collin Anderson]

depends on #23429.

[Tim Graham]

Test fails on Python 3.

[Flavio Curella]

test fixed

[Tim Graham <timograham@…>]

In c53f2451a83fc081c1886d07c390cf0447b1143f:

Fixed #23398 -- Added helpful error message when runserver is accessed via HTTPS

[Aymeric Augustin]

Thanks for completing the patch!