Django violates RFC7230 when handling requests.

[Stavros Korokithakis]

For a request coming in with an absolute URI and a different host header, Django still uses the Host header value to service the request. RFC 7230 specifies:

If the request-target is in absolute-form, the effective request URI is the same as the request-target.

(​https://tools.ietf.org/html/rfc7230#section-5.5)

Thus, if a request comes in where the host header is different from the host in the absolute URI, Django should use the absolute URI, rather than the host value.

This is a problem when a request comes in looking like:

GET https://valid.hostname/ HTTP/1.1
Host: invalid.hostname

Django currently fails this as a violation of ALLOWED_HOSTS, but it shouldn't. Granted, we only see this in attacks, but nginx passes these requests through (because it should) and Django fails them because of the wonky host.

[Florian Apolloner]

I think this depends on how you configured Nginx. If Nginx works as a proxy for your application, this applies ( ​https://tools.ietf.org/html/rfc7230#section-5.4 ):

   When a proxy receives a request with an absolute-form of
   request-target, the proxy MUST ignore the received Host header field
   (if any) and instead replace it with the host information of the
   request-target.  A proxy that forwards such a request MUST generate a
   new Host field-value based on the received request-target rather than
   forward the received Host field-value.

If Nginx does not work as a proxy, it should reject requests which are not compliant to the RFC ( ​https://tools.ietf.org/html/rfc7230#section-5.3 ):

   When making a request directly to an origin server, other than a
   CONNECT or server-wide OPTIONS request (as detailed below), a client
   MUST send only the absolute path and query components of the target
   URI as the request-target.  If the target URI's path component is
   empty, the client MUST send "/" as the path within the origin-form of
   request-target.  A Host header field is also sent, as defined in
   Section 5.4.

In this case it is imo questionable if Nginx should pass an invalid request on at all, or answer with 400 already as per ( ​https://tools.ietf.org/html/rfc7230#section-5.4 ):

   A server MUST respond with a 400 (Bad Request) status code to any
   HTTP/1.1 request message that lacks a Host header field and to any
   request message that contains more than one Host header field or a
   Host header field with an invalid field-value.

It is questionable on whether an invalid request makes the Host header in this case invalid, but well.

Either way, I cannot see any reason for Django to consider such a request not as an attack, and therefore the last paragraph of ( ​https://tools.ietf.org/html/rfc7230#section-5.5 ) applies:

   Once the effective request URI has been constructed, an origin server
   needs to decide whether or not to provide service for that URI via
   the connection in which the request was received.  For example, the
   request might have been misdirected, deliberately or accidentally,
   such that the information within a received request-target or Host
   header field differs from the host or port upon which the connection
   has been made.  If the connection is from a trusted gateway, that
   inconsistency might be expected; otherwise, it might indicate an
   attempt to bypass security filters, trick the server into delivering
   non-public content, or poison a cache.  See Section 9 for security
   considerations regarding message routing.

Django chooses not to honour this request cause the host-header is not configured as it expects it too. This is a valid reason to reject the request, cause after all a valid client MUST send a valid host-header and technically is not allowed to send a full request URI. So unless there is any evidence that this has legitimate use cases, I suggest we leave it as is to keep the code simple.

[Stavros Korokithakis]

I would be fine with that, especially given the rationale, except for the fact that nginx does pass the request through, and I get thousands of emails. This has to be handled *somewhere*, otherwise I need to just completely disable invalid host reporting, and people in #nginx seemed to think that nginx is handling this compliantly.

[Florian Apolloner]

Replying to Stavros Korokithakis:

and I get thousands of emails. This has to be handled *somewhere*, otherwise I need to just completely disable invalid host reporting

Yes, disable it in your LOGGING config, can you confirm that this does not happen with the default Django Logging config? Ie set logging to {}

[Florian Apolloner]

Oh, I found something interesting in ( ​https://tools.ietf.org/html/rfc7230#section-5.3.2 ):

   To allow for transition to the absolute-form for all requests in some
   future version of HTTP, a server MUST accept the absolute-form in
   requests, even though HTTP/1.1 clients will only send them in
   requests to proxies.

That said, now that HTTP/2 exists and is the successor of 1.1, we should look what that spec requires. Either way, I think Django should not concern itself to much with it, since it is not a server but just a library rejecting an invalid request.

Arguably there is actually a bug, ie if a client sends:

GET http://valid.com/ HTTP/1.1
Host: alsovalid.com

but I cannot figure out a sane reason to do this :D

[Tim Graham]

I tried curl --verbose --header 'Host: www.example.com' 'https://www.djangoproject.com' and it didn't cause a problem so I guess we aren't affected by the problematic nginx configuration?

What's the next step in evaluating this ticket?

[Florian Apolloner]

That is weird, the response should be a 400, I get empty reply when I try via curl using your example.

[Tim Graham]

I thought it's expected that nginx doesn't give a reply for invalid hosts? I cannot find a source for this idea though.

[Florian Apolloner]

if you look into the ansible repo roles/webserver/files/sites/default should return 444.