ALLOWED_HOSTS requires adding absolute domain

[Michael Manfre]

The current ALLOWED_HOSTS implementation requires adding the absolute domain name (single trailing dot).

E.g.

ALLOWED_HOSTS = [
  '.src.org',
  '.src.org.',
]

[anonymous]

​https://github.com/django/django/pull/1036

[Carl Meyer]

I think it would _probably_ be safe for Django to automatically strip the final dot from an FQDN in a Host header before checking it against ALLOWED_HOSTS, but it's possible that could open an exploit in cases where the user's DNS setup results in the absolute and non-absolute versions of the domain name resolving to different hosts. So I think it's safer if we address this via documentation.

[Carl Meyer <carl@…>]

In 6bdeed1b811ddf9a920e925ad05d82cffbf13c3a:

Fixed #20338 - Document FQDN behavior with ALLOWED_HOSTS

[mdrozdziel@…]

This documentation is wrong and misleading. This behaviour should be changed. User has no control if the request will have a trailing dot or not. Its not a matter of supporting fqdn or not - this suggestion is incorrect. Currently everyone basically have to include both dot and non-dot version. If someone will link to the site with trailing dot the request will contain the trailing dot in the Host header, and if ALLOWED_HOSTS doesn't include version with the dot - exception will be raised.

I am not sure what what possible exploit you are reffering to, but current configuration causes sites to randomly throw exceptions with no clear explanation on what is going on.

[Aymeric Augustin]

In my admittedly limited experience, I haven't even encountered a site that used a trailing dot -- something like "​http://facebook.com./", I suppose. Could you show us some examples of sites that require being accessed in this fashion?

Is there a problem with redirecting to a canonical hostname without a trailing dot at the level of the front web server (ie. before the request reaches Django)?

[mdrozdziel@…]

I don't know if there are sites which _require_ trailing dot, but that is beside the point here. You can of course "sanitize" hostname(s) in the server configuration (I guess most if not all modern webservers allow such configuration). There are however problems with current situation.

If you ask to redirect to canonical hostname, why there is ALLOWED_HOSTS at all in the first place? You can as easily restrict the list of passed hostnames on the server level, before the request reaches Django. If you have no problem with stripping the final dot at the front web server level, why do you hesitate to strip that dot while processing the request in Django?

Currently Apache (I haven't tested other web servers) without special configuration will pass the host header with trailing dot to Django. It's enough that someone will post a link with a trailing period to your website and this will result in a error 500 and pretty weird stack trace without clear explanation why the exception was thrown. The main problem is, that this link will be entirely valid and lots of people will have to waste good chuck of time to investigate why the problem is there (we already blamed this to a random bug in Django, before we spotted the dot).

Basically current situation requires one of two solutions. Either place both versions (with and without dot) in the ALLOWED_HOSTS clause or setup a web server in a very special way to strip the trailing dot, just in case someone will someday will send that kind of request. Both solutions are pretty dull and unreasonable. This becomes even more unreasonable if the site is suppose to support lots of different domains.

Despite the fact, that stripping the dot at the front web server level is technically possible, its not always easily achievable. I guess this might be a problem on a shared hosting where you can't access that configuration. In other cases this my require unnecessary cooperation with other departments, etc. If your front web server setup for given app is complicated another rule like this will introduce huge complication.

We had no issues with current situation until someone posted a link to our site which ended up with dot (this could happen to the fact, that it was at the end of the sentence and probably got automatically parsed over to a html a-tag with the trailing dot included). Suddenly we started getting more and more stack traces, users started getting 500 errors. We had to spend time to investigate and fix that problem.

All I am trying to achieve here, is to prevent this kind of situation from happening to someone else again. Asking people to set a specific rewrite just to prevent this rare situation is not a solution, nor is asking everyone to include double entries in ALLOWED_HOSTS. Last one is actually against any common sense what so ever.

I believe that stripping the dot at the Django level is one reasonable option. I don't really see what kind of security hole could that possibly introduce. I am not a Python developer, but I guess that something like ​https://github.com/mdrozdziel/django/commit/db9e09b855f0307ac88a1af55dd2034d6339663a should solve the issue.

[Carl Meyer]

I don't have much objection to just automatically stripping the trailing dot; I think mariusz is right that in general that will do the right thing. I think I was just being paranoid earlier about opening up possible holes in the ALLOWED_HOSTS mechanism after we'd just gone to so much trouble to put it in place. I would like to get input from @dstufft and/or @PaulM before we make this change, though.

[Claude Paroz]

I'm in favour of ignoring this trailing dot. I have one site where I had to tweak ALLOWED_HOSTS because some strange bot was appending that famous dot. I guess I could easily trigger 500 errors on many Django sites...

[Mariusz Droździel <mdrozdziel@…>]

@claudep Indeed.

[Claude Paroz]

Pull request: ​https://github.com/django/django/pull/1805

[Claude Paroz <claude@…>]

In c052699be3637c22e3a26383a4bdabc8c3cc0feb:

Fixed #20338 -- Stripped ending dot during host validation

Thanks manfre for the report and Timo Graham for the review.

[Jaap Roes]

I really wish this fix would've been backported. It's not necessarily a security issue, but the behavior is annoying and can be abused to fill up error logs on sites that are known to use specific versions of Django.