Code

Opened 12 months ago

Closed 6 months ago

#20338 closed Bug (fixed)

ALLOWED_HOSTS requires adding absolute domain

Reported by: manfre Owned by: nobody
Component: HTTP handling Version: master
Severity: Normal Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

The current ALLOWED_HOSTS implementation requires adding the absolute domain name (single trailing dot).

E.g.

ALLOWED_HOSTS = [
  '.src.org',
  '.src.org.',
]

Attachments (0)

Change History (14)

comment:1 Changed 12 months ago by anonymous

  • Has patch set
  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

comment:2 Changed 12 months ago by carljm

  • Triage Stage changed from Unreviewed to Accepted

I think it would _probably_ be safe for Django to automatically strip the final dot from an FQDN in a Host header before checking it against ALLOWED_HOSTS, but it's possible that could open an exploit in cases where the user's DNS setup results in the absolute and non-absolute versions of the domain name resolving to different hosts. So I think it's safer if we address this via documentation.

comment:3 Changed 12 months ago by Carl Meyer <carl@…>

  • Resolution set to fixed
  • Status changed from new to closed

In 6bdeed1b811ddf9a920e925ad05d82cffbf13c3a:

Fixed #20338 - Document FQDN behavior with ALLOWED_HOSTS

comment:4 Changed 7 months ago by mdrozdziel@…

This documentation is wrong and misleading. This behaviour should be changed. User has no control if the request will have a trailing dot or not. Its not a matter of supporting fqdn or not - this suggestion is incorrect. Currently everyone basically have to include both dot and non-dot version. If someone will link to the site with trailing dot the request will contain the trailing dot in the Host header, and if ALLOWED_HOSTS doesn't include version with the dot - exception will be raised.

I am not sure what what possible exploit you are reffering to, but current configuration causes sites to randomly throw exceptions with no clear explanation on what is going on.

comment:5 Changed 7 months ago by anonymous

  • Resolution fixed deleted
  • Status changed from closed to new

comment:6 Changed 7 months ago by aaugustin

In my admittedly limited experience, I haven't ever encountered a site that used a trailing dot -- something like "http://facebook.com./", I suppose. Could you show us some examples of sites that require being accessed in this fashion?

Is there a problem with redirecting to a canonical hostname without a trailing dot at the level of the front web server (ie. before the request reaches Django)?

Last edited 7 months ago by aaugustin (previous) (diff)

comment:7 Changed 7 months ago by mdrozdziel@…

I don't know if there are sites which _require_ trailing dot, but that is beside the point here. You can of course "sanitize" hostname(s) in the server configuration (I guess most if not all modern webservers allow such configuration). There are however problems with current situation.

If you ask to redirect to canonical hostname, why there is ALLOWED_HOSTS at all in the first place? You can as easily restrict the list of passed hostnames on the server level, before the request reaches Django. If you have no problem with stripping the final dot at the front web server level, why do you hesitate to strip that dot while processing the request in Django?

Currently Apache (I haven't tested other web servers) without special configuration will pass the host header with trailing dot to Django. It's enough that someone will post a link with a trailing period to your website and this will result in a error 500 and pretty weird stack trace without clear explanation why the exception was thrown. The main problem is, that this link will be entirely valid and lots of people will have to waste good chuck of time to investigate why the problem is there (we already blamed this to a random bug in Django, before we spotted the dot).

Basically current situation requires one of two solutions. Either place both versions (with and without dot) in the ALLOWED_HOSTS clause or setup a web server in a very special way to strip the trailing dot, just in case someone will someday will send that kind of request. Both solutions are pretty dull and unreasonable. This becomes even more unreasonable if the site is suppose to support lots of different domains.

Despite the fact, that stripping the dot at the front web server level is technically possible, its not always easily achievable. I guess this might be a problem on a shared hosting where you can't access that configuration. In other cases this my require unnecessary cooperation with other departments, etc. If your front web server setup for given app is complicated another rule like this will introduce huge complication.

We had no issues with current situation until someone posted a link to our site which ended up with dot (this could happen to the fact, that it was at the end of the sentence and probably got automatically parsed over to a html a-tag with the trailing dot included). Suddenly we started getting more and more stack traces, users started getting 500 errors. We had to spend time to investigate and fix that problem.

All I am trying to achieve here, is to prevent this kind of situation from happening to someone else again. Asking people to set a specific rewrite just to prevent this rare situation is not a solution, nor is asking everyone to include double entries in ALLOWED_HOSTS. Last one is actually against any common sense what so ever.

I believe that stripping the dot at the Django level is one reasonable option. I don't really see what kind of security hole could that possibly introduce. I am not a Python developer, but I guess that something like https://github.com/mdrozdziel/django/commit/db9e09b855f0307ac88a1af55dd2034d6339663a should solve the issue.

comment:8 Changed 7 months ago by carljm

I don't have much objection to just automatically stripping the trailing dot; I think mariusz is right that in general that will do the right thing. I think I was just being paranoid earlier about opening up possible holes in the ALLOWED_HOSTS mechanism after we'd just gone to so much trouble to put it in place. I would like to get input from @dstufft and/or @PaulM before we make this change, though.

comment:9 Changed 7 months ago by claudep

I'm in favour of ignoring this trailing dot. I have one site where I had to tweak ALLOWED_HOSTS because some strange bot was appending that famous dot. I guess I could easily trigger 500 errors on many Django sites...

comment:10 Changed 7 months ago by Mariusz Droździel <mdrozdziel@…>

@claudep Indeed. http://f.cl.ly/items/1W1M0f441Z2y3E0f1I3y/Zrzut%20ekranu%202013-09-15%20o%2013.05.52.png

comment:11 Changed 7 months ago by timo

  • Component changed from Documentation to HTTP handling
  • Needs tests set
  • Version changed from 1.4 to master

comment:12 Changed 6 months ago by claudep

  • Needs tests unset

comment:13 Changed 6 months ago by timo

  • Triage Stage changed from Accepted to Ready for checkin

comment:14 Changed 6 months ago by Claude Paroz <claude@…>

  • Resolution set to fixed
  • Status changed from new to closed

In c052699be3637c22e3a26383a4bdabc8c3cc0feb:

Fixed #20338 -- Stripped ending dot during host validation

Thanks manfre for the report and Timo Graham for the review.

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.