Opened 7 years ago

Closed 7 years ago

Last modified 7 years ago

#28488 closed Bug (fixed)

Django 1.11+ raises CSRF verification failed if settings.DEBUG is False and an intermediate 404 page is requested

Reported by: Ruben Alves Owned by: nobody
Component: CSRF Version: dev
Severity: Release blocker Keywords: csrf failed settings debug false production
Cc: rubenanapu@…, Nicola, Jay Cox, Florian Apolloner, Berker Peksag Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Ruben Alves)

Hi all,

I'm using Django1.11 (and made tests also with Django1.11.4) and having problems when submitting a form with POST method.

I'm calling the {% csrf_token %} inside of the form, so, this is not the problem.
The problem when submitting the form is:

Forbidden (403)
CSRF verification failed. Request aborted.
More information is available with DEBUG=True.

Then, in order to see "more information", I've enabled settings.DEBUG to True and submitted the form again. At this moment, the problem didn't happens anymore.

So I've disabled settings.DEBUG, submitted again, and the problem was there. Enabled DEBUG=True again, problem has gone.

Initially I thought that could be some error in my code, but the same happens when I try to reset my password wit the django.contrib.auth.views.password_reset view.

In my settings.py, I have the following changes that were made recently:

SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True

I use AWS (Amazon Web Service) Elastic Beanstalk with https enabled.

The worst part is that I've discovered this only on production because I make all tests in my local machine with DEBUG=True, and on production, we set DEBUG=False. Also, the error message should be shown only if DEBUG=True.

It's something related to the same error mentioned on Google Groups https://groups.google.com/forum/#!searchin/django-users/CSRF$20verification$20failed.$20Request$20aborted.$20More$20information$20is$20available$20with$20DEBUG$3DTrue.%7Csort:relevance/django-users/ISoJ6CwHOXQ/pirLih0jBgAJ

Attachments (3)

error.png (9.9 KB ) - added by Ruben Alves 7 years ago.
Error message on production with settings.DEBUG=False
ticket28488.zip (18.4 KB ) - added by Nicola 7 years ago.
sample project that show the issue
t28488.diff (1.2 KB ) - added by Florian Apolloner 7 years ago.

Download all attachments as: .zip

Change History (44)

by Ruben Alves, 7 years ago

Attachment: error.png added

Error message on production with settings.DEBUG=False

comment:1 by Ruben Alves, 7 years ago

Description: modified (diff)

comment:2 by Tim Graham, 7 years ago

Did you experience a change in Django 1.11 compared to Django 1.10 or earlier? I think you'll need to debug the problem further and let us know why Django is at fault.

comment:3 by Ruben Alves, 7 years ago

Before Django1.11, I was using Django 1.8 and everything was working perfectly.

In order to use Django1.11, I did all the changes mentioned on https://docs.djangoproject.com/en/1.11/releases/1.11/

What I've done now was add settings.CSRF_FAILURE_VIEW = 'courses.views.csrf_failure' with DEBUG=False

I made my courses.views.csrf_failure raise an exception, so we could see the traceback.

The traceback is the following:

File "/opt/python/run/venv/lib64/python2.7/site-packages/django/core/handlers/exception.py" in inner

  1. response = get_response(request)

File "/opt/python/run/venv/lib64/python2.7/site-packages/django/core/handlers/base.py" in _legacy_get_response

  1. response = self._get_response(request)

File "/opt/python/run/venv/lib64/python2.7/site-packages/django/core/handlers/base.py" in _get_response

  1. response = middleware_method(request, callback, callback_args, callback_kwargs)

File "/opt/python/run/venv/lib64/python2.7/site-packages/django/middleware/csrf.py" in process_view

  1. return self._reject(request, REASON_BAD_TOKEN)

File "/opt/python/run/venv/lib64/python2.7/site-packages/django/middleware/csrf.py" in _reject

  1. return _get_failure_view()(request, reason=reason)

File "/opt/python/current/app/courses/views/init.py" in csrf_failure

  1. raise Exception('Lets see where this shit works:' + str(locals()))
Version 0, edited 7 years ago by Ruben Alves (next)

comment:4 by Tim Graham, 7 years ago

The traceback doesn't give me any ideas about the cause. I think you'll need to add further logging in django/middleware/csrf.py to find the root cause.

A large CSRF change between Django 1.8 and 1.11 is 5112e65ef2df1dbb95ff83026b6a962fb2688661. Perhaps you can test with Django 1.9 and 1.10 to help narrow the problem.

In the backwards-incompatible changes section of the 1.10 release notes, there's a note, "CSRF token values are now required to be strings of 64 alphanumerics; values of 32 alphanumerics, as set by older versions of Django by default, are automatically replaced by strings of 64 characters. Other values are considered invalid. This should only affect developers or users who replace these tokens." Maybe it's relevant to you.

Last edited 7 years ago by Tim Graham (previous) (diff)

comment:5 by Ruben Alves, 7 years ago

Ok, I've added some logs on django.middleware.csrf, and this is what I have:

Between lines 208 and 209 (https://github.com/django/django/blob/stable/1.11.x/django/middleware/csrf.py#L208) I've added a log and the value of csrf_token is vcdodHkRb8jlxQP5fnTjuQp5i3PMWYYEBOYfpOFkCEjWWwHMpJ9uqaPGI6vGi6hS

The if request.is_secure(): is True on line 228 (https://github.com/django/django/blob/stable/1.11.x/django/middleware/csrf.py#L228)

The value of request_csrf_token on line 311 (https://github.com/django/django/blob/stable/1.11.x/django/middleware/csrf.py#L311) before we call request_csrf_token = _sanitize_token(request_csrf_token) is tMgl4aTrdjOEShUax3Gz1CJLbvnhBWiEVzmzgTxSclDuP01lBfwoE2R6dDyljaCQ

After _sanitize_token be called, the value of request_csrf_token is the same, and the value of csrf_token is vcdodHkRb8jlxQP5fnTjuQp5i3PMWYYEBOYfpOFkCEjWWwHMpJ9uqaPGI6vGi6hS.

Finally _compare_salted_tokens(request_csrf_token, csrf_token) on line 313 returns False (https://github.com/django/django/blob/stable/1.11.x/django/middleware/csrf.py#L313).

comment:6 by Tim Graham, 7 years ago

Maybe you can try to minimize the middleware, etc. in your project and see if something like that is causing the issue.

comment:7 by Ruben Alves, 7 years ago

I've being testing to reload the page and see the Cookie named csrftoken.

On the system that runs with Django1.8, every time I reload the page, the csrftoken Cookie has the same value.

On the system that runs with Django1.11, every time I reload the page, the csrftoken Cookie has a different value.

Do you know if on Django1.11 is expected to have a different csrftoken Cookie every time I reload the page? If yes, why is expected to be different in Django1.11 and not in Django1.8?

comment:8 by Tim Graham, 7 years ago

Yes, that's expected after 5112e65ef2df1dbb95ff83026b6a962fb2688661.

comment:9 by Ruben Alves, 7 years ago

But even if they are different, the function _unsalt_cipher_token (https://github.com/django/django/blob/stable/1.11.x/django/middleware/csrf.py#L62) should return always the same value with the different values on the Cookie, right?

I've called the _unsalt_cipher_token function with the different values of the Cookie, and the returned values are ALWAYS different.

comment:10 by Ruben Alves, 7 years ago

I've created a template tag named uncypher that calls _unsalt_cipher_token as you can see below:

from django.middleware.csrf import _unsalt_cipher_token
from django.template import Library

register = Library()


@register.simple_tag
def uncypher(token):
    """
    Testing the CSRF problem
    """
    return _unsalt_cipher_token(token)

Then, on a html template I've added

var parser = new DOMParser();
var csrf = parser.parseFromString("{% csrf_token %}", "text/xml").firstChild;
console.error('token: {% uncypher csrf_token %}');

After set settings.DEBUG=True and reload the page 3 times, on the JavaScript Console was printed the same value lKnMEqisHw8mxMIXZgZTKt9cirypfv1f on the 3 times that I've reloaded the page.

Then I've set settings.DEBUG=False and reloaded the page 4 times again.
On the first time it printed the same value lKnMEqisHw8mxMIXZgZTKt9cirypfv1f. On the second YgnNvwBJJwAID17nl7FLhtA3rObtvGLX, third time printed hB1plGV688FpDPNu9waO1H959c04YLPM and fourth time printed C2sJ9e7coe74juO6HxAY4RIppBY4yi1Y.

comment:11 by Ruben Alves, 7 years ago

Tested the system with Django1.10 and Django1.11.

With Django1.10 everything works. With Django1.11 we have the error.

I'm going to see the diff between the two versions: https://github.com/django/django/compare/stable/1.10.x...stable/1.11.x

Last edited 7 years ago by Ruben Alves (previous) (diff)

comment:12 by Ruben Alves, 7 years ago

I finally solved my problem.

The problem is: If a user just get a 404 page, a new CSRF Token is generated, invalidating the CSRF Token that was originally loaded with {% csrf_token %}.

How I've discovered

On the WebSite that I work, we have a page with the URI /en/courses/.

On that page, on the .html file we include a JavaScript: <script src="/static/js/app.js"></script>

The app.js was trying to load a file named assets/js/particles.json using the relative path, so the final URL of the static file was /en/courses/assets/js/particles.json. Load the particles.json is the only thing that app.js was doing. Nothing else.

That particles.json doesn't exist on our system.

After successfully load /en/courses/, the Django Server was receiving a request for /en/courses/assets/js/particles.json that raises a 404 error for the static file, but this error 404 is not even noticed by the users because this JS file was doing nothing.

After I remove this <script src="/static/js/app.js"></script>, everything worked fine.

Then with debugs on the django.middleware.csrf.CsrfViewMiddleware I could confirm that if I reload a page 1000 times, the CSRF Token returned by _unsalt_cipher_token (​https://github.com/django/django/blob/stable/1.11.x/django/middleware/csrf.py#L62) is always the same. But if I just try to access a page that doesn't exist, then _unsalt_cipher_token returns a different value (a new token is generated).

Unbelievable that a single javascript that doesn't even exist has broken my system and took me 3 days to find out.

comment:13 by KwonHan Bae, 7 years ago

so it's not a bug?
I think if it a bug we can make another explicit ticket

comment:14 by Ruben Alves, 7 years ago

I consider a bug.

If a user has a form, and on the same page tries to load an image, css, or javascript with a wrong URI, the form will never be accepted.

comment:15 by Mark Lavin, 7 years ago

Based on the root cause of this problem, it sounds to be very similar to #22961 (StaticFilesHandler should not run middleware on 404). It has a patch to make that change but didn't include any tests for that new behavior.

comment:16 by Claude Paroz, 7 years ago

Is it right to assume that you are using the runserver server on your production instance?

comment:17 by Ruben Alves, 7 years ago

No, ./manage.py runserver is used only in localhost for development purposes.

In AWS we use Elastic Beanstalk with a .ebextensions/django.config file according to the instructions found on http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create-deploy-python-django.html

comment:18 by Claude Paroz, 7 years ago

Ah sorry, I didn't notice at first that the missing file has a path outside of the static directory. So it's not a StaticFilesHandler issue.

comment:19 by Claude Paroz, 7 years ago

I think this is clearly a bug if someone is able reproduce the issue in a test.

comment:20 by Tim Graham, 7 years ago

Resolution: needsinfo
Status: newclosed
Summary: Django 1.11 to 1.11.4 raises CSRF verification failed if settings.DEBUG is FalseDjango 1.11+ raises CSRF verification failed if settings.DEBUG is False and an intermediate 404 page is requested

If someone can provide a sample project with steps to reproduce, please reopen the ticket.

comment:21 by Nicola, 7 years ago

same problem here, in my case the problem is with the django admin site and the 404 page is caused by chrome trying to get /favicon.ico

comment:22 by Nicola, 7 years ago

Cc: Nicola added

comment:23 by Nicola, 7 years ago

this bug is really difficult to track down, for my problem I added the workaround in apache config, something like this:

ProxyPass /favicon.ico !
Alias /favicon.ico /path/to/favicon.ico
<Location "/favicon.ico">
    Require all granted
</Location> 

after this change the django admin site worked again, hope this helps someone other.

I think I have this issue on my web site since a favicon is defined for all pages but not /admin (django admin site) and so chrome try to get the /favicon.ico for admin site (the favicon is in /static/images/favicon.ico), django return 404 and admin site login does not work.

A fix in django itself would be really appreciated, thanks!

by Nicola, 7 years ago

Attachment: ticket28488.zip added

sample project that show the issue

comment:24 by Nicola, 7 years ago

Resolution: needsinfo
Status: closednew

attached a sample project that show the issue with the admin site

you can reproduce using runserver

visit the page (tested with firefox and chromium on linux)

http://127.0.0.1:8000/admin

look at the runserver logs and ensure that you see

GET /favicon.ico HTTP/1.1" 404 230

try to login to the django admin site with any user and pass

you'll get a csrf error

comment:25 by Tim Graham, 7 years ago

Thanks for the sample project. I bisected the changed in behavior to eef95ea96faef0b7dbbe0c8092202b74f68a899b which first appeared in Django 1.9, yet the report is about Django 1.11, so I'm not sure if everyone reporting this issue is encountering the same problem. It looks like the key to reproducing it is that the custom handler404 uses render() rather than render_to_response() or a plain HttpResponse(). I'll look more later.

comment:26 by Tim Graham, 7 years ago

Cc: Jay Cox added

Jay, as the author of eef95ea96faef0b7dbbe0c8092202b74f68a899b, can you look at this? It looks like before your patch, this condition in CsrfViewMiddleware.process_response() prevented a CSRF token from being set for the 404 requests:

if request.META.get("CSRF_COOKIE") is None:
    return response

After the patch, a cookie is being set for those requests.

comment:27 by Florian Apolloner, 7 years ago

Cc: Florian Apolloner added

@tim: If you run over similar security related bugs, please always CC me

I also agree that https://github.com/django/django/commit/eef95ea96faef0b7dbbe0c8092202b74f68a899b#diff-a3be722ce2831a8d11438021d44cedf1L200 probably should never have been dropped. I'll look into the root cause later this day.

Last edited 7 years ago by Florian Apolloner (previous) (diff)

comment:28 by Florian Apolloner, 7 years ago

I think the main issue here is that a 404 handler does not run through process_view of the csrf middleware and therefor does not know that a token already exists and creates a new one (more details later)

by Florian Apolloner, 7 years ago

Attachment: t28488.diff added

comment:29 by Florian Apolloner, 7 years ago

I've attached a patch, it is still not perfect and the underlying problem will still be there if there is a middleware doing something stupid before the CSRF middleware is run (ie run the http404 handler in process_request).

As for https://github.com/django/django/commit/eef95ea96faef0b7dbbe0c8092202b74f68a899b#diff-a3be722ce2831a8d11438021d44cedf1L200 -- I was wrong there, it is afaik not needed since if the "CSRF_COOKIE_USED" basically takes care of that too.

comment:30 by Florian Apolloner, 7 years ago

Triage Stage: UnreviewedAccepted

comment:31 by Florian Apolloner, 7 years ago

Version: 1.11master

comment:33 by Berker Peksag, 7 years ago

Cc: Berker Peksag added

comment:34 by Berker Peksag, 7 years ago

I have the same problem, but disabling the custom 404 handler didn't work for me as described in #28143 (closed as a duplicate of this issue)

Removing the custom 404 view from urls.py, or setting DEBUG to True (obviously), fixes the Django admin failed csrf error.

Test URL: https://staging.python.org/admin/
Source code: https://github.com/python/pythondotorg
Installation document: https://pythondotorg.readthedocs.io/install.html

comment:35 by Florian Apolloner, 7 years ago

Does my PR fix it for you?

comment:36 by Berker Peksag, 7 years ago

Does my PR fix it for you?

I can't reproduce the problem in development environment even if I set settings.DEBUG to False. I applied your patch manually on the server and unfortunately I still get 403 when I try to submit a form.

comment:37 by Berker Peksag, 7 years ago

I just tried Nicola's sample project and I was able to reproduce it locally.

For some strange reason, I can't reproduce it locally on my own project even if I made the same request:

[19/Sep/2017 02:00:34] "GET / HTTP/1.1" 200 42428
[19/Sep/2017 02:00:36] "GET /static/js/main-min.js HTTP/1.1" 404 31942
[19/Sep/2017 02:00:50] "GET /admin/login/?next=/admin/ HTTP/1.1" 200 1625
[19/Sep/2017 02:00:50] "GET /static/admin/css/base.css HTTP/1.1" 404 31950
[19/Sep/2017 02:00:50] "GET /static/admin/css/login.css HTTP/1.1" 404 31952

I removed all custom middlewares and copied the MIDDLEWARE setting from Nicola's project and I was still able to login successfully.

On both local and production environment, I see different csrfmiddlewaretoken values after each refresh but the value of csrfmiddlewaretoken doesn't match with the csrftoken cookie (it also stays same after I refreshed the page) However, the csrftoken cookie changes each request in Nicola's project.

comment:38 by Berker Peksag, 7 years ago

[...] I see different csrfmiddlewaretoken values after each refresh but the value of csrfmiddlewaretoken doesn't match with the csrftoken cookie [...]

Turns out I was reading an older version of the document and this behavior is expected in Django 1.11.

Also, I get 403 even if there was no 404 request in the middle so I'm beginning to think that my problem is different than this ticket.

comment:39 by Berker Peksag, 7 years ago

[...] so I'm beginning to think that my problem is different than this ticket.

Turns out the hostname of the referer should be an exact match of CSRF_COOKIE_DOMAIN and in my case is_same_domain('staging.python.org', 'staging.python.org:9000') returns False. I set CSRF_TRUSTED_ORIGINS = ['staging.python.org'] and everything works fine now.

Sorry for the noise!

comment:40 by Tim Graham <timograham@…>, 7 years ago

Resolution: fixed
Status: newclosed

In c4c128d:

Fixed #28488 -- Reallowed error handlers to access CSRF tokens.

Regression in eef95ea96faef0b7dbbe0c8092202b74f68a899b.

comment:41 by Tim Graham <timograham@…>, 7 years ago

In 42847327:

[1.11.x] Fixed #28488 -- Reallowed error handlers to access CSRF tokens.

Regression in eef95ea96faef0b7dbbe0c8092202b74f68a899b.

Backport of c4c128d67c7dc2830631c6859a204c9d259f1fb1 from master

Note: See TracTickets for help on using tickets.
Back to Top