Django 1.11+ raises CSRF verification failed if settings.DEBUG is False and an intermediate 404 page is requested

[Ruben Alves]

Hi all,

I'm using Django1.11 (and made tests also with Django1.11.4) and having problems when submitting a form with POST method.

I'm calling the {% csrf_token %} inside of the form, so, this is not the problem.
The problem when submitting the form is:

Forbidden (403)
CSRF verification failed. Request aborted.
More information is available with DEBUG=True.

Then, in order to see "more information", I've enabled settings.DEBUG to True and submitted the form again. At this moment, the problem didn't happens anymore.

So I've disabled settings.DEBUG, submitted again, and the problem was there. Enabled DEBUG=True again, problem has gone.

Initially I thought that could be some error in my code, but the same happens when I try to reset my password wit the django.contrib.auth.views.password_reset view.

In my settings.py, I have the following changes that were made recently:

SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True

I use AWS (Amazon Web Service) Elastic Beanstalk with https enabled.

The worst part is that I've discovered this only on production because I make all tests in my local machine with DEBUG=True, and on production, we set DEBUG=False. Also, the error message should be shown only if DEBUG=True.

It's something related to the same error mentioned on Google Groups ​https://groups.google.com/forum/#!searchin/django-users/CSRF$20verification$20failed.$20Request$20aborted.$20More$20information$20is$20available$20with$20DEBUG$3DTrue.%7Csort:relevance/django-users/ISoJ6CwHOXQ/pirLih0jBgAJ

[Ruben Alves]

Error message on production with settings.DEBUG=False

[Tim Graham]

Did you experience a change in Django 1.11 compared to Django 1.10 or earlier? I think you'll need to debug the problem further and let us know why Django is at fault.

[Ruben Alves]

Before Django 1.11, I was using Django 1.8 and everything was working perfectly.

In order to use Django 1.11, I did all the changes mentioned on ​https://docs.djangoproject.com/en/1.11/releases/1.11/

What I've done now was add settings.CSRF_FAILURE_VIEW = 'courses.views.csrf_failure' with DEBUG=False

I made my courses.views.csrf_failure raise an exception, so we could see the traceback.

The error message is:

Lets see where this shit happens: {'template_name': 'dooesntmatter', 'reason': u'CSRF token missing or incorrect.', 'request': <WSGIRequest: POST '/en/accounts/password_reset/'>}

The traceback is the following:

File "/opt/python/run/venv/lib64/python2.7/site-packages/django/core/handlers/exception.py" in inner
  41.             response = get_response(request)

File "/opt/python/run/venv/lib64/python2.7/site-packages/django/core/handlers/base.py" in _legacy_get_response
  249.             response = self._get_response(request)

File "/opt/python/run/venv/lib64/python2.7/site-packages/django/core/handlers/base.py" in _get_response
  178.             response = middleware_method(request, callback, callback_args, callback_kwargs)

File "/opt/python/run/venv/lib64/python2.7/site-packages/django/middleware/csrf.py" in process_view
  314.                 return self._reject(request, REASON_BAD_TOKEN)

File "/opt/python/run/venv/lib64/python2.7/site-packages/django/middleware/csrf.py" in _reject
  163.         return _get_failure_view()(request, reason=reason)

File "/opt/python/current/app/courses/views/__init__.py" in csrf_failure
  49.     raise Exception('Lets see where this shit happens: ' + str(locals()))

[Tim Graham]

The traceback doesn't give me any ideas about the cause. I think you'll need to add further logging in django/middleware/csrf.py to find the root cause.

A large CSRF change between Django 1.8 and 1.11 is 5112e65ef2df1dbb95ff83026b6a962fb2688661. Perhaps you can test with Django 1.9 and 1.10 to help narrow the problem.

In the backwards-incompatible changes section of the 1.10 release notes, there's a note, "CSRF token values are now required to be strings of 64 alphanumerics; values of 32 alphanumerics, as set by older versions of Django by default, are automatically replaced by strings of 64 characters. Other values are considered invalid. This should only affect developers or users who replace these tokens." Maybe it's relevant to you.

[Ruben Alves]

Ok, I've added some logs on django.middleware.csrf, and this is what I have:

Between lines 208 and 209 (​https://github.com/django/django/blob/stable/1.11.x/django/middleware/csrf.py#L208) I've added a log and the value of csrf_token is vcdodHkRb8jlxQP5fnTjuQp5i3PMWYYEBOYfpOFkCEjWWwHMpJ9uqaPGI6vGi6hS

The if request.is_secure(): is True on line 228 (​https://github.com/django/django/blob/stable/1.11.x/django/middleware/csrf.py#L228)

The value of request_csrf_token on line 311 (​https://github.com/django/django/blob/stable/1.11.x/django/middleware/csrf.py#L311) before we call request_csrf_token = _sanitize_token(request_csrf_token) is tMgl4aTrdjOEShUax3Gz1CJLbvnhBWiEVzmzgTxSclDuP01lBfwoE2R6dDyljaCQ

After _sanitize_token be called, the value of request_csrf_token is the same, and the value of csrf_token is vcdodHkRb8jlxQP5fnTjuQp5i3PMWYYEBOYfpOFkCEjWWwHMpJ9uqaPGI6vGi6hS.

Finally _compare_salted_tokens(request_csrf_token, csrf_token) on line 313 returns False (​https://github.com/django/django/blob/stable/1.11.x/django/middleware/csrf.py#L313).

[Tim Graham]

Maybe you can try to minimize the middleware, etc. in your project and see if something like that is causing the issue.

[Ruben Alves]

I've being testing to reload the page and see the Cookie named csrftoken.

On the system that runs with Django1.8, every time I reload the page, the csrftoken Cookie has the same value.

On the system that runs with Django1.11, every time I reload the page, the csrftoken Cookie has a different value.

Do you know if on Django1.11 is expected to have a different csrftoken Cookie every time I reload the page? If yes, why is expected to be different in Django1.11 and not in Django1.8?

[Tim Graham]

Yes, that's expected after 5112e65ef2df1dbb95ff83026b6a962fb2688661.

[Ruben Alves]

But even if they are different, the function _unsalt_cipher_token (​https://github.com/django/django/blob/stable/1.11.x/django/middleware/csrf.py#L62) should return always the same value with the different values on the Cookie, right?

I've called the _unsalt_cipher_token function with the different values of the Cookie, and the returned values are ALWAYS different.

[Ruben Alves]

I've created a template tag named uncypher that calls _unsalt_cipher_token as you can see below:

from django.middleware.csrf import _unsalt_cipher_token
from django.template import Library

register = Library()


@register.simple_tag
def uncypher(token):
    """
    Testing the CSRF problem
    """
    return _unsalt_cipher_token(token)

Then, on a html template I've added

var parser = new DOMParser();
var csrf = parser.parseFromString("{% csrf_token %}", "text/xml").firstChild;
console.error('token: {% uncypher csrf_token %}');

After set settings.DEBUG=True and reload the page 3 times, on the JavaScript Console was printed the same value lKnMEqisHw8mxMIXZgZTKt9cirypfv1f on the 3 times that I've reloaded the page.

Then I've set settings.DEBUG=False and reloaded the page 4 times again.
On the first time it printed the same value lKnMEqisHw8mxMIXZgZTKt9cirypfv1f. On the second YgnNvwBJJwAID17nl7FLhtA3rObtvGLX, third time printed hB1plGV688FpDPNu9waO1H959c04YLPM and fourth time printed C2sJ9e7coe74juO6HxAY4RIppBY4yi1Y.

[Ruben Alves]

Tested the system with Django1.10 and Django1.11.

With Django1.10 everything works. With Django1.11 we have the error.

Do you know where I can see the diff (code, not "Release Notes") between the two versions of Django (1.10 and 1.11)?

[Ruben Alves]

I finally solved my problem.

The problem is: If a user just get a 404 page, a new CSRF Token is generated, invalidating the CSRF Token that was originally loaded with {% csrf_token %}.

How I've discovered

On the WebSite that I work, we have a page with the URI /en/courses/.

On that page, on the .html file we include a JavaScript: <script src="/static/js/app.js"></script>

The app.js was trying to load a file named assets/js/particles.json using the relative path, so the final URL of the static file was /en/courses/assets/js/particles.json. Load the particles.json is the only thing that app.js was doing. Nothing else.

That particles.json doesn't exist on our system.

After successfully load /en/courses/, the Django Server was receiving a request for /en/courses/assets/js/particles.json that raises a 404 error for the static file, but this error 404 is not even noticed by the users because this JS file was doing nothing.

After I remove this <script src="/static/js/app.js"></script>, everything worked fine.

Then with debugs on the django.middleware.csrf.CsrfViewMiddleware I could confirm that if I reload a page 1000 times, the CSRF Token returned by _unsalt_cipher_token (​​https://github.com/django/django/blob/stable/1.11.x/django/middleware/csrf.py#L62) is always the same. But if I just try to access a page that doesn't exist, then _unsalt_cipher_token returns a different value (a new token is generated).

Unbelievable that a single javascript that doesn't even exist has broken my system and took me 3 days to find out.

[KwonHan Bae]

so it's not a bug?
I think if it a bug we can make another explicit ticket

[Ruben Alves]

I consider a bug.

If a user has a form, and on the same page tries to load an image, css, or javascript with a wrong URI, the form will never be accepted.

[Mark Lavin]

Based on the root cause of this problem, it sounds to be very similar to #22961 (StaticFilesHandler should not run middleware on 404). It has a patch to make that change but didn't include any tests for that new behavior.

[Claude Paroz]

Is it right to assume that you are using the runserver server on your production instance?

[Ruben Alves]

No, ./manage.py runserver is used only in localhost for development purposes.

In AWS we use Elastic Beanstalk with a .ebextensions/django.config file according to the instructions found on ​http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create-deploy-python-django.html

[Claude Paroz]

Ah sorry, I didn't notice at first that the missing file has a path outside of the static directory. So it's not a StaticFilesHandler issue.

[Claude Paroz]

I think this is clearly a bug if someone is able reproduce the issue in a test.

[Tim Graham]

If someone can provide a sample project with steps to reproduce, please reopen the ticket.

[Nicola]

same problem here, in my case the problem is with the django admin site and the 404 page is caused by chrome trying to get /favicon.ico

[Nicola]

this bug is really difficult to track down, for my problem I added the workaround in apache config, something like this:

ProxyPass /favicon.ico !
Alias /favicon.ico /path/to/favicon.ico
<Location "/favicon.ico">
    Require all granted
</Location> 

after this change the django admin site worked again, hope this helps someone other.

I think I have this issue on my web site since a favicon is defined for all pages but not /admin (django admin site) and so chrome try to get the /favicon.ico for admin site (the favicon is in /static/images/favicon.ico), django return 404 and admin site login does not work.

A fix in django itself would be really appreciated, thanks!

[Nicola]

sample project that show the issue

[Nicola]

attached a sample project that show the issue with the admin site

you can reproduce using runserver

visit the page (tested with firefox and chromium on linux)

​http://127.0.0.1:8000/admin

look at the runserver logs and ensure that you see

GET /favicon.ico HTTP/1.1" 404 230

try to login to the django admin site with any user and pass

you'll get a csrf error

[Tim Graham]

Thanks for the sample project. I bisected the changed in behavior to eef95ea96faef0b7dbbe0c8092202b74f68a899b which first appeared in Django 1.9, yet the report is about Django 1.11, so I'm not sure if everyone reporting this issue is encountering the same problem. It looks like the key to reproducing it is that the custom handler404 uses render() rather than render_to_response() or a plain HttpResponse(). I'll look more later.

[Tim Graham]

Jay, as the author of eef95ea96faef0b7dbbe0c8092202b74f68a899b, can you look at this? It looks like before your patch, this condition in CsrfViewMiddleware.process_response() prevented a CSRF token from being set for the 404 requests:

if request.META.get("CSRF_COOKIE") is None:
    return response

After the patch, a cookie is being set for those requests.

[Florian Apolloner]

@tim: If you run over similar security related bugs, please always CC me

I also agree that ​https://github.com/django/django/commit/eef95ea96faef0b7dbbe0c8092202b74f68a899b#diff-a3be722ce2831a8d11438021d44cedf1L200 probably should never have been dropped. I'll look into the root cause later this day.

[Florian Apolloner]

I think the main issue here is that a 404 handler does not run through process_view of the csrf middleware and therefor does not know that a token already exists and creates a new one (more details later)

[Florian Apolloner]

I've attached a patch, it is still not perfect and the underlying problem will still be there if there is a middleware doing something stupid before the CSRF middleware is run (ie run the http404 handler in process_request).

As for ​https://github.com/django/django/commit/eef95ea96faef0b7dbbe0c8092202b74f68a899b#diff-a3be722ce2831a8d11438021d44cedf1L200 -- I was wrong there, it is afaik not needed since if the "CSRF_COOKIE_USED" basically takes care of that too.

[Florian Apolloner]

-> ​https://github.com/django/django/pull/9100

[Berker Peksag]

I have the same problem, but disabling the custom 404 handler didn't work for me as described in #28143 (closed as a duplicate of this issue)

Removing the custom 404 view from urls.py, or setting DEBUG to True (obviously), fixes the Django admin failed csrf error.

Test URL: ​https://staging.python.org/admin/
Source code: ​https://github.com/python/pythondotorg
Installation document: ​https://pythondotorg.readthedocs.io/install.html

[Florian Apolloner]

Does my PR fix it for you?

[Berker Peksag]

Does my PR fix it for you?

I can't reproduce the problem in development environment even if I set settings.DEBUG to False. I applied your patch manually on the server and unfortunately I still get 403 when I try to submit a form.

[Berker Peksag]

I just tried Nicola's sample project and I was able to reproduce it locally.

For some strange reason, I can't reproduce it locally on my own project even if I made the same request:

[19/Sep/2017 02:00:34] "GET / HTTP/1.1" 200 42428
[19/Sep/2017 02:00:36] "GET /static/js/main-min.js HTTP/1.1" 404 31942
[19/Sep/2017 02:00:50] "GET /admin/login/?next=/admin/ HTTP/1.1" 200 1625
[19/Sep/2017 02:00:50] "GET /static/admin/css/base.css HTTP/1.1" 404 31950
[19/Sep/2017 02:00:50] "GET /static/admin/css/login.css HTTP/1.1" 404 31952

I removed all custom middlewares and copied the MIDDLEWARE setting from Nicola's project and I was still able to login successfully.

On both local and production environment, I see different csrfmiddlewaretoken values after each refresh but the value of csrfmiddlewaretoken doesn't match with the csrftoken cookie (it also stays same after I refreshed the page) However, the csrftoken cookie changes each request in Nicola's project.

[Berker Peksag]

[...] I see different csrfmiddlewaretoken values after each refresh but the value of csrfmiddlewaretoken doesn't match with the csrftoken cookie [...]

Turns out I was reading an older version of the document and this behavior is expected in Django 1.11.

Also, I get 403 even if there was no 404 request in the middle so I'm beginning to think that my problem is different than this ticket.

[Berker Peksag]

[...] so I'm beginning to think that my problem is different than this ticket.

Turns out the hostname of the referer should be an exact match of CSRF_COOKIE_DOMAIN and in my case is_same_domain('staging.python.org', 'staging.python.org:9000') returns False. I set CSRF_TRUSTED_ORIGINS = ['staging.python.org'] and everything works fine now.

Sorry for the noise!

[Tim Graham <timograham@…>]

In c4c128d:

Fixed #28488 -- Reallowed error handlers to access CSRF tokens.

Regression in eef95ea96faef0b7dbbe0c8092202b74f68a899b.

[Tim Graham <timograham@…>]

In 42847327:

[1.11.x] Fixed #28488 -- Reallowed error handlers to access CSRF tokens.

Regression in eef95ea96faef0b7dbbe0c8092202b74f68a899b.

Backport of c4c128d67c7dc2830631c6859a204c9d259f1fb1 from master