Opened 8 years ago

Closed 8 years ago

#9172 closed (wontfix)

CsrfMiddleware breaks django test client

Reported by: Gruffudd Williams Owned by: nobody
Component: Testing framework Version: 1.0
Severity: Keywords:
Cc: Dave Naffziger Triage Stage: Unreviewed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

The very excellent CsrfMiddleware has one undesired side-effect: It stops the django test client from working.
Is there a way of identifying that a POST has originated from django.test.client and, if so, disable the checking?

We're currently removing the CsrfMiddleware from settings at runtime in our test harness, but that's a bit naughty!

Attachments (4)

9172_r9084.diff (5.3 KB) - added by Carl Meyer 8 years ago.
make test client bypass CsrfMiddleware
django-trunk-csrf-test-client.patch (5.1 KB) - added by Markus Bertheau 8 years ago.
Updated patch for current trunk
django-1.0.2-csrf-test-client.patch (3.6 KB) - added by Markus Bertheau 8 years ago.
Updated patch for 1.0.2
django-trunk-remove-csrf-in-test-client.patch (916 bytes) - added by Markus Bertheau 8 years ago.
Remove csrf middleware for tests automatically

Download all attachments as: .zip

Change History (18)

comment:1 Changed 8 years ago by Carl Meyer

Removing CsrfMiddleware from settings in your test harness is the best solution. Injecting test-client-specific checks into CsrfMiddleware itself would be far uglier. I believe this should be closed as wontfix.

comment:2 Changed 8 years ago by Russell Keith-Magee

I'm not convinced that this should be a wontfix - a solution of some kind is clearly needed. Ideally, your tests should be able to run on the same configuration as your production site - every difference between production and test environment is potential source for error. However, Carl is also correct - we don't really want to be inserting "If test" logic into the CSRF Middleware (we haven't had to do this anywhere else).

I can't say I have any brilliant ideas off the top of my head, so some deeper thought will be required. Suggestions welcome.

comment:3 Changed 8 years ago by Carl Meyer

I'd love to see a nicer solution; I banged my head on this one for quite some time.

You _can_ run your tests on the same configuration as your production site, but then they have to follow the same rules as your production site (which is as it should be, else you aren't really testing). If you're using CsrfMiddleware that means your tests need to GET the form first, scrape the csrf token, and then use it for the POST (you actually only have to scrape once per session, since Django's Csrf token varies only with session id and SECRET_KEY). If you do that, everything will work fine.

Obviously, that's not a real attractive solution either. But if you want your tests to play by different rules, you should have to say so explicitly, and removing the middleware makes it explicit.

comment:4 Changed 8 years ago by Chris Beaven

Couldn't the test client check for the middleware and fetch (and insert) the token if necessary?

comment:5 in reply to:  4 Changed 8 years ago by Carl Meyer

Replying to SmileyChris:

Couldn't the test client check for the middleware and fetch (and insert) the token if necessary?

Ok, I'm dumb. I went a little ways down that road, but not far enough. Turns out that's really simple to do; in fact, I just put together a patch that does it, along with accompanying test and doc update.

Changed 8 years ago by Carl Meyer

Attachment: 9172_r9084.diff added

make test client bypass CsrfMiddleware

comment:6 Changed 8 years ago by Carl Meyer

Has patch: set

comment:7 Changed 8 years ago by Dave Naffziger

Cc: Dave Naffziger added

comment:8 Changed 8 years ago by Dave Naffziger

Maybe I'm missing something, but I don't see the patch attached to this ticket - just a blank diff file.

comment:9 in reply to:  8 Changed 8 years ago by Russell Keith-Magee

Replying to davenaff:

Maybe I'm missing something, but I don't see the patch attached to this ticket - just a blank diff file.

This is a common problem with Trac; some patches break the patch viewer. If you download the patch, it should be ok.

comment:10 Changed 8 years ago by Carl Meyer

I think it's patches that contain the "\ No newline at end of file" marker that seem to give Trac trouble - I could be wrong.

Changed 8 years ago by Markus Bertheau

Updated patch for current trunk

Changed 8 years ago by Markus Bertheau

Updated patch for 1.0.2

comment:11 Changed 8 years ago by Markus Bertheau

I'm using the patch locally and it works fine. I also attached updated versions of this patch for trunk and 1.0.2. Trac still refuses to show the 1.0.2 patch, but it's there and working.

comment:12 Changed 8 years ago by Luke Plant

Resolution: wontfix
Status: newclosed

The patch does almost the same thing as disabling the CsrfMiddleware -- it simply makes the incoming POST data match what the middleware wants (whether or not the actual page that a user would see would match what the middleware wants). I cannot see the advantage of using this over disabling the middleware.

I see it this way: presumably you are trusting the middleware to do its job of stopping CSRF attacks in the absence of the token (this is basic stuff, and it has its own tests, after all). What you might not trust is whether the middleware is inserting the token correctly, which is a possibility, and will be more of a problem when/if an updated middleware is used (as is planned), which doesn't insert the token into outgoing pages, but requires a template tag in every form that needs it. In that case, you will want to test *exactly* what happens when a user accesses the page, and check you don't get 403's when you shouldn't. The only way to do this properly is to do browser simulation, using something like twill. If you are using that, it will automatically submit hidden fields, as it must, and will get through the middleware just fine.

In short, the only reason to not disable the middleware for testing is if you are going to go the whole way and do browser simulation, in which case you don't need a patch. Also, the patch introduces a dependency on a contrib app which isn't very nice.

comment:13 Changed 8 years ago by Markus Bertheau

Resolution: wontfix
Status: closedreopened

What about doing that automatically?

Changed 8 years ago by Markus Bertheau

Remove csrf middleware for tests automatically

comment:14 Changed 8 years ago by Luke Plant

Resolution: wontfix
Status: reopenedclosed

It is simpler to just put the developer in control of the settings being used, and not mess with the settings in case they actually do want the middleware to be on and functioning (as in the case of full browser simulation). This special-casing doesn't add anything that cannot be achieved more simply. Also, in one sense, if you have the middleware enabled in your settings file, and you use the Django test client to test pages with sessions enabled, then your tests ought to fail -- the middleware is designed to stop the kind of access that is done by the test client. You can also manually monkey patch the middleware in/out for specific tests as required.

Note: See TracTickets for help on using tickets.
Back to Top