Signing and Signed Cookies

Note: signing facilities were introduced in Django 1.4: ​https://docs.djangoproject.com/en/dev/releases/1.4/#tools-for-cryptographic-signing. The discussion below is kept here for historical reasons.

Proposal for 1.2: Django core should include low-level code for signing and verifying signatures on arbitrary bytestrings. This should be used to support a new higher level signed cookie feature.

Under discussion here ​http://groups.google.com/group/django-developers/browse_thread/thread/133509246caf1d91

Experimental code here: ​http://github.com/simonw/django-signed

Justification

• Signing is a widely used web application security technique. Django uses it in a few places already (the form wizard and sessions contrib apps), and it is useful any time an application might want to pass data through an untrusted channel and ensure it hasn't been tampered on the other side. The Web is generally an untrusted channel.
• Signed cookies can replace sessions in many use cases, with the significant bonus that unlike sessions they do not require a round-trip to a persistent store.
• Signing is hard to do right, and most hand-rolled implementations make similar mistakes. Signing is best handled with hmac and sha1, but Django implementations currently tend to use the weaker MD5 without hmac. A signing implementation in Django core could be audited by expert cryptographers, providing a secure base on which other Django applications could build.

Potential uses for signing:

• Signed cookies
• Generating CSRF tokens
• Secure /logout/ and /change-language/ links
• Securing /login/?next=/some/path/
• Securing hidden fields in form wizards
• Recover-your-account links in e-mails

Signed cookie API

This is under heavy discussion on the mailing list. Current proposals for setting a signed cookie:

response.set_cookie(key, value, signed=True)

Or

response.set_signed_cookie(key, value)

(Blanket signing everything is probably not an option as some cookies, such as those used by Google Analytics, need to remain unsigned)

Reading a signed cookie is harder. Since cookies may be both signed and unsigned, it's not going to be possible to transparently verify signed cookies and return them using the same API as unsigned cookies. Benjamin Slavin pointed out that an attacker could then set bad data in an unsigned cookie and the auto-unsigning code would assume it was never signed in the first place. This means we need an explicit API for reading cookies that should have been signed.

Current suggestions include:

request.unsign_cookie(key) # reflects the lower-level API, but unintuitive
request.get_signed_cookie(key) # quite verbose
request.COOKIES.get_signed(key) # similar to request.POST.get_list
request.COOKIES.get_unsigned(key)
request.COOKIES.unsign(key)
request.get_cookie(key, signed=True) # for symmetry with response.set_cookie(..., signed=True)

Proposed signing API

Simon Willison has offered to donate the signing code from his django-openid library for the low level signing API:

​http://github.com/simonw/django-openid/blob/master/django_openid/signed.py

​http://github.com/simonw/django-openid/blob/master/django_openid/tests/signing_tests.py

The API would look like this:

>>> from django.utils import signed
>>> signed.sign('hello')
'hello.9asVJn9dfv6qLJ_BYObzF7mmH8c'

The signature is a URL-safe base64 encoded digest of the hmac/sha1. I used base64 rather than .hexdigest() for space reasons - base64 digests are 27 characters, hexadecimal digests are 40. When you're including signatures in cookies and URLs (especially account recovery URLs sent out in plain text, 80 character wide e-mails) every byte counts.

>>> signed.unsign('hello.9asVJn9dfv6qLJ_BYObzF7mmH8c')
'hello'
>>> signed.unsign('hello.badsignature')
Traceback (most recent call last):
...
BadSignature: Signature failed: badsignature

BadSignature would probably be a subclass of django.core.exceptions.SuspiciousOperation

>>> signed.dumps({"a": "foo"})
'KGRwMApTJ2EnCnAxClMnZm9vJwpwMgpzLg.mYepoYkzWwXRmsCTVJm3Mb0HHz4'
>>> signed.loads(_)
{'a': 'foo'}

Again, the pickle is URL-safe base64 encoded to take up less valuable cookie space and generally make it easier to pass around on the Web. A nice thing about URL-safe base64 is that it uses 64 out of the 65 URL-safe characters (by URL-safe I mean characters that are left unchanged by Python's urllib.urlencode function) - the remaining character is the period, which I use to separate the pickle from the signature.

signed.dumps takes a couple of extra optional arguments. The first is compress=True (default is False) which zlib compresses the pickle if doing so will save any space:

>>> import this # to get an object worth compressing
...
>>> len(signed.dumps(this.s))
1207
>>> len(signed.dumps(this.s, compress=True))
637

By default, all signatures use Django's SECRET_KEY. If you want to sign with a different key, you can pass it as an argument to the various functions:

>>> signed.sign('hello', key='sekrit')
'hello.o6MKehoOfZ2b2FU84wzibW6IWxI'
>>> signed.unsign(_, key='sekrit')
'hello'

The dumps and loads methods also take a key argument, as well as an additional optional extra_key argument for if you want to generate different signatures for different parts within your application (useful for the extra paranoid):

>>> signed.dumps('hello', key='sekrit', extra_key='ultra')
'UydoZWxsbycKcDAKLg.1XYDpILo5xqSwImfa3WuJJT4RPo'
>>> signed.loads(_, key='sekrit', extra_key='ultra')
'hello'

Potential additions to the above API

• Low level access to just the signature generation routine itself (at the moment sign() and unsign() append the signature to the string themselves)
• Maybe functions or parameters for creating timestamped signatures and failing the signature if it is older than a certain expiry timeout.
• A ​signer class that encapsulates common arguments to sign() and unsign() and allows for customization.

SECRET_KEY considerations

Simon says:

One thing that worries me slightly about increasing the amount of signing going on in Django is that it elevates the importance of the SECRET_KEY. I'm currently ignorant of best practices regarding protecting this kind of shared secret, but the steps we take (*ing it out from the debug pages and otherwise ignoring it) could almost certainly be improved.

One thing that's particularly interesting to me is what happens when you change your secret. If you're changing your secret because it's leaked then obviously you want stuff signed with the old secret to become invalid immediately, but I can imagine some users wanting to rotate their secret keys on a continual basis for added security against brute force attacks.

If you're rotating your secret, invalidating all of your users signed cookies etc is a bit of an annoyance. It might be worth supporting two secrets - the current SECRET_KEY and an optional OLD_SECRET_KEY - with unsigning operations falling back on the old key if the current key fails. This would allow users to deploy a new secret while keeping the old one valid for a week or so, upgrading any tokens that use the old key in the process. Amazon recently announced a similar feature for handling web service credentials, which inspired this suggestion:

​http://aws.typepad.com/aws/2009/09/aws-access-credential-rotation.html

This is probably all too much complication, but it's something that's been nagging at me since I started increasing my dependence on the SECRET_KEY setting.