Add an optional dependency on python-fastpbkdf2

[Terry Chia]

I maintain ​python-fastpbkdf2, a hashlib.pbkdf2_hmac compatible interface that's around 3 times faster on CPython and more than 10x faster on PyPy.

This is the benchmark I use:

#!/usr/bin/bash

echo "Benchmark hashlib..."
python -m timeit -n 100 -s "from hashlib import pbkdf2_hmac" "pbkdf2_hmac('sha1', b'password', b'salt', 100000)"

echo "Benchmark fastpbkdf2..."
python -m timeit -n 100 -s "from fastpbkdf2 import pbkdf2_hmac" "pbkdf2_hmac('sha1', b'password', b'salt', 100000)"

On CPython 3.4.1,

$ ./bench.sh
Benchmark hashlib...
100 loops, best of 3: 60.2 msec per loop
Benchmark fastpbkdf2...
100 loops, best of 3: 20.3 msec per loop

On PyPy 2.6.0:

$ ./bench.sh
Benchmark hashlib...
100 loops, best of 3: 242 msec per loop
Benchmark fastpbkdf2...
100 loops, best of 3: 19.2 msec per loop

A faster PBKDF2 implementation improves security because a higher work factor can be used for the same amount of computing power.

I propose adding an optional dependency on python-fastpbkdf2 ala how Django depends on bcrypt and modifying the code to prefer python-fastpbkdf2's implementation whenever it's available with a fallback on the current hashlib.pbkdf2_hmac and pure Python code.

If this idea seems favourable to the Django maintainers, I have a patch ready for review.

[Tim Graham]

Are your improvements suitable for inclusion in Python itself? It seems better to pursue that course of action to me. My initial reaction is reluctance to add a dependency in a security sensitive area, especially for a project which is only 1 month old. You can write to the DevelopersMailingList to get more feedback.

[Terry Chia]

Replying to timgraham:

Are your improvements suitable for inclusion in Python itself? It seems better to pursue that course of action to me. My initial reaction is reluctance to add a dependency in a security sensitive area, especially for a project which is only 1 month old. You can write to the DevelopersMailingList to get more feedback.

I'm not sure if it's suitable for inclusions into Python itself as I'm not a core dev. Even if it is, it won't benefit Python 2.7 or PyPy users (who gains the most from this library).

I understand the hesitation to add a dependency on a new project. If it's any help, I take test coverage very seriously and the project is currently at 100% coverage and are tested against the standard PBKDF2 vectors. The bindings rely on CFFI which is what the bcrypt dependency also currently using. The ​C library itself about 400 or so lines of pretty readable C if anyone would like to review it.

I'll write to the list seeking further feedback in a few days.

[Tim Graham]

I think the follow up on the mailing list hasn't happened yet.

[Tim Graham]

​django-developers thread