Encourage using persistent database connections

[Tom Carrick]

Sometimes I'm asked to review a Django project's performance. The most typical issues are probably N+1 selects and wasting time creating connections.

I propose encouraging setting CONN_MAX_AGE to a non-zero value by:

• Setting CONN_MAX_AGE to None or a reasonably high value in the generated settings.py - but keeping the default as-is for compatibility.
• Adding a deployment level system check to guard against a zero value.

Anyone who needs this to be 0 can easily change it and silence the system check, but I think most projects will benefit from this.

[Carlton Gibson]

Hi Tom. Thanks for the suggestion here.

It looks like this has come up before, and there's a problem with runserver compatibility: ​mailing list post from 2013.

(​Mailing list search for CONN_MAX_AGE)

Do these points still apply? If so, is there a way around them? Did anyone explore the idea from the mailing list thread?

Also relevant is #33497 -- "Database persistent connections do not work with ASGI in 4.0" — Question then: is it clear cut that persistent connections are better — not just for this ASGI case but in general? (If you have lots of traffic, sure, but... 🤷) From that ticket, I know Florian is thinking about a connection pool, which would be connected.

Whilst generally it seems Sure, yeah, I think some wider discussion is needed, even if that just results in Clearly, yeah. (DevelopersMailingList :)
I'll mark this needs info in the meantime.

[Tom Carrick]

Thanks, makes sense. I'll do a bit of digging and see where I end up.

[Florian Apolloner]

Replying to Carlton Gibson:

It looks like this has come up before, and there's a problem with runserver compatibility: ​mailing list post from 2013.

I think this still holds true. Unless we change runserver to use a threadpool (or db pool) it doesn't make much sense to use persistent connections there.

[Tom Carrick]

Okay, I think it's best to revisit this once (if?) we have connection pooling.