Database and memcached connections break after fork.
|Reported by:||sebastian_noack||Owned by:||nobody|
|Component:||Database layer (models, ORM)||Version:||1.4|
|Cc:||davidswafford||Triage Stage:||Design decision needed|
|Has patch:||yes||Needs documentation:||no|
|Needs tests:||yes||Patch needs improvement:||no|
If you have a management command that does CPU-heavy tasks, or when implementing a server for certain background tasks, its likely that you will use the multiprocessing module, to scale over multiple CPUs. However django implements connections to the database and memcached as singletons (created on first use, reused forever). So if you have used the database or memcached before forking, the child processes inherit the established connection. And when multiple processes use a connection at the same time (which can and will happen) the requests will fail in an ugly way.
However the multiprocessing module comes with a mechanism provided for such cases, that enables you to cleanup things after fork. My patch uses that mechanism, in order to reset the possibly created database and memcached connections after fork. So that the child process will create its own connection when it needs it.
Change History (8)
Changed 3 years ago by sebastian_noack
comment:1 Changed 3 years ago by lrekucki
- Needs documentation unset
- Needs tests set
- Patch needs improvement unset
- Triage Stage changed from Unreviewed to Design decision needed
comment:4 Changed 2 years ago by apollo13
- Resolution set to wontfix
- Status changed from new to closed
comment:6 Changed 23 months ago by davidswafford
- Cc davidswafford added
- Resolution wontfix deleted
- Status changed from closed to new