Code

Opened 11 months ago

Last modified 7 months ago

#20562 new Cleanup/optimization

Docs: How to use django ORM with multiprocessing

Reported by: guettli Owned by: nobody
Component: Documentation Version: 1.5
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

There are several tickets closed as "invalid" which were submitted because the user had problems to use the django ORM with the multiprocessing library.

Please add some documentation how to do use it.

Main part: restart the database connection after fork()....

Attachments (0)

Change History (5)

comment:1 Changed 11 months ago by bmispelon

  • Cc bmispelon@… added
  • Easy pickings set
  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted
  • Type changed from Uncategorized to New feature

I seem to have commented on the wrong ticket.

Sorry for the noise :/

Last edited 11 months ago by bmispelon (previous) (diff)

comment:2 Changed 11 months ago by bmispelon

  • Cc bmispelon@… removed
  • Easy pickings unset
  • Triage Stage changed from Accepted to Unreviewed
  • Type changed from New feature to Uncategorized

comment:3 Changed 11 months ago by akaariai

  • Triage Stage changed from Unreviewed to Accepted

I believe the only thing we can document is that "don't use fork()". Closing connection after fork() might be too late (who says that it is safe to close a connection from the child?). You need to do it before fork(). This might work. Or might not work. How about in-memory sqlite database, will that work? And so on...

The problem isn't that we aren't willing to make fork() work, or document how you can use fork() with Django. The problem is that in general the libraries used by Django aren't fork() safe. We can't work around that.

I am marking this as accepted. We should at least mention that you should not use fork(). In addition we should maybe recommend alternatives to fork(). I don't believe we should mention that "you can use fork() if you do the following things". It will be nearly impossible to actually guarantee that will be true.

comment:4 Changed 11 months ago by guettli

My rule of thumb: "fork() before connection.cursor is created. If it is None, it is safe to fork()". The same goes for other connections (for example memcached).

comment:5 Changed 7 months ago by timo

  • Type changed from Uncategorized to Cleanup/optimization

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as new
The owner will be changed from nobody to anonymous. Next status will be 'assigned'
as The resolution will be set. Next status will be 'closed'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.