Opened 16 years ago

Closed 15 years ago

#5632 closed (invalid)

Problems with DB objects on a multithreaded scenario

Reported by: claudio.marinozzi@… Owned by: nobody
Component: Database layer (models, ORM) Version: dev
Severity: Keywords: multi thread db
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


Discussion topic that started here in Django Developers

There are some critical functions like get_or_create, create, delete, add, remove, etc. that if used in a multithreaded scenario could be crash or "damage data" (some samples in the discussion above) when working on same DB-data, or even not, at the same time.

These operations should be serialized (mutexed) or something like that.

I attach a diff file containing the quick, but working, solution. It's a decorated function used on those critical functions that does the mutex thing. Maybe the decoration should be added to some other functions too. Still don't know Django that good.

Attachments (3)

dbmultithreaded.diff (5.8 KB) - added by claudio.marinozzi@… 16 years ago.
diff created with Django on Trunk at h12pm Italy time
dbmultithreaded2.diff (5.9 KB) - added by teepark 15 years ago.
dbmultithreaded3.diff (6.3 KB) - added by teepark 15 years ago.
based on SVN trunk r7364

Download all attachments as: .zip

Change History (9)

Changed 16 years ago by claudio.marinozzi@…

Attachment: dbmultithreaded.diff added

diff created with Django on Trunk at h12pm Italy time

comment:1 Changed 16 years ago by James Bennett

Patch needs improvement: set

comment:2 Changed 16 years ago by Ian Kelly

Please use the syntax "get_or_create = mutex(get_or_create)" for your decorators rather than the "@mutex" syntax. The reason for this is that the @ syntax is not supported in Python 2.3.

Also, please don't put your name in the code. If you want to be credited for your contributions, that's fine, but the proper place for it is in the AUTHORS file. See for more information.

On a more stylistic note, this code:

            res = func(*args, **kwargs)

is exactly equivalent to this:

            res = func(*args, **kwargs)

comment:3 Changed 16 years ago by Simon G <dev@…>

Has patch: set
Triage Stage: UnreviewedAccepted

Changed 15 years ago by teepark

Attachment: dbmultithreaded2.diff added

Changed 15 years ago by teepark

Attachment: dbmultithreaded3.diff added

based on SVN trunk r7364

comment:4 Changed 15 years ago by anonymous

Patch needs improvement: unset

sorry for the two tickets, I meant to include Claudio and myself in AUTHORS

comment:5 Changed 15 years ago by teepark

I don't have the necessary permission to modify/remove tickets otherwise I would remove mine.

This doesn't fix the multi-process situation - each process would just acquire it's own process-specific lock.

Also, get_or_create shouldn't be used with non-unique fields anyway (you get an AssertionError if more than one row comes up), and get_or_create in the development version catches IntegrityErrors, which will result from the unique constraint being violated by simultaneous get_or_creates.

So claudio's and my patches don't fix the whole problem, and the development version already does for sane cases. Sorry for all the unnecessary posts & patches.

comment:6 Changed 15 years ago by mrts

Resolution: invalid
Status: newclosed

This can be closed especially now that Malcolm has checked r8267 in.

Note: See TracTickets for help on using tickets.
Back to Top