Opened 16 years ago

Closed 16 years ago

Last modified 12 years ago

#6641 closed Uncategorized (fixed)

Concurrency issue with get_or_create

Reported by: JEFFGUINNESS Owned by: nobody
Component: Database layer (models, ORM) Version: dev
Severity: Normal Keywords: get_or_create
Cc: tomasz.zielinski@…, gerdemb Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

This is from Travis Terry post on http://groups.google.com/group/django-developers/browse_thread/thread/9730f4005bf08b67 and where he is much more eloquent then I am, I will post his some of his comments.

Two threads/processes/servers (let's call them P1 and P2) need to
concurrently create a unique object

  1. P1 calls get_or_create(), which tries to get the item (it doesn't exist)
  2. P2 calls get_or_create(), which tries to get the same item (it doesn't exist)
  3. P1's get_or_create() tries to create the item (this works and returns the item)
  4. P2's get_or_create() tries to create the item. One of two things happens:
    1. a second item is created with the same parameters if this doesn't violate a UNIQUE constraint
    2. the second create fails (because of a UNIQUE constraint) and raises an exception

The following pseudo-code handles the problem:

def get_or_create(**kwargs):
    try:
       obj = get(**kwargs)
    except:
       try:
          obj = create(**kwargs)
       except:
          obj = get(**kwargs)
    return obj 

Attachments (2)

query.diff (1.0 KB ) - added by JEFFGUINNESS 16 years ago.
Diff file
6641-get_or_create.diff (1.2 KB ) - added by Timothée Peignier <tim@…> 16 years ago.
Catch IntegrityError

Download all attachments as: .zip

Change History (11)

by JEFFGUINNESS, 16 years ago

Attachment: query.diff added

Diff file

comment:1 by James Bennett, 16 years ago

Resolution: wontfix
Status: newclosed

This really doesn't feel like it warrants inclusion in Django's core, since a site which has traffic patterns that would generate this problem can easily use something like Marty's suggestion in the mailing-list thread.

comment:2 by Malcolm Tredinnick, 16 years ago

Patch needs improvement: set
Resolution: wontfix
Status: closedreopened
Triage Stage: UnreviewedAccepted

This patch isn't too intrusive and I can't imagine how it will fail incorrectly. Plus it doesn't add a lot of overhead. So, sorry James... I realise it's borderline, but I think it might be worth it.

Patch needs improvement, though. Catching Exception is wrong. You're looking for IntegrityError, so catch it. That's one of the exceptions we carefully promote into django.db.backends so it's portable.

by Timothée Peignier <tim@…>, 16 years ago

Attachment: 6641-get_or_create.diff added

comment:3 by Philippe Raoult, 16 years ago

Patch needs improvement: unset
Triage Stage: AcceptedReady for checkin

Marking as ready since the new patch addresses malcom's comments.

comment:4 by Philippe Raoult, 16 years ago

Version: 0.96SVN

Changing the version, the patch applies to SVN (and qsrf by the look of it, the issue is still present there).

comment:5 by Malcolm Tredinnick, 16 years ago

Resolution: fixed
Status: reopenedclosed

(In [7289]) Fixed #6641 -- If we lose a race when creating a new object in get_or_create,
re-get the result from the database and return that. Thanks, Jeff Guinness and
Timothée Peignier.

comment:6 by Tomasz Zieliński, 13 years ago

Easy pickings: unset
Severity: Normal
Type: Uncategorized

comment:7 by Tomasz Zieliński, 13 years ago

Cc: tomasz.zielinski@… added

comment:9 by gerdemb, 12 years ago

Cc: gerdemb added
UI/UX: unset
Note: See TracTickets for help on using tickets.
Back to Top