Django

Code

Ticket #11906 (new)

Opened 6 months ago

Last modified 1 month ago

QuerySet._fill_cache is not thread-safe

Reported by: mrts Assigned to: nobody
Milestone: Component: Database layer (models, ORM)
Version: SVN Keywords: threading thread
Cc: Triage Stage: Accepted
Has patch: 0 Needs documentation: 0
Needs tests: 0 Patch needs improvement: 0

Description

If several threads iterate over the same queryset, the following error occurs: 

File "/some/path/__init__.py", line 624, in prerender
   for obj in objects: # <-- objects is a QuerySet

 File "/some/path/python2.5/site-packages/django/db/models/query.py", line 106, in _result_iter
   self._fill_cache()

 File "/some/path/python2.5/site-packages/django/db/models/query.py", line 692, in _fill_cache
   self._result_cache.append(self._iter.next())

ValueError: generator already executing

That happens as multiple threads concurrently both append to self._result_cahce and consume self._iter (see source:/django/trunk/django/db/models/query.py@11584#L692).

Attachments

Change History

09/18/09 10:27:25 changed by mrts

  • needs_better_patch changed.
  • needs_tests changed.
  • needs_docs changed.

The question is how and whether this should be fixed, as using a shared queryset is not necessarily a good thing (it makes mostly sense to consume it with list() and store the list or use cache instead).

09/18/09 10:28:01 changed by mrts

  • keywords set to threading thread.

09/18/09 11:18:30 changed by mrts

Summarized it as follows in the unofficial threadsafety wiki page: Django core is generally threadsafe as of 1.0.3 / 1.1. However, QuerySets are known not to be thread-safe, see #11906. Usually that does not pose problems as they are (or should be) not shared between threads in Django. The exception to that rule is the use of exotic global/class-level/shared instance variable querysets in your own code (e.g. when using the ORM outside of the Django dispatch system), where you are assumed to know what you are doing and protect them appropriately anyway.

02/03/10 21:21:26 changed by russellm

  • stage changed from Unreviewed to Accepted.

Add/Change #11906 (QuerySet._fill_cache is not thread-safe)




Change Properties
Action