id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
9006	QuerySet indexing by __getitem__ gets wrong answer in edge cases	Stefan Moluf	Malcolm Tredinnick	"I've just spent a few hours in pdb tracking this down. I've got about as far as I'm able into the Django libraries looking for the culprit, but I've run out of ideas due to my limited knowledge of Django's inner workings. I need someone with a bit more experience to look at this one.

I have a Membership model which relates Users to Groups.

{{{
#!python
# models.py
class Membership (models.Model):
    user = models.ForeignKey(User)
    group = models.ForeignKey(Group)
    school_year = models.ForeignKey(SchoolYear, related_name='memberships')
}}}

This query turns up the following results:

{{{
>>> Membership.objects.filter(user__username=""jammons"")
[<Membership: Office of Technology Services (Jeff Ammons, 2008 - 2009)>, <Membership: UPS Fencing Club (Jeff Ammons, 2008 - 2009)>, <Membership: Adelphian Concert Choir (Jeff Ammons, 2008 - 2009)>]
}}}

These results are expected and good. However, indexing does the Wrong Thing. For some reason, {{{__getitem__}}} returns the same model for {{{[0]}}} and {{{[2]}}}:

{{{
>>> qs[0]
<Membership: Adelphian Concert Choir (Jeff Ammons, 2008 - 2009)>
>>> qs[1]
<Membership: UPS Fencing Club (Jeff Ammons, 2008 - 2009)>
>>> qs[2]
<Membership: Adelphian Concert Choir (Jeff Ammons, 2008 - 2009)>
>>> 
}}}

The result is that the real {{{qs[0]}}} is unreachable, and furthermore, inline formset validation is therefore broken when this occurs, because the wrong self.instance is set on the first form, causing it to .exclude() the wrong instance from its unique_check and resulting in the form believing that it is a duplicate.

Engaging pdb and then entering ""continue"" (running the query in the debugger without actually inspecting the trace) returns the same answer.
{{{
>>> def test():
...     print Membership.objects.filter(user__username=""jammons"")[0]
...
>>> import pdb
>>> pdb.runcall(test)
> <console>(2)test()
(Pdb) continue
Adelphian Concert Choir (Jeff Ammons, 2008 - 2009)
}}}

__HOWEVER__: Running pdb and stepping through the code produces the __right__ answer:

{{{
>>> pdb.runcall(test)
> <console>(2)test()
(Pdb) step

...lots of stepping through the various nested calls...

(Pdb) step
Office of Technology Services (Jeff Ammons, 2008 - 2009)
}}}

Upon inspecting {{{django.db.connection.queries}}}, what seems to be happening is that the first two methods hit the _result_cache to do the indexing, while the third runs new queries instead.

Theory 1: This is some sort of bizarre race condition where the _result_cache is being deleted, but somehow lingers on long enough that execution speed is a factor in determining whether the indexing can access it or not.

Theory 2: By trial and error, it seemed that stepping through _clone() {{{(django/db/models/query.py, line 218)}}} in pdb gave the right answer, while allowing the debugger to run it at full speed gave the wrong. Perhaps there's something in {{{__clone()}}} that depends on execution speed. This is not 100%, though, as at least on one occasion it produced the wrong answer.

Theory 3: There is some issue with the laziness of !QuerySets and database query time.

Someone else is probably better qualified than I am to speculate on the potential causes. I'm happy to provide more detailed information if you need it. Good luck!"		closed	Core (Other)	1.0		wontfix	queryset index race	andrew.tennikoff@… Carl Meyer	Unreviewed	0	0	0	0	0	0