prefetch_related infinite recursion by default managers
|Reported by:||akaariai||Owned by:||nobody|
|Component:||Database layer (models, ORM)||Version:||1.3|
|Has patch:||no||Needs documentation:||yes|
|Needs tests:||no||Patch needs improvement:||no|
I haven't tested these two cases with the trunk version, so I don't know if these mentioned problems work with prefetch_related or not. At least both of these need tests, though.
Recursive default queryset prefetch_related calls need test. The simplest case is:
class PeopleDefManager(models.Manager): def get_query_set(self): return super(PeopleDefmanager, self).get_query_set().prefetch_related('reverse_best_friend') class People(models.Model): name = models.TextField() best_friend = models.ForeignKey('self', related_name='reverse_best_friend') objects = PeopleDefManager()
I did some work for #17000, and my version for that ticket had a very serious problem. The problem was indefinite recursion which ended up eating all the memory and going to OOM killer. That is a pretty bad failure condition... I ended up tracking which prefetches had been done and bailing out if recursion was detected. And having a hard limit of doing at most 200 prefetch_one_level calls in one queryset. The limit could be much lower while still allowing for infinite recursion. I haven't tested if the trunk version suffers from this. At least tests needed in any case.
Another condition that needs testing is:
A / \ through / \ through b_set / \ c_set B C \ / through d_set \ / through d_set \ / D
It is important that both B and C have a field with same name to D. I don't know if this works or not, but this also needs tests at least.
Change History (8)
comment:1 Changed 3 years ago by akaariai
- Cc anssi.kaariainen@… added
- Needs documentation set
- Needs tests unset
- Patch needs improvement unset
- Summary changed from prefetch_related tests needed: recursive default prefetch_related structures, diamond structures to prefetch_related infinite recursion by default managers
Changed 3 years ago by akaariai
comment:3 Changed 3 years ago by lukeplant
- Resolution set to fixed
- Status changed from new to closed