Opened 9 years ago

Closed 9 years ago

Last modified 9 years ago

#26338 closed Bug (needsinfo)

Getting Queryset item by index with PostgreSQL behaves randomly

Reported by: Paweł Adamczak Owned by: nobody
Component: Database layer (models, ORM) Version: 1.9
Severity: Normal Keywords: postgresql
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

I originally 'reported' it on StackOverflow, but this is probably where it should be.

This code on SQLite works as intended and gets two random items from queryset:

random.seed()
index1, index2 = random.sample(range(0, qs.count()), 2)

all_objects = qs.all()
object1 = all_objects[index1]
object2 = all_objects[index2]

but when run on PostgreSQL, getting items from Queryset by index seems to behave randomly.

Here are index1, index2 and object1, object2 pairs when that code is run 5 times:

9 12
ID: 2754 ID: 2365
15 11
ID: 1626147 ID: 200811
12 1
ID: 2365 ID: 203112
1 12
ID: 203112 ID: 2365
1 3
ID: 203112 ID: 203112

The last one is especially important, because what it means is qs.all()[1] == qs.all()[3] which just can't be (all ID's are unique). And this occurred quite regularly.

Converting queryset to list fixed the issue (all_objects = list(qs.all())), so I am guessing it has something to do with Queryset laziness in PostgreSQL implementation. I tried to look it up in the source code but with no luck.

Change History (7)

comment:1 by Tim Graham, 9 years ago

Does the queryset have an order_by()? If a query doesn’t have an ordering specified, results are returned from the database in an unspecified order.

comment:2 by Paweł Adamczak, 9 years ago

Yes, both the model and queryset has order_by set.

But even if it's in unspecified order, could the order change between accessing it's items? Because from my understanding, event if it's random, qs.all()[1] == qs.all()[3] should never be true.

comment:3 by Tim Graham, 9 years ago

I believe all_objects[index1] and all_objects[index2] are running two separate queries. When you first cast to a list, you only have one query. Please verify by checking connections.queries.

Version 0, edited 9 years ago by Tim Graham (next)

comment:4 by Tim Graham, 9 years ago

Resolution: needsinfo
Status: newclosed

Please reopen with a test for Django's test suite that demonstrates the problem.

comment:5 by Paweł Adamczak, 9 years ago

You were right with the order_by, but why setting it on model level isn't taking care of that?
Shouldn't that be enough?

class Meta:
        ordering = ['-field1, '-field2', 'field3']

comment:6 by Tim Graham, 9 years ago

Again, I'd ask if you could please provide more details such as a test for Django's test suite (ideally) or a sample project to reproduce the issue. Although I suspect a bug in your code rather than in Django, there aren't enough details here currently to investigate the issue. Ideally, "is it a bug?" questions would first be directed to our support channels (TicketClosingReasons/UseSupportChannels) and reported here only if someone else confirms the problem as a probable bug. Thanks!

comment:7 by Paweł Adamczak, 9 years ago

OK, sorry for the inconvenience.

Note: See TracTickets for help on using tickets.
Back to Top