Support Negative Indexing on QuerySets

[Adam Nelson]

Add support for negative indexing of QuerySets:

Model.objects.all()[:-1]

The result is:

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/var/www/pinax-env/lib/python2.6/site-packages/django/db/models/query.py", line 126, in __getitem__
    "Negative indexing is not supported."
AssertionError: Negative indexing is not supported.

[David Cramer]

I'm not sure I entirely agree with support, and I think I understand why it's not available: it makes assumptions.

It can also easily be done by doing Model.objects.all().order_by('-id')[0:1]

[Adam Nelson]

I don't see what assumption a negative index makes that a positive index would not. Feel free to kill if there is an assumption I'm unaware of.

[James Bennett]

This is a long-standing explicit design decision in the ORM, and so gets a wontfix; if you disagree, feel free to bring it up on the django-developers mailing list and try to build consensus for changing it.

[Adam Nelson]

It's not worth the time to rally support with so many more important issues out there. I just figured it was good to get a permanent yes/no in Trac since there's nothing regarding this design decision documented elsewhere. I figured it was a feature that is just difficult to do and would be worth doing at some point.

[a.mamish@…]

I think this is more important than many think. It's a Python idiom that gets lost in translation. You are emulating lists with the sets and their indexing, it's only natural to support negative indexing as the language does. It surprised me when I started using Django that this didn't work, and I was so happy to be able to do that with Python while typing it too!

[anonymous]

bump

[anonymous]

Is there any justification for the design decision? I don't really see how halfway supporting indexing (unless there's some good DB-related reason for it) is good design.

[loic84]

FWIW I'm +1 on this.

I don't like when we stop halfway when emulating a python idiom; either we use our own unrelated API or we comply with the basic expectations that come with the idiom.

In any case, if you care about this, you need to post to the ML; closed tickets receive very little attention so they aren't a good place to debate this kind of decisions.

[django@…]

Replying to loic84:

FWIW I'm +1 on this.

I don't like when we stop halfway when emulating a python idiom; either we use our own unrelated API or we comply with the basic expectations that come with the idiom.

In short: querysets are not == lists.

The problem with saying the python negative indexing idiom should be supported is that negative indexing applies nicely to sets, lists, dicts etc where you know the length of the set, list, dict etc.

AFAIK, you can't negative index a generator without first exhausting the generator ... that would be a similar (but not identical) analogy to negative indexing a queryset (ie you have to retrieve the whole queryset before you can know the size).
I realise that the ORM could invert the sort and then re-write the slice for you.

[loic84]

Replying to django@…:

AFAIK, you can't negative index a generator without first exhausting the generator ... that would be a similar (but not identical) analogy to negative indexing a queryset (ie you have to retrieve the whole queryset before you can know the size).

Negative indexing could be implemented without performance issues, just look at the implementation of QuerySet.__getitem__().

The fancy slicing operations like [::2] which don't translate well to SQL are the real problem, I don't think we need to support those.

[loic84]

POC ​https://github.com/loic/django/compare/ticket13089

[anonymous]

+1
violates principle of least surprise

[yscumc]

+1
I was surprised by this and had to use qs[len(qs)-1] as a workaround.

[Florian Verdet]

It's even more surprising for values_list() -- especially for values_list( 'field', flat=True), since it has list in its name...

[Daniel Hahler]

Just for reference: it was discussed on the mailing list at ​https://groups.google.com/d/msg/django-developers/x86iWFnZiik/wyeI5woWkVYJ.