Inconsistent behavior when a QuerySet is sliced

[Luis A. Arce]

I think that the current behavior of the QuerySet.slice method is not consistent. Just an example:

def limited_filtering(max_results=10, *args, **kwargs){
    """
    Returns a Queryset
    """

    returns SomeModel.filter(*args, **kwargs)[:max_results]
}

# ...

def other_method_somewhere(queryset){
    """
    Expects a Queryset of SomeModel and returns the same Queryset filtered again
    """

    return queryset.filter(some_other_criteria)
}

other_method_somewhere(limited_filtering(*args, **kwargs)); # raises "AssertionError: Cannot filter a query once a slice has been taken."

Given the previous code, it is legit and fair that other_method_somewhere expects, as in most situations, a filtrable QuerySet. Not allowing a sliced QuerySet to be filtered after being sliced adds an edge case difficult to handle when methods using a QuerySet type are somehow semantically far away from each other (and thus, they don't know if the other is returning an sliced QuerySet).

Without knowing the exact reasons why this was implemented this way, the possible solutions I can come with are:

• The most natural one, to make the sliced QuerySets filtrable.
• To make all the returned query sets in which can_filter == False to return a new type of QuerySet, something like InmutableQuerySet in order to take advantage of other methods related to them (values_list, etc.).
• To make QuerySet's slices return list or other type of collection (I guess this is impossible without breaking backwards compatibility).

Actually, the last case is supposedly the way one would expect it to work according to the documentation:

​https://docs.djangoproject.com/en/dev/ref/models/querysets/#when-querysets-are-evaluated

Slicing. As explained in Limiting QuerySets, a QuerySet can be sliced, using Python’s array-slicing syntax. Slicing an unevaluated QuerySet usually returns another unevaluated QuerySet, but Django will execute the database query if you use the “step” parameter of slice syntax, and will return a list. Slicing a QuerySet that has been evaluated (partially or fully) also returns a list.

By the way, there is also no reference in the documentation(or at least I could not find it) to the public method QuerySet.can_filter.

[Tim Graham]

There was a recent ​django-dev thread about this. I have not taken time to fully read it and determine if any consensus was reached.

[Tim Graham]

If we don't do this, let's document why not.

[jorgecarleitao]

I will map the discussion in the thread to here: I believe the consensus was that this cannot be done:

Assume other_method_somewhere uses a filter. The sequence would be

queryset.filter(first_clause)[:10].filter(second_clause)

Most people would assume (as pointed out in the thread) that the second clause would be applied to the last 10 entries, when it is not. Doing that is not possible in SQL without
subqueries and thus we explicitly forbid it.

[Thomas C]

See also ​https://groups.google.com/forum/#!topic/django-developers/ELqU2xt_Qo0 (sliced querysets not usable in a ModelChoiceField because not filterable).

[Tim Graham <timograham@…>]

In 327df551e89a505c5756becee97c40198f38aff2:

Fixed #23817 -- Updated docs on QuerySet evaluation

Removed inaccurate info about partial evaluation after refs #18702.
Added information on modifying sliced QuerySets; refs #22503.

[Tim Graham <timograham@…>]

In 614dd44d0d2e5f8f12256835c3453f420f54c3b4:

[1.6.x] Fixed #23817 -- Updated docs on QuerySet evaluation

Removed inaccurate info about partial evaluation after refs #18702.
Added information on modifying sliced QuerySets; refs #22503.

Backport of 327df551e89a505c5756becee97c40198f38aff2 from master

[Tim Graham <timograham@…>]

In 593353d8af5b4b8d1a1e627712fe68ed593961d0:

[1.7.x] Fixed #23817 -- Updated docs on QuerySet evaluation

Removed inaccurate info about partial evaluation after refs #18702.
Added information on modifying sliced QuerySets; refs #22503.

Backport of 327df551e89a505c5756becee97c40198f38aff2 from master

[Дилян Палаузов]

Is this fixed? The last comments suggest so, but the status is "new".

[Adam Johnson]

I think the documentation edit fixed this.