Opened 4 hours ago

Last modified 23 minutes ago

#36606 assigned Cleanup/optimization

Optimize QuerySet.values_list(flat=True) with no fields

Reported by: Adam Johnson Owned by: Adam Johnson
Component: Database layer (models, ORM) Version: dev
Severity: Normal Keywords:
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Adam Johnson)

Currently, QuerySet.values_list() ensures that no more than 1 field is set (source):

        if flat and len(fields) > 1:
            raise TypeError(
                "'flat' is not valid when values_list is called with more than one "
                "field."
            )

However, it also allows the case where *no* fields are declared, for which all fields are fetched, only to throw away all but the first one (source):

class FlatValuesListIterable(BaseIterable):
    """
    Iterable returned by QuerySet.values_list(flat=True) that yields single
    values.
    """

    def __iter__(self):
        queryset = self.queryset
        compiler = queryset.query.get_compiler(queryset.db)
        for row in compiler.results_iter(
            chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size
        ):
            yield row[0]

I think we can optimize this case to select only the first field in the model instead, maintaining semantics while avoiding overfetching.

This case also seems untested with the values_list() tests in tests/lookup/tests.py, so we'd want to add a test there.

Change History (3)

comment:1 by Adam Johnson, 4 hours ago

Description: modified (diff)

comment:2 by Adam Johnson, 42 minutes ago

Owner: set to Adam Johnson
Status: newassigned

comment:3 by Simon Charette, 23 minutes ago

I wonder if we should take the opportunity to deprecate values_list(flat=True) instead by changing the check to be len(fields) != 1 as it seems like an omission in the API and should be a rare occurrence in code bases so I'm not convinced it's worth supporting it.

Note: See TracTickets for help on using tickets.
Back to Top