Opened 2 months ago
Closed 2 months ago
#36606 closed Cleanup/optimization (fixed)
Optimize QuerySet.values_list(flat=True) with no fields
| Reported by: | Adam Johnson | Owned by: | Adam Johnson |
|---|---|---|---|
| Component: | Database layer (models, ORM) | Version: | dev |
| Severity: | Normal | Keywords: | |
| Cc: | Triage Stage: | Accepted | |
| Has patch: | yes | Needs documentation: | no |
| Needs tests: | no | Patch needs improvement: | no |
| Easy pickings: | no | UI/UX: | no |
Description (last modified by )
Currently, QuerySet.values_list() ensures that no more than 1 field is set (source):
if flat and len(fields) > 1: raise TypeError( "'flat' is not valid when values_list is called with more than one " "field." )
However, it also allows the case where *no* fields are declared, for which all fields are fetched, only to throw away all but the first one (source):
class FlatValuesListIterable(BaseIterable): """ Iterable returned by QuerySet.values_list(flat=True) that yields single values. """ def __iter__(self): queryset = self.queryset compiler = queryset.query.get_compiler(queryset.db) for row in compiler.results_iter( chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size ): yield row[0]
I think we can optimize this case to select only the first field in the model instead, maintaining semantics while avoiding overfetching.
This case also seems untested with the values_list() tests in tests/lookup/tests.py, so we'd want to add a test there.
Change History (8)
comment:1 by , 2 months ago
| Description: | modified (diff) |
|---|
comment:2 by , 2 months ago
| Owner: | set to |
|---|---|
| Status: | new → assigned |
comment:3 by , 2 months ago
comment:4 by , 2 months ago
Yes, I'm inclined to agree. It's not clear when reading what values_list(flat=True) would do.
I think we can make the optimization *and* deprecate, though.
comment:5 by , 2 months ago
| Has patch: | set |
|---|
comment:6 by , 2 months ago
| Triage Stage: | Unreviewed → Accepted |
|---|
Agree we should deprecate, happy to take a follow-up PR as Refs ... this one.
I wonder if we should take the opportunity to deprecate
values_list(flat=True)instead by changing the check to belen(fields) != 1as it seems like an omission in the API and should be a rare occurrence in code bases so I'm not convinced it's worth supporting it.