id,summary,reporter,owner,description,type,status,component,version,severity,resolution,keywords,cc,stage,has_patch,needs_docs,needs_tests,needs_better_patch,easy,ui_ux 17030,Remove special handling of deferred models in queryset iterators,Anssi Kääriäinen,nobody,"Suor gave me a great idea in ticket #17025: use attname cache for `__init__`. This turned out to work really well for deferred models: we don't need to special case them any more in queryset iterator loops (we do of course need to get the right deferred model class still). Because the model `__init__` did need to get the data in the same order as the model._meta.fields for *args based initialization, we couldn't use the fast init for deferred models. But if the model's options know the selected columns in the queryset, we suddenly can use the *args based `__init__` which is a lot faster. A new model._meta.get_init_attnames() method was added for this. One might wonder if this is backwards compatible. Well, the `__init__` behaviour surely changed for deferred models, but I don't think this matters: the deferred classes are only for internal use of querysets, and thus the user should never call `__init__` of deferred models directly. The speedup from djangobench for raw_deferred is 2.5x. The speedup for query_all and query_all_multifield is about 1.05. There isn't a djangobench test for .defer, but according to my test .only('id')[0] is about 2x faster, and .only('id)[0:100] is about 1.6x faster. The attached patch passes all test. There is some cleanup needed in get_cached_row, but there is a separate patch for optimizing those parts of the queryset, so I didn't bother to clean it up. However there is something strange going on with select_related querysets and its handling of load_fields. It seems that sometimes the field is passed as field.attname and sometimes as field.name to deferred_model_factory, and I don't believe that to be correct. The patch just checks for both, but that is the wrong cure. I believe the load_fields should always be field.name, not field.attname. Another tickets problem IMHO. Suor also suggested the idea of using `__dict__`.update in model.`__init__`. That would speed up model initialization somewhat, but I believe that is backwards incompatible as `__setattr__` would not be called. Is this corret? We are talking about 10-15% speed increase in the case the model has a lot of columns.",Cleanup/optimization,closed,"Database layer (models, ORM)",1.3,Normal,wontfix,,anssi.kaariainen@…,Accepted,1,0,0,0,0,0