id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
14700	Speed up RawQuerySet iterator	Anssi Kääriäinen	nobody	"Currently `RawQuerySet` uses a lot of cycles doing repeatedly the same calculations inside the iterator loop. The attached patch corrects this problem with the following results, using the test found in #14697 (with Test2.objects.all() replaced by Test2.objects.raw('select * from test`__`test2'):

Before patch: 0.9 seconds to 1.0 seconds.

After patch 0.195 seconds to 0.205 seconds. (This is just slightly faster than using Test2.objects.all(), with #14697 applied)

There is another patch, which is unfortunately most likely backwards incompatible. The idea of the another patch is to speed up model instance initialization by passing in a dict(attname->val) containing all the values needed, and then using self.`__`dict`__`.update(attname, val). This however adds a new keyword argument to the `__`init`__`. Before the patch, all the kwargs were attr -> value, but after that there can be a kwarg '`_`use_dict', which contains a dict of attname -> val. The patch cuts off another 0.06 seconds (or 30%) from the test case, leaving around 0.14 seconds left. There might be some other incompatibilities, too... The same amount of performance increase could be achieved for standard `QuerySet` iterator using the same hack.

Just fetching the data from db, creating 10000 raw objects and updating 10 attributes for each of those objects results in 0.01 seconds used. Hence there is just 0.04, or 40% overhead left when using the second patch, and 100% overhead when using the first patch.

I will try to find the time to write django-bench benchmark for this case.
"		closed	Database layer (models, ORM)	dev		fixed	rawqueryset, iterator, performance	Marti Raudsepp	Accepted	1	0	0	1	0	0