id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
26063	"Regression in Django 1.9: SQLite can no longer handle more than 2000 values in a ""foo__in"" filter"	Raphaël Hertzog	Aymeric Augustin	"Commit 4f6a7663bcddffb114f2647f9928cbf1fdd8e4b5 introduced a regression in Django 1.9 because you can no longer have a queryset of the form {{{ Model.objects.filter(foo__in=array) }}} with more than 2000 items in {{{array}}}.

The above changes generates a “SELECT QUOTE(?), ..., QUOTE(?)” query with as many values as items in the array. This will hit the SQLite limit SQLITE_MAX_COLUMN which defaults to 2000 and can only be increased up to 32767.

Before this change we were only limited by SQLITE_MAX_VARIABLE_NUMBER which is lower by default (999) but which can be increased to a much higher value. In Debian for example we use a value of 250000.

While the latter limit was unreasonably low (it just costs a bit of memory and you can expect to have many parameters), the former limit make much more sense IMO and I don't expect distributions to override the upstream value.

I'm not sure what's the best way forward. Possibly process parameters by batch of 2000.

(This bug has been initially reported in https://bugs.debian.org/809211 )"	Bug	closed	Database layer (models, ORM)	1.9	Release blocker	fixed		d@…	Accepted	1	0	0	0	0	0