id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
35975	New feature: faster fixture loading via loaddata command	JorisBenschop	JorisBenschop	"As per [https://forum.djangoproject.com/t/feature-proposal-faster-fixture-loading-via-loaddata-command/36972/4 this forum discussion], I have created a patch to improve load times for the loaddata command under some circumstances.  

Currently the “loaddata” management command uses the obj.save() method for each deserialized object within a fixture. This function first tries an UPDATE statement and, if that fails, tries an INSERT statement.

I propose to add two optional flags to the loaddata command:

    –force-insert adds the “force_insert=True” to the save() method. This reduces load time by ~50% but has an increased risk of the upload failing in case the record already exists.

    –bulk_create: This option groups records in the fixture by the model, and insert the group as a single bulk_create statement. For large fixtures, I have achieved a 1000-fold improvement in loading time. It has a number of risks (already described in the bulk_create section), notably that it skips some routines covered in the save() method of the model.

Both these flags are supposed to be run by people who know that above-mentioned issues will not be a problem for them, and should not be enabled by default.

"	Uncategorized	closed	Core (Management commands)	5.1	Normal	duplicate	loaddata		Unreviewed	1	0	0	0	0	0