RunPython parameter "atomic" is ineffective.

[Vernon Cole]

This report is similar to #24630, but differs from it in detail. The fix for #24630 is insufficient for this scenario.

I am attempting to apply a data migration to a Postgres table with millions of records -- my first attempt (using RunSQL) failed and rolled back after about two days of run time. It appears that the job must be done in smaller chunks, so I rewrote the migration using RunPython, limiting the size of the result set with a slicing, with set_autocommit(False) and explicit commit calls.

It appears from the documentation that the built-in transaction control can be overridden:
​https://docs.djangoproject.com/en/1.8/topics/db/transactions/

This should be safe, but may cause a crash if you attempt to use the schema_editor provided on these backends; in this case, please set atomic=False.

But "atomic=False" has no effect on the operation of the migration. Stepping through the code using a debugger, I failed to find anyplace where that parameter is actually used, nor could I find a way to short-cut the atomic transaction. It appears that I will need to write a stand-alone program to migrate the data.

Unless someone more talented than me can find a way to make "atomic=False" actually work, it should be removed from the code and from all documentation; or the documentation made more explicit to explain when and where it does work.

[Vernon Cole]

My attempt to break up the migration into smaller chunks.

[Tim Graham]

The atomic parameter doesn't have any effect for databases like PostgreSQL which have features.can_rollback_ddl (see the implementation in 5a917cfef319df33ca30a3b27bd0b0533b8e63bb). I updated my PR in #24630 to make this explicit in the documentation.

[Tim Graham]

If you could review my patch and reopen this ticket or leave a comment there if you believe the issue isn't sufficiently addressed, that would be appreciated.