Changes between Initial Version and Version 2 of Ticket #28231
- Timestamp:
- May 23, 2017, 12:09:21 AM (8 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
Ticket #28231
- Property Cc added
- Property Summary bulk_create: avoid iterating `objs` more than necessary when `bulk_size` is provided → bulk_create: avoid iterating `objs` more than necessary when `batch_size` is provided
-
Ticket #28231 – Description
initial v2 1 When `b ulk_size` provided in `bulk_create`, a user might assume (As I myself have) that `objs` iterable will not be iterated more than `bulk_size` times at once, and that no more than roughly `bulk_size` Model objects reside in memory at any given time.1 When `batch_size` provided in `bulk_create`, a user might assume (As I myself have) that `objs` iterable will not be iterated more than `batch_size` times at once, and that no more than roughly `batch_size` Model objects reside in memory at any given time. 2 2 3 3 When using `bulk_create` for relatively big sets of objects provided by a generator object, it would be prefered to avoid iterating over the entire generator object. Moreover, if not iterating over the generator object is deemed unnecessary or out-of-scope for django it would be prefered to make a comment on said behavior in documentation. … … 5 5 I suggest two possible solutions: 6 6 1. Document this behavior (`bulk_create` converts passed `objs` iterator to a list), or 7 2. Avoid doing so when `b ulk_size` is given (or default is other than `None`. i.e. sqlite).7 2. Avoid doing so when `batch_size` is given (or default is other than `None`. i.e. sqlite). 8 8 9 9 I did not research the possibility of avoiding the list conversion, but if that solution is accepted by the community I volunteer to investigate further and claim this ticket.