Opened 4 years ago

Closed 4 years ago

#31749 closed New feature (invalid)

Unable to find effective method to bulk_update else create

Reported by: Vishesh Mangla Owned by: nobody
Component: Database layer (models, ORM) Version: 3.0
Severity: Normal Keywords: bulk_update, bulk_create
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Vishesh Mangla)

I 'm making this code where on daily basis after preprocessing a csv file 100, 000 -200, 000 or even more rows will be read. Each row will get converted to a model object. Now the csv says me nothing about whether those ids existed in the database or not. I have been said
"update if key already present else create a new one". And the client has demands that it should be done in seconds. I said that it's not possible and somehow he is ready to wait for 15 minutes at the max. That's why I 'm refraining from using create_or_update or other functions. Is there any way to achieve the speed and accomplish the task at the same time? There are around 100 columns and the file is being read in small chunks because I can't read it whole in the memory. It's whopping 140 mbs.

Change History (2)

comment:1 by Vishesh Mangla, 4 years ago

Description: modified (diff)

comment:2 by Mariusz Felisiak, 4 years ago

Resolution: invalid
Status: newclosed

Please don't use Trac as a support channel. Closing per TicketClosingReasons/UseSupportChannels.

Note: See TracTickets for help on using tickets.
Back to Top