Code

Changes between Version 7 and Version 8 of AppEngine


Ignore:
Timestamp:
02/12/09 02:10:18 (5 years ago)
Author:
wkornewald
Comment:

batch middleware

Legend:

Unmodified
Added
Removed
Modified
  • AppEngine

    v7 v8  
    6363Datastore writes are very expensive. App Engine provides batch operations for saving and deleting lots of model instances at once (no more than 500 entries, though). Django should provide such an API, too, so code can be optimized. 
    6464 
    65 The API would be most flexible if it worked like a transaction handler where all save() calls within a function call are collected and then committed afterwards. The implementation wouldn't be trivial, though. It requires maintaining a cache of to-be-saved instances, so filter() calls can check the cache. Also, when a real transaction starts the cache must be flushed and disabled because in transactions we have to interact with the DB directly in order to lock an entity group. 
     65The API would be most flexible if it worked like a transaction handler where all save() calls within a function call are collected and then committed afterwards. The implementation wouldn't be trivial, though. It requires maintaining a cache of to-be-saved instances, so filter() calls can check the cache. Also, when a real transaction starts the cache must be flushed and disabled because in transactions we have to interact with the DB directly in order to lock an entity group. Instead of a decorator we could also provide a middleware, but this could lead to problems if, for instance, the view issues an http request (e.g., to start a task) and thus requires that the data has already been stored. 
    6666 
    6767There are batch operations for getting lots of model instances by key. This could be emulated with