Changes between Version 16 and Version 17 of AppEngine
- Timestamp:
- Mar 19, 2009, 11:06:02 AM (15 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
AppEngine
v16 v17 12 12 13 13 At the Django level we need support for: 14 * setting an owner model for !ManyToManyFields (i.e., there is no intermediary table and the field's data could be stored on either the model defining the relationor on the other model) and !ModelForm should handle that case efficiently (i.e., try to save the relations together with the model instead of afterwards in save_m2m)14 * setting an owner model for !ManyToManyFields (i.e., there is no intermediary table and the field's data could be stored on either the model defining the !ManyToManyField or on the other model) and !ModelForm should handle that case efficiently (i.e., try to save the relations together with the model instead of afterwards in save_m2m) 15 15 * !ListField (stores a list of values of a certain type; DB backends could use this internally for !ManyToManyFields without an intermediary table) and [http://code.djangoproject.com/ticket/2417 BinaryField] 16 16 * batch save() and delete() … … 19 19 * Permission and !ContentType should be fake models which are stored as a string in a !CharField 20 20 21 == Reminder: Datastore and request limitations ==22 23 You can't have requests that take longer than 10 seconds and you can't retrieve more than 1000 model instances at once from the datastore. It's also impossible to run more than 30 queries without hitting the 10 sec request limit.24 25 A single entity (actually: a whole entity group) can't handle more than 5 writes per second (writes = save or delete).26 27 Unique properties can only be emulated via the key_name, but this means their values can't be changed afterwards, so we might have to fall back to non-guaranteed uniqueness. Since you can't issue queries from within a transaction we have the problem that we can't even do a simple check in all cases. Probably we can only rely on checking on the !ModelForm level, then. Alternatively, we have to document that you can't use transactions on models that have unique properties apart from the !PrimaryKey (which can be emulated with the key_name and thus gives us a 100% uniqueness guarantee because it can be used in a transaction).28 29 An entity may not have more than 5000 index entries.30 31 All query filter rules are connected via the AND operator. The OR operator is not supported.32 33 Transactions can only run on a single entity group and you can't run queries within a transaction.34 35 Also not supported:36 * JOINs (could be done manually for small datasets)37 * sub-queries (ditto)38 * DISTINCT queries (i.e., no queryset.dates(), etc.)39 * referential integrity40 41 21 == Emulation of SQL features == 42 22 43 23 If we emulate certain SQL features, it must be possible to detect when something gets emulated. This is important because you might mistakenly develop code that doesn't scale when your database grows beyond a certain size. In settings.py we could have a flag which specifies whether to throw warnings for emulated features (ENABLE_DB_EMULATION_WARNINGS?). By default, warnings should be enabled, so you never miss potential sources of problems. 44 24 25 Alternatively, one could have to activate emulation by calling a special function (Model.objects.with_emulation().filter(...)). That's more explicit and less error-prone. 26 45 27 == Schemas == 46 28 47 Since tables are flexible and don't have schema definitions running "manage.py syncdb" shouldn't be necessary. 29 Since tables are flexible and don't have schema definitions running "manage.py syncdb" shouldn't be necessary. It can still be supported by emulating a remote DB connection. 48 30 49 31 == Indexes == … … 55 37 == Keys, key_name, key id, parents == 56 38 57 Django should always assign a key_name to each newly created entity instead of letting App Engine choose a key id, so data can be exported from and imported into the datastore more easily and migrations to other providers become less problematic. A transaction can be used to ensure that no existing entity with the generated key_name gets overwritten.39 In general, the "pk" field should never be assumed to be a number. If an entity uses a key_name the application should continue to work. 58 40 59 An interface to the underlying key and key id should be provided, too, but it shouldn't be recommended due to its problems.41 The key_name and parent could be emulated with a !CharField(primary_key=True) that automatically prefixes the given string with a character, internally. A special function could be provided for encoding a primary_key that consists of parents and a key_name or key id. This API would allow for reducing the key_name and parent into a single pk property and staying compatible with existing Django code which wouldn't work if we had separate pk and parent properties. 60 42 61 The key_name and parent could be emulated with a !CharField(primary_key=True) that automatically prefixes the given string with a character, internally. If you only need a key_name it's sufficient to specify a string. If you want to also specify a parent you could create a special encoded string by passing the parent and (optionally) the desired key_name to that function and then passing the result to the !CharField. This API would allow for reducing the key_name and parent into a single pk property and staying compatible with existing Django code which wouldn't work if we had separate pk and parent properties. 62 63 The pk property should return an url-safe string that contains the key_name without the safety prefix (i.e., the value of the !CharField(primary_key=True)) and the parent pk. This is more portable than using the str(Key) because it doesn't contain the model and app name. Moreover, in case you specified a pk manually , the URLs will be much nicer. Even if you don't specify a key_name the URL is still shorter than the str(Key) version. 43 The pk property should return an url-safe string that contains the key_name without the safety prefix (i.e., the value of the !CharField(primary_key=True)) and the parent pk. This is more portable than using the str(Key) because it doesn't contain the model and appid. Moreover, in case you specified a pk manually, the URLs will be much nicer. Even if you don't specify a key_name the URL is still shorter than the str(Key) version. 64 44 65 45 Queries should support ancestor conditions. 66 46 67 Every model should provide these properties: key, key_name, key_id, parent, parent_key47 Every model should provide properties for: key, key_name, key_id, parent, parent_key 68 48 69 49 == Transactions == 70 50 71 Django could emulate transactions with the commit_on_success decorator. Manual transaction handling and checkpoints can't be implemented with App Engine's current API, though. We might ask Google for help. The problem with commit_on_success is that it should run only once, but App Engine 's run_in_transaction runs up to three times if an error occurs. The worst that can happen is that someone uses a custom decorator which calls commit_on_success multiple times because this could quickly hit a request limit. Maybe Django should officially change commit_on_success to issue retries?51 Django could emulate transactions with the commit_on_success decorator. Manual transaction handling and checkpoints can't be implemented with App Engine's current API, though. We might ask Google for help. The problem with commit_on_success is that it should run only once, but App Engine requires that it runs multiple times if an error occurs. The worst that can happen is that someone uses a custom decorator which calls commit_on_success multiple times because this could quickly hit a request limit. Maybe Django should officially change commit_on_success to issue retries? 72 52 73 53 == Datastore batch operations ==