Changes between Version 4 and Version 5 of QuerysetRefactorBranch

Timestamp:

Mar 13, 2008, 7:54:35 PM (17 years ago)

Author:

Malcolm Tredinnick

Comment:

Updated to reflect the current state of play.

QuerysetRefactorBranch

@@ -1 +1 @@
 = The queryset-refactor branch =
+
+(Up to date as of 14 March, 2008)
 
 This branch contains a major refactoring of the {{{django.db.models.query.QuerySet}}} class to fix a group of SQL problems and make SQL generation easier for database backends requiring customization.
@@ -15 +17 @@
 Branch was created on 13 September, 2007.
 
-At the moment, the branch is fairly unusable. All the tests pass, but there are a lot of other places in the code that attempt to construct SQL fragments and pass them around and they will tend to break.
+At the moment, the branch should be fairly usable for people wishing to test (with the exception of the Oracle backend, as noted below). It is not appropriate for production usage. Some of the new features, such as model inheritance still have rough edges or are incomplete, but all the porting work is complete and code that fixed existing bugs should all work. There are still some internal changes that need to be made prior to merging with trunk, so any code that tries to rely on internal methods may need to change before the next release.
 
-The focus right now is on correctness, but some significant performance profiling and tuning will need to be done. Current performance is quite a bit slower than the existing code. The good news here is that there are some obvious places to speed things up and, no doubt, some unobvious as well once we get to that point. An optimisation pass is still some way off at the moment; getting all the relevant bugs fixed is the current focus.
+Bug reports against code not listed in the [#Workinprogress work in progress] section should be filed in Trac. Please take some extra effort to check for duplicates first.
 
 Tickets of interest for this branch are marked with ''qs-rf'' (and ''qs-rf-fixed'' when fixed) in the keywords field in Trac. [http://code.djangoproject.com/query?status=new&status=assigned&status=reopened&keywords=%24qs-rf&order=priority This report] shows all the tickets being worked on (To view ''qs-rf'' with the ''qs-rf-fixed'' tagged tickets, [http://code.djangoproject.com/query?status=new&status=assigned&status=reopened&keywords=%7Eqs-rf&order=priority view this report]).
 
-== Changes introduced on the branch ==
+== New features ==
+
+Along with, and as part of, all the bug fixes mentioned above there are a number of new features in the branch. A number of these features are purely internal details, but there are a few that add extra public functionality.
+
+ 1. Ordering querysets across related models has a new syntax that is the same as the way you specify relations in a filter: {{{field1__field2__field3}}}, etc. The new syntax is more natural and consistent, as well as helping solve a few bugs. See the {{{order_by()}}} documentation in db-api.txt for more information and some examples.
+ 2. Model inheritance is now possible. Both abstract base classes and multi-table inheritance are possible. See the model-api.txt documentation for details.
+ 3. The {{{__iter__()}}} method on querysets does not pull all the results into memory immediately. This reduces memory usage for large querysets where you don't end up accessing all the results. Queryset caching still occurs, though, so a single queryset object will only hit the database once. This change means that testing the boolean value of a queryset will only pull in the first few rows of the result, not all of them.
+ 4. Slicing a queryset from a particular value to the end of a queryset is possible.
+ 5. Querysets have a {{{reverse()}}} method that reverses whatever the current ordering is.
+ 6. The queryset {{{values()}}} method can retrieve fields that are related via a {{{ForeignKey}}} or {{{OneToOneField}}} relation.
+ 7. A new {{{valueslist()}}} method has been added to querysets. This is similar to {{{values()}}} except that it returns a list of tuples, rather than a list of dictionaries.
+ 8. You can specify a list of related fields to traverse in a {{{select_related()}}} call. This provides a way to select only the related data you are interested in. Only single-valued relations can be selected in this way, however (not {{{ManyToManyFields}}}).
+ 9. Filtering a queryset by checking if a field attribute is {{{None}}} is equivalent to testing if the corresponding database column is {{{NULL}}}. So {{{qs.filter(foo=None)}}} is now identical to {{{qs.filter(foo__isnull=True)}}}.
+ 10. An {{{update()}}} method has been added to querysets to allow multiple objects to have an attribute updated in a single SQL query.
+
+== Backwards incompatible changes ==
 
 A few backwards incompatible changes are created by the changes in this branch. Most people won't be affected by many of these, and porting code is reasonably straightforward.
 
  * The {{{Options.get_order_sql()}}} method is now gone in {{{django.db.models.options}}}. There appears to be no use for this method any longer.
- * Ordering querysets across related models has a new syntax that is the same as the way you specify relations in a filter: {{{field1__field2__field3}}}, etc. Previously, you had to specify the database table name and the ''model'' field name, separated by a period. This led to some problems with multiple joins, as well as requiring you to remember the database table name, which depended on things like the application name. The new syntax is more natural and consistent, as well as helping solve a few bugs. See the {{{order_by()}}} documentation in db-api.txt for more information and some examples.
  * {{{Q}}} objects have changed internally. This is only relevant if you have created custom Q-like objects. You would have created a {{{get_sql()}}} method that returned a data structure that was inserted into the query. In the new code, you create a {{{add_to_query()}}} method that accepts one argument -- the {{{django.db.models.sql.query.Query}}} instance for the current query. Your Q-like object can then add to the various attributes of this class (`select`, `where`, etc) to have whatever effect it likes on the result. Note that the {{{add_to_query()}}} method is called when the object is added to the {{{Query}}} object and more changes may be made before it is turned into SQL and executed against the database.
+ * The {{{OneToOneField}}} class has finally been updated, as the documentation has indicated would be happening for a long while. There are few externally visible changes, with one exception: a {{{OneToOneField}}} is no longer automatically the primary key for a model that includes it. It still accepts the {{{primary_key}}} attribute, however, so you should add {{{primary_key=True}}} to the declaration of any existing {{{OneToOneField}}} instances in your code to preserve backwards compatibility.
+ * If you pass a bad field name into a filter, Django now raises {{{FieldError}}} (from {{{django.core.exceptions}}}), rather than Python's built in {{{TypeError}}}. Also, the list of legal field names is now sorted alphabetically for easier searching. This should have no effect on most production code, however some test suites may need to be updated to accommodate the changed traceback output.
+
+== Work in progress ==
+
+If you are testing this branch, there are a few areas that need caution. These are known places that are work-in-progress, so expecting them to work perfectly (or even at all) would be optimistic. Obviously, in addition to these points there are a lot of little things being tweaked and fixed, as well as optimisation work going on. That's to be expected on a branch.
+
+ * Model inheritance has not been integrated into the admin. It's unclear at this time whether it will be worth doing this for existing admin or just port straight to newforms-admin. In any case, trying to use multi-table inheritance via the admin interface won't work.
+ * {{{OneToOneField}}} in the admin interface has similar problems.
+ * {{{select_related()}}} with inherited models has not yet been integrated.
+ * The Oracle backend has not been ported yet. This won't be particularly difficult, but it's still work to be done. Justin Bronn has started the ball rolling with a patch in #6161 that will be used as the base.
+ * {{{exclude()}}} handling for particularly twisted filter combinations is still being bullet-proofed. It should work in almost all cases, but check the results carefully at the moment if you're doing something complex.