= The queryset-refactor branch = This branch was used to develop a major refactoring of the {{{django.db.models.query.QuerySet}}} class to fix a group of SQL problems and make SQL generation easier for database backends requiring customization. == Status == The branch was created on 13 September, 2007, and merged into trunk on 26 April, 2008 (in [7477]). == New features == Along with, and as part of, all the bug fixes mentioned above there are a number of new features in the branch. A number of these features are purely internal details, but there are a few that add extra public functionality. 1. Ordering querysets across related models has a new syntax that is the same as the way you specify relations in a filter: {{{field1__field2__field3}}}, etc. The new syntax is more natural and consistent, as well as helping solve a few bugs. See the [http://www.djangoproject.com/documentation/db-api/#order-by-fields order_by() documentation] for more information and some examples.[http://www.mustuniversity.com/Must/UniversityDegree.asp university degrees] 2. Model inheritance is now possible. Both abstract base classes and multi-table inheritance are possible. See the [http://www.djangoproject.com/documentation/model-api/#model-inheritance model-api documentation] for details. [http://www.mustuniversity.com/Schools-Majors/Psychology.html Psychology Degree] 3. The {{{__iter__()}}} method on querysets does not pull all the results into memory immediately. This reduces memory usage for large querysets where you don't end up accessing all the results. Queryset caching still occurs, though, so a single queryset object will only hit the database once. This change means that testing the boolean value of a queryset will only pull in the first few rows of the result, not all of them. 4. Slicing a queryset from a particular value to the end of a queryset is possible. 5. Querysets have a {{{reverse()}}} method that reverses whatever the current ordering is. 6. The queryset {{{values()}}} method can retrieve fields that are related via a {{{ForeignKey}}} or {{{OneToOneField}}} relation. 7. A new {{{values_list()}}} method has been added to querysets. This is similar to {{{values()}}} except that it returns a list of tuples, rather than a list of dictionaries. 8. You can specify a list of related fields to traverse in a {{{select_related()}}} call. This provides a way to select only the related data you are interested in. Only single-valued relations can be selected in this way, however (not {{{ManyToManyFields}}}). 9. Filtering a queryset by checking if a field attribute is {{{None}}} is equivalent to testing if the corresponding database column is {{{NULL}}}. So {{{qs.filter(foo=None)}}} is now identical to {{{qs.filter(foo__isnull=True)}}}. 10. An {{{update()}}} method has been added to querysets to allow multiple objects to have an attribute updated in a single SQL query. 11. `Q` classes now fully support `&`, `|` and `~` to combine them in pairs as conjunctions or disjunctions or to negate the sense of a filter, respectively (`&` and `|` were previously supported, but returned a different type of class). Thus the `QAnd`, `QOr` and `QNot` classes are no longer required and have been deprecated. Using them raises a warning (although they still work as before). == Backwards incompatible changes == A few backwards incompatible changes are created by the changes in this branch. Most people won't be affected by many of these, and porting code is reasonably straightforward. === Most visible === * The {{{OneToOneField}}} class has finally been updated, as the documentation has indicated would be happening for a long while. There are few externally visible changes, with one exception: a {{{OneToOneField}}} is no longer automatically the primary key for a model that includes it. It still accepts the {{{primary_key}}} attribute, however, so you should add {{{primary_key=True}}} to the declaration of any existing {{{OneToOneField}}} instances in your code to preserve backwards compatibility.[http://www.mustuniversity.com/Programs/Degree/Masters-Degree.html online Masters degree] * If you pass a bad field name into a filter or {{{order_by()}}}, Django now raises {{{FieldError}}} (from {{{django.core.exceptions}}}), rather than Python's built in {{{TypeError}}}. Also, the list of legal field names is now sorted alphabetically for easier searching. This should have no effect on most production code, however some test suites may need to be updated to accommodate the changed traceback output. * There is a slight difference between these two filter statements {{{ #!python qs.objects.filter(f1).filter(f2) qs.objects.filter(f1, f2) }}} This difference only applies when `f1` and `f2` are referencing the same multi-valued relationship (a `ManyToManyField` or reverse `ForeignKey`). The first version allows filtering against different items from the relationship (things that match `f1` on one object in the related table as well as `f2` on another object in the related table), whereas the second version's filters will be applied to the same item. See the [http://www.djangoproject.com/documentation/db-api/#lookups-that-span-relationships database api documentation] for details. [http://www.mustuniversity.com/Programs/Degree/degree.asp Online degrees] | [http://www.mustuniversity.com/Programs/Degree/PhD-Degree.html phd degree] * It is possible to use extra select fields -- those included via {{{extra(select=...)}}} -- for ordering the results. Previously, those fields could be specified to the {{{order_by()}}} method. Due to increased error checking, that is no longer practical. Instead, pass those extra columns to the {{{order_by}}} argument of the {{{extra()}}} method: {{{ #!python qs.extra(select={'a': ...}).order_by('a') # Old style qs.extra(select={'a': ...}, order_by=('a',)) # New style }}} This only applies to ordering by items in an `extra(select={...})` dictionary. Normal ordering is still done with the `order_by()` method to querysets, there have been no changes there. So this change will affect relatively few people. * A couple of changes to `order_by()`: * Previously, if you passed no arguments to `order_by()`, it did nothing. Now, it clears the ordering (previously there was no way to remove any existing ordering). So if you are expecting the "do nothing" behaviour, make sure to check the arguments you are passing to `order_by()` and don't call it if you have no arguments. * `order_by()` will raise an error if you try to order by a related model and resolving that model's standard ordering leads to an infinite loop (ordering by a related model tacks the target model's default ordering onto the end). This commonly comes up when ordering by a `ForeignKey` to "self". The previous code wouldn't raise an error here, but the results weren't particularly intuitive. However, if you still want the old behaviour, specify the ordering constraint explicitly so that it ends in a non-related field. For the "reference to self" case, you could make this change: {{{ #!python class Tag(models.Model): parent = models.ForeignKey('self', blank=True, null=True) class Meta: # ordering = ('parent',) # Old code now causes an infinite loop. ordering = ('parent__id',) # Last field is a non-relation. No problem. }}} === Other === * The {{{Options.get_order_sql()}}} method is now gone in {{{django.db.models.options}}}. There appears to be no use for this method any longer. * {{{Q}}} objects have changed internally. This is only relevant if you have created custom Q-like objects. You would have created a {{{get_sql()}}} method that returned a data structure that was inserted into the query. In the new code, you create a {{{add_to_query()}}} method that accepts two arguments -- the {{{django.db.models.sql.query.Query}}} instance for the current query and a set of aliases used in the current `filter()` call. Your Q-like object can then add to the various attributes of this class (`select`, `where`, etc) to have whatever effect it likes on the result. Note that the {{{add_to_query()}}} method is called when the object is added to the {{{Query}}} object and more changes may be made before it is turned into SQL and executed against the database. * Still on {{{extra(select=...)}}}... if you want to substitute parameters into these extra selection columns, use the {{{select_params}}} argument to {{{extra()}}}. The {{{params}}} argument is only applied to the extra where conditions. * {{{select_related(False)}}} is no longer possible. Don't worry. You didn't know this existed, so you won't miss it. It was never part of the official API. * The internal method {{{DatabaseOperations.limit_offset_sql()}}} has been dropped (as of r7732 on trunk). == Things to Note == * Model inheritance has not been integrated into the existing admin. It will eventually be implemented in the newforms-admin branch. * {{{OneToOneField}}} in the admin interface has a similar status.