Opened 14 years ago
Last modified 11 years ago
#15559 new Bug
Distinct queries will cause errors with some custom model fields
Reported by: | urandom | Owned by: | nobody |
---|---|---|---|
Component: | Database layer (models, ORM) | Version: | dev |
Severity: | Normal | Keywords: | |
Cc: | Michael Radziej | Triage Stage: | Accepted |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
Starting with 1.3 rc1, some queries are marked as distinct by the system.
If a model now uses a custom field, that relies on a special DB column (like Postgresql's 'point' column),
the database will immediately throw an error, since such a column cannot be compared, and thus unique values cannot be found.
It would probably be a good idea to allow fields to specify whether they cannot be distinct:
custom = MyCustomField(distinctless=True)
which would turn this ordinary distinct query:
SELECT DISTINCT id, number, custom FROM foo;
into:
SELECT DISTINCT ON(id, number) FROM FOO;
Discussion link:
https://groups.google.com/forum/#!topic/django-developers/iO8z3iLpgg4
Change History (15)
comment:1 by , 14 years ago
Cc: | added |
---|
comment:2 by , 14 years ago
Component: | Uncategorized → Database layer (models, ORM) |
---|---|
Keywords: | regression blocker added |
milestone: | → 1.3 |
Triage Stage: | Unreviewed → Accepted |
Version: | 1.3-beta → 1.3-rc1 |
follow-up: 15 comment:3 by , 14 years ago
comment:4 by , 14 years ago
I discovered another problem with the distinct queries from [15607]. Using Oracle, it breaks foreign keys to models that have TextFields. This is because Oracle won't let you do SELECT DISTINCT with a select list that includes a LOB column.
+1 for reverting.
follow-up: 6 comment:5 by , 14 years ago
ikelly, does Oracle support SELECT DISTINCT ON (id) if there's a LOB column in the SELECT list?
comment:6 by , 14 years ago
Replying to mir:
ikelly, does Oracle support SELECT DISTINCT ON (id) if there's a LOB column in the SELECT list?
No. The analytic query that emulates DISTINCT ON works using a regular DISTINCT query and selecting for each column the first value in that column for the corresponding DISTINCT ON partition in place of the actual value for the row. The result is that rows that are equivalent over the DISTINCT ON partition become equivalent over the entire row and so are filtered out by the DISTINCT keyword. That means you're still effectively doing a SELECT DISTINCT with a LOB column.
comment:9 by , 14 years ago
As you can see, I decided to revert [15607], because of the support for reverting listed above and on the mailing list, and because of the closeness of the 1.3 release.
follow-up: 11 comment:10 by , 14 years ago
Severity: | → Normal |
---|---|
Type: | → Uncategorized |
There are other places where we are using distinct (see [15526]), so it is possible that this bug is only fixed in the one instance where the reporter experienced a regression. A more satisfying general solution is probably still needed, but a new bug should be opened for that.
comment:11 by , 14 years ago
Keywords: | regression blocker removed |
---|---|
milestone: | 1.3 |
Resolution: | fixed |
Status: | closed → reopened |
Type: | Uncategorized → Bug |
Version: | 1.3-rc1 → SVN |
Replying to lukeplant:
There are other places where we are using distinct (see [15526]), so it is possible that this bug is only fixed in the one instance where the reporter experienced a regression. A more satisfying general solution is probably still needed, but a new bug should be opened for that.
I don't see any need to open a new bug - it would just be a copy-paste of this one. The description for this bug is still fully applicable (and was never fixed, just worked around). Reopening and removing 1.3-blocker keywords.
comment:14 by , 12 years ago
Status: | reopened → new |
---|
comment:15 by , 11 years ago
Replying to lukeplant:
We could also revert the fix in [15607], and either:
I'm revisiting #11707 on a sprint. I implemented the duplicate elimination 2 years ago with ojii, but it never got RFC'ed.
Since then I thought: DISTINCT logically equivalent with a GROUP BY on all fields. Not all databases support DISTINCT ON, but a GROUP BY on just the pk should do the trick (at least in #11707).
Not sure we want to be adding a new option between now and 1.3 final, but seems like we need to do something (minimum document the change as a backwards-incompatibility?).