Opened 14 years ago

Closed 14 years ago

Last modified 12 years ago

#12429 closed (fixed)

RawQuerySet doesn't call ops.convert_values in the backend

Reported by: Erin Kelly Owned by: nobody
Component: Database layer (models, ORM) Version: dev
Severity: Keywords: oracle raw
Cc: Matt Boersma Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Because RawQuerySet doesn't invoke the sql compiler, the ops.convert_values method in the Oracle backend never gets called. As a result, several values come through in the wrong format -- most notably, TextField values get pulled in as a cx_Oracle LOB object rather than as a Unicode string.

The complication with just adding a call to the method is that when using the other backends, it only ever gets called in an aggregation query. Outside of this context, it breaks. The method needs a closer examination to determine when it should be called and exactly what it should be doing.

Also, we need a test case for this. The existing raw_query tests miss it.

Attachments (2)

t12429-r12905.diff (615 bytes ) - added by Russell Keith-Magee 14 years ago.
Possible fix for Oracle problems in raw_query
t12429-r12905.2.diff (1.1 KB ) - added by Russell Keith-Magee 14 years ago.
Second attempt at an Oracle fix

Download all attachments as: .zip

Change History (10)

comment:1 by Russell Keith-Magee, 14 years ago

Triage Stage: UnreviewedAccepted

comment:2 by George Vilches, 14 years ago

We don't necessarily need to call the other backends .convert_values directly. We could still potentially do it through the SQLCompiler on the backend so that it functions equivalently to the main QuerySet class. However, post-1.2 it's been discussed on IRC that the base convert_values method does need a review and update to handle cases outside of aggregates more intelligently.

comment:3 by Russell Keith-Magee, 14 years ago

Resolution: fixed
Status: newclosed

(In [12904]) Fixed #12429 -- Ensure that raw queries call resolve_columns if the backend defines it. This ensures (as much as possible) that the model values returned by a raw query match that in normal queries. Thanks to Ian Kelly for the report.

comment:4 by Karen Tracey, 14 years ago

Resolution: fixed
Status: closedreopened

r12904 doesn't quite work for Oracle: the raw_query tests now all result in an error due to the fact that RawQuery does not have a high_mark attribute. From the line of code in the traceback (I only included one, they are all the same) I suspect low_mark is going to be a problem too...

No fixtures found.
testAnnotations (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testDBColumnHandler (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testExtraConversions (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testFkeyRawQuery (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testGetItem (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testInvalidQuery (modeltests.raw_query.tests.RawQueryTests) ... ok
testManyToMany (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testMissingFields (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testMissingFieldsWithoutPK (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testMultipleIterations (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testOrderHandler (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testParams (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testRawQueryLazy (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testSimpleRawQuery (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testTranslations (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testWhiteSpaceQuery (modeltests.raw_query.tests.RawQueryTests) ... ERROR
test_inheritance (modeltests.raw_query.tests.RawQueryTests) ... ERROR

======================================================================
ERROR: testAnnotations (modeltests.raw_query.tests.RawQueryTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\u\kmt\django\trunk\tests\modeltests\raw_query\tests.py", line 170, in testAnnotations
    self.assertSuccessfulRawQuery(Author, query, authors, expected_annotations)
  File "C:\u\kmt\django\trunk\tests\modeltests\raw_query\tests.py", line 17, in assertSuccessfulRawQuery
    results = list(model.objects.raw(query, params=params, translations=translations))
  File "C:\u\kmt\django\trunk\django\db\models\query.py", line 1358, in __iter__
    yield self.transform_results(row)
  File "C:\u\kmt\django\trunk\django\db\models\query.py", line 1422, in transform_results
    values = compiler.resolve_columns(values, fields)
  File "C:\u\kmt\django\trunk\django\db\backends\oracle\compiler.py", line 9, in resolve_columns
    if self.query.high_mark is not None or self.query.low_mark:
AttributeError: 'RawQuery' object has no attribute 'high_mark'

by Russell Keith-Magee, 14 years ago

Attachment: t12429-r12905.diff added

Possible fix for Oracle problems in raw_query

comment:5 by Russell Keith-Magee, 14 years ago

I've just uploaded a possible fix for the oracle problems; if anyone with an Oracle setup could take it for a spin, I'd be much obliged.

comment:6 by Karen Tracey, 14 years ago

With the t12429-r12905 patch applied test_invalid_query in the raw_query tests now works:

testInvalidQuery (modeltests.raw_query.tests.RawQueryTests) ... ok

But the other 16 tests generate an error:

======================================================================
ERROR: test_inheritance (modeltests.raw_query.tests.RawQueryTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\u\kmt\django\trunk\tests\modeltests\raw_query\tests.py", line 218, in test_inheritance
    [o.pk for o in FriendlyAuthor.objects.raw(query)], [f.pk]
  File "C:\u\kmt\django\trunk\django\db\models\query.py", line 1358, in __iter__
    yield self.transform_results(row)
  File "C:\u\kmt\django\trunk\django\db\models\query.py", line 1422, in transform_results
    values = compiler.resolve_columns(values, fields)
  File "C:\u\kmt\django\trunk\django\db\backends\oracle\compiler.py", line 17, in resolve_columns
    values.append(self.query.convert_values(value, field, connection=self.connection))
AttributeError: 'RawQuery' object has no attribute 'convert_values'

by Russell Keith-Magee, 14 years ago

Attachment: t12429-r12905.2.diff added

Second attempt at an Oracle fix

comment:7 by Russell Keith-Magee, 14 years ago

Resolution: fixed
Status: reopenedclosed

(In [12907]) Fixed #12429 -- Modified RawQuery to provide some facilities required by Oracle. Thanks to Karen Tracey for the testing help.

comment:8 by Jacob, 12 years ago

milestone: 1.2

Milestone 1.2 deleted

Note: See TracTickets for help on using tickets.
Back to Top