RawQuerySet doesn't call ops.convert_values in the backend

[Erin Kelly]

Because RawQuerySet doesn't invoke the sql compiler, the ops.convert_values method in the Oracle backend never gets called. As a result, several values come through in the wrong format -- most notably, TextField values get pulled in as a cx_Oracle LOB object rather than as a Unicode string.

The complication with just adding a call to the method is that when using the other backends, it only ever gets called in an aggregation query. Outside of this context, it breaks. The method needs a closer examination to determine when it should be called and exactly what it should be doing.

Also, we need a test case for this. The existing raw_query tests miss it.

[George Vilches]

We don't necessarily need to call the other backends .convert_values directly. We could still potentially do it through the SQLCompiler on the backend so that it functions equivalently to the main QuerySet class. However, post-1.2 it's been discussed on IRC that the base convert_values method does need a review and update to handle cases outside of aggregates more intelligently.

[Russell Keith-Magee]

(In [12904]) Fixed #12429 -- Ensure that raw queries call resolve_columns if the backend defines it. This ensures (as much as possible) that the model values returned by a raw query match that in normal queries. Thanks to Ian Kelly for the report.

[Karen Tracey]

r12904 doesn't quite work for Oracle: the raw_query tests now all result in an error due to the fact that RawQuery does not have a high_mark attribute. From the line of code in the traceback (I only included one, they are all the same) I suspect low_mark is going to be a problem too...

No fixtures found.
testAnnotations (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testDBColumnHandler (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testExtraConversions (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testFkeyRawQuery (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testGetItem (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testInvalidQuery (modeltests.raw_query.tests.RawQueryTests) ... ok
testManyToMany (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testMissingFields (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testMissingFieldsWithoutPK (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testMultipleIterations (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testOrderHandler (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testParams (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testRawQueryLazy (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testSimpleRawQuery (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testTranslations (modeltests.raw_query.tests.RawQueryTests) ... ERROR
testWhiteSpaceQuery (modeltests.raw_query.tests.RawQueryTests) ... ERROR
test_inheritance (modeltests.raw_query.tests.RawQueryTests) ... ERROR

======================================================================
ERROR: testAnnotations (modeltests.raw_query.tests.RawQueryTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\u\kmt\django\trunk\tests\modeltests\raw_query\tests.py", line 170, in testAnnotations
    self.assertSuccessfulRawQuery(Author, query, authors, expected_annotations)
  File "C:\u\kmt\django\trunk\tests\modeltests\raw_query\tests.py", line 17, in assertSuccessfulRawQuery
    results = list(model.objects.raw(query, params=params, translations=translations))
  File "C:\u\kmt\django\trunk\django\db\models\query.py", line 1358, in __iter__
    yield self.transform_results(row)
  File "C:\u\kmt\django\trunk\django\db\models\query.py", line 1422, in transform_results
    values = compiler.resolve_columns(values, fields)
  File "C:\u\kmt\django\trunk\django\db\backends\oracle\compiler.py", line 9, in resolve_columns
    if self.query.high_mark is not None or self.query.low_mark:
AttributeError: 'RawQuery' object has no attribute 'high_mark'

[Russell Keith-Magee]

Possible fix for Oracle problems in raw_query

[Russell Keith-Magee]

I've just uploaded a possible fix for the oracle problems; if anyone with an Oracle setup could take it for a spin, I'd be much obliged.

[Karen Tracey]

With the t12429-r12905 patch applied test_invalid_query in the raw_query tests now works:

testInvalidQuery (modeltests.raw_query.tests.RawQueryTests) ... ok

But the other 16 tests generate an error:

======================================================================
ERROR: test_inheritance (modeltests.raw_query.tests.RawQueryTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\u\kmt\django\trunk\tests\modeltests\raw_query\tests.py", line 218, in test_inheritance
    [o.pk for o in FriendlyAuthor.objects.raw(query)], [f.pk]
  File "C:\u\kmt\django\trunk\django\db\models\query.py", line 1358, in __iter__
    yield self.transform_results(row)
  File "C:\u\kmt\django\trunk\django\db\models\query.py", line 1422, in transform_results
    values = compiler.resolve_columns(values, fields)
  File "C:\u\kmt\django\trunk\django\db\backends\oracle\compiler.py", line 17, in resolve_columns
    values.append(self.query.convert_values(value, field, connection=self.connection))
AttributeError: 'RawQuery' object has no attribute 'convert_values'

[Russell Keith-Magee]

Second attempt at an Oracle fix

[Russell Keith-Magee]

(In [12907]) Fixed #12429 -- Modified RawQuery to provide some facilities required by Oracle. Thanks to Karen Tracey for the testing help.

[Jacob]

Milestone 1.2 deleted