MySQL query with CONVERT_TZ is not properly formated

[Luboš Truhlář]

Hi,

I think that there is a bug in ORM if queryset using 'date' and TIMEZONE.

This code:

queryset = queryset.filter(created_at__date__lte=created_at_to.date())

produces this SQL query:

SELECT ... FROM `payments_payment` WHERE DATE(CONVERT_TZ(`payments_payment`.`created_at`, 'UTC', Europe/Prague)) <= 2016-06-30)

The problem is that 'Europe/Prague' is not in the quotes! Mysql returns this error:

#1054 - Unknown column 'Europe' in 'where clause'

my settings.py

TIME_ZONE = "Europe/Prague"

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'reality_15062016',
        'PORT': '3306',
        'TIME_ZONE': 'Europe/Prague',
        'OPTIONS': {
            'charset': 'utf8',
            'use_unicode': True,
        }
    }
}

[Tim Graham]

Can you try to construct a test for Django's test suite? A found ​a similar one which generates this query:

SELECT `model_fields_datetimemodel`.`id`, `model_fields_datetimemodel`.`d`, `model_fields_datetimemodel`.`dt`, `model_fields_datetimemodel`.`t`
FROM `model_fields_datetimemodel`
WHERE DATE(CONVERT_TZ(`model_fields_datetimemodel`.`dt`, 'UTC', America/Vancouver)) = 2014-03-12

Does it pass for you? Which version of MySQL are you using?

Did you load the ​MySQL timezone definitions?

[Luboš Truhlář]

Using Mysql: 5.5.49-0+deb8u1 - (Debian)
MySQL timezone definitions have been loaded properly.

Strange behavior. I pass all 6 tests in the suite, which you point. But when I try the single SQL query, the Error appears.

I modified the code a little to see the query which Django produce.

    ...
        with self.settings(TIME_ZONE='UTC'):
            # But in UTC, the __date only matches one of them.
            q  = DateTimeModel.objects.filter(dt__date=d)
            print q.query
            self.assertQuerysetEqual(DateTimeModel.objects.filter(dt__date=d), [repr(m1)])

and the output is this:

Using existing test database for alias 'default'...
.sSELECT `payments_datetimemodel`.`id`, `payments_datetimemodel`.`d`, `payments_datetimemodel`.`dt`, `payments_datetimemodel`.`t` FROM `payments_datetimemodel` WHERE DATE(CONVERT_TZ(`payments_datetimemodel`.`dt`, 'UTC', UTC)) = 2014-03-12
....
----------------------------------------------------------------------
Ran 6 tests in 0.734s

OK (skipped=1)

Then trying to use the same query (without quotes) directly in mysql client.

mysql> SELECT `payments_datetimemodel`.`id`, `payments_datetimemodel`.`d`, `payments_datetimemodel`.`dt`, `payments_datetimemodel`.`t` FROM `payments_datetimemodel` WHERE DATE(CONVERT_TZ(`payments_datetimemodel`.`dt`, 'UTC', UTC)) = 2014-03-12;
ERROR 1054 (42S22): Unknown column 'UTC' in 'where clause'

and with quotes:

mysql> SELECT `payments_datetimemodel`.`id`, `payments_datetimemodel`.`d`, `payments_datetimemodel`.`dt`, `payments_datetimemodel`.`t` FROM `payments_datetimemodel` WHERE DATE(CONVERT_TZ(`payments_datetimemodel`.`dt`, 'UTC', 'UTC')) = 2014-03-12;
Empty set (0.00 sec)

[Tim Graham]

str(queryset.query) returns a representation of the query, not necessarily one that can be executed. Is that the issue in the original report?

[Luboš Truhlář]

No, it's not the issue, which I wan't to report. My issue is that the '_date' filter is not working correctly in my configuration.

Replying to timgraham:

str(queryset.query) returns a representation of the query, not necessarily one that can be executed. Is that the issue in the original report?

[Simon Charette]

What did you use to print the query you pasted into the mysql client?

[Luboš Truhlář]

queryset = Payments.objects.filter(created_at__date__lte=created_at_to.date())
print queryset.query

Replying to charettes:

What did you use to print the query you pasted into the mysql client?

[Simon Charette]

As mentioned by Tim above str(queryset.query) returns ​a representation of the query and not what is actually executed as the escaping part is left to the backend which escapes parameters such as the timezone name.

Please re-open if you can provide an actual failing testcase or at least a traceback that does not rely on executing a representation of Queryset.query.