Test failures with MySQL >= 5.6.4 and MySQLdb < 1.2.5

[Jon Dufresne]

This combination does not support database microsecond precision. Mircosecond precision was added to MySQL in version 5.6.4, but due to a bug in older MySQLdb, is still not always available. See ​https://github.com/farcepest/MySQLdb1/issues/24.

Before any changes, with this combination, when saving the date 2008-12-31 23:59:59.999999 the database result is rounded to 2009-01-01 00:00:00. However, (to my understanding) Django expects the date to be truncated, not rounded.

Additionally, MySQL INTERVAL would promote a datetime without microseconds to a datetime with microseconds if the INTERVAL was a float or contained microseconds. Older MySQLdb was unable to properly parse the result to a Python datetime.

FWIW, I ran into this using the system libraries of a default Fedora 21 install. Fedora 21 ships with following versions:

community-mysql-server-5.6.23-1.fc21.x86_64
MySQL-python-1.2.3-13.fc21.x86_64

Individuals can workaround this by using virtualenv to install a newer MySQLdb or mysqlclient.

The following tests fail before any changes:

======================================================================
FAIL: test_mixed_type_annotation_date_interval (annotations.tests.NonAggregateAnnotationTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jon/devel/django/tests/annotations/tests.py", line 147, in test_mixed_type_annotation_date_interval
    self.assertEqual(t.expires, expires)
AssertionError: None != datetime.datetime(2015, 3, 20, 15, 0)

======================================================================
FAIL: test_get_next_prev_by_field (model_regress.tests.ModelTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jon/devel/django/tests/model_regress/tests.py", line 178, in test_get_next_prev_by_field
    e.get_previous_by_when().when, datetime.datetime(2000, 1, 1, 6, 1, 1)
AssertionError: datetime.datetime(2000, 1, 1, 12, 0, 20) != datetime.datetime(2000, 1, 1, 6, 1, 1)

======================================================================
FAIL: test_mixed_comparisons1 (expressions.tests.FTimeDeltaTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jon/devel/django/django/test/testcases.py", line 1023, in skip_wrapper
    return test_func(*args, **kwargs)
  File "/home/jon/devel/django/tests/expressions/tests.py", line 765, in test_mixed_comparisons1
    self.assertEqual(test_set, self.expnames[:i + 1])
AssertionError: Lists differ: [] != [u'e0']

Second list contains 1 additional elements.
First extra element 0:
e0

- []
+ [u'e0']

======================================================================
FAIL: test_extra_method_select_argument_with_dashes_and_values (basic.tests.ModelTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jon/devel/django/tests/basic/tests.py", line 304, in test_extra_method_select_argument_with_dashes_and_values
    [[('dashed-value', 1), ('headline', 'Article 11')], [('dashed-value', 1), ('headline', 'Article 12')]])
AssertionError: Lists differ: [[(u'dashed-value', 1L), (u'he... != [[(u'dashed-value', 1), (u'hea...

Second list contains 1 additional elements.
First extra element 1:
[(u'dashed-value', 1), (u'headline', u'Article 12')]

- [[(u'dashed-value', 1L), (u'headline', u'Article 11')]]
?                      -                                ^

+ [[(u'dashed-value', 1), (u'headline', u'Article 11')],
?                                                      ^

+  [(u'dashed-value', 1), (u'headline', u'Article 12')]]

======================================================================
FAIL: test_year_lookup_edge_case (basic.tests.ModelTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jon/devel/django/tests/basic/tests.py", line 236, in test_year_lookup_edge_case
    ["<Article: Article 11>", "<Article: Article 12>"])
  File "/home/jon/devel/django/django/test/testcases.py", line 901, in assertQuerysetEqual
    return self.assertEqual(list(items), values, msg=msg)
AssertionError: Lists differ: ['<Article: Article 11>'] != [u'<Article: Article 11>', u'<...

Second list contains 1 additional elements.
First extra element 1:
<Article: Article 12>

- ['<Article: Article 11>']
+ [u'<Article: Article 11>', u'<Article: Article 12>']

----------------------------------------------------------------------

PR to follow.

[Jon Dufresne]

​https://github.com/django/django/pull/4453

[Claude Paroz]

Can you look at [9e746c13e81241fbf1ae64ec118edaa491790046] and see if partially reverting that patch (stripping microseconds based on connection.features.supports_microsecond_precision) does the job?
If we could prevent touching to get_prep_value, it would be better.

[Jon Dufresne]

Thanks!

I have updated the patch based on your suggestion.

[Claude Paroz]

Thanks Jon, the patch looks good!
Tim, are you OK for backporting (considering current test failures with this specific MySQL/MySQLdb combination)?

[Tim Graham]

Yes, backport is fine with me. Let's add release notes, of course.

[Jon Dufresne]

Release notes added to PR. Thanks.

[Claude Paroz <claude@…>]

In 2cf58e8:

Fixed #24584 -- Fixed microsecond handling with older MySQLdb

[Claude Paroz <claude@…>]

In 551d4bb4:

[1.8.x] Fixed #24584 -- Fixed microsecond handling with older MySQLdb

Backport of 2cf58e80d from master.