Nested foreign key test failures on Django 3.0 with SQLite 3.30.

[Chris Lamb]

Hi,

I am seeing the following test failures for the 3.0~beta1 release:

======================================================================
FAIL: test_explicit_ForeignKey (nested_foreign_keys.tests.DeeplyNestedForeignKeysTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.7/unittest/case.py", line 59, in testPartExecutor
    yield
  File "/usr/lib/python3.7/unittest/case.py", line 628, in run
    testMethod()
  File "/<<PKGBUILDDIR>>/tests/nested_foreign_keys/tests.py", line 176, in test_explicit_ForeignKey
    self.assertEqual(Package.objects.exclude(screening__movie__director=self.director).count(), 1)
  File "/usr/lib/python3.7/unittest/case.py", line 852, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib/python3.7/unittest/case.py", line 845, in _baseAssertEqual
    raise self.failureException(msg)
AssertionError: 0 != 1

======================================================================
FAIL: test_inheritance (nested_foreign_keys.tests.DeeplyNestedForeignKeysTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.7/unittest/case.py", line 59, in testPartExecutor
    yield
  File "/usr/lib/python3.7/unittest/case.py", line 628, in run
    testMethod()
  File "/<<PKGBUILDDIR>>/tests/nested_foreign_keys/tests.py", line 153, in test_inheritance
    self.assertEqual(Event.objects.exclude(screening__movie__director=self.director).count(), 1)
  File "/usr/lib/python3.7/unittest/case.py", line 852, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib/python3.7/unittest/case.py", line 845, in _baseAssertEqual
    raise self.failureException(msg)
AssertionError: 0 != 1

======================================================================
FAIL: test_explicit_ForeignKey (nested_foreign_keys.tests.NestedForeignKeysTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.7/unittest/case.py", line 59, in testPartExecutor
    yield
  File "/usr/lib/python3.7/unittest/case.py", line 628, in run
    testMethod()
  File "/<<PKGBUILDDIR>>/tests/nested_foreign_keys/tests.py", line 100, in test_explicit_ForeignKey
    self.assertEqual(Package.objects.exclude(screening__movie=self.movie).count(), 1)
  File "/usr/lib/python3.7/unittest/case.py", line 852, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib/python3.7/unittest/case.py", line 845, in _baseAssertEqual
    raise self.failureException(msg)
AssertionError: 0 != 1

======================================================================
FAIL: test_explicit_ForeignKey_NullFK (nested_foreign_keys.tests.NestedForeignKeysTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.7/unittest/case.py", line 59, in testPartExecutor
    yield
  File "/usr/lib/python3.7/unittest/case.py", line 628, in run
    testMethod()
  File "/<<PKGBUILDDIR>>/tests/nested_foreign_keys/tests.py", line 121, in test_explicit_ForeignKey_NullFK
    self.assertEqual(PackageNullFK.objects.exclude(screening__movie=self.movie).count(), 2)
  File "/usr/lib/python3.7/unittest/case.py", line 852, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib/python3.7/unittest/case.py", line 845, in _baseAssertEqual
    raise self.failureException(msg)
AssertionError: 1 != 2

======================================================================
FAIL: test_inheritance (nested_foreign_keys.tests.NestedForeignKeysTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.7/unittest/case.py", line 59, in testPartExecutor
    yield
  File "/usr/lib/python3.7/unittest/case.py", line 628, in run
    testMethod()
  File "/<<PKGBUILDDIR>>/tests/nested_foreign_keys/tests.py", line 53, in test_inheritance
    self.assertEqual(Event.objects.exclude(screening__movie=self.movie).count(), 1)
  File "/usr/lib/python3.7/unittest/case.py", line 852, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib/python3.7/unittest/case.py", line 845, in _baseAssertEqual
    raise self.failureException(msg)
AssertionError: 0 != 1

======================================================================
FAIL: test_inheritance_null_FK (nested_foreign_keys.tests.NestedForeignKeysTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.7/unittest/case.py", line 59, in testPartExecutor
    yield
  File "/usr/lib/python3.7/unittest/case.py", line 628, in run
    testMethod()
  File "/<<PKGBUILDDIR>>/tests/nested_foreign_keys/tests.py", line 72, in test_inheritance_null_FK
    self.assertEqual(Event.objects.exclude(screeningnullfk__movie=self.movie).count(), 2)
  File "/usr/lib/python3.7/unittest/case.py", line 852, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib/python3.7/unittest/case.py", line 845, in _baseAssertEqual
    raise self.failureException(msg)
AssertionError: 1 != 2

======================================================================
FAIL: test_ticket7076 (queries.tests.Queries1Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.7/unittest/case.py", line 59, in testPartExecutor
    yield
  File "/usr/lib/python3.7/unittest/case.py", line 628, in run
    testMethod()
  File "/<<PKGBUILDDIR>>/tests/queries/tests.py", line 806, in test_ticket7076
    ['<Tag: t1>', '<Tag: t4>', '<Tag: t5>']
  File "/<<PKGBUILDDIR>>/django/test/testcases.py", line 1056, in assertQuerysetEqual
    return self.assertEqual(list(items), values, msg=msg)
  File "/usr/lib/python3.7/unittest/case.py", line 852, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib/python3.7/unittest/case.py", line 1058, in assertListEqual
    self.assertSequenceEqual(list1, list2, msg, seq_type=list)
  File "/usr/lib/python3.7/unittest/case.py", line 1040, in assertSequenceEqual
    self.fail(msg)
  File "/usr/lib/python3.7/unittest/case.py", line 693, in fail
    raise self.failureException(msg)
AssertionError: Lists differ: ['<Tag: t4>', '<Tag: t5>'] != ['<Tag: t1>', '<Tag: t4>', '<Tag: t5>']

First differing element 0:
'<Tag: t4>'
'<Tag: t1>'

Second list contains 1 additional elements.
First extra element 2:
'<Tag: t5>'

- ['<Tag: t4>', '<Tag: t5>']
?          ^

+ ['<Tag: t1>', '<Tag: t4>', '<Tag: t5>']
?          ^           +++++++++++++


----------------------------------------------------------------------
Ran 13071 tests in 180.966s

FAILED (failures=7, skipped=891, expected failures=4)

The full build log can be found here: ​https://buildd.debian.org/status/fetch.php?pkg=python-django&arch=all&ver=2%3A3.0~beta1-1&stamp=1571079369&file=log

[Simon Charette]

I noticed these failures when running the suite against SQLite 3.30 and I think you ran the suite against SQLite 3.30.1 here.

Given how long some of these tests have been around I suspect this is a regression in recent versions of SQLite.

I'll try bisecting the exact version of SQLite that is causing issue here as I get similar failures on Django 2.1 and 2.2.

[Chris Lamb]

Indeed, I am using 3.30.1:

Setting up libsqlite3-0:amd64 (3.30.1-1) ...

[Mariusz Felisiak]

I've checked and they work fine with SQLite 3.29.

[Simon Charette]

I confirm that the issues are solved with the current SQLite master which is currently considered 3.31 probably by ​https://www.sqlite.org/src/info/aa57d7abac0bb92d.

I'm not sure of what SQLite's backporting policy here though, is there's something we can do to get this in a possible 3.30.2 since it's a regression?

[Mariusz Felisiak]

Chris thanks for the report. Simon thanks for the investigation. I'm not sure if they will backport this fix, nevertheless it's not an issue in Django so we can close this ticket.