#29187 closed Bug (fixed)
test_delete_signals occasionally fails when running all django tests
| Reported by: | Jacob Mulford | Owned by: | Jacob Walls |
|---|---|---|---|
| Component: | Core (Other) | Version: | 2.0 |
| Severity: | Normal | Keywords: | tests, unit tests |
| Cc: | Hans-Peter Jansen | Triage Stage: | Ready for checkin |
| Has patch: | yes | Needs documentation: | no |
| Needs tests: | no | Patch needs improvement: | no |
| Easy pickings: | no | UI/UX: | no |
Description
I'm unable to replicate this consistently (outside of random chance) but occasionally test_delete_signals fails with the following error message when running all of the django tests:
(djangodev) jakemulf@jakemulf-Aspire-E5-574G:~/projects/open_source_projects/django$ python tests/runtests.py
Testing against Django installed in '/home/jakemulf/projects/open_source_projects/django/django' with up to 4 processes
Creating test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
Creating test database for alias 'other'...
Cloning test database for alias 'other'...
Cloning test database for alias 'other'...
Cloning test database for alias 'other'...
Cloning test database for alias 'other'...
System check identified no issues (14 silenced).
.....................................................................................................................................................................................................................................................................................................................................x......................................................................................................................................................................................................s..........s.......ss..................................s..............................................................................................s.......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................sssssssssssssssssssssssssssssssssssss.............................................sss..s...........................................s.............................................................................................................s......................................................................................................................................................................................s....................................................................s..s......................s.............................................s..............s..............................................................s.................................................s.s.......s.s.......................................................................s..............................................................................................................................................................................................s....................................................sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss.F.....sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss..s..sssss............................s................................s.....................................................................................................s........ssss................................s.....s.s..........................................s.......................................................................s......s........................................s.........s....................................................................sssss.....................................................................................................................................................................s.............sss.........................................................................................................................ssssssssssss.............................................................................................................................................................................................................................................s......s......................sss.s....s.............................x..s..............x.............................................................................................s..........................................................................................................s.....................................................................................................s........................................................................................................................................................................................s.............................................................................................ssssssssssssssssssssssssss......................s.......ss............................................................................................................................s......................................................................................................................................................s.......................................................................................................................................................................sssssssssssssssssssssssssssssssssssssssssssssssssssssss.......ssssssssssssssssssssssssssssssssssssssssssssssssssssss.............................................................................................................................................................................................................................s........................................................................ssssss..............................................................................................................................................................................................................................s.....s...ssssssssssssssssssssss..........................................................................................................................................................................s..................................................................s............s..s.....ss.ssss.....ss.s......s...........s....s....................................s.........s...............ss.........................................................................................................................................................sss..........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................s.......................................................................................................................................................................................................................................................................................................................................................................................ss...................................................................................................................................ssssss......................................................................................................................................................................s.................................................................................................................................................................................................................................................................s..s.......................................................................................................................................................s...................................................s........................................................................................................sss.sssssssssss.....s...s..........s.................................................................................................................................................................s..............................................................................................................s..............ss............................sssssss..sssss..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................s....ssssssssss..............................sss......s............................s................ss...ss...s.s...........................................................................................................s.............s...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................s.............................................................................................................................................................................................s............................................................................x.............................ss...........................................................................sssssssssss.s.sssss.ssssss.........................................................................................................................ssssssss.....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
======================================================================
FAIL: test_delete_signals (signals.tests.SignalTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python3.5/unittest/case.py", line 58, in testPartExecutor
yield
File "/usr/lib/python3.5/unittest/case.py", line 603, in run
self.tearDown()
File "/home/jakemulf/projects/open_source_projects/django/tests/signals/tests.py", line 32, in tearDown
self.assertEqual(self.pre_signals, post_signals)
File "/usr/lib/python3.5/unittest/case.py", line 820, in assertEqual
assertion_func(first, second, msg=msg)
File "/usr/lib/python3.5/unittest/case.py", line 1029, in assertTupleEqual
self.assertSequenceEqual(tuple1, tuple2, msg, seq_type=tuple)
File "/usr/lib/python3.5/unittest/case.py", line 1000, in assertSequenceEqual
self.fail(msg)
File "/usr/lib/python3.5/unittest/case.py", line 665, in fail
raise self.failureException(msg)
AssertionError: Tuples differ: (1, 0, 2, 1) != (1, 0, 2, 0)
First differing element 3:
1
0
- (1, 0, 2, 1)
? ^
+ (1, 0, 2, 0)
? ^
----------------------------------------------------------------------
Ran 12143 tests in 135.413s
FAILED (failures=1, skipped=804, expected failures=4)
Destroying test database for alias 'default'...
Destroying test database for alias 'default'...
Destroying test database for alias 'default'...
Destroying test database for alias 'default'...
Destroying test database for alias 'default'...
Destroying test database for alias 'other'...
Destroying test database for alias 'other'...
Destroying test database for alias 'other'...
Destroying test database for alias 'other'...
Destroying test database for alias 'other'...
(djangodev) jakemulf@jakemulf-Aspire-E5-574G:~/projects/open_source_projects/django$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean
(djangodev) jakemulf@jakemulf-Aspire-E5-574G:~/projects/open_source_projects/django$ python
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
Running the same test command again passes all tests.
I have not yet gotten the same error when isolating these signal tests (python tests/runtests.py signals.tests.SignalTests)
Change History (12)
comment:1 by , 8 years ago
comment:2 by , 8 years ago
| Component: | Testing framework → Core (Other) |
|---|---|
| Triage Stage: | Unreviewed → Someday/Maybe |
I haven't seen that failure before, but if someone identifies the cause, please let us know.
comment:3 by , 7 years ago
| Cc: | added |
|---|
I see this very issue in OBS builds of Django, which typically build with the Django testsuite enabled.
[ 264s] ====================================================================== [ 264s] FAIL: test_delete_signals (signals.tests.SignalTests) [ 264s] ---------------------------------------------------------------------- [ 264s] Traceback (most recent call last): [ 264s] File "/usr/lib64/python3.6/unittest/case.py", line 59, in testPartExecutor [ 264s] yield [ 264s] File "/usr/lib64/python3.6/unittest/case.py", line 608, in run [ 264s] self.tearDown() [ 264s] File "/home/abuild/rpmbuild/BUILD/Django-2.1.5/tests/signals/tests.py", line 32, in tearDown [ 264s] self.assertEqual(self.pre_signals, post_signals) [ 264s] File "/usr/lib64/python3.6/unittest/case.py", line 829, in assertEqual [ 264s] assertion_func(first, second, msg=msg) [ 264s] File "/usr/lib64/python3.6/unittest/case.py", line 1039, in assertTupleEqual [ 264s] self.assertSequenceEqual(tuple1, tuple2, msg, seq_type=tuple) [ 264s] File "/usr/lib64/python3.6/unittest/case.py", line 1010, in assertSequenceEqual [ 264s] self.fail(msg) [ 264s] File "/usr/lib64/python3.6/unittest/case.py", line 670, in fail [ 264s] raise self.failureException(msg) [ 264s] AssertionError: Tuples differ: (1, 0, 2, 1) != (1, 0, 2, 0) [ 264s] [ 264s] First differing element 3: [ 264s] 1 [ 264s] 0 [ 264s] [ 264s] - (1, 0, 2, 1) [ 264s] ? ^ [ 264s] [ 264s] + (1, 0, 2, 0) [ 264s] ? ^ [ 264s] [ 264s] [ 264s] ---------------------------------------------------------------------- [ 264s] Ran 12368 tests in 172.836s
These builds are in public here:
https://build.opensuse.org/package/show/home:frispete:python/python-Django
comment:4 by , 4 years ago
Is anyone still seeing this test failure? Is there a way to move this ticket forward, or should it remain open? It doesn't seem to match the description of the "Someday/Maybe" triage stage here: https://docs.djangoproject.com/en/dev/internals/contributing/triaging-tickets/#someday-maybe
I can't reproduce the test failure locally.
comment:6 by , 3 weeks ago
| Owner: | changed from to |
|---|---|
| Status: | new → assigned |
| Triage Stage: | Someday/Maybe → Accepted |
We see this frequently in our CI runs now. Based on the analysis in comment:1, I think we just need to fix setUp to clear dead receivers.
Note:
See TracTickets
for help on using tickets.
I added a print statement in 'tests/signals/tests.py' to see the difference between 'signals.pre_delete.receivers' and 'signals.post_delete.receivers' and noticed that they were not the same when running all of the unit tests compared to running just the 'signals.tests.SignalTests' tests.
Isolating the signals.tests.SignalTests unit tests
Running all unit tests with success
Running all tests with failure on test_delete_signals
(djangodev) jakemulf@jakemulf-Aspire-E5-574G:~/projects/open_source_projects/django$ python tests/runtests.py Testing against Django installed in '/home/jakemulf/projects/open_source_projects/django/django' with up to 4 processes Creating test database for alias 'default'... Cloning test database for alias 'default'... Cloning test database for alias 'default'... Cloning test database for alias 'default'... Cloning test database for alias 'default'... Creating test database for alias 'other'... Cloning test database for alias 'other'... Cloning test database for alias 'other'... Cloning test database for alias 'other'... Cloning test database for alias 'other'... System check identified no issues (14 silenced). .....................................................................................................................................................................................................................................................................................................................................x....................................................................................................................................................................................s.s.........................................ss.........................................................s.........................................s..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................sssssssssssssssssssssssssssssssssssss.............................................sss..s...........................................s...................................................................................................s................................................................................................................................................................................................s................................................................................s..s....s...........................s...................................................s.s.......s.s..................................................s..................................s.......................................................................................s............................................................................................................................................................................................s......................................[((139947131390288, 38479880), <weakref at 0x7f47fb0d8ef8; to 'function' at 0x7f47fb0dc950 (clear_site_cache)>), ((139947111254080, 45829352), <weakref at 0x7f47f9d38408; to 'function' at 0x7f47f9da8840 (prevent_deletes)>)] [((139946820203112, 47003928), <weakref at 0x7f47ead241d8; dead>)] [((139947131390288, 38479880), <weakref at 0x7f47fb0d8ef8; to 'function' at 0x7f47fb0dc950 (clear_site_cache)>), ((139947111254080, 45829352), <weakref at 0x7f47f9d38408; to 'function' at 0x7f47f9da8840 (prevent_deletes)>)] [((139946820203112, 47003928), <weakref at 0x7f47ead241d8; dead>)] [((139947131390288, 38479880), <weakref at 0x7f47fb0d8ef8; to 'function' at 0x7f47fb0dc950 (clear_site_cache)>), ((139947111254080, 45829352), <weakref at 0x7f47f9d38408; to 'function' at 0x7f47f9da8840 (prevent_deletes)>)] [((139946820203112, 47003928), <weakref at 0x7f47ead241d8; dead>)] [((139947131390288, 38479880), <weakref at 0x7f47fb0d8ef8; to 'function' at 0x7f47fb0dc950 (clear_site_cache)>), ((139947111254080, 45829352), <weakref at 0x7f47f9d38408; to 'function' at 0x7f47f9da8840 (prevent_deletes)>)] [((139946820203112, 47003928), <weakref at 0x7f47ead241d8; dead>)] [((139947131390288, 38479880), <weakref at 0x7f47fb0d8ef8; to 'function' at 0x7f47fb0dc950 (clear_site_cache)>), ((139947111254080, 45829352), <weakref at 0x7f47f9d38408; to 'function' at 0x7f47f9da8840 (prevent_deletes)>)] [((139946820203112, 47003928), <weakref at 0x7f47ead241d8; dead>)] .....[((139947131390288, 38479880), <weakref at 0x7f47fb0d8ef8; to 'function' at 0x7f47fb0dc950 (clear_site_cache)>), ((139947111254080, 45829352), <weakref at 0x7f47f9d38408; to 'function' at 0x7f47f9da8840 (prevent_deletes)>)] [((139946820203112, 47003928), <weakref at 0x7f47ead241d8; dead>)] .........sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss[((139947131390288, 38479880), <weakref at 0x7f47fb0d8ef8; to 'function' at 0x7f47fb0dc950 (clear_site_cache)>), ((139947111254080, 45829352), <weakref at 0x7f47f9d38408; to 'function' at 0x7f47f9da8840 (prevent_deletes)>)] [] sssssssssssssssssssssss[((139947131390288, 38479880), <weakref at 0x7f47fb0d8ef8; to 'function' at 0x7f47fb0dc950 (clear_site_cache)>), ((139947111254080, 45829352), <weakref at 0x7f47f9d38408; to 'function' at 0x7f47f9da8840 (prevent_deletes)>)] s[] ssssss[((139947131390288, 38479880), <weakref at 0x7f47fb0d8ef8; to 'function' at 0x7f47fb0dc950 (clear_site_cache)>), ((139947111254080, 45829352), <weakref at 0x7f47f9d38408; to 'function' at 0x7f47f9da8840 (prevent_deletes)>)] [] ssss[((139947131390288, 38479880), <weakref at 0x7f47fb0d8ef8; to 'function' at 0x7f47fb0dc950 (clear_site_cache)>), ((139947111254080, 45829352), <weakref at 0x7f47f9d38408; to 'function' at 0x7f47f9da8840 (prevent_deletes)>)] [] ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss[((139947131390288, 38479880), <weakref at 0x7f47fb0d8ef8; to 'function' at 0x7f47fb0dc950 (clear_site_cache)>), ((139947111254080, 45829352), <weakref at 0x7f47f9d38408; to 'function' at 0x7f47f9da8840 (prevent_deletes)>)] [] sssssssssssssss[((139947131390288, 38479880), <weakref at 0x7f47fb0d8ef8; to 'function' at 0x7f47fb0dc950 (clear_site_cache)>), ((139947111254080, 45829352), <weakref at 0x7f47f9d38408; to 'function' at 0x7f47f9da8840 (prevent_deletes)>)] [] ssssssssssssssssssssssssssssssssssssssss.F.....sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss..s..............................s...........................................................................................................s..........................s........ssss................................s.....s.s..........................................s......................................................................................s..................s.............s...s..........................................................................sssss.....................................................................................................................................................................s.............sss.........................................................................................................................ssssssssssss.............................................................................................................................................................................................................................................s......s.......s...............................x....s..............x...................sss.s.........................................................................................s..................................................................................s......................................................s...............................................................................................................................................................................................................................................................s.............................................................................................ssssssssssssssssssssssssss...................s.........ss............................................................................................................................................................................................................................................................s.......................s.......................................................................................................................................................................sssssssssssssssssssssssssssssssssssssssssssssssssssssss.......ssssssssssssssssssssssssssssssssssssssssssssssssssssss.............................................................................................................................................................................................................................s........................................................................ssssss..............................................................................................................................................................................................................................s.....s....................ssssssssssssssssssssss..........................................................................................................................................................................s..................................................................................ss....................................................................................................................................................sss.............................................................................................................................................................................................................................................................................................................................................................................................................................s............s..s.....ss.ssss.....ss.s......s...........s....s....................................s.........s..............................................................................................................s..................................................................................................................................................................................................................................................................................................................................................................................ss..........................................................................................................................................ssssss...............................................................................................................................................................................s...............................................................................................................................................................................................................................................................s..s................................................................................................................................................s...................................................s...........................................................................................................sss.sssssssssss..s...s..............................................................................................................................s.............................................s........................................................................................................s........................ss......sssssss..sssss....................................................................................................................................................................................................................................................................................................................................................................................................................................................................s....ssssssssss....................................................................................ssss........s................ss...ss...s.s......................................................................................................s.............s................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................s....................................................................................................................................................................................sssssssssss.s.sssss.ssssss........s............................................................................x.............................ss................................................................................................................................................................................................ssssssss.......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... ====================================================================== FAIL: test_delete_signals (signals.tests.SignalTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python3.5/unittest/case.py", line 58, in testPartExecutor yield File "/usr/lib/python3.5/unittest/case.py", line 603, in run self.tearDown() File "/home/jakemulf/projects/open_source_projects/django/tests/signals/tests.py", line 34, in tearDown self.assertEqual(self.pre_signals, post_signals) File "/usr/lib/python3.5/unittest/case.py", line 820, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib/python3.5/unittest/case.py", line 1029, in assertTupleEqual self.assertSequenceEqual(tuple1, tuple2, msg, seq_type=tuple) File "/usr/lib/python3.5/unittest/case.py", line 1000, in assertSequenceEqual self.fail(msg) File "/usr/lib/python3.5/unittest/case.py", line 665, in fail raise self.failureException(msg) AssertionError: Tuples differ: (1, 0, 2, 1) != (1, 0, 2, 0) First differing element 3: 1 0 - (1, 0, 2, 1) ? ^ + (1, 0, 2, 0) ? ^ ---------------------------------------------------------------------- Ran 12143 tests in 129.548s FAILED (failures=1, skipped=804, expected failures=4) Destroying test database for alias 'default'... Destroying test database for alias 'default'... Destroying test database for alias 'default'... Destroying test database for alias 'default'... Destroying test database for alias 'default'... Destroying test database for alias 'other'... Destroying test database for alias 'other'... Destroying test database for alias 'other'... Destroying test database for alias 'other'... Destroying test database for alias 'other'... (djangodev) jakemulf@jakemulf-Aspire-E5-574G:~/projects/open_source_projects/django$It looks like the value in 'signals.post_delete.receivers' becomes a dead reference
The unit test still fails since the unit test is checking the length of the signals, and this dead reference is still counted in the length
def setUp(self): # Save up the number of connected signals so that we can check at the # end that all the signals we register get properly unregistered (#9989) self.pre_signals = ( len(signals.pre_save.receivers), len(signals.post_save.receivers), len(signals.pre_delete.receivers), len(signals.post_delete.receivers), ) def tearDown(self): # All our signals got disconnected properly. post_signals = ( len(signals.pre_save.receivers), len(signals.post_save.receivers), len(signals.pre_delete.receivers), len(signals.post_delete.receivers), ) self.assertEqual(self.pre_signals, post_signals)