Opened 11 years ago

Closed 11 years ago

#20700 closed Bug (duplicate)

Tests does not report missing fixtures

Reported by: Vlastimil Zíma Owned by: nobody
Component: Testing framework Version: dev
Severity: Normal Keywords:
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

If a test contains fixtures, which do not exists, it does not report it as as failure. This is serious problem as fixtures are part of the test and their absence may cause false-positive success.

Consider this example:

from django.test import TestCase

class TestFoo(TestCase):
    fixtures = ('/invalid/filename', ) # Assume the fixture should load FooModel(pk=1)

    def test_foo_delete(self):
        delete_if_foo_exists(pk=1)
        self.assertFalse(FooModel.objects.filter(pk=1).exists())

Such a test would pass even if the delete_if_foo_exists does nothing.

This test is a very simple example, but ignoring the fact that fixtures are not loaded, can easily hide less obvious and more severe errors. Mere typo in fixture name can very well lead to a serious error.

Change History (5)

comment:1 by jacobh, 11 years ago

Resolution: duplicate
Status: newclosed

Duplicate of #10200

comment:2 by Vlastimil Zíma, 11 years ago

Version: 1.5master

#10200 did not fixed this error. Tested on master.

class TestFoo(unittest.TestCase):
    fixtures = ('/invalid/filename', )
    def test_foo(self):
        self.fail('Should not run')

The main problem here is the loaddata does not report any error when file is missing.

comment:3 by Karen Tracey, 11 years ago

Resolution: duplicate
Status: closednew

Reopen for further investigation given comment that the fix for noted dupe ticket does not fix this particular problem.

comment:5 by Karen Tracey, 11 years ago

Resolution: duplicate
Status: newclosed

OK so you now do get a warning when running a test such as the one in the original description:

(django2) kmtracey@caktus006 10:26:34: ~/software/web/playground
--> ./manage.py test
Creating test database for alias 'default'...
/home/kmtracey/django/git-django/django/core/management/commands/loaddata.py:218: UserWarning: No fixture named 'filename' found.
  warnings.warn("No fixture named '%s' found." % fixture_name)

.
----------------------------------------------------------------------
Ran 1 test in 0.004s

OK
Destroying test database for alias 'default'...

That seems sufficient to me, even though the test does still "pass", I don't think it's too much to expect that devs will note the warning and investigate. So re-closing this as a dupe of #18990.

Note: See TracTickets for help on using tickets.
Back to Top