#20700 closed Bug (duplicate)

Tests does not report missing fixtures

Reported by: vzima Owned by: nobody
Component: Testing framework Version: master
Severity: Normal Keywords:
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

If a test contains fixtures, which do not exists, it does not report it as as failure. This is serious problem as fixtures are part of the test and their absence may cause false-positive success.

Consider this example:

from django.test import TestCase

class TestFoo(TestCase):
    fixtures = ('/invalid/filename', ) # Assume the fixture should load FooModel(pk=1)

    def test_foo_delete(self):
        delete_if_foo_exists(pk=1)
        self.assertFalse(FooModel.objects.filter(pk=1).exists())

Such a test would pass even if the delete_if_foo_exists does nothing.

This test is a very simple example, but ignoring the fact that fixtures are not loaded, can easily hide less obvious and more severe errors. Mere typo in fixture name can very well lead to a serious error.

Change History (5)

comment:1 Changed 22 months ago by jacobh

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Resolution set to duplicate
  • Status changed from new to closed

Duplicate of #10200

comment:2 Changed 22 months ago by vzima

  • Version changed from 1.5 to master

#10200 did not fixed this error. Tested on master.

class TestFoo(unittest.TestCase):
    fixtures = ('/invalid/filename', )
    def test_foo(self):
        self.fail('Should not run')

The main problem here is the loaddata does not report any error when file is missing.

comment:3 Changed 22 months ago by kmtracey

  • Resolution duplicate deleted
  • Status changed from closed to new

Reopen for further investigation given comment that the fix for noted dupe ticket does not fix this particular problem.

comment:5 Changed 21 months ago by kmtracey

  • Resolution set to duplicate
  • Status changed from new to closed

OK so you now do get a warning when running a test such as the one in the original description:

(django2) kmtracey@caktus006 10:26:34: ~/software/web/playground
--> ./manage.py test
Creating test database for alias 'default'...
/home/kmtracey/django/git-django/django/core/management/commands/loaddata.py:218: UserWarning: No fixture named 'filename' found.
  warnings.warn("No fixture named '%s' found." % fixture_name)

.
----------------------------------------------------------------------
Ran 1 test in 0.004s

OK
Destroying test database for alias 'default'...

That seems sufficient to me, even though the test does still "pass", I don't think it's too much to expect that devs will note the warning and investigate. So re-closing this as a dupe of #18990.

Note: See TracTickets for help on using tickets.
Back to Top