Tests does not report missing fixtures

[Vlastimil Zíma]

If a test contains fixtures, which do not exists, it does not report it as as failure. This is serious problem as fixtures are part of the test and their absence may cause false-positive success.

Consider this example:

from django.test import TestCase

class TestFoo(TestCase):
    fixtures = ('/invalid/filename', ) # Assume the fixture should load FooModel(pk=1)

    def test_foo_delete(self):
        delete_if_foo_exists(pk=1)
        self.assertFalse(FooModel.objects.filter(pk=1).exists())

Such a test would pass even if the delete_if_foo_exists does nothing.

This test is a very simple example, but ignoring the fact that fixtures are not loaded, can easily hide less obvious and more severe errors. Mere typo in fixture name can very well lead to a serious error.

[jacobh]

Duplicate of #10200

[Vlastimil Zíma]

#10200 did not fixed this error. Tested on master.

class TestFoo(unittest.TestCase):
    fixtures = ('/invalid/filename', )
    def test_foo(self):
        self.fail('Should not run')

The main problem here is the loaddata does not report any error when file is missing.

[Karen Tracey]

Reopen for further investigation given comment that the fix for noted dupe ticket does not fix this particular problem.

[Ramiro Morales]

See also #18990 and ​https://github.com/django/django/commit/7a99d1e167f81c5fcd1b9f7f548c5d6959cef2ef

[Karen Tracey]

OK so you now do get a warning when running a test such as the one in the original description:

(django2) kmtracey@caktus006 10:26:34: ~/software/web/playground
--> ./manage.py test
Creating test database for alias 'default'...
/home/kmtracey/django/git-django/django/core/management/commands/loaddata.py:218: UserWarning: No fixture named 'filename' found.
  warnings.warn("No fixture named '%s' found." % fixture_name)

.
----------------------------------------------------------------------
Ran 1 test in 0.004s

OK
Destroying test database for alias 'default'...

That seems sufficient to me, even though the test does still "pass", I don't think it's too much to expect that devs will note the warning and investigate. So re-closing this as a dupe of #18990.