Opened 8 months ago

Last modified 8 months ago

#29127 new Bug

Running tagged tests hides any with syntax errors

Reported by: Kryštof Řeháček Owned by:
Component: Testing framework Version: 2.0
Severity: Normal Keywords: tests, tagged-tests, SyntaxError
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

When I execute tests with syntax error without any argument then it shows SyntaxError: invalid syntax as expected but when I execute tagged tests, then it shows that it ran 0 tests and it is OK and even error. So when tests with syntax executed with the --tag="some_tag" argument and they have syntax error then they complete with OK status.

Change History (4)

comment:1 Changed 8 months ago by Kryštof Řeháček

Summary: manage.py executting tagged tests with syntax error does not show themmanage.py executing tagged tests with syntax error does not show them

comment:2 Changed 8 months ago by Tim Graham

Summary: manage.py executing tagged tests with syntax error does not show themRunning tagged tests hides any with syntax errors
Triage Stage: UnreviewedAccepted

The problem may be in django.test.runner.filter_tests_by_tags().

comment:3 Changed 8 months ago by Daniel Hones

Owner: changed from nobody to Daniel Hones
Status: newassigned

comment:4 Changed 8 months ago by Daniel Hones

Owner: Daniel Hones deleted
Status: assignednew

It looks like this behavior is ultimately due to Python's unittest TestLoader and not Django's test framework. In case of a SyntaxError when loading a test suite, it hits this line, which creates a _FailedTest object from it without raising an error. Then the django.test.runner.filter_tests_by_tags() sees a _FailedTest object which won't have any tags and so it won't be run at all. Since tests are tagged using a decorator, tests in a file with a syntax error won't have any tags recognized for them anyway because the file cannot be evaluated. I don't think there's a reasonable fix for this and I'm not sure it should even be considered a bug, although the behavior is a bit misleading.

Note: See TracTickets for help on using tickets.
Back to Top