assertWarnsMessage() only checks the first warning for a matching message

[Mike Edmunds]

assertWarnsMessage(category, message) fails if the first warning issued in category does not contain message, even if some later warning does match.

As a result, test cases depend on the specific order warnings are issued. This can become confusing when there are several related deprecations going in at once, or when a feature being tested uses some other feature that also issues a warning.

For example, you might expect this test to pass, but it fails with AssertionError: 'Deprecated in MyApp' not found in 'Deprecated in Django':

class MyAppTests(SimpleTestCase):
    def test_deprecated_in_my_app(self):
        with self.assertWarnsMessage(DeprecationWarning, "Deprecated in MyApp"):
            # MyApp calls something in Django that happens to be deprecated:
            warnings.warn("Deprecated in Django", RemovedInNextVersionWarning)
            # Then MyApp issues the warning being tested:
            warnings.warn("Deprecated in MyApp", DeprecationWarning)

If you switch the order of the two warnings, the test passes. (And the second warning is then ignored—see #37072.)

There is a workaround by combining a very carefully crafted ignore_warnings() context with assertWarnsMessage(). (It requires knowing that the semi-documented ignore_warnings() also accepts a message argument. And that it interprets it quite differently from assertWarnsMessage().)