Formatters run on `TemplateCommand` make `admin_scripts` tests brittle

[Natalia Bidart]

Some admin_scripts tests, such as those for startproject, assert literal file contents generated from templates. Because TemplateCommand runs formatters before exiting (run_formatters(...)), the output can differ depending on environment details such as temporary directory path length or whether black is installed.

For example, when black reformats long lines during parallel test runs, strings in the generated manage.py file are wrapped in parentheses, causing assertion failures.

Tests should not depend on the exact output of formatters. Mocking or disabling them during test runs would make the suite more stable.

[Natalia Bidart]

There are other tests failures that need to be logged at.

[Jacob Walls]

I expect some more discussion about the merit of the approach, but I pushed some edits that should handle those failures.

[Jacob Walls <jacobtylerwalls@…>]

In 6436ec32:

Fixed #36680 -- Parametrized formatter discovery in AdminScriptTestCase.

[Natalia <124304+nessita@…>]

In 32ef9579:

[6.0.x] Fixed #36680 -- Parametrized formatter discovery in AdminScriptTestCase.

Backport of 6436ec321073bf0622af815e0af08f54c97f9b30 from main.

[GitHub <noreply@…>]

In 3939cd27:

Refs #36680 -- Fixed admin_scripts tests crash when black is not installed.

Regression in 6436ec321073bf0622af815e0af08f54c97f9b30.

[Natalia <124304+nessita@…>]

In 11cfb57:

[6.0.x] Refs #36680 -- Fixed admin_scripts tests crash when black is not installed.

Regression in 6436ec321073bf0622af815e0af08f54c97f9b30.
Backport of 3939cd279569fde44f557d79f20bb5b1a02440af from main.

[Hal Blackburn]

I realise this ticket is closed, but I ran into this issue again (despite it being fixed) when I was working on the PR for ticket:36704. I found several admin_scripts tests were failing on a clean checkout after following the repo's instructions to run tests. The fix here didn't stop the tests failing when black is on the PATH in more than one place. (e.g. when it's installed globally via pipx or a system package manager). I've got a PR to extend the current fix to handle multiple instances of black on the PATH, but I've not discussed here ahead of time as it was a small thing to fix while working on the other ticket: ​https://github.com/django/django/pull/20054

[Jacob Walls <jacobtylerwalls@…>]

In e78420c2:

Refs #36680 -- Avoided manipulating PATH in AdminScriptTestCase.

This mostly reverts 6436ec321073bf0622af815e0af08f54c97f9b30,
which was fragile. Instead, if black is present, we use it to format the
expected and actual results, instead of hard-coding the expected
formatted value.

Co-authored-by: Natalia <124304+nessita@…>

[Jacob Walls <jacobtylerwalls@…>]

In a26b660c:

Refs #36680 -- Avoided manipulating PATH in AdminScriptTestCase.

This mostly reverts 6436ec321073bf0622af815e0af08f54c97f9b30,
which was fragile. Instead, if black is present, we use it to format the
expected and actual results, instead of hard-coding the expected
formatted value.

Co-authored-by: Natalia <124304+nessita@…>