ManagementUtility instantiates CommandParser without passing already-computed prog argument

[William Schwartz]

ManagementUtility ​goes to the trouble to parse the program name from the argv it's passed rather than from sys.argv:

    def __init__(self, argv=None):
        self.argv = argv or sys.argv[:]
        self.prog_name = os.path.basename(self.argv[0])
        if self.prog_name == '__main__.py':
            self.prog_name = 'python -m django'

But then when it needs to parse --pythonpath and --settings, it ​uses the program name from sys.argv:

        parser = CommandParser(usage='%(prog)s subcommand [options] [args]', add_help=False, allow_abbrev=False)

Above "%(prog)s" ​refers to sys.argv[0]. Instead, it should refer to self.prog_name. This can fixed as follows:

        parser = CommandParser(
            prog=self.prog_name,
            usage='%(prog)s subcommand [options] [args]',
            add_help=False,
            allow_abbrev=False)

I'm aware that execute_from_command_line is a private API, but it'd be really convenient for me if it worked properly in my weird embedded environment where sys.argv[0] is ​incorrectly None. If passing my own argv to execute_from_command_line avoided all the ensuing exceptions, I wouldn't have to modify sys.argv[0] globally as I'm doing in the meantime.

[Mariusz Felisiak]

Tentatively accepted, looks valid but I was not able to reproduce and invalid message (even with mocking sys.argv), so a regression test is crucial.

[William Schwartz]

Replying to Mariusz Felisiak:

...I was not able to reproduce and invalid message (even with mocking sys.argv), so a regression test is crucial.

I've created a patch with a test case in ​PR. (I didn't see Douglas Cueva's patch GH-13652 before I pushed mine. Sorry.) Is this a backport candidate?

The ​test I added passes now, and failed like below before I applied the fix.

(venv) ➜  django git:(mgmt-argv0) ✗ tox -e py38 -- admin_scripts.tests.ExecuteFromCommandLine
<snip>
py38 run-test: commands[0] | django/.tox/py38/bin/python runtests.py admin_scripts.tests.ExecuteFromCommandLine
Testing against Django installed in 'django/django'
System check identified no issues (0 silenced).
E
======================================================================
ERROR: test_prog_name_from_argv0 (admin_scripts.tests.ExecuteFromCommandLine)
Program name is computed once from argv argument, not sys.argv (#32177).
----------------------------------------------------------------------
Traceback (most recent call last):
  File "django/tests/admin_scripts/tests.py", line 1882, in test_prog_name_from_argv0
    execute_from_command_line([''] + args)
  File "django/django/core/management/__init__.py", line 414, in execute_from_command_line
    utility.execute()
  File "django/django/core/management/__init__.py", line 347, in execute
    parser = CommandParser(usage='%(prog)s subcommand [options] [args]', add_help=False, allow_abbrev=False)
  File "django/django/core/management/base.py", line 54, in __init__
    super().__init__(**kwargs)
  File "/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/argparse.py", line 1660, in __init__
    prog = _os.path.basename(_sys.argv[0])
  File "/usr/local/opt/python@3.8/bin/../Frameworks/Python.framework/Versions/3.8/lib/python3.8/posixpath.py", line 142, in basename
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType

----------------------------------------------------------------------
Ran 1 test in 0.005s

FAILED (errors=1)

[Mariusz Felisiak]

Thanks for the patch.

Is this a backport candidate?

No, this doesn't qualify for a backport.

[Mariusz Felisiak <felisiak.mariusz@…>]

In cc226935:

Fixed #32177 -- Made execute_from_command_line() use program name from the argv argument.

This caused crash in environments where sys.argv[0] is incorrectly set
to None.

[Niklas Mohrin]

I believe the same problem exists with the CommandParser used by subcommands: At the end of ManagementUtility.execute (see 1), the subcommand is run with self.fetch_command(subcommand).run_from_argv(self.argv). In run_from_argv, a parser is created with self.argv[0] via parser = self.create_parser(argv[0], argv[1]) (see 2). Thus, when running python -m django check --help (or any other subcommand), the output starts with usage: __main__.py check.

Maybe it would be better to just update self.argv[0] in ManagementUtility instead of having prog_name?

Either way, should I open a separate ticket and/or implement the proposed change?

• (1): ​https://github.com/django/django/blob/4a3ad9eebbc16ce80b348644b557c84ecc741be7/django/core/management/__init__.py#L436
• (2): ​https://github.com/django/django/blob/4a3ad9eebbc16ce80b348644b557c84ecc741be7/django/core/management/base.py#L408