Bash completion helper always with exit-code 1

[Daniel Hahler]

The handler for (Bash) autocompletion exists with code 1 always, which is not helpful in case you want to catch errors, e.g. because some imports failed.

Code reference: ​https://github.com/django/django/blob/bac13074ee5bf6b35e45370fc8b9b14448c765dd/django/core/management/__init__.py#L270

It should exit with 0 by default instead.

It was added in [677ddcbb04], and it does not look like it would be handled / expected by the Bash wrapper, but I have not checked it.

[Tim Graham]

I haven't looked into the issue, but it seems reasonable to at least document why it exits with status code 1 if it's not a bug.

[Daniel Roschka]

Ok, this is going to be exciting! Really, trust me!

The reporter of this issue noticed that the autocomplete method of django.core.management.ManagementUtility exits from Python with exit code 1 and feared that this might cause problems, because no distinction between successful and unsucessful management command runs was possible because of that.

The autocomplete method is always called when running a Django management command. So every management command exits with exit code 1, right? Well, not really: If it would, management commands wouldn't do anything useful, because the autocomplete method is called before the specified management command is run. Looking closer into autocomplete reveals why it does work instead:

        # Don't complete if user hasn't sourced bash_completion file.
        if 'DJANGO_AUTO_COMPLETE' not in os.environ:
            return

Of course DJANGO_AUTO_COMPLETE usually isn't set when you run a Django management command manually, that's why autocomplete just returns instead of exiting and the management command runs as expected and returning its real exit code. So, when not running in "command autocompletion mode" Djangos management commands don't even execute the sys.exit(1) in autocomplete. Perfect.

But what about that famous "command autocompletion mode" I already mentioned? To get this mode to work you need slightly more than setting the environment variable DJANGO_AUTO_COMPLETE and therefore there is a ready-to-use script in django/extras: ​https://github.com/django/django/blob/master/extras/django_bash_completion

To understand the following we have to dig a little bit into what happens when the bash-completion script for Django is sourced:

• The script defines a shell function, which sets some environment variables and calls the Django management command typed before the autocompletion started (that's the $1):

  _django_completion()
  {
      COMPREPLY=( $( COMP_WORDS="${COMP_WORDS[*]}" \
                     COMP_CWORD=$COMP_CWORD \
                     DJANGO_AUTO_COMPLETE=1 $1 ) )
  }
  

• Then it runs the Bash builtin complete command with the shell function as an argument (-F _django_completion) which registers the code in the shell function for autocompletion. The second argument (-o default django-admin.py manage.py django-admin) defines for which commands this autocompletion should run (don't worry, there is additionally logic for cases where you run it by specifying the python interpreter like python manage.py ...):

  complete -F _django_completion -o default django-admin.py manage.py django-admin
  

Once registered the function will be executed whenever you type one of commands the autocompletion function is registered for and press TAB afterwards. Bash runs the whole autocompletion logic in the background (but its output will be still printed to stdout and stderr) so there is nothing it could return its exit code to.

As there is nothing to forward the exit code for, the problem is solved, right? Kind of. I began to wonder how Bash processes the autocompletion functions and what it does with the exit codes of the functions. To figure that out I had to dig deep into the source code of Bash (remark: no fun if you're used to Python) and found something interesting:
There is indeed a different behavior for different exit codes, but you'd never guessed for which ones: If the autocompletion function returns with exit code 124 (execution timed out) or 127 (function to execute wasn't found) the behavior is different than with any other exit code (if somebody is interested in details, check gen_shell_function_matches in pcomplete.c). When changing Django to exit with exit code 124 or 127 instead of 1 the autocompletion completely breaks and offers only file and directory names in the current directory instead of the usual completion options. Beside this special behavior the exit codes of the autocomplition functions are completly ignored.

So this was a fun adventure, but just proved that it doesn't really matter which exit code the autocomplete method in Django use as long as it's not 124 or 127. I suggest leaving it as it is or simply changing it to 0 to make a bit clearer that it's nothing to worry about. I'm also not sure if and how that should be better documented, so I suggest somebody else decides that.

tl;dr: Everything is fine as the exit code of the autocompletion method is never returned to the shell and is not 124 or 127 (as that would change what Bash returns for autocompletion).

[Russell Keith-Magee <russell@…>]

In 9378229:

Merge pull request #6389 from Dunedan/ticket_25420

Fixed #25420 -- Documented bash completion exit code behavior.

[Daniel Hahler]

Thanks for fixing this! (by changing it to 0 and not only document it)

Just for information/reference:

Everything is fine as the exit code of the autocompletion method is never returned to the shell and is not 124 or 127 (as that would change what Bash returns for autocompletion).

I am using the autocompletion in a custom (not yet released) completion function for Zsh, and the exit code of 0 helps me to know that it was successful.
I have not checked the (shipped) Bash completion, but also there it might make a difference in case Django fails before reaching the autocompletion code (e.g. because of import errors etc).

[Daniel Roschka]

Ah, now with discussion and Github and here it might become slightly confusing. Anyway, I posted my general comments already on Github (​https://github.com/django/django/pull/6389#issuecomment-206457800) and just want to add one note here:

I have not checked the (shipped) Bash completion, but also there it might make a difference in case Django fails before reaching the autocompletion code (e.g. because of import errors etc).

That's what I checked and the return code doesn't matter at all (beside the weird edge cases with 124 and 127). If the notes from my original investigation weren't clear enough regarding that please drop me a note and I'll try to clear that out.

And regarding your zsh completion support I suggest to add different exit codes with the ticket implementing the zsh completion.