Opened 6 years ago

Closed 6 years ago

#27059 closed Bug (needsinfo)

autoreload loose the sys.path in some cases (buildout install of python for instance, maybe others)

Reported by: Annakan Owned by: nobody
Component: Core (Management commands) Version: 1.10
Severity: Normal Keywords: reloader runserver buildout isolation pythonpath sys.path
Cc: Triage Stage: Unreviewed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Annakan)

In my configuration, aiming for isolation, I installed django with pyenv (virtualenv) and buildout.
In that case when using manage.py runserver the reloading loose the sys.path (and thus the eggs installed in the buildout)
In the case of a buildout sys.executable returns the path of the python executable the buildout was build with, and not the python script in <buildout-root>/bin/

I am not sure why nobody had the problem before but I devised a hack since restart_with_reloader properly copy the environment I patched the code to copy the sys.path into the pythonpath envar of the process.

The modified code is below :

(From Django-1.9.8-py3.5.egg/django/utils/autoreload.py)

def restart_with_reloader():
    while True:
        args = [sys.executable] + ['-W%s' % o for o in sys.warnoptions] + sys.argv
        if sys.platform == "win32":
            args = ['"%s"' % arg for arg in args]
        # Simply restarting the executable won't preserve virtualenv/pyenv/buildout path properly in case of a "buildout"
        # sys.executable point at the  the python binary used to  create the buildout 
        # and not the bin/python script used to launch the manage.py script
        cur_python_path = ':'.join(sys.path)
        if 'PYTHONPATH' in os.environ:
            if cur_python_path not in os.environ['PYTHONPATH']:
                os.environ['PYTHONPATH'] += cur_python_path
        else:
            os.environ['PYTHONPATH'] = cur_python_path
        new_environ = os.environ.copy()
        new_environ["RUN_MAIN"] = 'true'
        exit_code = os.spawnve(os.P_WAIT, sys.executable, args, new_environ)
        if exit_code != 3:
            return exit_code

The 6 lines added are

        cur_python_path = ':'.join(sys.path)
        if 'PYTHONPATH' in os.environ:
            if cur_python_path not in os.environ['PYTHONPATH']:
                os.environ['PYTHONPATH'] += cur_python_path
        else:
            os.environ['PYTHONPATH'] = cur_python_path

I am not sure there are side effects for project without a buildout though but is seems to me that if site.py is not too messed up with (we are looking at you ubuntu and thanks pyenv ;) ) pythonpath is second in peeking order and we should aim to preserve the original sys.path when reloading.
The worse that could happen is that the sys.path contains the same path twice, but I checked and Python seems to properly clean up the sys.path to keep module paths de-duplicated.

Change History (4)

comment:1 Changed 6 years ago by Annakan

Description: modified (diff)

comment:2 Changed 6 years ago by Annakan

Description: modified (diff)

comment:3 Changed 6 years ago by Tim Graham

I'm unfamiliar with buildout. Could you please give steps to reproduce the issue?

comment:4 Changed 6 years ago by Tim Graham

Easy pickings: unset
Resolution: needsinfo
Status: newclosed
Note: See TracTickets for help on using tickets.
Back to Top