Unit test model_regress.test_pickle.ModelPickleTestCase fails on RHEL6 SCL

[David D. Riddle]

We are running RHEL6 with Python 2.7 running in a Software Collection. To use Python 2.7 on RHEL6 it is necessary to use RedHat Software Collections. This bug effects both the master branch and version 1.8. Here is the error message I am receiving:

~/Django-1.8/tests$ scl enable python27 bash
~/Django-1.8/tests$ PYTHONPATH=..:$PYTHONPATH ./runtests.py model_regress.test_pickle
Testing against Django installed in '/services/scratch/ddriddle/tmp/Django-1.8/django'
Creating test database for alias 'default'...
Creating test database for alias 'other'...
./opt/rh/python27/root/usr/bin/python: error while loading shared libraries: libpython2.7.so.1.0: cannot open shared object file: No such file or directory
F.
======================================================================
FAIL: test_unpickling_when_appregistrynotready (model_regress.test_pickle.ModelPickleTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/services/scratch/ddriddle/tmp/Django-1.8/tests/model_regress/test_pickle.py", line 100, in test_unpickling_when_appregistrynotready
    self.fail("Unable to reload model pickled data")
AssertionError: Unable to reload model pickled data

----------------------------------------------------------------------
Ran 3 tests in 0.017s

FAILED (failures=1)
Destroying test database for alias 'default'...
Destroying test database for alias 'other'...

[Tim Graham]

I see you assigned the ticket to yourself. Does that mean you plan to investigate and propose a fix? I am not sure if it's a problem with Django or a problem with your Python install.

[David D. Riddle]

Replying to timgraham:

I see you assigned the ticket to yourself. Does that mean you plan to investigate and propose a fix? I am not sure if it's a problem with Django or a problem with your Python install.

I do plan to purpose a fix. I have written a patch and I plan to submit it later today after I finish testing the patch on Windows and a vanilla Debian install. The problem is with the unit test itself. Django and the Python environment I am running in are fine.

[David D. Riddle]

I have created a branch with a fix for this issue here: ​https://github.com/ddriddle/django/tree/ticket_24717.

Please review the commit and tell me if I need to make any modifications. Here are the notes from the commit:

    Fixed #24717 -- Fixed test failures for RHEL6 SCL
    
    The test test_unpickling_when_appregistrynotready in the package
    model_regress.test_pickle fails on RHEL6 systems running python 2.7
    from a RedHat Software Collection (SCL) because this test runs an
    external python script with a stripped system environment. RedHat SCLs
    work by setting a number of system environment variables when these are
    stripped out by this test the python 2.7 interpreter is no longer able
    to function properly because it can not find the system libraries needed
    from the SCL. The solution I propose is to use mock to modify the system
    environment directly instead of passing a stripped environment. I tested
    this commit using using python 3.4.3 on Windows 7, and python 2.7 on
    Windows 7, Debian 7.8, and RHEL6 with SCLs enabled.
    
    Note that on Unix/Linux systems the code as it was before could also
    break on local installs of python to a home directory or /usr/local for
    example since it was not passing variables such as LD_LIBRARY_PATH to
    the python script it executes.

[Tim Graham]

Left some comments on the ​pull request.

[David D. Riddle]

I made the change you requested.

[Tim Graham <timograham@…>]

In 31e6c9c8:

Fixed #24717 -- Fixed model_regress test failure on RHEL6 SCL

The test failed on RHEL6 systems running python 2.7 from a RedHat
Software Collection (SCL) because this test runs an external python
script with a stripped system environment. RedHat SCLs work by setting
a number of system environment variables when these are stripped out by
this test the python 2.7 interpreter is no longer able to function
properly because it can not find the system libraries needed.

Now we use use mock to modify the system environment directly.

[Tim Graham <timograham@…>]

In 8c58caaf:

[1.8.x] Fixed #24717 -- Fixed model_regress test failure on RHEL6 SCL

The test failed on RHEL6 systems running python 2.7 from a RedHat
Software Collection (SCL) because this test runs an external python
script with a stripped system environment. RedHat SCLs work by setting
a number of system environment variables when these are stripped out by
this test the python 2.7 interpreter is no longer able to function
properly because it can not find the system libraries needed.

Now we use use mock to modify the system environment directly.

Backport of 31e6c9c8e310e68a96c1f76b9657242c6cecaa4d from master

[Tim Graham <timograham@…>]

In 29e90f3:

Fixed model_regress test failure on non-ASCII path; refs #24717.

[Tim Graham <timograham@…>]

In 556b1c8:

[1.8.x] Fixed model_regress test failure on non-ASCII path; refs #24717.

Backport of 29e90f3ae0b0b72bbdf342e0136414bff917c6a6 from master