PermissionError due to unjustified attempt to access '/usr'.

[tytusd]

After updating Django from 3.0.8 to 3.1.0 it became unusable - I cannot call any manage.py management command in the production environment, where the user does not have root access and has very limited access to the /usr directory (it is a managed environment). Normally everything works fine, as Django projects run within virtual environments. Unfortunately, after the update, the Django code clearly attempts to access /usr directory (even though it is running inside of a virtual environment located elsewhere, within the user home directory). After downgrading back to 3.0.8 everything works fine again.

Stack trace below:

(web) [XYZ@s39]:<~/domains/XXX/public_python>$ python manage.py migrate
Traceback (most recent call last):
  File "manage.py", line 21, in <module>
    main()
  File "manage.py", line 17, in main
    execute_from_command_line(sys.argv)
  File "/usr/home/XYZ/.virtualenvs/web/lib/python3.6/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/usr/home/XYZ/.virtualenvs/web/lib/python3.6/site-packages/django/core/management/__init__.py", line 377, in execute
    django.setup()
  File "/usr/home/XYZ/.virtualenvs/web/lib/python3.6/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/home/XYZ/.virtualenvs/web/lib/python3.6/site-packages/django/apps/registry.py", line 114, in populate
    app_config.import_models()
  File "/usr/home/XYZ/.virtualenvs/web/lib/python3.6/site-packages/django/apps/config.py", line 211, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/home/XYZ/.virtualenvs/web/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/home/XYZ/.virtualenvs/web/lib/python3.6/site-packages/django/contrib/auth/models.py", line 2, in <module>
    from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
  File "/usr/home/XYZ/.virtualenvs/web/lib/python3.6/site-packages/django/contrib/auth/base_user.py", line 8, in <module>
    from django.contrib.auth import password_validation
  File "/usr/home/XYZ/.virtualenvs/web/lib/python3.6/site-packages/django/contrib/auth/password_validation.py", line 160, in <module>
    class CommonPasswordValidator:
  File "/usr/home/XYZ/.virtualenvs/web/lib/python3.6/site-packages/django/contrib/auth/password_validation.py", line 170, in CommonPasswordValidator
    DEFAULT_PASSWORD_LIST_PATH = Path(__file__).resolve(strict=True).parent / 'common-passwords.txt.gz'
  File "/usr/local/lib/python3.6/pathlib.py", line 1141, in resolve
    s = self._flavour.resolve(self, strict=strict)
  File "/usr/local/lib/python3.6/pathlib.py", line 346, in resolve
    return _resolve(base, str(path)) or sep
  File "/usr/local/lib/python3.6/pathlib.py", line 330, in _resolve
    target = accessor.readlink(newpath)
  File "/usr/local/lib/python3.6/pathlib.py", line 440, in readlink
    return os.readlink(path)
PermissionError: [Errno 13] Permission denied: '/usr'
(web) [XYZ@s39]:<~/domains/XXX/public_python>$ pip install Django==3.0.8
Collecting Django==3.0.8
  Downloading Django-3.0.8-py3-none-any.whl (7.5 MB)
     |████████████████████████████████| 7.5 MB 4.5 MB/s 
Requirement already satisfied: sqlparse>=0.2.2 in /usr/home/XYZ/.virtualenvs/web/lib/python3.6/site-packages (from Django==3.0.8) (0.3.1)
Requirement already satisfied: asgiref~=3.2 in /usr/home/XYZ/.virtualenvs/web/lib/python3.6/site-packages (from Django==3.0.8) (3.2.10)
Requirement already satisfied: pytz in /usr/home/XYZ/.virtualenvs/web/lib/python3.6/site-packages (from Django==3.0.8) (2020.1)
Installing collected packages: Django
  Attempting uninstall: Django
    Found existing installation: Django 3.1
    Uninstalling Django-3.1:
      Successfully uninstalled Django-3.1
Successfully installed Django-3.0.8
(web) [XYZ@s39]:<~/domains/XXX/public_python>$ python manage.py migrate
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, database, pages, sessions
Running migrations:
  No migrations to apply.
(web) [XYZ@s39]:<~/domains/XXX/public_python>$

[Mariusz Felisiak]

Duplicate of #31912 (both are related with PermissionError raised by Path.resolve().

[Carlton Gibson]

OK, let's re-open just to take a look. Still seems like a permission issue to me, but it's come up twice, so let's double-check.

edeec1247e52de6fc32cee93e96d4ce36003ea4b added the strict parameter. Does removing that solve the issue?

I'll CC Jon, who made most of the pathlib related updates here.

[Carlton Gibson]

Sorry. Wrong ticket. Doh.

[Mariusz Felisiak]

It's a separate issue, accepted based on comment#6 and comment#8. I think we should remove strict=True from CommonPasswordValidator.

Regression in edeec1247e52de6fc32cee93e96d4ce36003ea4b.

[Claude Paroz]

Maybe someone could report an issue in the Python documentation. The strict=True documentation doesn't say anything about permissions on intermediate directories.

[Mariusz Felisiak]

​PR

[Mariusz Felisiak]

OK let's fix both and keep them in the first ticket, sorry for the noise.