Django cannot load when Python is compiled with --without-doc-strings enabled

[Jon Janzen]

I'm not sure that this is even a supported configuration for Django, but CPython's ./configure script offers an option called ​--without-doc-strings which improves memory utilization slightly.

Quoting the relevant portion from CPython docs:

--without-doc-strings
Disable static documentation strings to reduce the memory footprint (enabled by default). Documentation strings defined in Python are not affected.

I have an use-case where I need to deploy a Django service on a device with low available memory. I'd like to disable built-in doc strings as part of an overall strategy to reduce memory but compiling CPython with that option and running the django service crashes:

  File ".../asgi.py", line 3, in <module>
    from django.core.asgi import get_asgi_application
  File ".../lib/python3.11/site-packages/django/core/asgi.py", line 2, in <module>
    from django.core.handlers.asgi import ASGIHandler
  File ".../lib/python3.11/site-packages/django/core/handlers/asgi.py", line 11, in <module>
    from django.core.handlers import base
  File ".../lib/python3.11/site-packages/django/core/handlers/base.py", line 11, in <module>
    from django.urls import get_resolver, set_urlconf
  File ".../lib/python3.11/site-packages/django/urls/__init__.py", line 1, in <module>
    from .base import (
  File ".../lib/python3.11/site-packages/django/urls/base.py", line 8, in <module>
    from .exceptions import NoReverseMatch, Resolver404
  File ".../lib/python3.11/site-packages/django/urls/exceptions.py", line 1, in <module>
    from django.http import Http404
  File ".../lib/python3.11/site-packages/django/http/__init__.py", line 2, in <module>
    from django.http.request import (
  File ".../lib/python3.11/site-packages/django/http/request.py", line 8, in <module>
    from django.core import signing
  File ".../lib/python3.11/site-packages/django/core/signing.py", line 43, in <module>
    from django.utils.crypto import constant_time_compare, salted_hmac
  File ".../lib/python3.11/site-packages/django/utils/crypto.py", line 85, in <module>
    if func_supports_parameter(hashlib.md5, 'usedforsecurity'):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/django/utils/inspect.py", line 74, in func_supports_parameter
    return any(param.name == name for param in _get_callable_parameters(func))
                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/django/utils/inspect.py", line 16, in _get_callable_parameters
    return _get_func_parameters(func, remove_first=is_method)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/django/utils/inspect.py", line 7, in _get_func_parameters
    parameters = tuple(inspect.signature(func).parameters.values())
                       ^^^^^^^^^^^^^^^^^^^^^^^
  File ".../cpython/Lib/inspect.py", line 3270, in signature
    return Signature.from_callable(obj, follow_wrapped=follow_wrapped,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../cpython/Lib/inspect.py", line 3018, in from_callable
    return _signature_from_callable(obj, sigcls=cls,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../cpython/Lib/inspect.py", line 2510, in _signature_from_callable
    return _signature_from_builtin(sigcls, obj,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../cpython/Lib/inspect.py", line 2317, in _signature_from_builtin
    raise ValueError("no signature found for builtin {!r}".format(func))
ValueError: no signature found for builtin <built-in function openssl_md5>

(irrelevant information lightly censored)

An easy way to repro is to follow this script:

git clone https://github.com/python/cpython.git
cd cpython
git checkout 3.11 # or whatever version you have that already has django installed
./configure --without-doc-strings
make -j
./python -c "import django.utils.crypto"

Looking through this stack trace the problem looks to be ​some code that executes at module import to determine whether or not md5 supports the usedforsecurity parameter.

If this ticket is accepted I'm happy to put up a PR to fix this, I've included my proposed solution on this ticket to save a roundtrip in discussion:

diff --git a/django/utils/inspect.py b/django/utils/inspect.py
index 28418f7312..d8622a22df 100644
--- a/django/utils/inspect.py
+++ b/django/utils/inspect.py
@@ -70,4 +70,12 @@ def method_has_no_args(meth):
 
 
 def func_supports_parameter(func, name):
-    return any(param.name == name for param in _get_callable_parameters(func))
+    try:
+        callable_parameters = _get_callable_parameters(func)
+    except ValueError:
+        # When Python is compiled with the --without-doc-strings
+        # argument to ./configure the signatures for built-in
+        # functions are not available. In such a case, be
+        # conservative and assume no:
+        return False
+    return any(param.name == name for param in callable_parameters)

[Mariusz Felisiak]

Thanks for this report, however it seems to be an issue in Python not in Django itself. According to Python's docs, the ​--without-doc-strings option should not affect builtins. Moreover, we check signatures in other places, I don't think it's worth the extra complexity.

[Jon Janzen]

Hey Mariusz,

According to Python's docs, the ​--without-doc-strings option should not affect builtins.

I must be missing something because as I read the docs you linked it fails to mention this limitation, but that is not the same thing as saying that it should not affect builtins.

In fact, when you read the CPython source code there are a few places that make it more explicit. For example, the tests for parsing the signatures of builtins are disabled when docstrings are disabled:

​https://github.com/python/cpython/blob/7a0f3c1d92ef0768e082ace19d970b0ef12e7346/Lib/test/test_capi/test_misc.py#L164

Additionally, there is some commentary in-lined into the inspect module about this:

​https://github.com/python/cpython/blob/7a0f3c1d92ef0768e082ace19d970b0ef12e7346/Lib/inspect.py#L2528-L2534

I'll open a ticket with CPython when I have a chance to ask them to document the limitation, but this is an intentional part of compiling with --without-doc-strings.

Moreover, we check signatures in other places, I don't think it's worth the extra complexity.

Is this referring to the specific solution I offered or the concept of fixing this issue overall?

Sorry if reopening isn't the proper procedure, I'm still trying to learn my way around Django's process

[Jon Janzen]

CPython issue xref: ​https://github.com/python/cpython/issues/100150

[Mariusz Felisiak]

Is this referring to the specific solution I offered or the concept of fixing this issue overall?

Almost all hooks in the django.utils.inspect module are affected, not just func_supports_parameter(). Adding official support for CPython compiled with --without-doc-strings would required catching ValueError in all of them. I still don't think it's worth the extra complexity and it would be clunky and confusing for users as we cannot return false positives/negatives in all these places.

I appreciate you'd like to reopen the ticket, but please ​follow the triaging guidelines with regards to wontfix tickets and take this to DevelopersMailingList, where you'll reach a wider audience and see what other think,