Opened 54 minutes ago
#37106 new Uncategorized
pylibmc install fails from tests requirements on Python 3.12+
| Reported by: | Kiko Barr | Owned by: | |
|---|---|---|---|
| Component: | Documentation | Version: | 6.0 |
| Severity: | Normal | Keywords: | test, pylibmc |
| Cc: | Kiko Barr | Triage Stage: | Unreviewed |
| Has patch: | no | Needs documentation: | no |
| Needs tests: | no | Patch needs improvement: | no |
| Easy pickings: | no | UI/UX: | no |
Description
π Hello from the Django sprint at PyCon 2026!
High Level Overview
When following the Unit Tests walkthrough, python -m pip install -r tests/requirements/py3.txt is failing to install pylibmc. This is because Django pyproject.toml requires CPython 3.12+, but wheel for pylimbc hosted on PyPI only goes up to 3.11. See PyPI: βhttps://pypi.org/project/pylibmc/#files
Six people at the sprint encountered this same issue. I think this is important to document because new contributors are encouraged to start by running tests and then they run into this issue. While many of the tests can still be run, some will produce errors if pylibmc is not pre-installed.
There have been several previous Django-related posts discuss removing pylibmc as a dependency, but have not come to a conclusion. While the discussion is occurring, I think it would be good to update the test documentation with some short-term workarounds in the meantime.
Django closed issues:
Django forum posts:
- βhttps://forum.djangoproject.com/t/error-when-installing-requirements-for-django-tests-fatal-error-libmemcached-memcached-h-file-not-found-include-libmemcached-memcached-h/1685
- βhttps://forum.djangoproject.com/t/should-we-stop-recommending-pylibmc/42993
pylibmc issue:
Reproducing the Issue
The issue appears while running commands from the "Quickstart" instruction in the "Unit Tests" documentation, found here: βhttps://docs.djangoproject.com/en/dev/internals/contributing/writing-code/unit-tests/#quickstart
Based on the documentation, I created a virtual environment and ran these commands:
git clone https://github.com/YourGitHubName/django.git django-repo # created and activated the venv here and ran the rest of the commands cd django-repo/tests python -m pip install -e ..
However, the next command will fail:
python -m pip install -r requirements/py3.txt
This is the error message:
Building wheels for collected packages: pylibmc
Building wheel for pylibmc (pyproject.toml) ... error
error: subprocess-exited-with-error
Γ Building wheel for pylibmc (pyproject.toml) did not run successfully.
β exit code: 1
β°β> [6 lines of output]
In file included from src/_pylibmcmodule.c:34:
src/_pylibmcmodule.h:42:10: fatal error: 'libmemcached/memcached.h' file not found
42 | #include <libmemcached/memcached.h>
| ^~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
error: command '/usr/bin/clang' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pylibmc
Failed to build pylibmc
error: failed-wheel-build-for-install
Γ Failed to build installable wheels for some pyproject.toml based projects
β°β> pylibmc
Issue Description
Issue for those running Python 3.12 with no libmemcached installed:
- When running
python -m pip install -r requirements/py3.txt, pip readsrequirements/py3.txtand sees pylibmc. - pip queries PyPI for pylibmc distributions, including wheel files (.whl) and source archive (.tar.gz / .zip, called sdist)
- Since there is no wheel for Python 3.12, pip does not use the wheel install and instead downloads the sdist to build pylibmc locally.
- The source build compiles C code for pylibmc, which needs libmemcached headers/libraries. If libmemcached is not installed, then this step fails, producing the error from above,
src/_pylibmcmodule.h:42:10: fatal error: 'libmemcached/memcached.h' file not found
Additional issue for some on macOS:
- Even after libmemcached was installed globally, people on macOS continued to run on the same error. This is because Homebrew, a very common package manager for macOS, often installs under a non-default prefix (usually /opt/homebrew). If that prefix isnβt in the compiler/linker search paths for that shell/build, compile fails even though libmemcached is installed.
- People on macOS had to run additional commands to direct clang to the right path using
CPPFLAGSto tell compiler where headers are andLDFLAGSto tell linker where libs are.
Short-term Workaround
Of the six people at the sprint who encountered this same issue, there were several different versions of the same workaround.
For four of us on Macs, we needed to first install libmemcached headers (someone else also had to install memcached, zlib, and pkg-config to get it to work, which may be worth looking into):
# install globally outside of venv brew install libmemcached # run these inside of the venv export CPPFLAGS="-I$(brew --prefix)/include" export LDFLAGS="-L$(brew --prefix)/lib" python -m pip install -r requirements/py3.txt
Someone on Linux Omarchy only had to run:
sudo pacman -S libmemcached-awesome
Someone else on Linux Debian ran:
sudo apt install libmemcached-dev
Listing all these variations for different operating systems is not the most elegant solution, but it does facilitate a smoother onboarding experience for new contributors.