Opened 11 years ago
Closed 8 years ago
#22446 closed Cleanup/optimization (fixed)
Add tox support
Reported by: | Jeremy Bowman | Owned by: | Tobias McNulty |
---|---|---|---|
Component: | Testing framework | Version: | dev |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Accepted | |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
tox (https://testrun.org/tox/) makes it easier to run tests in multiple versions of Python, either all in the same test run or one at a time (among other features). Given that Django currently supports multiple Python versions straddling 2.x and 3.x, making this easier seems like a significant win. Key points:
- A tox.ini file would be added which defines all the tox environments
- The default environment could be run just with
tox
- Specific versions could be run via
tox -e py27
,tox -e py33
,tox -e pypy
, etc. - Multiple versions could be tested in a single run via commands like
tox -e py27,py32,py33,pypy
ortox -e ALL
(especially handy if you'll be stepping away from your computer for a while) - The desired python interpreter(s) would need to be installed beforehand, but tox itself would create the virtualenv(s) and install any necessary dependencies into it
- Documentation could be built with
tox -e docs
- In comparison to the current documented commands for running tests, this would be faster to type and easier to remember
- The documentation would be updated to mention this as one way (the recommended way?) to run tests and build documentation
- No other changes would be needed beyond the creation of tox.ini, so all the old ways of running tests and such would still work unchanged (tox itself would still run those commands)
One attempt at adding tox support was made 2 years ago (https://github.com/django/django/pull/110), but had been lumped together with TravisCI support and was rejected on those grounds. It seems more valuable now with the desire to be able to test Python 2, Python 3, and PyPy (and others?) easily.
Change History (11)
comment:2 by , 11 years ago
Easy pickings: | unset |
---|
I'm in favor of recommending the djangocore-box, but tox could probably be leveraged there as well.
comment:3 by , 11 years ago
Triage Stage: | Unreviewed → Accepted |
---|
Accepting the idea of exploration in this area.
comment:4 by , 10 years ago
Is this still needed now that we have http://djangoci.com/?
Not sure what the approach would be now. Maybe a docker container with all of the databases (kind of outside the scope of django itself)? Or an ansible script to set it all up?
comment:5 by , 10 years ago
+1 for tox support. tox is really helpful when you want to run specific tests on multiple Python versions.
Here is my tox.ini
as a reference point: https://gist.github.com/berkerpeksag/1f644aa3454f99bb4c51 (It doesn't support different backends such as MySQL.)
comment:6 by , 8 years ago
(Also made a similar post to #19891)
I created a "lightweight" Travis/tox setup for my own purposes so I can check pull requests before submitting them to Django proper. The goal is not to replace djangoci.com but to provide a lightweight sanity check and help avoid clogging up Travis when making frequent commits.
If there is a desire to include a tox.ini with Django itself, this file might work as a foundation: https://github.com/tobiasmcnulty/django/blob/travis/tox.ini
As I type this, I'm wondering if we could supply a generic tox.ini that includes all the builds Jenkins runs. Then Jenkins could run tox -e for each applicable environment, and anyone wanting to run their own Travis build server could do the same for whatever environments they care about.
In the meantime, anyone who stumbles across this ticket is welcome to copy what I did to your own GitHub/Travis account or submit a PR to my fork to see if it builds, before submitting to Django proper (Jenkins will only fully build your PRs if your username is on the whitelist). I will not, of course, review your PR or merge it into this fork. There are no guarantees that it'll be completely up to date, etc. (create a fork for yourself if needed).
The repo is here, on the default "travis" branch: https://github.com/tobiasmcnulty/django
comment:7 by , 8 years ago
Owner: | changed from | to
---|---|
Status: | new → assigned |
comment:10 by , 8 years ago
Patch needs improvement: | unset |
---|
As discussed with Jannis, who closed the referenced PR, at PyCon2014 sprints we might want to move this discussion the django-developers mailing list.
While it could be useful to provide a simple way for contributors to run the test suite against multiple versions of Python we'd need to add a big warning this is by no mean a way of achieve (given all the supported environments {Unix, Win} X {SQLite3, MySQL, PostGres, Oracle, ...} X {Py2.7, ..., Py3.4})
In my case I would definitely leverage this when reviewing patches, I actually have a
tox.ini
file I keep of VCS tracking using.git/info/exclude
.