Opened 17 months ago

Last modified 8 months ago

#22446 new Cleanup/optimization

Add tox support

Reported by: jmbowman Owned by: jmbowman
Component: Testing framework Version: master
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

tox (https://testrun.org/tox/) makes it easier to run tests in multiple versions of Python, either all in the same test run or one at a time (among other features). Given that Django currently supports multiple Python versions straddling 2.x and 3.x, making this easier seems like a significant win. Key points:

  • A tox.ini file would be added which defines all the tox environments
  • The default environment could be run just withtox
  • Specific versions could be run via tox -e py27, tox -e py33, tox -e pypy, etc.
  • Multiple versions could be tested in a single run via commands like tox -e py27,py32,py33,pypy or tox -e ALL (especially handy if you'll be stepping away from your computer for a while)
  • The desired python interpreter(s) would need to be installed beforehand, but tox itself would create the virtualenv(s) and install any necessary dependencies into it
  • Documentation could be built with tox -e docs
  • In comparison to the current documented commands for running tests, this would be faster to type and easier to remember
  • The documentation would be updated to mention this as one way (the recommended way?) to run tests and build documentation
  • No other changes would be needed beyond the creation of tox.ini, so all the old ways of running tests and such would still work unchanged (tox itself would still run those commands)

One attempt at adding tox support was made 2 years ago (https://github.com/django/django/pull/110), but had been lumped together with TravisCI support and was rejected on those grounds. It seems more valuable now with the desire to be able to test Python 2, Python 3, and PyPy (and others?) easily.

Change History (5)

comment:1 Changed 17 months ago by charettes

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Version changed from 1.6 to master

As discussed with Jannis, who closed the referenced PR, at PyCon2014 sprints we might want to move this discussion the django-developers mailing list.

While it could be useful to provide a simple way for contributors to run the test suite against multiple versions of Python we'd need to add a big warning this is by no mean a way of achieve (given all the supported environments {Unix, Win} X {SQLite3, MySQL, PostGres, Oracle, ...} X {Py2.7, ..., Py3.4})

In my case I would definitely leverage this when reviewing patches, I actually have a tox.ini file I keep of VCS tracking using .git/info/exclude.

Last edited 17 months ago by charettes (previous) (diff)

comment:2 Changed 17 months ago by timo

  • Easy pickings unset

I'm in favor of recommending the djangocore-box, but tox could probably be leveraged there as well.

comment:3 Changed 16 months ago by timo

  • Triage Stage changed from Unreviewed to Accepted

Accepting the idea of exploration in this area.

comment:4 Changed 8 months ago by collinanderson

Is this still needed now that we have http://djangoci.com/?

Not sure what the approach would be now. Maybe a docker container with all of the databases (kind of outside the scope of django itself)? Or an ansible script to set it all up?

comment:5 Changed 8 months ago by berkerpeksag

+1 for tox support. tox is really helpful when you want to run specific tests on multiple Python versions.

Here is my tox.ini as a reference point: https://gist.github.com/berkerpeksag/1f644aa3454f99bb4c51 (It doesn't support different backends such as MySQL.)

Note: See TracTickets for help on using tickets.
Back to Top