Opened 2 years ago

Closed 2 months ago

#22446 closed Cleanup/optimization (fixed)

Add tox support

Reported by: jmbowman Owned by: Tobias McNulty
Component: Testing framework Version: master
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

tox (https://testrun.org/tox/) makes it easier to run tests in multiple versions of Python, either all in the same test run or one at a time (among other features). Given that Django currently supports multiple Python versions straddling 2.x and 3.x, making this easier seems like a significant win. Key points:

  • A tox.ini file would be added which defines all the tox environments
  • The default environment could be run just withtox
  • Specific versions could be run via tox -e py27, tox -e py33, tox -e pypy, etc.
  • Multiple versions could be tested in a single run via commands like tox -e py27,py32,py33,pypy or tox -e ALL (especially handy if you'll be stepping away from your computer for a while)
  • The desired python interpreter(s) would need to be installed beforehand, but tox itself would create the virtualenv(s) and install any necessary dependencies into it
  • Documentation could be built with tox -e docs
  • In comparison to the current documented commands for running tests, this would be faster to type and easier to remember
  • The documentation would be updated to mention this as one way (the recommended way?) to run tests and build documentation
  • No other changes would be needed beyond the creation of tox.ini, so all the old ways of running tests and such would still work unchanged (tox itself would still run those commands)

One attempt at adding tox support was made 2 years ago (https://github.com/django/django/pull/110), but had been lumped together with TravisCI support and was rejected on those grounds. It seems more valuable now with the desire to be able to test Python 2, Python 3, and PyPy (and others?) easily.

Change History (11)

comment:1 Changed 2 years ago by Simon Charette

Needs documentation: unset
Needs tests: unset
Patch needs improvement: unset
Version: 1.6master

As discussed with Jannis, who closed the referenced PR, at PyCon2014 sprints we might want to move this discussion the django-developers mailing list.

While it could be useful to provide a simple way for contributors to run the test suite against multiple versions of Python we'd need to add a big warning this is by no mean a way of achieve (given all the supported environments {Unix, Win} X {SQLite3, MySQL, PostGres, Oracle, ...} X {Py2.7, ..., Py3.4})

In my case I would definitely leverage this when reviewing patches, I actually have a tox.ini file I keep of VCS tracking using .git/info/exclude.

Last edited 2 years ago by Simon Charette (previous) (diff)

comment:2 Changed 2 years ago by Tim Graham

Easy pickings: unset

I'm in favor of recommending the djangocore-box, but tox could probably be leveraged there as well.

comment:3 Changed 2 years ago by Tim Graham

Triage Stage: UnreviewedAccepted

Accepting the idea of exploration in this area.

comment:4 Changed 21 months ago by Collin Anderson

Is this still needed now that we have http://djangoci.com/?

Not sure what the approach would be now. Maybe a docker container with all of the databases (kind of outside the scope of django itself)? Or an ansible script to set it all up?

comment:5 Changed 21 months ago by Berker Peksag

+1 for tox support. tox is really helpful when you want to run specific tests on multiple Python versions.

Here is my tox.ini as a reference point: https://gist.github.com/berkerpeksag/1f644aa3454f99bb4c51 (It doesn't support different backends such as MySQL.)

comment:6 Changed 4 months ago by Tobias McNulty

(Also made a similar post to #19891)

I created a "lightweight" Travis/tox setup for my own purposes so I can check pull requests before submitting them to Django proper. The goal is not to replace djangoci.com but to provide a lightweight sanity check and help avoid clogging up Travis when making frequent commits.

If there is a desire to include a tox.ini with Django itself, this file might work as a foundation: https://github.com/tobiasmcnulty/django/blob/travis/tox.ini

As I type this, I'm wondering if we could supply a generic tox.ini that includes all the builds Jenkins runs. Then Jenkins could run tox -e for each applicable environment, and anyone wanting to run their own Travis build server could do the same for whatever environments they care about.

In the meantime, anyone who stumbles across this ticket is welcome to copy what I did to your own GitHub/Travis account or submit a PR to my fork to see if it builds, before submitting to Django proper (Jenkins will only fully build your PRs if your username is on the whitelist). I will not, of course, review your PR or merge it into this fork. There are no guarantees that it'll be completely up to date, etc. (create a fork for yourself if needed).

The repo is here, on the default "travis" branch: https://github.com/tobiasmcnulty/django

comment:7 Changed 3 months ago by Tobias McNulty

Owner: changed from jmbowman to Tobias McNulty
Status: newassigned

comment:8 Changed 3 months ago by Tobias McNulty

Has patch: set

comment:9 Changed 3 months ago by Tim Graham

Patch needs improvement: set

Comments for improvement on the PR.

comment:10 Changed 2 months ago by Tim Graham

Patch needs improvement: unset

comment:11 Changed 2 months ago by Tim Graham <timograham@…>

Resolution: fixed
Status: assignedclosed

In 09d3874:

Fixed #22446 -- Added tox.ini to automate pull request checks.

Note: See TracTickets for help on using tickets.
Back to Top