Opened 8 years ago

Closed 8 years ago

Last modified 7 years ago

#26677 closed Cleanup/optimization (fixed)

Run i18n tests on disposable FS tree instead of source code

Reported by: Ramiro Morales Owned by: Ramiro Morales
Component: Internationalization Version: dev
Severity: Normal Keywords: tests isolation i18n serializemixin unit unittest
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Ramiro Morales)

Currently, tests that exercise translatable literal extraction to .po files by makemessages (located on test_extraction.py) and test that exercise compilation of .po files to .mo files by compilemessages (test_compilation.py) are run under the test/i18n test source code tree creating subdirectories and files under the different work projects/applications located there.

Proposal is to copy the relevant filesystem tree to a tempdir as provided by runtests.py + tempfile.mkdtemp() as part of every test case setup.

This has the following advantages:

  • Avoid weird bogus errors and failures caused by stray directories left by previous runs/debugging sessions.
  • Allows us to solve the fact that currently tests on test_compilation.py aren't marked as needing seralized execution and are actually being run in parallel. This, for instance, causes a couple of consecutive runs of the i18n test on my Ubuntu 1.4.04 system to report misleading failures, most probably because of race conditions.
  • Ensure actual isolation among test cases
  • Allow each of these test cases to be actually run in in parallel by our existing machinery. This allows us to be able to stop relying on django.test.testcases.SerializeMixin

These changes possibly aren't visible on our CI infrastructure as every job there always use a clean tree but can be of help to those running the test suite and/or working on i18n tests.

Other tests (tests.py, test_percents.py) don't have write access to disk so they are unaffected.

Change History (14)

comment:1 by Ramiro Morales, 8 years ago

comment:2 by Ramiro Morales, 8 years ago

Owner: changed from nobody to Ramiro Morales
Status: newassigned

comment:3 by Ramiro Morales, 8 years ago

Summary: Run i18n tests on disposablle FS tree instead of source codeRun i18n tests on disposable FS tree instead of source code

comment:4 by Tim Graham, 8 years ago

Component: Testing frameworkInternationalization
Has patch: set
Triage Stage: UnreviewedAccepted

comment:5 by Ramiro Morales, 8 years ago

Description: modified (diff)

comment:6 by Tim Graham, 8 years ago

Patch needs improvement: set

Comment for improvement on the PR.

comment:7 by Aymeric Augustin, 8 years ago

I did that in a bunch of other places when I implemented test parallelization. To test it, I just made the entire source tree readonly, ran the tests and saw what broke.

I gave up for i18n tests because it required too extensive changes. It's great to see that you're picking up the flag! Indeed, once you've managed to isolate each test in its own temporary directory, you can remove SerializeMixin, and the tests will be faster.

While you're there you could check how much I/O these tests perform. I/O may be a bottleneck even with a SSD e.g. if each test copies 50 files where 5 would suffice. IIRC the total runtime of the i18n tests was about 4 seconds with a SSD, so there isn't a huge lot to gain and I didn't spend much time optimizing it. Over a few thousands runs, improvements would compound, though.

in reply to:  7 comment:8 by Ramiro Morales, 8 years ago

Patch needs improvement: unset

Replying to aaugustin:

Indeed, once you've managed to isolate each test in its own temporary directory, you can remove SerializeMixin, and the tests will be faster.

Exactly, that's one of the included changes.

While you're there you could check how much I/O these tests perform. I/O may be a bottleneck even with a SSD e.g. if each test copies 50 files where 5 would suffice. IIRC the total runtime of the i18n tests was about 4 seconds with a SSD, so there isn't a huge lot to gain and I didn't spend much time optimizing it. Over a few thousands runs, improvements would compound, though.

Will do some testing. No SDD here but will see what conclussion can be made.

Thanks!

comment:9 by Tim Graham, 8 years ago

Triage Stage: AcceptedReady for checkin

comment:10 by Ramiro Morales <cramm0@…>, 8 years ago

Resolution: fixed
Status: assignedclosed

In faeeb84e:

Fixed #26677 -- Converted some i18n tests to use disposable FS tree.

This allows makemessages/compilemessages tests in test_extraction.py
and test_compilation.py to actually run isolated from each other
(unaffected by stray FS objects left by cleanup actions failures, debug
sessions, etc.) and to take advantage of the parallel tests execution
feature like most of the Django test suite.

test_percents.py gets slightly refactored to not inherit from the new
machinery which sets up every test case to copy and run under a
temporary tree.

comment:11 by Ramiro Morales <cramm0@…>, 8 years ago

In bb7bb37:

Refs #26677 -- Simplified i18n test cleanups.

The fact that we aren't dealing with the Django source tree anymore
allows us to drop several tearDown()/addCleanup() calls that were
concerned with removing apiece files/dirs/symlinks created by test
cases, as we are covered by the removal of the parent temporary tree
anyways.

Thanks Tim Graham for advice and review.

comment:12 by Tim Graham <timograham@…>, 7 years ago

In c1bd4679:

[1.10.x] Fixed #26677 -- Converted some i18n tests to use disposable FS tree.

This allows makemessages/compilemessages tests in test_extraction.py
and test_compilation.py to actually run isolated from each other
(unaffected by stray FS objects left by cleanup actions failures, debug
sessions, etc.) and to take advantage of the parallel tests execution
feature like most of the Django test suite.

test_percents.py gets slightly refactored to not inherit from the new
machinery which sets up every test case to copy and run under a
temporary tree.

Backport of faeeb84edfebecf5a5f40df9ef816e5f1cd457c6 from master

comment:13 by Tim Graham <timograham@…>, 7 years ago

In a079d5ce:

[1.10.x] Refs #26677 -- Simplified i18n test cleanups.

The fact that we aren't dealing with the Django source tree anymore
allows us to drop several tearDown()/addCleanup() calls that were
concerned with removing apiece files/dirs/symlinks created by test
cases, as we are covered by the removal of the parent temporary tree
anyways.

Thanks Tim Graham for advice and review.

Backport of bb7bb379e8cd91a91336946829519d64e919a1d2 from master

comment:14 by Tim Graham, 7 years ago

The commits were backported to 1.10.x at the request of #27526.

Note: See TracTickets for help on using tickets.
Back to Top