id,summary,reporter,owner,description,type,status,component,version,severity,resolution,keywords,cc,stage,has_patch,needs_docs,needs_tests,needs_better_patch,easy,ui_ux 31431,Coverage integration with multiprocessed Django tests.,Ichlasul Affan,Ichlasul Affan,"Currently, to use {{{Coverage.py}}} for line coverage analysis on Django tests, we can use this command: {{{ coverage run --omit='....' manage.py test ..... coverage report -m }}} This method is valid on single-processed testing. {{{Coverage.py}}} will analyze which lines that Python interpreter has gone through when the testing process runs. Unfortunately, on multi-processed testing, coverage analysis result becomes invalid as it marks some lines as ""uncovered"", given the same code and test suites. Based on my initial observation, looks like the {{{Coverage.py}}} only analyze lines run by main Python interpreter process. The child processes are untouched. So, only tests run by the first process that are being analyzed. For example, I have two different test suites: - {{{AccountTests}}} for {{{account}}} app, and - {{{HistoryTests}}} for {{{history}}} app. Let's say the first test process will run {{{AccountTests}}}. The result will only mark as ""covered"" on code lines in {{{account}}} app, and some of the {{{models.py}}}, {{{urls.py}}}, and {{{apps.py}}} code of {{{history}}} app. The latter happened because Django initialized App registry, test models, and migrations using codes in {{{apps.py}}} and {{{models.py}}}. The parent {{{urls.py}}} also imports {{{urls.py}}} inside each app. These are subfeatures that I propose in this ticket: 1. Integration of Coverage.py into Django Test engine, so we can use coverage analysis by using: {{{python manage.py test --check-coverage --coverage-omit='....' ......}}}. The execution of Coverage.py is now handled by Django test using different test runner called {{{CoverageTestRunner}}}, that extends the ordinary {{{DiscoverRunner}}}. 2. Implementation of {{{CoverageTestRunner}}} itself, by utilizing {{{Coverage.py}}}'s own API. This includes initialization process for coverage analysis inside the test runner. 3. Implementation of multiprocessed test runner support. It might be implemented using different init worker than the usual {{{ParallelTestSuite}}}, or there might be any better idea in the future. When merging test results into parent process, it will also need to merge the coverage analysis result of each processes. I have no idea yet for a concrete Proof of Concept. The main idea is initializing coverage analysis API for each of test processes, then merge the results using {{{OR}}} operator. That said, a line counts as ""covered"" when it is executed by minimum of one test process. Any suggestions about this idea will be very much appreciated.",New feature,closed,Testing framework,dev,Normal,wontfix,coverage; parallel testing,,Unreviewed,0,0,0,0,0,0