Opened 12 years ago

Last modified 9 years ago

#18586 closed Cleanup/optimization

Rewrite unit tests migrated from doctests — at Version 16

Reported by: Michal Petrucha Owned by: Mark Smith
Component: Testing framework Version: dev
Severity: Normal Keywords: unit tests
Cc: Peter Zsoldos, amizya@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: no

Description (last modified by Julien Phalip)

There's quite a lot of unit tests that have been rewritten 1:1 from doctests. These tests tend to verify all aspects of a certain feature at once, they modify the state of the database throughout the whole test and by the end of the test no one really knows what the database state is supposed to be.

For example, modeltests.generic_relations.GenericRelationsTests.test_generic_relations walks through most of the features supported by GenericForeignKeys, it is 130 lines long and in case the last assertion fails, you'll have to spend ridiculous amounts of time figuring out what happened.

A likely incomplete list of similar tests:
modeltests.basic.ModelTest.test_lookup
modeltests.basic.ModelTest.test_object_creation
modeltests.custom_columns.CustomColumnsTests.test_db_column
modeltests.custom_managers.CustomManagerTests.test_manager
modeltests.custom_pk.CustomPKTests.test_custom_pk
modeltests.defer.DeferTests.test_defer
modeltests.expressions.ExpressionsTests.test_filter
modeltests.field_subclassing.CustomField.test_custom_field
modeltests.files.FileStorageTests.test_files
modeltests.get_or_create.GetOrCreateTests.test_get_or_create
modeltests.m2m_recursive.RecursiveM2MTests.test_recursive_m2m
modeltests.m2m_signals.ManyToManySignalsTest
modeltests.m2m_through.M2mThroughTests
modeltests.model_formsets.ModelFormsetTests
modeltests.model_forms.OldFormForXTests.test_with_data (This one has more than 500 lines!)
modeltests.model_inheritance.ModelInheritanceTests
modeltests.model_package.ModelPackageTests.test_model_packages
modeltests.order_with_respect_to.OrderWithRespectToTests.test_basic
modeltests.signals.SignalTests.test_basic

I only went through modeltests/*/tests.py; I might have overlooked a few tests.

This ticket is meant to track all of them; each time a test is updated, we can strike it out in the list above and once there is no item left, we can close this. The point is to make the test suite help developers as much as possible.

Change History (16)

comment:1 by anonymous, 12 years ago

Owner: changed from nobody to anonymous
Status: newassigned

comment:2 by Mark Smith, 12 years ago

Owner: changed from anonymous to Mark Smith
Status: assignednew

comment:3 by Mark Smith, 12 years ago

Owner: changed from Mark Smith to anonymous
Status: newassigned
Triage Stage: UnreviewedAccepted

comment:4 by Mark Smith, 12 years ago

Owner: changed from anonymous to Mark Smith
Status: assignednew

Last attempt to get the status of this ticket correct.

comment:5 by Mark Smith, 12 years ago

Status: newassigned

Basic test refactorings now committed at https://github.com/bedmondmark/django/tree/test_refactor - I'll issue a Pull Request when I've finished the whole lot.

comment:6 by Michal Petrucha, 12 years ago

Found another candidate: regressiontests.defer_regress.DeferRegressionTest.test_basic

IMO, this test is broken because its results depend on the order of execution of tests as it accesses the global model class storage. This means it won't be that easy to rewrite this test.

comment:7 by Aymeric Augustin, 11 years ago

Component: UncategorizedTesting framework

comment:8 by modi.konark@…, 11 years ago

Tried Merging the pull request by koniiik, there were lot of conflicts. The test directory layout has changed and also the conventions used needed to be re-worked.

Will be re-writing the tests and issue a pull request as and when they are done.

https://github.com/django/django/pull/923 , for the test " Custom Columns " .

comment:9 by Claude Paroz, 11 years ago

I'd just like to note that cutting single tests in multiple tests has a cost for the test suite, as the database is flushed between tests. So it's right and fine to separate tests when testing different features, but don't just create test methods for each single database request.

comment:10 by Carl Meyer <carl@…>, 11 years ago

In 483e1b807e7a3c9e49fe5fcbf9b1300ef98a381c:

Refs #18586 -- Split out long custom_columns lookup test into multiple tests.

comment:11 by Carl Meyer, 11 years ago

Description: modified (diff)

comment:12 by Carl Meyer, 11 years ago

Description: modified (diff)

comment:13 by Carl Meyer, 11 years ago

@claudep If you want to look over the initial PR I just committed and comment on whether you think these tests are split up too finely, I'd like to make sure we have consensus on the right level of test granularity before @modi.konark moves forward with more PRs.

comment:14 by Claude Paroz, 11 years ago

Carl, I guess we're also entering personal style here.
For example in that commit, I would have regrouped test_get_first_name/test_filter_first_name and probably also the 3 last (test_get_all_authors_for_an_article/test_get_all_articles_for_an_author/test_get_author_m2m_relation), particularly as they are only querying data.
But once again, this is just me, and I can understand others have different approaches.

comment:15 by Tim Graham <timograham@…>, 10 years ago

In 0d98422b1365ae1e75051036936aab31248d5ee1:

Refactored RecursiveM2MTests into smaller pieces, refs #18586.

comment:16 by Julien Phalip, 10 years ago

Description: modified (diff)
Note: See TracTickets for help on using tickets.
Back to Top