#26198 closed Bug (invalid)
partition_suite() removing non duplicate tests
Reported by: | Luke Scott | Owned by: | nobody |
---|---|---|---|
Component: | Testing framework | Version: | 1.8 |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Unreviewed | |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
In django 1.8 partition_suite() in django.test.runner is only including my behave tests from the first app.
e.g. ./manage.py test foo bar baz
will test all unit tests for apps foo, bar and baz, but will only test behave tests for app foo.
I am using django-behave==0.1.5 and behave==1.2.5 with django==1.8.9
This setup was working fine - tested in django==1.7.11 and was not a problem - upgrading to 1.8 is when the problem began.
All tests appear to be passed correctly to reorder_suite() but in partition_suite() only behave tests of the first app listed are being added to be tested.
In the 'django.tests.runner' module this appears to be due to reorder_suite() using an OrderedSet and subsequent '<django_behave.runner.DjangoBehaveTestCase testMethod=runTest>' objects not getting added to this set in partition_suite(). To confirm - the subsequent objects that should be added do have different object IDs and are in fact different tests.
def partition_suite(suite, classes, bins, reverse=False): suite_class = type(suite) if reverse: suite = reversed(tuple(suite)) for test in suite: if isinstance(test, suite_class): partition_suite(test, classes, bins, reverse=reverse) else: for i in range(len(classes)): if isinstance(test, classes[i]): bins[i].add(test) # subsequent behave test objects not being added here break else: bins[-1].add(test)
Looking at this commit:
https://github.com/django/django/commit/2ca0870b672413cf5bc6e441bda58839f902f5b0
In partition_suite() I changed:
bins = [OrderedSet() for i in range(class_count + 1)]
back to
bins = [suite_class() for i in range(class_count + 1)]
and I changed
bins[i].add(test)
to
bins[i].addTest(test)
I retested using my test suite for my project and all the tests ran as expected.
Not sure what the further implications of changing this are with regards to preventing duplicated tests from being added to the suite.
Thanks
Change History (3)
comment:1 by , 9 years ago
Resolution: | → invalid |
---|---|
Status: | new → closed |
comment:2 by , 9 years ago
Thanks charettes - I appreciate your feedback.
I will follow this up on django-behave
Thanks
Luke
comment:3 by , 9 years ago
That worked - thanks charettes. I have raised a PR over on django-behave.
Hi lukeaus,
Thanks for your report but it looks like this is bug in
django-behave
.Python
set
(and Django'sOrderedSet
) use the__hash__
method of objects that are added to them to identify them uniquely.In the case of
unittest.TestCase
instance this method returns thetype
of the testcase and the name of the the test method:It looks like
django-behave
adds multiple instance ofDjangoBehaveTestCase
to its suite but all of them haverunTest
as test method name resulting in the samehash()
output and making them indistinguishable.From a quick look the
DjangoBehaveTestCase.__hash__()
method should be adjusted to also includeself.features_dir
: