Warn against modifying objects created in setUpTestData

[Markus Holtermann]

Given the following models:

from django.db import models


class RelObj(models.Model):
    def __repr__(self):
        return '<RelObj %d at id %x>' % (self.pk, id(self))


class Obj(models.Model):
    relobj = models.OneToOneField(RelObj)
    name = models.CharField(max_length=10, unique=True)
    val = models.IntegerField()

    def __repr__(self):
        return '<Obj %d at id %x>' % (self.pk, id(self))

And these tests:

from django.db.utils import IntegrityError
from django.test import TestCase

from .models import Obj, RelObj


class ModelTests(TestCase):

    @classmethod
    def setUpTestData(cls):
        cls.relobj = RelObj.objects.create()
        cls.obj = Obj.objects.create(relobj=cls.relobj, name='foo', val=1)

    def test_1(self):
        self.assertEqual(self.obj.relobj, self.relobj)
        self.assertEqual(self.relobj.obj, self.obj)
        self.assertEqual(self.obj.name, 'foo')
        self.assertEqual(self.obj.val, 1)

    def test_2(self):
        with self.assertRaisesRegex(IntegrityError, 'app_obj_name_key'):
            Obj.objects.create(relobj=self.relobj, name='foo', val=2)

    def test_3(self):
        self.assertEqual(self.obj.relobj, self.relobj)
        self.assertEqual(self.relobj.obj, self.obj)
        self.assertEqual(self.obj.name, 'foo')
        self.assertEqual(self.obj.val, 1)

I get the following traceback on PostgreSQL

Creating test database for alias 'default'...
..F
======================================================================
FAIL: test_3 (app.tests.ModelTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/markus/Coding/django-test-app/app/tests.py", line 26, in test_3
    self.assertEqual(self.relobj.obj, self.obj)
AssertionError: <app.models.Obj object at 0x7f579e2afe10> != <Obj 1 at id 7f579e2afc18>

----------------------------------------------------------------------
Ran 3 tests in 0.004s

FAILED (failures=1)
Destroying test database for alias 'default'...

The problem seems to be within the Obj creation in test_2 which assigns self.relobj / cls.relobj the instance of the newly created object that couldn't be saved.

I'm on Django 1.9.dev20150818235245.

I know that I could use refresh_from_db(), but that seems counter productive, given the intention of setUpTestData(). Using setUp() instead of setUpTestData() also works.

[Tim Graham]

I'm not sure what we could do here besides adding a documentation warning that you shouldn't modify objects created in setUpTestData() in your test methods. Modifications to in-memory objects from setup work done at the class level will inevitably persist between test methods.

[Markus Holtermann]

Yeah, I see that. The "solution" I came up with now is

def test_2(self):
    try:
        with self.assertRaisesRegex(IntegrityError, 'app_obj_name_key'):
            Obj.objects.create(relobj=self.relobj, name='foo', val=2)
    finally:
        self.relobj = self.obj

[Tim Graham]

​PR

[Tim Graham <timograham@…>]

In 81006b9:

Fixed #25290 -- Warned against modifying objects created in setUpTestData() in tests.

[Tim Graham <timograham@…>]

In bd55e80:

[1.8.x] Fixed #25290 -- Warned against modifying objects created in setUpTestData() in tests.

Backport of 81006b9657534a21c3eadb2bc4a87c46db54c099 from master

[Tim Graham <timograham@…>]

In 1eb2ff3:

[1.9.x] Fixed #25290 -- Warned against modifying objects created in setUpTestData() in tests.

Backport of 81006b9657534a21c3eadb2bc4a87c46db54c099 from master