StaticLiveServerTestCase does not properly respect data migrations

[Ruben Nielsen]

When data migrations exist for a project, it is expected that each test case has these migrations run before running the test.
That is indeed the case for django.tests.TestCase (unit tests), but it is NOT working for django.contrib.staticfiles.testing.StaticLiveServerTestCase (functional tests).

Migrations seem to be run for each class, e.g. MyTestCase(StaticLiveServerTestCase), but not for the methods of this class. That is, if MyTestCase has 3 methods with tests, the migrations are only loaded for the first one, and then the database is wiped, and the objects from the data migration are no longer present for the next two methods.

A minimal reproduction of this issue - along with a guide to the issue and which files to look at - can be found here:
​https://github.com/eldamir/django_migration_bug

[Baptiste Mispelon]

Hi,

I can reproduce the reported issue and it does appear to be a bug indeed.

Thanks.

[Greg Chapple]

The problem appears to be with django.test.testcases.LiveServerTestCase which inherits from TransactionTestCase. Changing this class to inherit from TestCase seems to solve the issue, though I'm not sure if this is ideal. The docstring on LiveServerTestCase mentions a reason as to why it doesn't inherit from TestCase:

Note that it inherits from TransactionTestCase instead of TestCase because
the threads do not share the same transactions (unless if using in-memory
sqlite) and each thread needs to commit all their transactions so that the
other thread can see the changes.

I'm not sure how true this is, as when changing LiveServerTestCase to inherit from TestCase, the issue is fixed and there are no failing tests. Not sure whether this means the above statement is no longer true, or whether there simply are not enough tests to prove it.

[Thomas C]

It looks like a more general problem with TransactionTestCase which does not restore properly the data from migrations between tests.

I added a test demonstrating the problem : ​https://github.com/tchaumeny/django/compare/ticket_23640#diff-3d02be4b6d6ab6d1ac6fe6bbddc3a844R1

[Greg Chapple]

It appears that this problem only exists with migrations. When I use fixtures generated by manage.py dumpdata the problem goes away. The example below works without issue.

class MyTests(TransactionTestCase):
    fixtures = ['data.json']

    def test_fixtures_exist(self):
        self.assertEqual(MyModel.objects.count(), 4)

    def test_fixtures_still_exist(self):
        self.assertEqual(MyModel.objects.count(), 4)

Removing fixtures = ['data.json'] causes the second test to fail.

[Thomas C]

According to the documentation this might be the expected behavior, not a bug: Any initial data loaded in migrations will only be available in TestCase tests and not in TransactionTestCase tests, and additionally only on backends where transactions are supported (the most important exception being MyISAM). — see ​https://docs.djangoproject.com/en/1.7/topics/testing/overview/#rollback-emulation

[Greg Chapple]

You can use fixtures generated by dumpdata, though, if you need.

[Greg Chapple]

Resolving this ticket per tchaumeny's comment above.

[Ruben Nielsen]

Well, with Django 1.7 we were finally presented with an alternative to fixtures, since fixtures are very costly to maintain, whereas data migrations are very easy.

If this issue is intended behaviour as stated, isn't there a way to do the functional testing like with StaticLiveTestCase? Is there another base case I can inherit from, so that I won't have to deal with this issue?

[Greg Chapple]

Inheriting from TestCase and StaticLiveServerTestCase appears to work, though I don't know what other issues that will cause.

class MyTests(TestCase, StaticLiveServerTestCase):
    ...

Does that work for your use case?

[Thomas C]

You should not inherit from both those classes as they are doing very different things. What you need is to set serialized_rollback = True in the body of your test case, which should make the migrated data restored for each test function.

[Greg Chapple]

Yeah, just after writing that I realised that was probably a bad idea.

For me, I also need to set available_apps = ['<app-name>'] in order for serialized_rollback to work.

[Ruben Nielsen]

Thank you kindly for the last two comments. My tests now run as expected after setting

serialized_rollback = True

and

available_apps = ['<app-name>']

Is there a note of these settings in the documentation? Maybe there should be. It is not something I would have found myself. Thanks for your assist!

[Ruben Nielsen]

I do get another error now when using the solution mentioned above, which seems a bit weird too.

After the first test case runs, before getting to the setUp method of the second test case, I get

Error
Traceback (most recent call last):
  File "/home/ruben/v3env/lib/python3.4/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/home/ruben/v3env/lib/python3.4/site-packages/django/db/backends/sqlite3/base.py", line 486, in execute
    return Database.Cursor.execute(self, query, params)
sqlite3.IntegrityError: UNIQUE constraint failed: django_content_type.app_label, django_content_type.model

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ruben/v3env/lib/python3.4/site-packages/django/test/testcases.py", line 182, in __call__
    self._pre_setup()
  File "/home/ruben/v3env/lib/python3.4/site-packages/django/test/testcases.py", line 754, in _pre_setup
    self._fixture_setup()
  File "/home/ruben/v3env/lib/python3.4/site-packages/django/test/testcases.py", line 797, in _fixture_setup
    connections[db_name]._test_serialized_contents
  File "/home/ruben/v3env/lib/python3.4/site-packages/django/db/backends/creation.py", line 428, in deserialize_db_from_string
    obj.save()
  File "/home/ruben/v3env/lib/python3.4/site-packages/django/core/serializers/base.py", line 173, in save
    models.Model.save_base(self.object, using=using, raw=True)
  File "/home/ruben/v3env/lib/python3.4/site-packages/django/db/models/base.py", line 618, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/home/ruben/v3env/lib/python3.4/site-packages/django/db/models/base.py", line 699, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/home/ruben/v3env/lib/python3.4/site-packages/django/db/models/base.py", line 732, in _do_insert
    using=using, raw=raw)
  File "/home/ruben/v3env/lib/python3.4/site-packages/django/db/models/manager.py", line 92, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/ruben/v3env/lib/python3.4/site-packages/django/db/models/query.py", line 921, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/home/ruben/v3env/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 920, in execute_sql
    cursor.execute(sql, params)
  File "/home/ruben/v3env/lib/python3.4/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/home/ruben/v3env/lib/python3.4/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/home/ruben/v3env/lib/python3.4/site-packages/django/utils/six.py", line 549, in reraise
    raise value.with_traceback(tb)
  File "/home/ruben/v3env/lib/python3.4/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/home/ruben/v3env/lib/python3.4/site-packages/django/db/backends/sqlite3/base.py", line 486, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.IntegrityError: UNIQUE constraint failed: django_content_type.app_label, django_content_type.model

What seems to cause this is some automatically generated SQL calls from Django:

UPDATE "django_content_type" SET "name" = ?, "app_label" = ?, "model" = ? WHERE "django_content_type"."id" = ? ('log entry', 'admin', 'logentry', 1)
INSERT INTO "django_content_type" ("id", "name", "app_label", "model") SELECT ? AS "id", ? AS "name", ? AS "app_label", ? AS "model" (1, 'log entry', 'admin', 'logentry')

As far as I can tell, a row is updated, setting some values, and then a row is inserted which violates the UNIQUE constraint.

I don't have the insight to work around this. I will just use fixtures for now :(