Parallel testing: Please implement a signal or call checks after test DB creation but before cloning it.

[László Károlyi]

Hi,

I tried to reach someone on IRC, but actually it's better to document my issue here, maybe there will be a solution to it eventually.

I'm trying to use the new and shiny parallel testing functionality of Django 1.11, and I've run into problems with it.

So I have functionality implemented on check and the post_migrate signal, where I conditionally insert stuff into the DB that can't be inserted by fixtures, and also conditionally modify DB fields that can't be done from the ORM. (the latter is basically changing collation on a column)

It seems that the post_migrate signal is run randomly before and after the test creation, and checks are only run *after* the DB is cloned, so I don't have a sufficient point where I can have my logic run.

Monkey-patching has shown that the best solution would be to run call_command('check') ​here, so basically after create_test_db but before clone_test_db.

I can quickly add that, but I'm not sure how it could be tested, and what implications it would have, so I'm asking you guys, if you think it's doable and feasible. Because of this right now, I can't use parallel testing.

[László Károlyi]

Update:

It seems I managed to consistently insert the needed data into the DB before it gets cloned. But that brought another problem I discovered, and which maybe could be fixed.

So the first part of what I'm doing at check and post_migrate is, I update my own permission models, which subclass the original django Permission model. Formerly, I had a function that would give a ContentType model of django.contrib.auth:

def _get_perm_contenttype(app_label: str, model_name: str):
    """
    Return a `ContentType`.
    """
    from django.contrib.contenttypes.models import ContentType
    return ContentType.objects.get_for_model(
        apps.get_model(app_label=app_label, model_name=model_name))

Because ContentType.objects.get_for_model() caches, and because the nature of the parallel testing, that cached result sometimes brought models with wrong IDs, and I got constraint violations because no such IDs existed in my DB.

I fixed my code by not using get_for_model(), and using get() instead. I'm just leaving this information here for your consideration.

[Tim Graham]

#15610 and #16281 might be related to the contenttypes issue. Can we close this ticket then? If there's some issue with contenttypes, it's likely more clear to open a separate ticket about that.

[László Károlyi]

Replying to Tim Graham:

Can we close this ticket then?

I think so, yes. Thx, closing.