Opened 9 years ago
Last modified 3 years ago
#24778 new New feature
Data Migration from Fixture
Reported by: | Eugene | Owned by: | nobody |
---|---|---|---|
Component: | Migrations | Version: | dev |
Severity: | Normal | Keywords: | |
Cc: | Eugene, Alex Dehnert | Triage Stage: | Someday/Maybe |
Has patch: | yes | Needs documentation: | yes |
Needs tests: | yes | Patch needs improvement: | yes |
Easy pickings: | no | UI/UX: | no |
Description (last modified by )
Providing data via fixtures has been deprecated. In the past, we used to execute the loaddata manually. After Django introduce migration, the recommended way to import data is to create an empty migration and use RunPython
migration operations to load the data.
This is a very common use case for data migration via fixture. We create the function just to call_command loaddata
. http://stackoverflow.com/a/25981899/764592
In my opinion, instead of having to create the function, we can actually simplify this into a migration operation on its own.
As follow:
# Module: django.db.migrations.operations.base.special from django.core.management import call_command class LoadFixture(Operation): reduces_to_sql = False reversible = False def __init__(self, *fixtures): self.fixtures = fixtures def state_forwards(self, app_label, state): pass def database_forwards(self, app_label, schema_editor, from_state, to_state): for fixture in self.fixtures: call_command('loaddata', fixture, app_label=app_label) def database_backwards(self, app_label, schema_editor, from_state, to_state): pass def describe(self): return "Load Fixture Operation"
The implication of LoadFixture
operations can be shown in the following example:
Assuming we have the fixture in foobar/fixtures/book_data.json
# File: foobar/migrations/0002_auto_load_book.py class Migration(migrations.Migration): dependencies = [ ('foobar', '0001_initial'), ] operations = [ migrations.LoadFixture('book_data'), ]
The migration script is now much simpler.
PS: This is my first time creating a ticket and involved in Django internal. Let me know if I should make a PR for this feature.
Change History (11)
comment:1 by , 9 years ago
Cc: | added |
---|---|
Description: | modified (diff) |
comment:2 by , 9 years ago
Description: | modified (diff) |
---|
comment:3 by , 9 years ago
Needs documentation: | set |
---|---|
Needs tests: | set |
Patch needs improvement: | set |
Triage Stage: | Unreviewed → Someday/Maybe |
comment:4 by , 9 years ago
Markus, what do think about allowing an apps
kwarg to be passed to the loaddata
command and serializers' initializers?
If supplied the command and the serializers would either use the provided apps
or default to django.apps.apps
.
From that point we could provide a RunPython
subclass that simply calls call_command
with apps=apps
?
comment:5 by , 9 years ago
That would probably work. I even started going down that rabbit hole of adding apps
to the serializers, but revoked the changes because I didn't see the benefit and it got kind of ugly.
comment:6 by , 9 years ago
And there are even already apps out there that try to implement fixture loading during migrations with, of course, the exact same problem I mentioned before:
- https://github.com/alexhayes/django-migration-fixture
- https://github.com/doctormo/django-fast-fixtures
There even is a library out there that provides that feature for South: https://github.com/sebleier/django-alpaca
comment:7 by , 8 years ago
I ran into the app state not matching the model.py state while loading fixtures and ended up creating a branch of django to handle this use case. I issued a PR to a feature branch in my fork of django. I am more than willing to work on this issue given the appropriate guidance. I will finish up reading the contributor guidelines.
comment:9 by , 8 years ago
You should probably write to the DevelopersMailingList about this. I'm uncertain about the design and whether or not it should be included in Django.
comment:10 by , 8 years ago
As mentioned above in comments 3 to 5 and suggested by charettes, the "right" approach to this issue is likely adding apps
as an optional argument to the serializers and using that in a next step.
A command that dumps a fixture file or a database as a database migrations feels ugly here. You're essentially creating (arbitrary) python code that follows absolutely no patterns (how do you handle multi line strings, byte strings, UUIDs, ...)?
comment:12 by , 3 years ago
Cc: | added |
---|
I'm a bit torn about this feature. I can understand peoples request to easily load (existing) fixtures. It's convenient. On the other hand this will inevitably lead to the very same problem we wanted to prevent by using
RunPython
in its intended form:Apart from that, as soon as a model changes your
LoadFixture
operation will fail:call_command
will use the modelmyapp.models.MyModel
whereas the database has an older state because the respective changes to the database would happen in "0003_add_somefield".