Opened 8 years ago

Closed 2 years ago

#9549 closed New feature (wontfix)

Give initial_data.* files an option for not over writing current data.

Reported by: Dana Woodman Owned by: nobody
Component: Core (Serialization) Version: 1.0
Severity: Normal Keywords: initial data, syncdb, json, xml, yaml
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: yes
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


When using initial_data in conjunction with syncdb, any data that was in a table with the same pk will get over written with the contents of the initial data file, even if an administrator has made changes to the content. This is intended behavior but in certain situations not always desirable. It would be useful (to me, as Im sure it would be to others) to be able to override this behavior within the file by supplying an option to the JSON/XML/YAML file that the contents in the database be respected and to only write to the database if there is no object with a matching private key.

Attachments (1)

9549-0.diff (2.5 KB) - added by Mikhail 8 years ago.

Download all attachments as: .zip

Change History (12)

comment:1 Changed 8 years ago by Russell Keith-Magee

Component: Database layer (models, ORM)Serialization
Needs documentation: unset
Needs tests: unset
Patch needs improvement: unset
Triage Stage: UnreviewedAccepted

As I've noted on the mailing list, Implementing a --noclobber option seems like a reasonable idea to me. As a note to anyone attempting to implement this; loaddata and syncdb would both need to have this option added. Providing an efficient implementation is also an important concern. Keep in mind that the naive approach (lookup PK, only write if PK doesn't exist) could potentially double the amount of SQL traffic required..

comment:2 Changed 8 years ago by James Bennett

I guess I'm more on the fence about this: the whole point of initial-data fixtures is that they represent the state your DB is supposed to be in after a syncdb. If your initial data fixture might leave your DB in a state you don't want to have after syncdb, then it's not an initial data fixture...

comment:3 Changed 8 years ago by Russell Keith-Magee

James- I agree with you: syncdb should overwrite initial_data fixtures and guarantee the state of the database. I'm not suggesting that we change the default overwrite behaviour of syncdb. Personally, I see the real benefit of --noclobber is in the loaddata case. However, if we're adding --noclobber to loaddata, I don't see the harm in passing the option on to syncdb - with a big honking note in the docs that say "you probably don't want to use this option".

Changed 8 years ago by Mikhail

Attachment: 9549-0.diff added

comment:4 Changed 8 years ago by Mikhail

Has patch: set
Needs documentation: set
Resolution: fixed
Status: newclosed

I don't think efficient implementation is so important in this case. This operation only used from time to time by administrator, so it has no real impact in production environment. I belive simplicity of implementation is much more important here.

comment:5 Changed 8 years ago by Malcolm Tredinnick

Resolution: fixed
Status: closedreopened

@uptimebox: the ticket is closed or fixed yet. That happens if/when the patch is committed. Reopening.

comment:6 Changed 5 years ago by Luke Plant

Severity: Normal
Type: New feature

comment:7 Changed 5 years ago by Jaime Irurzun

Easy pickings: unset
UI/UX: unset

I'm very interested in this feature, so I'd be happy to finish whatever the ticket needs to be committed. Is documentation what has blocked this ticket for 3 years now?

comment:8 Changed 5 years ago by Aymeric Augustin

#15926 is related.

comment:9 Changed 4 years ago by Aymeric Augustin

Status: reopenednew

comment:10 Changed 3 years ago by chrisv

Not sure whether this is the right place for discussing this, but I think the --noclobber approach is not really the optimal solution for this problem. IMHO, Django should differentiate between different types of initial database content:

1.) data which is similar to code in a way that it is expected to exist all the time and not be changed at all, for example a list of countries or mime-types. This is what initial_data provides at the moment, and for this kind of data it is correct to reset it after each syncdb.

2.) data which is similar to settings in a way that it is expected to be there, but can be modified in a project, for example notification texts which can be customized for a specific project/site. I believe this is what this ticket was meant for originally.

3.) data which only serves as sample content, and is expected to be deleted or modified in most cases, for example the "first post" in a fresh Wordpress install. This is not necessarily in the scope of this ticket and only mentioned for completeness.

With --noclobber, it is not really possible to differentiate between 1.), 2.) and 3.), instead it is only possible to tell syncdb to treat all initial_data files to be one or the other. But how would the admin know which data is what, and when to overwrite or not?

A much better approach would IMHO be to put this data into different files and treat them respectively in syncdb, for example

  • system_data.(json|yaml|...) contains data which is expected to be overwritten by updates to the app (like current behaviour of initial_data)
  • custom_data.(json|yaml|...) contains data which is only imported when the pk does not exist, but not touched otherwise
  • demo_data.(json|yaml|...) is only imported when the table has just been created (I admit that one might be overkill, though - one can always put it into another fixture which has to be loaded manually)

I'm willing to write a patch for this if core devs think this is worthwhile.

comment:11 Changed 2 years ago by Aymeric Augustin

Resolution: wontfix
Status: newclosed

The new migrations framework doesn't load initial data any more.

Note: See TracTickets for help on using tickets.
Back to Top