Opened 8 years ago

Closed 2 years ago

#9549 closed New feature (wontfix)

Give initial_data.* files an option for not over writing current data.

Reported by: MrNarwhal Owned by: nobody
Component: Core (Serialization) Version: 1.0
Severity: Normal Keywords: initial data, syncdb, json, xml, yaml
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: yes
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


When using initial_data in conjunction with syncdb, any data that was in a table with the same pk will get over written with the contents of the initial data file, even if an administrator has made changes to the content. This is intended behavior but in certain situations not always desirable. It would be useful (to me, as Im sure it would be to others) to be able to override this behavior within the file by supplying an option to the JSON/XML/YAML file that the contents in the database be respected and to only write to the database if there is no object with a matching private key.

Attachments (1)

9549-0.diff (2.5 KB) - added by uptimebox 8 years ago.

Download all attachments as: .zip

Change History (12)

comment:1 Changed 8 years ago by russellm

  • Component changed from Database layer (models, ORM) to Serialization
  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted

As I've noted on the mailing list, Implementing a --noclobber option seems like a reasonable idea to me. As a note to anyone attempting to implement this; loaddata and syncdb would both need to have this option added. Providing an efficient implementation is also an important concern. Keep in mind that the naive approach (lookup PK, only write if PK doesn't exist) could potentially double the amount of SQL traffic required..

comment:2 Changed 8 years ago by ubernostrum

I guess I'm more on the fence about this: the whole point of initial-data fixtures is that they represent the state your DB is supposed to be in after a syncdb. If your initial data fixture might leave your DB in a state you don't want to have after syncdb, then it's not an initial data fixture...

comment:3 Changed 8 years ago by russellm

James- I agree with you: syncdb should overwrite initial_data fixtures and guarantee the state of the database. I'm not suggesting that we change the default overwrite behaviour of syncdb. Personally, I see the real benefit of --noclobber is in the loaddata case. However, if we're adding --noclobber to loaddata, I don't see the harm in passing the option on to syncdb - with a big honking note in the docs that say "you probably don't want to use this option".

Changed 8 years ago by uptimebox

comment:4 Changed 8 years ago by uptimebox

  • Has patch set
  • Needs documentation set
  • Resolution set to fixed
  • Status changed from new to closed

I don't think efficient implementation is so important in this case. This operation only used from time to time by administrator, so it has no real impact in production environment. I belive simplicity of implementation is much more important here.

comment:5 Changed 8 years ago by mtredinnick

  • Resolution fixed deleted
  • Status changed from closed to reopened

@uptimebox: the ticket is closed or fixed yet. That happens if/when the patch is committed. Reopening.

comment:6 Changed 5 years ago by lukeplant

  • Severity set to Normal
  • Type set to New feature

comment:7 Changed 5 years ago by etox

  • Easy pickings unset
  • UI/UX unset

I'm very interested in this feature, so I'd be happy to finish whatever the ticket needs to be committed. Is documentation what has blocked this ticket for 3 years now?

comment:8 Changed 5 years ago by aaugustin

#15926 is related.

comment:9 Changed 3 years ago by aaugustin

  • Status changed from reopened to new

comment:10 Changed 3 years ago by chrisv

Not sure whether this is the right place for discussing this, but I think the --noclobber approach is not really the optimal solution for this problem. IMHO, Django should differentiate between different types of initial database content:

1.) data which is similar to code in a way that it is expected to exist all the time and not be changed at all, for example a list of countries or mime-types. This is what initial_data provides at the moment, and for this kind of data it is correct to reset it after each syncdb.

2.) data which is similar to settings in a way that it is expected to be there, but can be modified in a project, for example notification texts which can be customized for a specific project/site. I believe this is what this ticket was meant for originally.

3.) data which only serves as sample content, and is expected to be deleted or modified in most cases, for example the "first post" in a fresh Wordpress install. This is not necessarily in the scope of this ticket and only mentioned for completeness.

With --noclobber, it is not really possible to differentiate between 1.), 2.) and 3.), instead it is only possible to tell syncdb to treat all initial_data files to be one or the other. But how would the admin know which data is what, and when to overwrite or not?

A much better approach would IMHO be to put this data into different files and treat them respectively in syncdb, for example

  • system_data.(json|yaml|...) contains data which is expected to be overwritten by updates to the app (like current behaviour of initial_data)
  • custom_data.(json|yaml|...) contains data which is only imported when the pk does not exist, but not touched otherwise
  • demo_data.(json|yaml|...) is only imported when the table has just been created (I admit that one might be overkill, though - one can always put it into another fixture which has to be loaded manually)

I'm willing to write a patch for this if core devs think this is worthwhile.

comment:11 Changed 2 years ago by aaugustin

  • Resolution set to wontfix
  • Status changed from new to closed

The new migrations framework doesn't load initial data any more.

Note: See TracTickets for help on using tickets.
Back to Top