Allow manage.py dumpdata to dump YAML in block style

[sampablokuper]

Using

python manage.py dumpdata --format=yaml

produces output in ​flow style. That's OK for automated processing, but it isn't anything like as human-readable as block style YAML. For people who, like me, need to edit the fixtures manually, it would be very useful to be able to do something like:

python manage.py dumpdata --format=yaml --flowstyle=false

[sampablokuper]

Added to CC list.

[Karen Tracey]

It's not fixed until the code is checked into SVN.

[Ramiro Morales]

Current patch simply changes situation from hardcoded flow style output to hardcoded not flow styled output and this isn't really an overall enhancement. This should be selectable from the command line (defaulting to the current style?) as proposed by the OP. Otherwise IMHO it has little changes of being committed.

[Aviral Dasgupta]

Okay, I've added a new patch. With this one, you can use --flowstyle for flow style and --blockstyle for block style YAML dumps.

[Łukasz Rekucki]

It still needs docs and tests. Also, it's a feature, so not going to make it for 1.3;

[Russell Keith-Magee]

I'm also not wild about the idea of having 2 options to control one flag. You can't specify both --flow and --block, so why have two options? I'd rather see just --block (enabling the new block style), or if there are likely to be more styling options, a --style=block argument.

[Aymeric Augustin]

Change UI/UX from NULL to False.

[Aymeric Augustin]

Change Easy pickings from NULL to False.

[blockstyle@…]

--blockstyle option not found?!! I'm dumping my fixtures as yaml. The flowstyle is a pita to see.
Why isn't it available? I'm using Django 1.6

[Simon Charette]

This ticket patch needs improvement based on @russellm comments, tests and documentation.

[Matthijs Kooijman]

I've taken the time time to redo this patch and address the comments on the previous patch. The patch now also adds tests and documentation.

The PyYAML docs say the following:

By default, PyYAML chooses the style of a collection depending on whether it has nested collections. If a collection has nested collections, it will be assigned the block style. Otherwise it will have the flow style.

If you want collections to be always serialized in the block style, set the parameter default_flow_style of dump() to False. For instance,

In other words, this option configures the style for collections, and there are three options: Always use the flow style, always use the block style, or the default of automatically selecting. To match that, I've added a --yaml-collection-style option that takes auto, flow or block option.

Some uncertainties / notes when reviewing the patch:

• The option name is a lot more verbose than the --style option Russel proposed. However, since it is only relevant to the yaml formatter, I've made the option name more verbose.
• The code in django/core/management/commands/dumpdata.py only passes the option on to the serializer when the current formatter is 'yaml'. This might be a bit hardcoded, but I didn't see an easy and elegant way to do this otherwise?
• I've left the default at "auto" as before, but I suspect that a default of "block" might be more appropriate for loaddata. The current mixed style likely produces more user surprise that the more verbose block style would, as well as being more appropriate for version control. Would it make sense to change the default? If so, should the default of the serializer class be changed as well, or just the dumpdata command?

Given this is a new feature, I've targeted the docs at 2.2.

The patch is available on github: ​https://github.com/django/django/pull/10117

[Matthijs Kooijman]

Meanwhile, on github Tim Graham has reviewed the patch and suggested to generalize the concept of format-specific commandline options, rather than hardcoding it as I had done. I've implemented this, but ran into some problems with the testing framework. See the PR for comments and further discussion.

Meanwhile, I also tried changing the default style from the current mixed style, to always using the block style (which I think is a better style for dumping model data). However, I noticed that this style also changes plain lists, not just mappings. E.g., a model with a many2manyfield ends up being serialized in block style too, which might not be a good default. For example the categories field in:

 - model: serializers.article
   pk: 1
   fields:
     author: 2
     headline: Poker has no place on ESPN
     pub_date: 2006-06-16 11:00:00
     categories:
     - 3
     - 1
     meta_data: []

Better would be to use the flow style for such simple lists:

    categories: [3, 1]

My current patch has three styles, 'flow' (everything flow style, single line), 'block' (everything block style, like above) and 'mixed' (the PyYAML default with flow style for containers with scalars and block style for containers with other containers.

I can imagine a fourth style, which uses block for all mappings, flow style for sequences with scalars and block style for sequences with containers. This is is essentially the PyYAML default style, except that mappings always use the block style. A quick test shows this is easy to implement by overriding the represent_mapping() method on DjangSafeDumper to force flow_style=False. Not sure how to name this style, though. It could also be described as "mixed", but I do not want to drop the PyYAML default style. The latter could be renamed to "default" or even "pyyaml-default", or perhaps this new style could be more explicitly called "block-mappings" to indicate it uses block style for mappings (without indicating the rest).

Any other thoughts?

[Carlton Gibson]

Given the difficulty of progressing this, and the amount of time it's sat untouched, I can't see that adding pass-through CLI options for serialisers is a good way forward.

Rather, I'd think that creating a serialiser subclass, encapsulating your project options and using the existing ​SERIALIZATION_MODULES setting is the way forward.