Opened 5 months ago

Last modified 5 months ago

#29249 new New feature

Add option to dumpdata to allow unicode JSON or YAML

Reported by: hakib Owned by: nobody
Component: Core (Management commands) Version: 2.0
Severity: Normal Keywords: dumpdata, unicode
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

It is currently not (easily) possible to use the dumpdata managemet command on models with unicode data.
The JSON serializer used by dumpdata is not accepting the ensure_ascii argument used by json.dumps as an argument.

Since ensure_ascii=True is the default, I suggest adding a --dont-ensure-ascii flag to the dumpdata managemet command so it will be easier to use dumpdata with unicode.

./manage.py dumpdata app.model --dont-ensure-ascii

I'm not sure what are the implications on other serializers such as YAML, XML etc.

Change History (4)

comment:1 Changed 5 months ago by Tim Graham

Component: UtilitiesCore (Management commands)
Summary: Add option to dumpdata with unicode dataAdd option to dumpdata with unicode JSON

I believe this would only apply to the JSON serializer, and I'm not sure about adding a dumpdata option that's specific to a particular serializer. I think your best solution is to subclass the JSON serializer, register it as a custom format, and then use that format in dumpdata.

comment:2 Changed 5 months ago by hakib

The JSON serializer has a ensure_ascii attribute and the YAML serializer has a allow_unicode attribute. I already submitted a PR implementing the flag in both serializers.

I haven't looked at the XML serializer yet but I'm sure it will be possible there as well.

As someone who works with unicode as the primary language for most apps (as I'm sure a lot of other developers do) it's a very a useful feature to be able to dump fixtures directly from local db in a readable format.

Last edited 5 months ago by Tim Graham (previous) (diff)

comment:3 Changed 5 months ago by Tim Graham

Has patch: set
Patch needs improvement: set
Triage Stage: UnreviewedAccepted

Okay. My main concern is that an option calls --allow_unicode may suggest to readers that all serializers prohibit unicode by default. That may not be true. Your patch also needs documentation.

comment:4 Changed 5 months ago by Tim Graham

Summary: Add option to dumpdata with unicode JSONAdd option to dumpdata to allow unicode JSON or YAML
Note: See TracTickets for help on using tickets.
Back to Top