Fixtures fail for UTF-8 encoded data

[colin@…]

Data loaded using initial_data.json does not correctly handle UTF-8 characters. Similarly if you use the --format XML flag an error is generated:

python manage.py dumpdata --format xml winereport
Unable to serialize database: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)

[Simon G. <dev@…>]

Colin, is this on the unicode branch or the standard branch?

[colin@…]

Hi Simon,

This is on the trunk (revision 5420). I take it that some work is going on in this area on the unicode branch?

Thanks,

Colin.

[James Bennett]

Yes, the "unicode" branch in the Django SVN repository is a reworking of a lot of Django's internals (nothing that affects end-user apps very much) to drastically improve Unicode handling. Before reporting bugs related to Unicode, please test against that branch to make sure the problem hasn't already been fixed there (the branch is nearing completion and will hopefully be merged fairly soon, so it's important to know about any remaining bugs it hasn't fixed).

[colin@…]

Hi,

I've re-tested this on the unicode branch. Both of the issues are resolved on this branch for newly created data dump files (files produced from trunk code don't work, but that's probably OK).

Is there a way to mark the ticket as being fixed on a particular branch?

Thanks,

Colin.

[Malcolm Tredinnick]

This is essentially a dupe of #4227.

(Triagers: I've been putting in the unicode-branch keyword for tickets that are open on trunk but fixed on the unicode branch. Then I'll close them all when the branch is merged.)