Opened 2 years ago

Last modified 2 years ago

#26721 new Cleanup/optimization

Document redirecting dumpdata output may result in incorrect encoding in Windows PowerShell

Reported by: OBu Owned by: nobody
Component: Documentation Version: 1.9
Severity: Normal Keywords: dumpdata loaddata
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by OBu)

When I "dumpdata" my database on my Windows 8.1 machine and load the dumped file with "loaddata", I get encoding errors.

Opening the dumped file with notepad++, setting the encoding to UTF-8 and saving it again helps.

A django-users thread describes the problem in detail so it seems I'm not alone...

It might not be a Django problem but a windows pipeline-issue (I was using the powershell), but it is a bit embarrassing.

Change History (5)

comment:1 Changed 2 years ago by Tim Graham

Description: modified (diff)

comment:2 Changed 2 years ago by Baptiste Mispelon

Hi,

Are you able to try to do dumpdata with both Python 2 and Python 3? Does it make any difference?

Thanks.

comment:3 Changed 2 years ago by OBu

Sorry, I don't. I was using python 3.5.1 and django 1.9.7

I just tried dir > test.txt in my windows command line, just to check the encoding of the resulting file, and notepad++ tells me it is UTF-8.

When I do the same in Windows Powershell, encoding is UCE-2 BE BOM

I assume this is exactly what happened to me, so it's not really a Django issue but a problem resulting from the character encoding in Windwos Powershell.

I'm not sure whether there is a way to fix this other then using the --output option (which results in a correctly encoded UTF-8 file no matter which shell was used), but maybe this problem should be mentioned in the docs since the --output option is new and most help web sites are using > .

Last edited 2 years ago by OBu (previous) (diff)

comment:4 Changed 2 years ago by OBu

Description: modified (diff)

comment:5 Changed 2 years ago by Tim Graham

Component: Core (Serialization)Documentation
Summary: "loaddata" can not load data generated by "dumpdata" due to encoding issuesDocument redirecting dumpdata output may result in incorrect encoding in Windows PowerShell
Triage Stage: UnreviewedAccepted
Type: BugCleanup/optimization

I'm not knowledgeable about Windows to confirm, but assuming some can confirm it, a documentation note seems okay.

Note: See TracTickets for help on using tickets.
Back to Top