Opened 6 years ago

Closed 6 years ago

#11420 closed (wontfix)

Test suite and default encoding on Windows

Reported by: richardb Owned by: nobody
Component: Testing framework Version: master
Severity: Keywords: BrokenUnicodeMethod
Cc: Triage Stage: Design decision needed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:


Test: run the Django test suite (tests/ on Windows platform with a python encoding set to something other that the default (ascii).

Outcome: one test fails (regressiontests\model_regress\, line 147).

The test assumes that the python encoding is ascii. I've attached a patch that only runs this test if the encoding *is* indeed ascii.

Note: this bug has already been mentionned on the django-developers mailing list:

Attachments (1)

patch-11420.diff (860 bytes) - added by richardb 6 years ago.

Download all attachments as: .zip

Change History (6)

Changed 6 years ago by richardb

comment:1 Changed 6 years ago by kmtracey

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Design decision needed

As I mentioned in that thread, running Python with a default encoding other than ascii is likely to hide errors. In the case posted in that thread the wrong encoding (apparently latin1) is used to translate a utf-8 bytestring to unicode, resulting in corrupt data. Why would be desirable to change the test suite to run cleanly on such a configuration?

comment:2 Changed 6 years ago by richardb

My thinking here was that the test specifically relied on Python running with a default encoding of ascii - which will often not be the case.

Changing the default encoding (as you mention in your email) is not a practical proposition for many people, so I thought that the next best option would be for Django to skip that test.

This behaviour is already displayed with, for example, the Markdown library - if it's not installed, the relevant tests are skipped.

comment:3 Changed 6 years ago by mtredinnick

Running Python with a default encoding other than ASCII is a bug in the way you are running Python. It is not intended that people should change the default encoding. It was a bug that the method to do so was left exposed as public API in the first place.

I'm -1 on adding extra code in Django to work around broken Python usage.

comment:4 Changed 6 years ago by anonymous

Running Python with a default encoding other than ASCII is a bug in the way you are running Python. It is not intended that people should change the default encoding.

That's maybe a bit harsh? Changing the default encoding is common practise - for example, it's detailed in the well-known 'Dive Into Python' book.
Likewise, if I search on google's french page for 'python unicode', within the first 5 hits, 2 include explanations on how to change the default encoding.

Note: I'm richardb, but posting under 'anonymous' as I'm not at home.

comment:5 Changed 6 years ago by kmtracey

  • Resolution set to wontfix
  • Status changed from new to closed

I don't know if it's common practice, but if it is, it should be discouraged. It certainly cannot be common practice for anyone writing code intended to be re-usable. Unfortunately mention of how to do it in a popular book may have lured people into using a technique that is a really bad idea. It may be tempting to change the default encoding to one where you'll never get UnicodeXXcodeErrors, but you are swapping nice obvious hard failures for the possibility of silently corrupted data. And you are making your code non-portable. Some Googling around brings up:

which make for good reading on the issue.

With Malcolm's -1 and my own feeling that it would be a bad idea to make the test suite hide this, I'm going to close this wontfix.

Note: See TracTickets for help on using tickets.
Back to Top