Django

Code

Ticket #11420 (closed: wontfix)

Opened 9 months ago

Last modified 8 months ago

Test suite and default encoding on Windows

Reported by: richardb Assigned to: nobody
Milestone: Component: Testing framework
Version: SVN Keywords: BrokenUnicodeMethod
Cc: Triage Stage: Design decision needed
Has patch: 1 Needs documentation: 0
Needs tests: 0 Patch needs improvement: 0

Description

Test: run the Django test suite (tests/runtests.py) on Windows platform with a python encoding set to something other that the default (ascii).

Outcome: one test fails (regressiontests\model_regress\models.py, line 147).

The test assumes that the python encoding is ascii. I've attached a patch that only runs this test if the encoding *is* indeed ascii.

Note: this bug has already been mentionned on the django-developers mailing list: http://groups.google.com/group/django-developers/browse_thread/thread/3e751cd4edf9a968

Attachments

patch-11420.diff (0.8 kB) - added by richardb on 07/03/09 09:41:38.

Change History

07/03/09 09:41:38 changed by richardb

  • attachment patch-11420.diff added.

07/03/09 10:30:29 changed by kmtracey

  • needs_better_patch changed.
  • stage changed from Unreviewed to Design decision needed.
  • needs_tests changed.
  • needs_docs changed.

As I mentioned in that thread, running Python with a default encoding other than ascii is likely to hide errors. In the case posted in that thread the wrong encoding (apparently latin1) is used to translate a utf-8 bytestring to unicode, resulting in corrupt data. Why would be desirable to change the test suite to run cleanly on such a configuration?

07/04/09 17:36:11 changed by richardb

My thinking here was that the test specifically relied on Python running with a default encoding of ascii - which will often not be the case.

Changing the default encoding (as you mention in your email) is not a practical proposition for many people, so I thought that the next best option would be for Django to skip that test.

This behaviour is already displayed with, for example, the Markdown library - if it's not installed, the relevant tests are skipped.

07/04/09 17:48:08 changed by mtredinnick

Running Python with a default encoding other than ASCII is a bug in the way you are running Python. It is not intended that people should change the default encoding. It was a bug that the method to do so was left exposed as public API in the first place.

I'm -1 on adding extra code in Django to work around broken Python usage.

07/11/09 09:06:54 changed by anonymous

Running Python with a default encoding other than ASCII is a bug in the way you are running Python. It is not intended that people should change the default encoding.

That's maybe a bit harsh? Changing the default encoding is common practise - for example, it's detailed in the well-known 'Dive Into Python' book.
Likewise, if I search on google's french page for 'python unicode', within the first 5 hits, 2 include explanations on how to change the default encoding.

Note: I'm richardb, but posting under 'anonymous' as I'm not at home.

07/11/09 09:53:54 changed by kmtracey

  • status changed from new to closed.
  • resolution set to wontfix.

I don't know if it's common practice, but if it is, it should be discouraged. It certainly cannot be common practice for anyone writing code intended to be re-usable. Unfortunately mention of how to do it in a popular book may have lured people into using a technique that is a really bad idea. It may be tempting to change the default encoding to one where you'll never get UnicodeXXcodeErrors, but you are swapping nice obvious hard failures for the possibility of silently corrupted data. And you are making your code non-portable. Some Googling around brings up:

http://tarekziade.wordpress.com/2008/01/08/syssetdefaultencoding-is-evil/

http://faassen.n--tree.net/blog/view/weblog/2005/08/02/0

which make for good reading on the issue.

With Malcolm's -1 and my own feeling that it would be a bad idea to make the test suite hide this, I'm going to close this wontfix.

Add/Change #11420 (Test suite and default encoding on Windows)




Change Properties
Action