non-english filenames broken on Windows after r10693

[vrehak]

The changeset r10693 fixing #11030 (File uploads break on non english
filesystem encoding) introduced a problem with non english filenames
in my setup (Windows XP or 2003, Python2.6, Apache). If I upload a
file with czech characters (e.g. čeština.txt - the first character is
ccaron or \u010d, the 3rd is scaron or \u0161) the file created on
Windows filesystem has a crippled name (Ä«eÅ·tina.txt).

The problem is caused by smart_str converting the unicode string into
bytestring. If I revert r10693 and let the filename be passed to
os.open() as unicode string, the file is correctly created on the
filesystem. I think that smart_str should be called only if the
underlying filesystem cannot handle unicode characters in the
filename.

Originally reported at ​http://groups.google.com/group/django-developers/browse_thread/thread/a8672274e4de5bad

[Karen Tracey]

(In [12662]) [1.1.X] Fixed #12898: Reverted a change that assumed the file system encoding was utf8, and changed a test to demonstrate how that assumption corrupted uploaded non-ASCII file names on systems that don't use utf8 as their file system encoding (Windows for one, specifically). Thanks for the report to vrehak.

r12661 from trunk.

[Karen Tracey]

(In [12676]) Fixed the files test to not care whether the names are reported as bytestrings or unicode. Refs #12898.

[Karen Tracey]

(In [12677]) [1.1.X] Fixed the files test to not care whether the names are reported as bytestrings or unicode. Refs #12898.

r12676 from trunk.