id,summary,reporter,owner,description,type,status,component,version,severity,resolution,keywords,cc,stage,has_patch,needs_docs,needs_tests,needs_better_patch,easy,ui_ux 23317,StaticFileStorage Encoding error on '\xf4',Klemen Sever,nobody,"While using the `collectstatic` command with `django-pipeline` and a i18n js library (containing lots of foreign characters), I stumbled on an encoding error: {{{ Traceback (most recent call last): File ""./dev_manage.py"", line 10, in execute_from_command_line(sys.argv) File ""/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4/site-packages/django/core/management/__init__.py"", line 385, in execute_from_command_line utility.execute() File ""/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4/site-packages/django/core/management/__init__.py"", line 377, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File ""/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4/site-packages/django/core/management/base.py"", line 288, in run_from_argv self.execute(*args, **options.__dict__) File ""/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4/site-packages/django/core/management/base.py"", line 337, in execute output = self.handle(*args, **options) File ""/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4/site-packages/django/core/management/base.py"", line 532, in handle return self.handle_noargs(**options) File ""/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py"", line 168, in handle_noargs collected = self.collect() File ""/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py"", line 114, in collect for original_path, processed_path, processed in processor: File ""/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4/site-packages/pipeline/storage.py"", line 32, in post_process packager.pack_stylesheets(package) File ""/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4/site-packages/pipeline/packager.py"", line 94, in pack_stylesheets variant=package.variant, **kwargs) File ""/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4/site-packages/pipeline/packager.py"", line 105, in pack self.save_file(output_filename, content) File ""/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4/site-packages/pipeline/packager.py"", line 117, in save_file return self.storage.save(path, ContentFile(""\xf4"")) File ""/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4/site-packages/django/core/files/storage.py"", line 51, in save name = self._save(name, content) File ""/Users/achedeuzot/Documents/Projects/project/www.project.net/lib/python3.4/site-packages/django/core/files/storage.py"", line 228, in _save _file.write(chunk) UnicodeEncodeError: 'ascii' codec can't encode character '\xf4' in position 0: ordinal not in range(128) }}} After some enquiries, the same error could be reproduced using the following lines: {{{ from django.contrib.staticfiles.storage import StaticFilesStorage from django.core.files.base import ContentFile storage = StaticFilesStorage() storage.save(""test-output.txt"", ContentFile(""ô"")) }}} I was able to ""solve"" the issue by using `bytearray`: {{{ storage.save(""test-output.txt"", ContentFile(bytearray(""ô"", 'utf-8'))) }}} Also, another solution is/was to add `LANG=en_US.utf-8` into the environment. The line that trows the error is the following: {{{ mode = 'wb' if isinstance(chunk, bytes) else 'wt' _file = os.fdopen(fd, mode) _file.write(chunk) # <= this one }}} We can see that if I send bytes, the write() happens without errors while if I send a string, the write() fails. How come Django fails on a ""standard"" utf-8 string ? Is this a Django issue or from something else (OS write(), ...)?",Bug,closed,File uploads/storage,1.7-rc-2,Normal,invalid,UnicodeEncodeError,,Unreviewed,0,0,0,0,0,0