Opened 11 years ago

Closed 10 years ago

#20375 closed Bug (duplicate)

Post-processing CSS file with Unicode chars fails

Reported by: Bouke Haarsma Owned by: nobody
Component: contrib.staticfiles Version: dev
Severity: Normal Keywords:
Cc: yeo.eugene.oey@…, joel@… Triage Stage: Accepted
Has patch: no Needs documentation: yes
Needs tests: yes Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

Say I have a static file like this: (icons.css)

.icon-home:before { content: '\2302'; } /* '⌂' */
.icon-play-circle2:before { content: '\e048'; } /* '' */
.icon-location:before { content: '\e724'; } /* '' */
.icon-map:before { content: '\e727'; } /* '' */
.icon-twitter-1:before { content: '\f309'; } /* '' */
.icon-facebook-1:before { content: '\f30c'; } /* '' */
.icon-doc:before { content: '📄'; } /* '\1f4c4' */
.icon-calendar:before { content: '📅'; } /* '\1f4c5' */
.icon-search:before { content: '🔍'; } /* '\1f50d' */

When this file is being post-processed (in collectstatic), an exception is raised:

Traceback (most recent call last):
  File "/Applications/PyCharm.app/helpers/pydev/pydevd.py", line 1481, in <module>
    debugger.run(setup['file'], None, None)
  File "/Applications/PyCharm.app/helpers/pydev/pydevd.py", line 1124, in run
    pydev_imports.execfile(file, globals, locals) #execute the script
  File "/Volumes/Users/bouke/Sites/gct/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "lib/python2.7/site-packages/django/core/management/__init__.py", line 443, in execute_from_command_line
    utility.execute()
  File "lib/python2.7/site-packages/django/core/management/__init__.py", line 382, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "lib/python2.7/site-packages/django/core/management/base.py", line 196, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "lib/python2.7/site-packages/django/core/management/base.py", line 232, in execute
    output = self.handle(*args, **options)
  File "lib/python2.7/site-packages/django/core/management/base.py", line 371, in handle
    return self.handle_noargs(**options)
  File "lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 163, in handle_noargs
    collected = self.collect()
  File "lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 120, in collect
    for original_path, processed_path, processed in processor:
  File "lib/python2.7/site-packages/django/contrib/staticfiles/storage.py", line 226, in post_process
    content = pattern.sub(converter, content)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 500: ordinal not in range(128)

Attachments (1)

testproj.zip (7.7 KB ) - added by Markus Amalthea Magnuson 10 years ago.
Test project for testing collectstatic

Download all attachments as: .zip

Change History (12)

comment:1 by anguslturner, 11 years ago

Component: Uncategorizedcontrib.staticfiles
Needs documentation: set
Needs tests: set
Triage Stage: UnreviewedAccepted
Version: 1.4master

comment:2 by djangsters, 11 years ago

How can I reproduce this. Here it works just fine.

comment:3 by Łukasz Balcerzak, 11 years ago

Resolution: invalid
Status: newclosed

Cannot reproduce too. Am closing for now as it doesn't seem like a bug in Django.

Please reopen once you can reproduce. Hints: run collectstatic outside of PyCharm and check css file encoding.

comment:4 by Eugene, 11 years ago

Cc: yeo.eugene.oey@… added
Patch needs improvement: set
Resolution: invalid
Status: closednew
Type: UncategorizedBug
Version: master1.4

Reproduce the Error

I encounter the same issues. And after tracing the type,
It happened on https://github.com/django/django/blob/1.4/django/contrib/staticfiles/storage.py#L226,
when trying to replace a unicode type into a string type that has a unicode character.

import re

patt = re.compile(r"""(url\(['"]{0,1}\s*(.*?)["']{0,1}\))""")

patt.sub(lambda x: u'The replacement is a function that return unicode', 
    """The content is str with at least a unicode character głó, 
    and don't forget to match url(../some/path)""")

Patching??

I tried to convert the type into string at https://github.com/django/django/blob/1.4/django/contrib/staticfiles/storage.py#L172 and it works

return str('url("%s")' % unquote(relative_url))

comment:5 by Eugene, 11 years ago

Version: 1.4master

by Markus Amalthea Magnuson, 10 years ago

Attachment: testproj.zip added

Test project for testing collectstatic

comment:6 by Markus Amalthea Magnuson, 10 years ago

I can't reproduce this. These are the steps I tried:

  1. Start a new project.
  2. Start a new app.
  3. Create a static folder in the new app, and put the icons.css from above in it.
  4. Edit settings, setting STATIC_ROOT to e.g. a local folder staticfiles and adding your new app to INSTALLED_APPS`
  5. Run collectstatic in your project folder

All files are copied, including icons.css, without errors.

I tested this using the current master branch of Django, on Python 2.7.6, on a Mac.

Attached is my test project, as described.

comment:7 by anonymous, 10 years ago

Duplicated this error when attempting to 'collectstatic' on:
http://cdn.jsdelivr.net/semantic-ui/0.18.0/css/semantic.css

Verified that ye0eugene0ey's patch from above resolves the issue.

Python 2.7.3 on Fedora

comment:8 by anonymous, 10 years ago

Cc: joel@… added

comment:9 by Claude Paroz, 10 years ago

Those having succeeded to reproduce, could you please also tell the Django version you are using?

comment:10 by slumtrimpet <joel@…>, 10 years ago

Good question... it errors on 1.4.1 and works on 1.5.4.
(those are the two env's I have handy)
(I'm 'anonymous' from 6 hours ago)

comment:11 by Claude Paroz, 10 years ago

Resolution: duplicate
Status: newclosed

Then I guess this is a duplicate of #18430.

Note: See TracTickets for help on using tickets.
Back to Top