Opened 2 years ago

Closed 15 months ago

#20375 closed Bug (duplicate)

Post-processing CSS file with Unicode chars fails

Reported by: bouke Owned by: nobody
Component: contrib.staticfiles Version: master
Severity: Normal Keywords:
Cc: yeo.eugene.oey@…, joel@… Triage Stage: Accepted
Has patch: no Needs documentation: yes
Needs tests: yes Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

Say I have a static file like this: (icons.css)

.icon-home:before { content: '\2302'; } /* '⌂' */
.icon-play-circle2:before { content: '\e048'; } /* '' */
.icon-location:before { content: '\e724'; } /* '' */
.icon-map:before { content: '\e727'; } /* '' */
.icon-twitter-1:before { content: '\f309'; } /* '' */
.icon-facebook-1:before { content: '\f30c'; } /* '' */
.icon-doc:before { content: '📄'; } /* '\1f4c4' */
.icon-calendar:before { content: '📅'; } /* '\1f4c5' */
.icon-search:before { content: '🔍'; } /* '\1f50d' */

When this file is being post-processed (in collectstatic), an exception is raised:

Traceback (most recent call last):
  File "/Applications/PyCharm.app/helpers/pydev/pydevd.py", line 1481, in <module>
    debugger.run(setup['file'], None, None)
  File "/Applications/PyCharm.app/helpers/pydev/pydevd.py", line 1124, in run
    pydev_imports.execfile(file, globals, locals) #execute the script
  File "/Volumes/Users/bouke/Sites/gct/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "lib/python2.7/site-packages/django/core/management/__init__.py", line 443, in execute_from_command_line
    utility.execute()
  File "lib/python2.7/site-packages/django/core/management/__init__.py", line 382, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "lib/python2.7/site-packages/django/core/management/base.py", line 196, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "lib/python2.7/site-packages/django/core/management/base.py", line 232, in execute
    output = self.handle(*args, **options)
  File "lib/python2.7/site-packages/django/core/management/base.py", line 371, in handle
    return self.handle_noargs(**options)
  File "lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 163, in handle_noargs
    collected = self.collect()
  File "lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 120, in collect
    for original_path, processed_path, processed in processor:
  File "lib/python2.7/site-packages/django/contrib/staticfiles/storage.py", line 226, in post_process
    content = pattern.sub(converter, content)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 500: ordinal not in range(128)

Attachments (1)

testproj.zip (7.7 KB) - added by giuliettamasina 16 months ago.
Test project for testing collectstatic

Download all attachments as: .zip

Change History (12)

comment:1 Changed 2 years ago by anguslturner

  • Component changed from Uncategorized to contrib.staticfiles
  • Needs documentation set
  • Needs tests set
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted
  • Version changed from 1.4 to master

comment:2 Changed 2 years ago by djangsters

How can I reproduce this. Here it works just fine.

comment:3 Changed 2 years ago by LukaszBalcerzak

  • Resolution set to invalid
  • Status changed from new to closed

Cannot reproduce too. Am closing for now as it doesn't seem like a bug in Django.

Please reopen once you can reproduce. Hints: run collectstatic outside of PyCharm and check css file encoding.

comment:4 Changed 2 years ago by ye0eugene0ey

  • Cc yeo.eugene.oey@… added
  • Patch needs improvement set
  • Resolution invalid deleted
  • Status changed from closed to new
  • Type changed from Uncategorized to Bug
  • Version changed from master to 1.4

Reproduce the Error

I encounter the same issues. And after tracing the type,
It happened on https://github.com/django/django/blob/1.4/django/contrib/staticfiles/storage.py#L226,
when trying to replace a unicode type into a string type that has a unicode character.

import re

patt = re.compile(r"""(url\(['"]{0,1}\s*(.*?)["']{0,1}\))""")

patt.sub(lambda x: u'The replacement is a function that return unicode', 
    """The content is str with at least a unicode character głó, 
    and don't forget to match url(../some/path)""")

Patching??

I tried to convert the type into string at https://github.com/django/django/blob/1.4/django/contrib/staticfiles/storage.py#L172 and it works

return str('url("%s")' % unquote(relative_url))

comment:5 Changed 2 years ago by ye0eugene0ey

  • Version changed from 1.4 to master

Changed 16 months ago by giuliettamasina

Test project for testing collectstatic

comment:6 Changed 16 months ago by giuliettamasina

I can't reproduce this. These are the steps I tried:

  1. Start a new project.
  2. Start a new app.
  3. Create a static folder in the new app, and put the icons.css from above in it.
  4. Edit settings, setting STATIC_ROOT to e.g. a local folder staticfiles and adding your new app to INSTALLED_APPS`
  5. Run collectstatic in your project folder

All files are copied, including icons.css, without errors.

I tested this using the current master branch of Django, on Python 2.7.6, on a Mac.

Attached is my test project, as described.

comment:7 Changed 15 months ago by anonymous

Duplicated this error when attempting to 'collectstatic' on:
http://cdn.jsdelivr.net/semantic-ui/0.18.0/css/semantic.css

Verified that ye0eugene0ey's patch from above resolves the issue.

Python 2.7.3 on Fedora

comment:8 Changed 15 months ago by anonymous

  • Cc joel@… added

comment:9 Changed 15 months ago by claudep

Those having succeeded to reproduce, could you please also tell the Django version you are using?

comment:10 Changed 15 months ago by slumtrimpet <joel@…>

Good question... it errors on 1.4.1 and works on 1.5.4.
(those are the two env's I have handy)
(I'm 'anonymous' from 6 hours ago)

comment:11 Changed 15 months ago by claudep

  • Resolution set to duplicate
  • Status changed from new to closed

Then I guess this is a duplicate of #18430.

Note: See TracTickets for help on using tickets.
Back to Top