#32319 closed New feature (fixed)
Add support to HashedFilesMixin for ES modules
Reported by: | gilmrjc | Owned by: | blighj |
---|---|---|---|
Component: | contrib.staticfiles | Version: | dev |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Ready for checkin | |
Has patch: | yes | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
The current HashedFilesMixin only supports importing static files in css. When importing es modules in javascript files and using ManifestStaticFilesStorage to collect statics, the javascript code stops working because its using a wrong url for the modules needed.
This change already has a PR (https://github.com/django/django/pull/13843).
Change History (25)
comment:1 by , 4 years ago
Version: | 3.1 → master |
---|
comment:2 by , 4 years ago
Component: | Uncategorized → contrib.staticfiles |
---|---|
Owner: | changed from | to
Status: | new → assigned |
Triage Stage: | Unreviewed → Accepted |
comment:3 by , 3 years ago
Needs documentation: | set |
---|---|
Needs tests: | set |
Patch needs improvement: | set |
comment:4 by , 3 years ago
Needs documentation: | unset |
---|---|
Needs tests: | unset |
Patch needs improvement: | unset |
Triage Stage: | Accepted → Ready for checkin |
comment:9 by , 3 years ago
Has patch: | unset |
---|---|
Resolution: | fixed |
Status: | closed → new |
Triage Stage: | Ready for checkin → Accepted |
We decided to revert the original patch.
comment:10 by , 3 years ago
Replying to Mariusz Felisiak <felisiak.mariusz@…>:
In b7b3bbc8:
Why is there support for export
? I can't find any mention of export with a url here:
https://developer.mozilla.org/en-US/docs/web/javascript/reference/statements/export
Could you please provide an example of how export
with a url works?
Please see my attempt which I am using in my code base:
https://github.com/django/django/pull/15177
It works with the following import statements:
import { name, draw, reportArea, reportPerimeter } from './modules/square.js'; import randomSquare from './modules/square.js'; import {default as randomSquare} from './modules/square.js'; import { newFunctionName, anotherNewFunctionName } from './modules/module.js'; import { function1 as newFunctionName, function2 as anotherNewFunctionName } from './modules/module.js'; import { name, draw, reportArea, reportPerimeter } from './modules/square.js'; import { name, draw, reportArea, reportPerimeter } from './modules/circle.js'; import { name, draw, reportArea, reportPerimeter } from './modules/triangle.js'; import { name as squareName, draw as drawSquare, reportArea as reportSquareArea, reportPerimeter as reportSquarePerimeter } from './modules/square.js'; import { squareName, drawSquare, reportSquareArea, reportSquarePerimeter } from './modules/square.js'; import * as Canvas from './modules/canvas.js'; import { Square } from './modules/square.js'; import { Square, Circle, Triangle } from './modules/shapes.js'; import colors from './modules/getColors.js';
comment:11 by , 3 years ago
I also think that this ticket should more clearly mention why the commit was reverted, that is expose the rationale and failure examples provided by Hervé Le Roy.
follow-up: 13 comment:12 by , 2 years ago
I'd really see this issue going forward. What about re-committing Hervé's patches and creating new release blockers (for 4.2) tickets for the failing use cases?
comment:13 by , 2 years ago
Replying to Claude Paroz:
I'd really see this issue going forward. What about re-committing Hervé's patches and creating new release blockers (for 4.2) tickets for the failing use cases?
I'm afraid that introducing a release blocker on purpose is not something that is acceptable. The main issue with this ticket is the extensive way in which import
/export
syntax is used in JavaScript. As far as I'm aware analyzing JS files with regular expressions is not enough to only sift the import
/ export
statements related with ES modules. Maybe limiting a patch to the .mjs
files would solve the issue, hard to say 😞
follow-up: 15 comment:14 by , 2 years ago
Could we report on the ticket the discussion that led to the commit reverts?
comment:15 by , 2 years ago
Replying to Claude Paroz:
Could we report on the ticket the discussion that led to the commit reverts?
Most of the short discussion is in this PR (I should be more detailed in my comments, sorry.)
follow-up: 24 comment:16 by , 2 years ago
I think it is possible to do this with a regex and still work with the kinds of statements found in higlight.js from #33253. You can tighten up what is matched by being more specfic on the url it matches, based on the specs of what a browser will support for a module specifier.
https://v8.dev/features/modules#specifiers
// Supported: import {shout} from './lib.mjs'; import {shout} from '../lib.mjs'; import {shout} from '/modules/lib.mjs'; import {shout} from 'https://simple.example/modules/lib.mjs';For now, module specifiers must be full URLs, or relative URLs starting with /, ./, or ../.
The collectstatic command should not be changing absolute URLs, only relative ones, so we can and an extra requirement to the (?P<url>) matcher so it must start with a dot or forward slash (?P<url>[\.\/].*?)
This limits what the regex matches, so that the code types in highlight.js, and some other issues I've seen in videojs and ace, are no longer matched.
But it doesn't have to worry about all possible values of the import/export part of the expression, which would be very hard to do with regex.
Here is what I'm using in my project for the js patterns
( "*.js", ( ( r"""(?P<matched>import(?P<import>[\s\{].*?)\s*from\s*['"](?P<url>[\.\/].*?)["']\s*;)""", """import%(import)s from "%(url)s";""", ), ( r"""(?P<matched>export(?P<exports>[\s\{].*?)\s*from\s*["'](?P<url>[\.\/].*?)["']\s*;)""", """export%(exports)s from "%(url)s";""", ), ( r"""(?P<matched>import\s*['"](?P<url>[\.\/].*?)["']\s*;)""", """import"%(url)s";""", ), ), ),
I'm enforcing the need for a semicolon for extra safety, and I've added support for
import{shout}from './lib.mjs';
which was needed for some js coming out of esbuild.
I don't know if //
is supported by browsers for module specifiers, if it is then you'd have to update the regex to not capture those, I haven't needed it for my own projects yet, so I'm keeping my regex simpler for now.
comment:18 by , 2 years ago
Has patch: | set |
---|
comment:19 by , 2 years ago
Owner: | changed from | to
---|---|
Status: | new → assigned |
comment:20 by , 2 years ago
This works with the higlight.js file but not the type definitions from this review, PR15058 (review). Since they are a feature of typescript and not javascript, I think that's a fair trade off.
comment:21 by , 2 years ago
Triage Stage: | Accepted → Ready for checkin |
---|
It's not clear the proposed patch covers all usages, but it looks OK for common cases.
I think we need to be prepared to say where we're not going to try more complex solutions.
But let's see... 😬
comment:24 by , 7 months ago
Replying to blighj:
I'm enforcing the need for a semicolon for extra safety, and I've added support for
Enforcing the ;
for "safety" actually breaks import assertions, e.g.:
import l from"/static/skin/skin/x-triangle.min.css"assert{type:"css"};
I think the following 2 steps will make it much for robust and cover much more use cases in modern web development:
- Remove the ending
;
from the regex's - Add the extra regex from my previous post.
comment:25 by , 5 months ago
Thank for fixing the issue, I just came across a mistake in the regex I posted above:
( r"""(?P<matched>(?P<import_as>import\s*\*as\s\S)+\s+from\s*["'](?P<url>[./].*?)["']\s*;)""", """%(import_as)s from "%(url)s";""", ),
Should be (the plus should be for one of more chars of the import as name, not groups, so the plus moved one char to the left:
( r"""(?P<matched>(?P<import_as>import\s*\*as\s\S+)\s+from\s*["'](?P<url>[./].*?)["']\s*;)""", """%(import_as)s from "%(url)s";""", ),
OK, yes, modules are increasingly important. We should at least see if we can support them. (Initial glance at the PR looks simple enough — too simple? 🤔 :)
Thanks for the report