Opened 3 months ago

Closed 2 months ago

#35669 closed Cleanup/optimization (fixed)

Improve `RuntimeError: Max post-process passes exceeded.` error

Reported by: Michael Owned by: Jae Hyuck Sa
Component: contrib.staticfiles Version: 5.1
Severity: Normal Keywords: collect static errors
Cc: Michael Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: no

Description (last modified by Michael)

Having just spend 3 hours trying to debug a collect static issue, to help future travellers, I recommend printing the files that caused max depth to be exceeded, often when a file references itself it causes recursion, but the only clue it currently prints is 'All' which does not narrow down the problem. We can surely do better than that! The proposed change prints the problem files only that keep chaning and can't be resolved:

So instead of getting:

Post-processing 'All' failed!

We get the new and improved:

Post-processing 'jsapp/jsapp/notify.min.js' failed!

Or if more than one file:

Post-processing 'jsapp/jsapp/notify.min.js, jsapp/jsapp/somethingelse.min.js' failed!

I recommend changing from:

# contrib/staticfiles/storage.py line 313: in def post_process(self, paths, dry_run=False, **options):
        unresolved_paths = []
        for i in range(self.max_post_process_passes):
            substitutions = False
            for name, hashed_name, processed, subst in self._post_process(
                paths, adjustable_paths, hashed_files
            ):
                # Overwrite since hashed_name may be newer.
                processed_adjustable_paths[name] = (name, hashed_name, processed)
                if subst and i == self.max_post_process_passes - 1:
                    unresolved_paths.append(name)
                substitutions = substitutions or subst

            if not substitutions:
                break

        if substitutions:
            problem_paths_str = ", ".join(unresolved_paths) if unresolved_paths else "All"
            yield problem_paths_str, None, RuntimeError("Max post-process passes exceeded.")

I recommend changing to (see the comments on the right of the proposed changed lines 1/5 to 5/5):

# contrib/staticfiles/storage.py line 313: in def post_process(self, paths, dry_run=False, **options):
        unresolved_paths = []                                            # < -- add this line 1/5
        for i in range(self.max_post_process_passes):
            substitutions = False
            for name, hashed_name, processed, subst in self._post_process(
                paths, adjustable_paths, hashed_files
            ):
                # Overwrite since hashed_name may be newer.
                processed_adjustable_paths[name] = (name, hashed_name, processed)
                if subst and i == self.max_post_process_passes - 1:      # < -- add this line 2/5
                    unresolved_paths.append(name)                        # < -- add this line 3/5
                substitutions = substitutions or subst

            if not substitutions:
                break

        if substitutions:
            problem_paths_str = ", ".join(unresolved_paths) if unresolved_paths else "All"     # < -- add this line 4/5
            yield problem_paths_str, None, RuntimeError("Max post-process passes exceeded.")   # < -- change this line 5/5

Change History (9)

comment:1 by Michael, 3 months ago

Description: modified (diff)

comment:2 by Michael, 3 months ago

Description: modified (diff)

comment:3 by Michael, 3 months ago

Description: modified (diff)

comment:4 by Sarah Boyce, 3 months ago

Triage Stage: UnreviewedAccepted
Type: New featureCleanup/optimization

Adding the suggestion as a diff

  • django/contrib/staticfiles/storage.py

    diff --git a/django/contrib/staticfiles/storage.py b/django/contrib/staticfiles/storage.py
    index 04a5edbd30..5c41fd6828 100644
    a b class HashedFilesMixin:  
    309309
    310310        paths = {path: paths[path] for path in adjustable_paths}
    311311        substitutions = False
    312 
     312        unresolved_paths = []
    313313        for i in range(self.max_post_process_passes):
    314314            substitutions = False
    315315            for name, hashed_name, processed, subst in self._post_process(
    class HashedFilesMixin:  
    318318                # Overwrite since hashed_name may be newer.
    319319                processed_adjustable_paths[name] = (name, hashed_name, processed)
    320320                substitutions = substitutions or subst
     321                if subst and i == self.max_post_process_passes - 1:
     322                    unresolved_paths.append(name)
    321323
    322324            if not substitutions:
    323325                break
    324326
    325327        if substitutions:
    326             yield "All", None, RuntimeError("Max post-process passes exceeded.")
     328            problem_paths_str = ", ".join(unresolved_paths) if unresolved_paths else "All"
     329            yield problem_paths_str, None, RuntimeError("Max post-process passes exceeded.")
    327330
    328331        # Store the processed paths
    329332        self.hashed_files.update(hashed_files)

Appreciate clear error messages so sounds good to me

comment:5 by Jae Hyuck Sa , 3 months ago

Has patch: set
Owner: set to Jae Hyuck Sa
Status: newassigned

comment:6 by Sarah Boyce, 3 months ago

Patch needs improvement: set

comment:7 by Jae Hyuck Sa , 2 months ago

Patch needs improvement: unset

comment:8 by Sarah Boyce, 2 months ago

Triage Stage: AcceptedReady for checkin

comment:9 by Sarah Boyce <42296566+sarahboyce@…>, 2 months ago

Resolution: fixed
Status: assignedclosed

In 2ff0025:

Fixed #35669 -- Improved max post-process passes exceeded error message in HashedFilesMixin.

Signed-off-by: SaJH <wogur981208@…>

Note: See TracTickets for help on using tickets.
Back to Top