djangodocs Sphinx extension incompatible with other themes

[Mike Edmunds]

The docs/_ext/djangodocs.py catchall Sphinx extension has three problems that make it difficult or impossible to use with other Sphinx themes. These need to be fixed before the docs/_theme/djangodocs Sphinx theme can be replaced with a third-party, well-maintained theme as proposed in #37138.

1. Outdated workaround to remove table borders crashes other themes

The DjangoHTMLTranslator.visit_table() and depart_table() are working around <table border=1> created by the docutils html4 writer. But Sphinx has used the html5 writer (which doesn't emit the obsolete border attribute) since 2.0. The workaround depends on undocumented Sphinx internal attrs and causes an IndexError when used with (e.g.,) sphinx-pydata-theme.

It can and should be removed.

2. console-tabs implementation creates asymmetric DOM

The console-tabs renders structurally different html for the Unix/macOS tab:

<section class="c-content-unix" id="c-content-0-unix">
  <div class="highlight-console notranslate">
    <div class="highlight">...</div>
  </div>
</section>

… compared to the Windows tab:

<section class="c-content-win" id="c-content-0-win">
  <div class="highlight">...</div>
</section>

Notice the expected wrapper <div class="highlight-doscon notranslate">) is missing on the Windows tab. Although this doesn't matter for the djangodocs and djangoproject.com themes (or they have specific workarounds?), it affects spacing in themes like furo and pydata-sphinx-theme. (It would also break a Pygments style that had special rules for the doscon lexer, and it might result in translation tools treating the Windows command shell syntax as English text to be translated.)

One solution is to borrow Sphinx's own machinery to render the Windows literal block, rather than calling the Pygments highlighter directly as now. This would also also simplify the code a bit.

3. console-tabs extension is tightly coupled to djangodocs theme templates

The console-tabs extension relies on a console-tabs.css file that is conditionally included in docs/_theme/djangodocs/layout.html based on context provided by the extension. That doesn't work for third-party themes with their own layout templates. (Or requires maintaining a custom theme inheriting from the third party.)

We should extract console-tabs.css from the djangodocs theme and make it available in the djangodocs extension for any theme. Options:

1. Move console-tabs.css to the project Sphinx static dir (docs/_static) and add it to all pages in docs/conf.py, whether or not they contain console tabs. (Note that docs.djangoproject.com essentially already does this with its customized console-tabs.scss.)
2. Move console-tabs.css to docs/_static but inject it conditionally only in pages that need it in the djangodocs extension. (Extensions can call app.add_css_file() in page-specific hooks since Sphinx 3.5.)
3. Move console-tabs.css to docs/_ext/static and make the djangodocs extension responsible for copying it into the static output. (Requires Sphinx 9.1 or later for app.add_static_dir() support in extensions.)

Also, as part of any rework here, we should eliminate the vendored fa-brands font in the Django docs source (it's used only for console-tabs icons) and update the console-tabs rendering for improved accessibility (see #35874 stage 1)

djangoproject.com considerations

Items 1 and 2 should not affect docs.djangoproject.com.

docs.djangoproject.com will likely want to track any changes to console-tabs.css in its custom console-tabs.scss. That file is used for all recently-rendered versions on docs.djangoproject.com, and older versions are rendered with the older djangodocs Sphinx extension (unless this gets backported). If the rendered DOM for the tabs is changing significantly, we should make it possible for djangoproject.com styles to differentiate docs built with the older tabs from the new tabs (e.g., by changing class="console-tabs" to class="dj-console-tabs").