[SizzleUnrlsd]

Summary

Closes #273.

Each Django template was read on its own, so the inspection could not see what a {% block %} or an {% include %} puts where it stands:

• F: a child's block that prints {{ name }} inside its parent's <script> was read as HTML data in the child.
• H: an include placed inside an attribute value was read on its own as harmless text, even when that text ends the value (" onclick=").

In both, Django renders a variable that runs as script, and no xss was reported.

Each page is now also put together as Django renders it, then read once more by the reader of #272:

• the templates it extends, each block taking the definition of the most derived one, and {{ block.super }} the next one up;
• the page of each template it includes, in place;
• a child's text outside its blocks is left out, as Django does not render it.

A template escapes when the inspection of each file on its own establishes it, as before, and its pages, put together, read as escaping. The composition adds a check; it removes none.

Rule

A name several files have. Django renders the first file its loaders find, which the engine does not know.

• As a parent: the page is put together with each file, at each level of the chain, and must escape with each.
• As an include: the page inserts a marker standing for each file. The marker is refused in any attribute (name or value, quoted or not, whatever the attribute), a tag name, an end tag, a <script> or <style> block and a comment, and right after <, </, <! or <!-, where a file could end the construct or complete one. In HTML data, it is accepted when each file escapes and ends in data, with no construct, no comment and no <, </, <! or <!- left open at its end.
• Reading the marker like a variable is not enough. In the variant of H where q.html exists in two template directories, one of them holding " onclick=", a variable is allowed in the quoted title. Yet Django 5.1.15 renders <a title="" onclick="alert(1)">x</a> when that directory comes first.

A template rendering several pages, because a name in its chain has several files, inserts the same marker where another includes it.

An include on the way to itself, directly or through others, inserts a marker standing for the template's own page, under the same conditions. Property tests prove it on generated templates (below).

Bounds, checked while a page is put together:

• 8 MiB per page;
• 64 pages per template, and 16 MiB for its pages together.

Beyond a bound, the template is not established, and it is listed in Coverage.unread_templates with the reason.

Commit	Change
refactor(templates): parse a template into its markup, blocks and includes	the structure the composition needs; verdicts unchanged
fix(templates): read the page each template renders, its blocks and includes in place	F, H, H's variant, the 8 MiB bound
fix(templates): accept in HTML data an include of a name several files have	the marker, its rules, the splicing property
fix(templates): check a page with each file the name of its parent may be	each parent candidate, 64 pages and 16 MiB
fix(templates): insert a marker for an include on the way to itself	recursion, the unfolding properties
feat(templates): list the pages that cannot be put together among templates not read	template_escaping, Coverage.unread_templates
docs: describe how a page is put together to read Django templates	docs/usage.md, docstring, CHANGELOG.md
fix(templates): keep the alert when block.super is printed through a filter or a tag	review: {{ block.super|escape }}
fix(templates): put each block definition of a page together once	review: work doubling with {{ block.super }}
fix(templates): leave out the files already in the chain when a template extends its own name	review: a template extending its own name
docs: state how block.super and a template extending its own name are read	docs/usage.md, CHANGELOG.md
fix(templates): keep the alert wherever the variable block is named but in a bare block.super	second review: an alias of the block
docs: state that the variable block is read only in a bare block.super	docs/usage.md, CHANGELOG.md

Review

Three points on the first seven commits.

{{ block.super }} through a filter (P1). Only a bare {{ block.super }} was put in place; with a filter, it read as an escaped variable. Django still prints the parent's markup there, marked safe, and escape or default keep it as it is. With <a title="{% block attr %}" onclick="{% endblock %}">x</a> as the parent, {{ block.super|escape }}{{ name }} in the child renders <a title="" onclick="alert(1)">x</a>, and nothing was reported. The same holds for {% with parent=block.super %}{{ parent }}{% endwith %}. A use of block.super other than a bare {{ block.super }}, in a variable or a tag, now leaves the template not established. All 532 uses in both corpora are bare.

Work doubling before any bound (P2). The definitions a page renders were walked again at every {{ block.super }}, before the bounds applied. In a chain whose every block renders its parent twice, the work doubled at each level, though the page stayed empty: 20 levels took 16 s, against 0.3 s on main. Each definition is now put together once, then reused, and the includes are collected visiting each definition once. 60 levels take 0.1 s, and the 8 MiB bound applies to each definition as it grows.

A template extending its own name (P2). Django's loaders skip the files an extends chain already holds, so a page.html extending 'page.html' extends another file of that name. Such a chain was taken for a cycle, and the page reported as xss, though Django keeps the data escaped. The files already in the chain are now left out, and each remaining one is checked. With none left, the page is not established, as Django raises an error.

An alias of the block (second round). The check looked for block.super itself, and an alias of the block slipped past it: {% with parent=block %}{{ parent.super }}{{ name }}{% endwith %}, through |escape too. A template naming the variable block anywhere but in a bare {{ block.super }} is now not established, whatever the use: an alias, an attribute, a filter, a tag. This includes an {% include … with b=block %}, a prudent false positive: Django renders an include apart from the blocks around it, and block.super gives "" there. For the same reason, a bare {{ block.super }} in an included template renders nothing, as in Django. Across both corpora, the variable block is named elsewhere only by wagtail templates, which already use tags of their own and are not established, and the name is never an alias of the block there.

Results

Corpora.

• Local: 14 Django packages (wagtail, django-oscar, django-cms, weblate, Mezzanine…), Django 5.1.4 and the regression projects.
• Remote battery: 14 public projects, among them django, sentry, saleor, zulip and superset.

The check for a Python file turning autoescaping off is ignored here, so that every project's templates count.

Corpus	Established on main	With this change	Lost	Gained
local	634	614	20	0
remote	609	587	22	0

What is lost. Every loss is a template that includes django/forms/widgets/attrs.html inside an <input> tag: Django's form widgets, and wagtail's single_checkbox.html. attrs.html prints {{ name }} as an attribute name, which the inspection already refuses when a single template writes it. The time of escaped_templates does not grow on any project.

Reproductions.

• templates: a block or include inside a parent's script inserts unchecked markup there #273: F, H and H's variant are reported.
• templates: constructs left open, and template tags inside comments, hide cross-site scripting; unclosed ones make the inspection quadratic #272: the eight projects stay reported.
• Controls: these stay established:
  • a base template with <script>{% block js %}{% endblock %}</script>, rendered alone;
  • a page whose js block holds no variable, and a sibling that prints {{ name }} in its body;
  • {{ name }} in a block placed inside class="…";
  • {{ block.super }} in a script block;
  • an include without a quote inside a title;
  • nested blocks;
  • a tree template including itself in HTML data.

Tests

The new file tests/test_template_page_composition.py holds 25 tests, 51 cases. No test existing on main changes.

Before each fix:

• Composition: on the commit before it, the 7 tests of the new behaviour fail and the 4 controls pass.
• Includes of a name several files have: acceptance in HTML data and the splicing property fail; the 14 refusals pass already, the previous commit refusing every such include.
• Parents several files give: 5 tests fail, and the guard passes.
• Recursion: the established tree and the end-to-end unfolding property fail.
• Listing: the 3 listing tests fail.
• Review: on the seventh commit, the three tests of block.super through a filter or a tag fail; the doubling chain does not end within 60 s; the template extending its own name, rendered escaped by Django, is reported. The bound on a chain doubling its content, and the two guards beside the template extending its own name, pass already. On the eleventh commit, the two aliases of the block fail, and the bare {{ block.super }} of an included template passes.

Proof by tests for the marker.

• Splicing: every text of up to two pieces around a marker, and 20,000 random ones, are checked. Where the reader accepts the marker, any accepted text that ends in data, put in its place, keeps the page accepted and ending in data.
• Unfolding, at the level of the reader: every text of up to two pieces before and after a marker is checked. A text accepted with the marker where it inserts itself, ending in data, stays accepted unfolded three times.
• Unfolding, end to end: 3,000 generated templates include themselves, with or without a condition around the include; escaped_templates establishes 1,260 of them. For each, the recursion written out as templates including each other, four levels down, is established too.

Mutations. Each of these makes at least one test fail:

• the marker allowed in a start tag, a script or style block, a comment, an end tag, or after <, </, <! or <!-;
• the marker read as a variable;
• a text ending in an opener, a comment or a construct left open counted as ending in data;
• the files of an inserted name not required to end in data, including for a recursive include;
• a recursive include read as nothing;
• only the first parent candidate checked;
• a template rendering several pages put in place as its first;
• no bound on the number of pages, or on their size together;
• block.super allowed through a filter, or in a tag;
• each definition put together again every time it is rendered;
• a parent of the same name taken for a cycle;
• the variable block allowed in a variable, or in a tag;
• a bare {{ block.super }} refused too.

Compatibility

• Findings: xss may be reported where it was not, on the pages listed under Results and on any page of these shapes.
• API, additive: template_escaping(root) returns the names of the templates established to escape and the entries of the pages that could not be put together. escaped_templates(root) is unchanged.
• Coverage.unread_templates may list such pages. A VEX statement that relies on every template having been read then stays under investigation.

Not in this change

• The includer rule of templates: constructs left open, and template tags inside comments, hide cross-site scripting; unclosed ones make the inspection quadratic #272. A template including one that ends inside a tag or an end tag is still refused, though the composition now sees the markup that follows, which is harmless in three of the cases its tests pin. Dropping the rule would change those tests, so it stays.
• Two problems found while proving the marker, each in its own issue:
  • A comment holding a template tag (templates: a comment holding a template tag, closed later inside an attribute, hides an xss #278). The comment ends at its first template tag (templates: constructs left open, and template tags inside comments, hide cross-site scripting; unclosed ones make the inspection quadratic #272). A tag that renders nothing leaves the comment running to a later -->, and the markup after that --> is read from the wrong state. This is a regression of templates: constructs left open, and template tags inside comments, hide cross-site scripting; unclosed ones make the inspection quadratic #272, reported on 0.23.0, and not released.
  • Filter calls collected in exponential time (templates: collecting filter calls takes time exponential in nested includes #279). project_templates follows each path through nested includes. Two includes of one template at each of 18 levels take 0.8 s, and the time quadruples every two levels.

Verification

Python 3.12, on main at abd028c:

• ruff check . and mypy: clean on each commit;
• unit suite: 2,576, 2,587, 2,603, 2,609, 2,613, 2,616 and 2,616 passed, commit after commit, then 2,619, 2,621, 2,624 and 2,624 with the review, and 2,627 and 2,627 with its second round;
• regression: 19 of 19, snapshots unchanged;
• Enterprise plugins: 978 passed.