[SizzleUnrlsd]

Summary

Closes #284.

To collect the filters a render reaches, project_templates walked each include and extends, and cut a cycle at a template already on the way. Two consequences followed.

Filters missed. Django renders a cycle as far as the project's conditions let it turn, so the walk missed what the next levels apply:

• a template including itself with other bindings;
• a recursive tree;
• a parent including the child that extends it, where Django applies the parent's filters to what the include binds.

Time. A walk the cycle cut was walked again on every path. 20 levels of doubled includes closed into a cycle took 56 s.

What each template reaches, with the blocks the templates extending it override, is now the least fixpoint of what it applies and what its references reach, mapped through their bindings. The design and its measurements are in the issue.

Computation.

• Graph: the pairs (template, blocks overridden) a render reaches are grouped into strongly connected components, by Tarjan's algorithm without recursion.
• Order of components: each component is computed after those its references reach.
• Inside a component: every set starts empty. A pair is recomputed when a set it reads grows, until none grows.

Guarantees.

• Convergence: the sets only grow, in a finite domain. The result is the least fixpoint, whatever the order of the recomputations.

• No partial result: no list is built before its component has settled, and no cycle is cut.

• Order:

  • a pair outside any cycle keeps the order of a walk of each path, so where no cycle is reached, nothing changes, order included;
  • a pair in a cycle lists what it applies, then the rest by template, position, function and names.

  The order depends only on the settled sets and the templates, not on the order of the computation.

_reaching, _Walk and the cache of #279 are replaced.

Commit	Change
fix(templates): collect the filters a render reaches as a least fixpoint over cycles	the fixpoint, the two existing tests that pinned the cut, and the reproduction
test(templates): cover templates including themselves with other bindings, and trees	recursion and tree
test(templates): bound the time of collecting filters through cycles of includes	time
test(templates): check the filters are a fixpoint whatever the order of computation	closure and order of computation
docs: state that a cycle of includes passes its bindings at every level	docs/usage.md
docs(changelog): announce the filters reached through cycles of includes	CHANGELOG.md
fix(templates): join the names reaching each filter application in a cycle	review: one fact per filter application in a cycle
test(templates): bound the time of bindings permuted around a cycle of includes	review: the shape that blew up

Review

Each combination of the names reaching a filter's value and argument was its own fact. Bindings permuted around a cycle therefore made one fact of every combination. The review's template is 272 bytes: a filter chain over v0 to v6, including itself twice, once rotating 14 variables and once swapping v0 and v1.

Check	main	ad4dd04	With the fix
collecting the filters	0.0006 s, 7 filters	12.03 s, 60,762 filters	0.0023 s, 7 filters
full CLI analysis	0.35 s	stopped	0.35 s

The join. In a cycle, the names reaching the value, and those reaching the argument, are now joined for each filter application: one fact per application.

• Nothing is lost: the analysis reads those names position by position, joining what each brings. That is true of the taint flows and of the summaries, which also merge the calls at one place. Mapping through a reference distributes over the union.
• Cost: an application's names grow at most once per name, so a cycle settles in time polynomial in its pairs, applications and names.
• Outside cycles: nothing changes.

No bound cuts the computation short: the result is still the complete least fixpoint, joined by application in cycles.

Tests adapted.

• The recursion and tree tests now expect one call per filter, with the joined names.
• The fixpoint property checks closure by application in cycles, and one fact per application.
• The comparison with the walk path by path compares the names joined at each application.

The permuted shape, with 24 variables, is a new test of time. On ad4dd04, it exceeds 60 s, and the recursion, tree and fixpoint tests fail.

Mutations. Each of these makes tests fail:

• names replaced instead of joined: 4 tests;
• names joined outside cycles too: 4 tests, among them the order checks.

The design comment of #284 is updated with the join.

Results

Django. The bench generates 1,500 projects of four templates extending and including one another, with cycles. Django 5.1.15 renders each template, its filters recording the value they receive.

Templates Django renders	Missing something Django applies
without conditions, 1,249	main 0, this change 0
includes under a condition, 3,179	main 467, this change 0

The new tests expect what Django applies when it renders their templates.

Independent reference. It walks every path, letting each cycle turn 3, then 4 times.

• 1,495 projects: the reference converged, and the sets are equal.
• 5 projects: the reference exceeds its budget, so they are not verified by it.

Closure. On every pair of the 1,500 projects, the result is what the pair applies, with what its references reach mapped through them.

Order of computation. The lists are the same, order included, with the templates inserted in three other orders: 4,500 runs.

Against main.

• Generated templates no cycle reaches: identical, order included, on all 1,094.
• Local corpus: on 53 projects (14 Django packages, Django 5.1.4, the regression projects, the reproductions of templates: constructs left open, and template tags inside comments, hide cross-site scripting; unclosed ones make the inspection quadratic #272 and templates: a block or include inside a parent's script inserts unchecked markup there #273), the filters are identical, order included.

Time. project_templates alone, Python 3.12:

Shape	main	This change
chain of doubled includes, 60 levels	0.01 s	0.02 s
the same closed into a cycle, 20 levels	55.90 s	0.01 s
the same closed into a cycle, 60 levels	more than 60 s	0.01 s
9 templates each including every other	11.95 s	0.01 s
40 templates each including every other	more than 60 s	0.13 s
200 templates in a cycle, each binding the variable anew	4.23 s	0.19 s
the review's permuted bindings, 14 variables	0.0006 s	0.0023 s

End to end. The reproduction of #284 reports exploitable-vulnerability at app/views.py:5, through b.html:1, whether the view passes the request as v or as other.

Tests

Existing tests changed. Both pinned the cut walk, in tests/test_template_filter_walk.py, 3 cases. The module's docstring follows.

• test_a_child_its_parent_includes_back_gets_the_same_filters_whichever_is_walked_first, both cases. The parent's list gains striptags reached by other, last: Django renders b.html applying striptags to other. The comment describing the case changes accordingly.
• The comparison with the walk path by path. It is renamed test_the_filters_are_those_of_every_path_with_each_cycle_turned_until_it_settles. Its reference lets each cycle turn 2, then 3 times, within 20,000 steps.
  • Where both bounds give the same sets, every template is compared as a set, and those no cycle reaches also in order.
  • A project the reference does not settle is counted as not verified, never as equal: 10 of the 1,500.

New file. tests/test_template_filter_cycles.py holds 7 tests, 8 cases:

• Reproduction: the case of templates: the filter walk is exponential in a cycle of includes, and cuts a cycle Django renders through #284 end to end, with the request passed as other, and as v for the control.
• Recursion: a template including itself with v=n0, one call of upper reached by v and n0.
• Tree: the recursive tree, one call of upper reached by node.name and node.children.
• Time: a cycle of 60 levels of doubled includes, and 40 templates each including every other, each in a subprocess bounded at 60 s.
• Permuted bindings: the review's shape with 24 variables, bounded at 60 s.
• Fixpoint: on 500 generated projects, closure on every pair (joined by application in cycles), and the same lists whatever order the templates are inserted in.

Before the fix. On main, 9 tests fail:

• Reproduction: [] instead of a finding.
• Recursion and tree: n0 and node.children are missing.
• Time: both exceed 60 s.
• Fixpoint: _renders does not exist yet.
• The two existing tests changed: striptags reached by other is missing, and the sets differ from the reference.

The control case passes on main.

Mutations. Each of these makes at least one test fail:

• readers not queued again when a set grows;
• only one pair of a cycle computed first;
• a template including itself taken for acyclic;
• the order in a cycle left to the sets;
• the order outside cycles made canonical;
• a cycle cut, its members reading nothing from one another.

Verification

Python 3.12, on main at 926df83, in an environment installed as the CI installs it (pip install -e ".[dev]"):

• ruff check . and mypy: clean on each commit;
• unit suite: 2,651, 2,653, 2,655, 2,656, 2,656, 2,656, then 2,656 and 2,657 for the review's commits, commit after commit;
• regression: 19 of 19, snapshots unchanged;
• Enterprise plugins: 978 passed.

Out of scope

The walk counts a block as rendered when a template extending another defines it and no parent does. Django never renders such a block. That over-approximation is #285; this change leaves it as it is.