[SizzleUnrlsd]

Summary

Closes #279.

To collect the filters a render reaches, project_templates followed every path through includes and extends (_reaching). Two includes of one template at each of n levels made it follow 2^n paths, and every directory check runs that walk.

Each template is now walked once for each set of blocks the templates extending it override:

• Same result: a walk keeps its first occurrences, in order, and maps them through each reference. That gives the same filters in the same order as a walk path by path.
• One analysis only: the results are kept in a dictionary that each call to project_templates creates, so projects analysed one after the other in one process do not share them.
• Cycles: a walk cut short by a cycle of includes depends on where the cycle was entered. It is not kept, and it is walked again where it is needed.
• Order of the walk: a walk keeps the templates it went through. It is reused only where none of them is on the way; elsewhere a cycle would cut it, so it is walked again. The filters do not depend on which template is walked first.

Limit. Templates in a cycle of includes are still walked again wherever the cycle cuts their walk. With two includes of one template at each level of such a cycle, the cost stays exponential, as on main.

Commit	Change
test(templates): pin the filters reached through overridden blocks, cycles and successive analyses	the results to keep, those of main
fix(templates): collect each template's filters once per set of overridden blocks	the walk, and its test of time
docs(changelog): announce the linear walk of template filters	CHANGELOG.md
refactor(templates): return a template's walk of filters as a named tuple	_Walk, no behavior change
fix(templates): make the filters of a template independent of the order of the walk	the templates a walk went through, and the case of the review
test(templates): compare the walk of template filters with a walk path by path	1,500 generated projects against the walk of main
docs(changelog): state that templates in a cycle of includes are walked again	the limit above

Review

A walk that no cycle cut was reused even where a template it had gone through was on the way. There, a walk path by path is cut. In the case of the review:

• the child a.html extends b.html and overrides its body;
• b.html includes a.html with v=other in that block.

Walked from b.html, a.html stops at b.html. When a.html was walked first, its kept walk was reused inside b.html, which then got striptags reached with other. When b.html was walked first, it did not. The walk now keeps the templates it went through, and is reused only where none of them is on the way.

Results

Time. level0.html holds {{ name|upper }}, and level{n}.html includes level{n-1}.html twice:

Levels	main	This change
12	0.08 s	0.00 s
16	1.87 s	0.00 s
18	7.19 s	0.00 s
20	37.19 s	0.00 s
40	more than 60 s	0.01 s

The same, with level0.html also including the top level, which closes a cycle through every level:

Levels	main	This change
12	0.15 s	0.14 s
16	3.12 s	3.14 s
18	15.38 s	12.74 s
20	more than 60 s	57.23 s

Same filters.

• Corpus: project_templates(root).filters is identical, order included, on 50 projects: the local corpus (14 Django packages, Django 5.1.4, the regression projects) and the reproductions and controls of templates: constructs left open, and template tags inside comments, hide cross-site scripting; unclosed ones make the inspection quadratic #272 and templates: a block or include inside a parent's script inserts unchecked markup there #273.
• Walk time: it takes no longer: wagtail 0.58 s instead of 0.73 s, with its 1,573 filters.
• Generated projects: the filters are identical on the 1,500 generated projects of the new test.

Tests

The new file tests/test_template_filter_walk.py holds 6 tests, 7 cases. No existing test changes.

• Successive analyses: three projects analysed one after the other in one process, the first again after the second. Each test checks the filters, where they stand, the template applying each, the names reaching their value and argument (an include … with among them), and their order.
• Overridden blocks: a template reached directly and through a child overriding its block.
• Cycles: a template reached inside a cycle of includes, then from outside it.
• Order of the walk: the case of the review, under both namings, so that the child is walked first in one and the parent in the other.
• Walk path by path: the walk of main is written in the test. On 1,500 generated projects of four templates extending and including one another, inside and outside overridden blocks, with cycles among them, the filters are those of that walk. Each include binds the variable to a name of its own, so that what reaches a filter tells the paths apart.
• Time: 30 levels of doubled includes, in a subprocess bounded at 60 s.

Before the fix.

• On main: the test of time does not end within 60 s. The three tests of the first commit pass: they pin the results of main, which the fix keeps.
• On the first version of this change: the case of the review fails when the child is walked first, and so does the comparison with the walk path by path.

Mutations. Each of these makes the test written for it fail:

• the cache keyed without the blocks overridden;
• the results cut short by a cycle kept;
• one cache for every analysis;
• a kept walk reused without checking the templates it went through;
• those templates reduced to the template walked;
• those templates without the template walked.

Verification

Python 3.12, on main at 0da4cc8:

• ruff check . and mypy: clean on each commit;
• unit suite: 2,630, 2,631, 2,631, 2,631, 2,633, 2,634 and 2,634 passed, commit after commit;
• regression: 19 of 19, snapshots unchanged;
• Enterprise plugins: 978 passed.

This change and the fix of #278 touch different parts of taint/templates.py. Merged in either order, they give the same tree, on which ruff and mypy are clean, the unit suite passes (2,649) and the regression too (19 of 19).