[rparundekar]

Summary

New agentic recipe, agentic/screen_applicants_through_mcp_server: a blind first pass over applicants for a commercial HVAC technician role at a made-up facilities company.

• server.py is an MCP server built for this one job. It runs over streamable HTTP and needs a bearer token. Its tools are get_role, list_applicants, read_resume, find_evidence and search_applicants.
  • It calls three models on SIE: LightOnOCR-3-4B for scans, superlinked/pii-redact, and Qwen3-Reranker-4B.
  • Raw resume text goes only to the SIE models that read and mask it. Only the masked text reaches Claude.
  • A failed read or mask returns an error that carries no resume text.
• recipe.py is a Claude Agent SDK client.
  • Claude gets no built-in tools: only the server's tools and a local submit_decision. The run ignores the reader's own MCP servers and settings.
  • Python then checks that every quote is in the screened applicant's masked resume, that there is one verdict per must-have, and that the decision follows the rules.
  • Last, it checks that nothing Claude saw or wrote contains any applicant's name, email or phone number.
• The README links to SIE's general sie-mcp server, for the general document case.

Results

Four live runs using SIE Serverless on 9 Nov 2026: three before the self-review and one after. All four gave 6 of 7 applicants as expected, the same each time, with no identity leaks.

• Cost: a run makes 30 SIE calls and costs about $0.31.
• The miss is A-104 (license suspended until 2027). The written rule ("valid license today") rejects it, where we expected it to go to a person. The README explains this rather than tuning anything to hide it.
• Sonnet 5.5 as the agent gave the same statuses for $0.16.

Self-review

All six passes ran: code, silent failures, tests, comments, types and security. The fixes are in 09cc783. The two that matter most:

• The leak check had a blind spot. It missed a name at the start of a line, and a name with an accent. It now decodes the JSON first. Re-run over every saved transcript, it found no leaks.
• Tool errors could carry SIE's message, which can echo raw resume text. They no longer include it.

The details are in the PR comments.

Checks

• uvx ruff check . && uvx ruff format --check .
• mypy --strict
• uv run selfcheck.py
• tools/check_tree.py, pre-commit
• write-like-me pass on the README prose
• live run after the fixes

Related: superlinked/sie#936 fixes the sie-mcp issues found while checking whether to use it here. This PR doesn't depend on it.

🤖 Generated with Claude Code

Generated using wayfare.

[rparundekar]

Self-Review

Six passes ran on the rebased head 12df1cf: code, silent failures, tests, comments, types and security.

Critical (3)

• [silent-failure-hunter] recipe.py:203: the leak check reads tool results as raw JSON, where a newline is the two characters \n, so a name at the start of a line, or a non-ASCII name in a decision note, is never matched. "No identity leaks" was checked with a blind spot.
• [silent-failure-hunter] server.py:324: a SIE error message can carry the request input (a 422 echoes it), and the ToolError passes up to 200 characters of it to Claude, which can include raw resume text.
• [comment-analyzer] server.py:13, README.md:30: "raw resume text never leaves this process" is false; it goes to SIE to be read and masked. It never reaches the MCP client.

Important (9)

• [code-reviewer, type-design-analyzer, silent-failure-hunter] recipe.py:279: quotes are checked against whichever resume Claude read last, not the screened applicant's, so a decision quoting another applicant's resume passes.
• [code-reviewer] recipe.py:282: total_cost_usd is a running total for the session, so a revision double-counts cost.
• [code-reviewer, silent-failure-hunter] recipe.py:492: on Ctrl-C or a stop during a cold start, wait(timeout=10) raises, hides the real error and leaves the server on the port; the log file is never closed.
• [silent-failure-hunter] recipe.py:450: a server that dies mid-run is not noticed; every later applicant comes out no_decision.
• [type-design-analyzer, silent-failure-hunter] recipe.py:277: any tool result containing "SIE refused", including resume text, stops the run.
• [type-design-analyzer, silent-failure-hunter] recipe.py:144: a repeated requirement id lets the CSV show a different verdict from the one checked.
• [silent-failure-hunter] server.py:308: OCR keeps only the first markdown entity, even if it is empty.
• [code-reviewer, comment-analyzer] recipe.py:124, rules.yaml: the verdict-to-decision mapping is in Python, while the README says the decision rules are in rules.yaml.
• [pr-test-analyzer] selfcheck.py: 27 of 29 one-line breaks to the logic go unnoticed, including the bearer-token check, the masking windows and the "met with no quote" check.

Suggestions (11)

• [silent-failure-hunter] recipe.py:290: hitting max_turns is not recorded anywhere.
• [silent-failure-hunter] recipe.py:342: health() catches only OSError.
• [silent-failure-hunter] server.py:152: a pii-redact span whose text does not match the window is accepted.
• [silent-failure-hunter] server.py:348: a malformed score reply raises AttributeError inside the validator.
• [silent-failure-hunter] server.py:454: require_token only guards http scopes; guard everything but lifespan.
• [silent-failure-hunter] recipe.py:179: an empty email in the record always reports a leak.
• [silent-failure-hunter] recipe.py:171: the leak check only knows the screened applicant's record, not the others search_applicants can show.
• [type-design-analyzer] server.py:318: cached reads wait behind another applicant's cold-start OCR.
• [comment-analyzer] README.md:217, 251: "the same phone number written two ways" is only in selfcheck; A-106's 608 is unclear, not missing.
• [comment-analyzer] server.py:7, server.py:3, rules.yaml:11, rules.yaml:72, sie_calls.py:22: docstrings and comments that overstate or are out of date.
• [comment-analyzer] README.md:154: the token and cost ranges leave out the first run.

Strengths

• Security pass: no findings. The token compare is constant-time, the Host check is on, paths are confined, CSV cells are escaped, and the agent has no tool that could send raw text out.
• Masking fails closed: a failed or malformed SIE reply returns an error and nothing half-masked is cached.
• The SDK validates submit_decision against the schema before the handler runs.

---

Generated using wayfare.

[rparundekar]

Self-Review: Improvements

Critical (3 / 3 fixed):

• recipe.py:203: leak check could not see a name at the start of a line. Fixed: tool results are decoded first, accents and quotes folded, and every applicant's record is checked. Re-run over all 29 saved transcripts: no leaks.
• server.py:324: SIE error text could carry raw resume text to Claude. Fixed: tool errors name only the error type; the message goes to server.log.
• server.py:13, README.md:30: wrong claim about where raw text goes. Fixed: it goes only to the SIE models that read and mask it, never to Claude.

Important (9 / 9 fixed):

• recipe.py:279: quotes checked against the last resume read. Fixed: only this applicant's read_resume counts.
• recipe.py:282: cost double-counted on a revision. Fixed: uses the session's running total.
• recipe.py:492: cleanup could hide the error and leave the server up. Fixed: kill after 10 s, close the log handle.
• recipe.py:450: a dead server went unnoticed. Fixed: checked before each applicant.
• recipe.py:277: resume text could stop the run. Fixed: only an error result starting "SIE refused" does.
• recipe.py:144: a repeated must-have passed. Fixed: one verdict per must-have.
• server.py:308: OCR kept only the first markdown entity. Fixed: joins every non-empty one.
• recipe.py:124: decision-rule location. Fixed: README and rules.yaml say the order lives in expected_decision.
• selfcheck.py: thin coverage. Fixed: token check, windows and offsets, the fail-closed store, decision, leak and output cases.

Suggestions (10 / 11 fixed):

• recipe.py:290: max_turns now gives an out_of_turns status.
• recipe.py:342: health() catches bad replies.
• server.py:152: a pii-redact span must match its text.
• server.py:348: score validator checks entries are dicts.
• server.py:454: require_token guards every scope but lifespan.
• recipe.py:179: an empty email no longer reports a leak.
• recipe.py:171: other applicants' details are checked too.
• server.py:318: cached reads skip the lock.
• README.md, docstrings, rules.yaml comments, token and cost ranges corrected.
• sie_calls.py: Calls is updated under a lock.

Skipped:

• recipe.py:290: an API error result still stops the whole run instead of failing one applicant. Skipped: a 5xx from the API usually hits every applicant, and stopping keeps the run honest.

Also found while re-running: the Claude Code CLI aborts an MCP tool call after 300 s, shorter than a cold OCR start. recipe.py now sets CLAUDE_CODE_MCP_TOOL_IDLE_TIMEOUT from agent.tool_wait_s (960 s).

Live run after the fixes: same seven statuses, 6 of 7 as expected, no leaks.

Commits: 09cc783

---

Generated using wayfare.