Skip to content

feat(insurance): add an agent for one-off reviews of a repair shop's claims - #32

Merged
rparundekar merged 3 commits into
mainfrom
feat/agent-insurance
Nov 11, 2026
Merged

rparundekar merged 3 commits into
mainfrom
feat/agent-insurance

Conversation

@rparundekar

Copy link
Copy Markdown
Collaborator

What this does

Adds agent.py beside the invoice check. An investigator asks about one repair shop's year of claims. The agent runs the invoice workflow on every claim as its verifier. Code builds the per-claim table from those results, and Claude only reports patterns, each labelled a finding or a question. Code then checks that:

  • the claim list is complete;
  • every quote is a value the workflow read;
  • every finding rests on a failed check.

That last check means a closed claim the workflow refused to check can't be turned into a finding.

Live results (5 runs, using SIE Serverless, 10 Nov 2026)

5 of 5 as expected.

  • The refusal: on one run the agent called the closed claim a finding. The code sent it back, and the revision made it a question.
  • Noise: every run also asked a needless question about the shop's own address. It's reported under "Where it fails".

recipe.py and data/ are unchanged. Depends on the docs PR.

🤖 Generated with Claude Code

rparundekar and others added 3 commits November 10, 2026 19:31
…claims

agent.py answers an investigator's question about one shop's year of claims.
It runs the invoice workflow on every claim as its verifier; code builds the
per-claim table from those results, and Claude only reports patterns, each
labelled a finding or a question. Code checks the claim list is complete, every
quote is a value the workflow read, and every finding rests on a failed check,
so a closed claim the workflow refused can't be called a finding.

5 of 5 runs came out as expected. Every run also asked a needless question about
the shop's own address, reported in the README.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The post isn't published yet, so its links returned 404, and its claims will
change in the blog update. The README now links the root README's test and
states each point on its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- An unread field is not a failed check: an invoice whose only flagged
  checks say a value wasn't read ("Not read", conflicting_values, "no
  claim number read") is now not_checked with "a field wasn't read",
  the job is incomplete, and a finding can't rest on it.
- Escape invoice-derived and Claude text in siu-report.md with md_safe;
  the request, summary and notes go in fenced blocks.
- Tools reach only the job's records: claims_for_shop takes only the
  job's shop and year, get_claim only a claim on that list.
- check_report keeps every invoice per claim, not the last one.
- Claude API errors: 408, 409, timeouts and connection errors end the
  job with the reason; any other API error stops the run, printing its
  own message first; fallback answers are priced on response.model.
- README: what each rule proves, the scoped tools, a finding may name a
  clear claim, numbers from five live reruns after the fixes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rparundekar

Copy link
Copy Markdown
Collaborator Author

Self-review

Two review passes ran on this PR. One read the code: correctness, silent failures, tests and comments. The other covered security and checked the README against the code. Every finding below was fixed in b6afbb2, and each fix has a selfcheck case that fails if the fix is undone.

Important (2, both fixed)

  • agent.py claim_rows / check_report: a field the reader missed counted as a finding against the shop. A check flagged only because the OCR missed a field ("Not read: tax", conflicting_values, "no claim number read") was filed as a finding against the shop. A finding pattern resting on it passed, and the job ended report_ready. Fixed: those claims are now not_checked with the reason "a field wasn't read", the job ends incomplete, and no finding can rest on them. A mix of unread and real mismatches stays a finding, citing only the real failures.
  • agent.py report_markdown: invoice text went into siu-report.md as raw Markdown or HTML. A shop could print an image link in its bill-to, and it would load when the analyst opened the report. Fixed: every cell goes through md_safe, and the summary, notes and request are fenced.

Suggestions (all fixed)

  • The tools could reach other shops and claims. claims_for_shop and get_claim now refuse anything outside the job's vendor, year and claim list, so the README's "can't reach siu-09" holds by construction.
  • A claim with two invoices kept only the last one. Each claim now keeps a list of results.
  • Anthropic errors: 408, 409 and 5xx count as transient, and any other error stops the run with its own message first. Tokens are priced on the model that actually answered.
  • README: a finding may name a clear claim next to the failed one, and the README now says so. The token figure is exact, not an average.

Reruns after the fixes (5 live runs, 11 Nov 2026)

All 5 runs came out as expected, with the same per-claim table each time. On run 1 the agent filed the closed claim as a finding; the code sent it back, and the revision made it a question. The noise question about the shop's own address now shows up on 3 runs in 5, down from 5 in 5. Claude cost about $0.09 a job.

@rparundekar
rparundekar marked this pull request as ready for review November 11, 2026 02:46
@rparundekar
rparundekar requested a review from a team as a code owner November 11, 2026 02:46
@rparundekar
rparundekar merged commit e6ca4f3 into main Nov 11, 2026
49 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant