Applied AI engineer working across predictive machine learning, generative AI, and tool-using agent systems.
I build the full loop from model behavior to system outcomes: data, modeling, retrieval, tools, routing, evaluation, deployment, monitoring, and production feedback. I publish the code, evidence, negative results, and claim boundaries behind current experiments.
Website · Agent Economics Lab · Evidence ledger · Google Scholar · LinkedIn
Can a production AI decision survive missing evidence, weak graders, and the full cost of the work it creates?
The project turns versioned traces, outcomes, costs, and counterfactuals into an evidence-bound decision to scale a system, supervise it, or stop it. The same feedback contract spans predictive models, generative components, and tool-using agents while preserving their different failure modes.
Run the released demonstration without an account or API key:
uvx --from git+https://github.com/Vyoma/agent-economics-lab@v0.10.1 agent-economics demoRelease v0.10.1 ·
source at 4ca8d70 ·
release checksums ·
public documentation
Recomputed evidence at release v0.10.1 |
Result |
|---|---|
| Public agent datasets audited under one protocol | 10 |
| Runs carrying both outcome and proxy signals | 31,389 |
| Controlled required-gate perturbations | 588 |
These are public-dataset audits and controlled software experiments. They are not model rankings, production prevalence estimates, or deployment recommendations.
| System layer | Executable artifact | What it establishes |
|---|---|---|
| Predictive ML and GenAI | Production feedback contract | Binds data, model, policy, decision, outcome, and monitoring versions without pretending the modalities share one failure model. |
| Agent systems | Scale decision contract | Declares the evidence required before an autonomy or routing decision can pass. |
| Evaluation | Instrument scorecard | Measures the auditor itself and records where the available evidence remains too weak. |
| Deployment | Decision-record verifier | Replays a machine-readable decision and fails closed when evidence or contract identity drifts. |
| Research practice | Corrections and limitations | Keeps negative results, retractions, and non-claims adjacent to the implementation. |
The compact artifacts are intended for inspection and reuse:
- machine-readable decision-record template
- verified generated decision review
- filled cross-modality feedback-loop example
- public-data findings with a claim ledger
Reproduce the full released snapshot:
git clone --branch v0.10.1 https://github.com/Vyoma/agent-economics-lab
make -C agent-economics-lab reproduceUse Python 3.10 or newer. Keep the full Git history: reproduction checks
revisit historical commits to verify recorded defects. If python3 is older,
pass a supported interpreter, for example make -C agent-economics-lab PYTHON=python3.12 reproduce.
- One feedback loop, three kinds of evidence
- What survived six failed novelty claims
- A negative held-out result on code the detector had never seen
- Known limitations and unresolved measurement problems
- How should predictive models, foundation models, and tool-using agents share a production feedback loop without hiding distinct failure modes?
- Which changes improve end-to-end behavior: data, model, retrieval, tool interfaces, routing, or control policy?
- When outcomes and cheap proxies disagree, which claims survive, and where should the system abstain or hand off?
Each record below is mapped to a bounded claim, source class, evidence locator, and verification date in the public evidence ledger.
- Semantic grouping with network graphs, US Patent 11,748,453
- Navigating the Complexities of Generative AIs, IBM Research
- Building Retrieval Augmented Generation, UCLA Extension
- AI Agent Well-Architected Review, ServiceNow (acknowledged contributor)
San Francisco Bay Area · gajjar.vyoma@gmail.com