← All workCase study 06
Materia
An AI agent that finds the errors in financial spreadsheets that actually change a decision — 100% precision at 93% recall on a 12-workbook test set.
Type
AI Engineering
Status
Shipped
Context
Hackathon build
Period
2026
Source
Public
Python
267 → 13
flagged cells to findings
100%
precision (rules alone: 5%)
93%
recall
(01) Overview
Rule-based spreadsheet checkers drown reviewers in false positives. Materia uses them only to propose candidates: deterministic detectors flag suspect cells, an LLM judges which of them would change a decision, and a recompute tool must confirm the impact before a finding is kept. The model is never the last word — the recompute is.
(02) Architecture
Python · deterministic detectors propose candidates · LLM judge filters for decision impact · recompute tool confirms before a finding is kept
(03) What was built
- 01Deterministic detectors propose candidate errors; the LLM judges which ones change a decision; a recompute tool must confirm the impact before a finding is kept.
- 02On a 12-workbook test set, reduced 267 flagged cells to 13 findings, raising precision from 5% (rules only) to 100% at 93% recall.
(04) Stack
Python / OpenAI / Agentic tool use / LLM evaluation