raymondyang.dev
← map

Why

Coding agents say they're done, and checking that claim usually means reading the code. Vantle tries two ways to check it with less reading: a chart of a function's decisions that is itself checked against the code, and a receipt that checks the agent's work against examples agreed before it started. It is aimed at developers reviewing code, especially code written by AI agents.

The receipt

Since late October, Vantle's focus has been the receipt, which checks an agent's “done” against a few example rows that the human approves before the work starts. Each row says “given X, expect Y” or says what must stay the same, and the receipt lists every row as matched or mismatched, with its evidence. In a first replay on 6 past bugs, rows written by an agent alone caught none of them, so the next test uses rows the human reviews, on new tasks.

The receipt has no code yet. So far it has been run by hand, with the skill and row format in the repository's docs/receipts/.

The write-up is Making Vantle, part 1.

How the charts work

  1. 01ExtractThe chart comes from the syntax tree, with no AI.
  2. 02LabelClaude rewrites raw conditions as plain language. A label that changes the meaning is rejected.
  3. 03VerifyAn LLM writes a test for each path. Vantle records the route the code took and flags code no test reached.
  4. 04MarkEach path is verified, failed or unverified.

"Verified" means a test drove the real code down that route. It is not a proof for every input.

Charts are drawn as SVG, laid out with elkjs, in the style of the chart in Making Vantle, part 1. Arrows run at right angles, with “yes” leaving each box from the bottom and “no” from the right. Mermaid output is still available.

First results

Tested on 60 functions from 20 open-source TypeScript repos: 16 charts were usable at first, against a 50% target, so it missed its own bar. After two rounds of fixes, 23 were usable if arrows that no run can take count as noise, or 10 if they count as wrong. Chart work is paused while the receipt is tested, apart from how charts are drawn.

Try it

Node 22.12 or later, from a clone of the repository:

npm install && npm run build
node dist/cli.js -p fixtures/slack-notify scan                  # functions with the most decision logic
node dist/cli.js -p fixtures/slack-notify generate --no-label   # chart the marked function, locally

Labels and verify call the Claude API, so they need an Anthropic API key and cost money.

The model proposes, a check decides

The LLM stays in the loop but is never trusted alone: every label and test it writes is checked by deterministic code before it counts. The tool is measured the same way. Each run has a pass bar written down before it starts, outputs get blind reviews against a fixed rubric, and runs are repeated after fixes.