Altrace

Available now · Open source

Your coding agent says the tests passed. What actually ran?

Rashomon keeps a local record of Claude Code’s tool calls, failed commands and subagent activity. It flags failures the agent’s final summary leaves out.

  • Open source
  • Claude Code on macOS and Linux
  • Local records
  • No telemetry
What it records, and what it never sends
Install Rashomon

Agent’s account

“Done. I refactored ParseConfig and all tests pass.”

Recorded

failed calls: 1
test runs: 1 (0 ok, 1 failed)
subagents: 1
  agent-a41f (Explore): 3 declarations, 3 executions, 1 Bash
  (these calls do not appear in the main transcript)

Flagged

the final message contains none of these 43 words: fail, failed, failing, error, errors, couldn't, could not, unable, not able, didn't, did not, blocked and 31 more of 43 (--json lists them all)

A call failed and the test run failed, and the summary uses none of Rashomon’s 43 failure words. The report lists the words that are absent. It does not guess why.

Show the excerpt exactly as rendered 
  the agent's account:
    "Done. I refactored ParseConfig and all tests pass."
  subagents: 1
    agent-a41f (Explore): 3 declarations, 3 executions, 1 Bash
    (these calls do not appear in the main transcript)
  failed calls: 1
    the final message contains none of these 43 words: fail, failed, failing, error, errors, couldn't, could not, unable, not able, didn't, did not, blocked and 31 more of 43 (--json lists them all)
  test runs: 1 (0 ok, 1 failed)

Example from Rashomon’s documentation, rendered by rashomon report.
See it in the README.

How it works

Record. Compare. Inspect.

  1. 01

    Record

    Hooks record each step Claude Code takes, its subagents’ included, and whether it worked, in a folder on your machine.

  2. 02

    Compare

    At the end of a turn, recorded failures are checked against the agent’s final message. When something is off, one line says so.

  3. 03

    Inspect

    rashomon report shows the full record: subagent calls, failed steps, test runs, and what Rashomon could not see.

  • Not a sandbox. It never stops or contains the agent.
  • Not exhaustive claim checking. It looks for 43 failure words in the summary and lists the ones that are absent.
  • Claude Code only today, on macOS and Linux. Open-source alpha.

In development

AutoSteer

Today: inspect what your agent did. Next: AutoSteer turns your corrections into guidance for future runs. Not yet shipped.

Follow on GitHub

Get started

Install it as a Claude Code plugin.

$ claude plugin marketplace add altrace-dev-role/rashomon
$ claude plugin install rashomon@rashomon
$ claude plugin enable rashomon@rashomon

Start a new Claude Code session, then run /rashomon:report. The README also covers a command-line install.

Install Rashomon