# harness
A coding agent harness in Go. Point it at a repository, tell it to make the
failing test pass, and measure how often it actually does.
The code is the byproduct. The deliverable is [METRICS.md](METRICS.md).
[PLAN.md](PLAN.md) is the build plan this follows.
## Quick start
```bash
export ANTHROPIC_API_KEY=sk-ant-...
go build -o harness ./cmd/harness
./harness -C /path/to/a/throwaway/clone # full-screen TUI
./harness -C /path/to/repo -p "why does TestFoo fail?" # one prompt
```
A different endpoint — a gateway, a proxy, a local server speaking the Messages
API:
```bash
./harness -base-url http://127.0.0.1:4000 -api-key ... -C .
# or ANTHROPIC_BASE_URL / ANTHROPIC_API_KEY / ANTHROPIC_AUTH_TOKEN
```
Flags override the environment. `-auth-token` sends `Authorization: Bearer`,
which is what OAuth tokens and most gateways want; `-api-key` sends
`x-api-key`. A non-default endpoint announces itself on stderr, because your
prompt and every file the agent reads go wherever that URL points.
Useful flags: `-model`, `-effort low|medium|high|xhigh|max`, `-max-turns`,
`-resume latest`, `-no-stream`, `-no-cache`, `-v`, `-yolo`, `-no-tui`, `-no-skills`.
## The TUI
On a terminal, the interactive mode is a full-screen [Bubble Tea](https://github.com/charmbracelet/bubbletea)
app. It opens on a welcome block naming the model, the endpoint and the root,
and settles into a transcript above a prompt:
```
⏺ Let me see what the test expects before changing anything.
⏺ ReadFile(path=internal/agent/loop_test.go)
⎿ 1: package agent
…
⏺ EditFile(path=internal/agent/compact.go)
4 additions, 1 removal
@@ -50,7 +50,9 @@
+ if window == 0 {
✢ Noodling… (14s · turn 3 · ↓ 1.3k tokens · esc to interrupt) ● high · /effort
╭──────────────────────────────────────────────────────────────────────────────╮
│ › _ │
╰──────────────────────────────────────────────────────────────────────────────╯
joni ✳ Opus 5 ◐ 62.2k (6%) ↑ 4.2k ⟳ 58.0k ↓ 1.3k $0.0835
⏵ approvals on (shift+tab to cycle) · /help for commands
```
Every event hangs off a dot in the left gutter, and everything belonging to it
is indented under that dot, so a screen of tool output still reads as a
sequence of things that happened. Diffs are shown in full and in colour with a
count of what changed; other output is elided after six lines.
The status bar reports the context gauge — the last turn's input size against
the window, which drops when the conversation is compacted — and separates
fresh input tokens from cache reads, because the gap between the two is what
[METRICS.md](METRICS.md) is about. The prompt box grows with what you type, up
to six lines.
| Key | Does |
| --- | --- |
| `enter` | send |
| `ctrl+j` | newline without sending |
| `shift+tab` | cycle the approval mode |
| `ctrl+r` | expand elided tool output |
| `pgup`/`pgdn` | scroll the transcript |
| `esc` | interrupt the turn |
| `ctrl+c` | interrupt the turn; again when idle to exit |
Commands: `/allowed`, `/clear`, `/help`, `/exit`, `/skills`.
`shift+tab` cycles between asking before every mutating call and accepting them
all, which is `-yolo` reached from the keyboard. The mode line says which is in
force at all times, and the switch is recorded in the transcript, because a
mode that silently lets the agent write to disk is not one to keep in a corner.
Approvals appear inline as a bordered prompt, answered by number or by the key
in brackets — `y`, `n`, `a` for this exact command, `t` for the whole tool. The
agent goroutine blocks until you answer, which is the point: the tool has not
run yet. Interrupting while that prompt is open denies the call and cancels the
turn rather than leaving the agent waiting on a channel nobody will read.
The TUI is skipped automatically when stdin or stdout is not a terminal, and by
`-p` and `-no-tui`, which use the line-based renderer instead. Both front ends
consume the same `agent.Event` stream, so neither can show something the other
cannot.
### Driving it without an API key
`internal/tui/testdata/mockapi` is a fake Messages API that streams a canned
reply and asks to run a shell command, which is enough to exercise streaming,
tool rendering, approve, deny and interrupt for nothing:
```bash
go run ./internal/tui/testdata/mockapi &
mkdir -p /tmp/demo && git -C /tmp/demo init -q
go build -o harness ./cmd/harness
./harness -C /tmp/demo -base-url http://127.0.0.1:8899 -api-key mock
```
Type anything, and the approval prompt appears. `-slow 150ms` stretches the
stream out if you want to watch the spinner or test `ctrl+c` mid-turn. It sits
under `testdata` so `./...` ignores it.
## Tools
| Tool | Approval | Notes |
| --- | --- | --- |
| `read_file` | automatic | Numbered lines, `offset`/`limit` for paging |
| `glob` | automatic | Newest first; `*.go` matches at any depth |
| `grep` | automatic | ripgrep when present, pure Go otherwise, identical output either way |
| `edit_file` | prompts | Requires a prior read; refuses ambiguous matches; returns a diff |
| `write_file` | prompts | Whole files only; separate from `edit_file` so the metrics stay honest |
| `bash` | prompts | Own process group, killed whole on timeout; output capped head and tail |
## Skills
A skill is a directory of instructions the agent pulls in when it is relevant:
```
~/.agents/skills/commit-helper/SKILL.md every repo
./.agents/skills/commit-helper/SKILL.md this one, checked in
```
```markdown
---
name: commit-helper
description: Use when writing commit messages for this repo
---
Read the last twenty commits for tone before writing anything.
Conventions are in references/conventions.md.
```
The directory name is the skill's name, and a `name:` that disagrees with it is
a load error. Only the description reaches the model until the skill is
invoked, so it is what decides whether the skill fires at all; the body can be
as long as you like.
Two ways in, one way to load. The agent calls the `skill` tool when a
description matches what it is doing; you can force one with `/commit-helper
fix the flaky test`, which the model answers by making the same call. `/skills`
lists what was found, project shadowing personal on a name collision.
Skill directories are readable by `read_file`, `glob` and `grep` even though
they sit outside the working root, so a skill can ship reference files and
point at them. They are not writable by anything: `edit_file`, `write_file` and
`bash` keep the working root as their only root. A skill is input, not a work
product.
Relative paths still anchor to the working root, so a skill pointing at its own
files is read with the directory the `skill` tool prints joined on. If the model
forgets, the "no such file" it gets back names the skill directories, which is
usually enough to recover on the next turn. A shadowed skill's directory is not
readable — only the skills actually in play widen anything.
Discovery happens at startup. The name-and-description listing goes in the
cached prompt prefix, so re-reading the directory each turn would cost a cache
miss on every turn for a file that never changes. Restart to pick up a new
skill. `-no-skills` turns the whole thing off; the eval runner never discovers
skills at all, so METRICS.md is unaffected by what is in your home directory.
## Evals
```bash
./evals/selftest.sh # no API calls: checks the tasks measure something
go run ./evals -n 3 # the real thing
```
Ten tasks, a fresh temp directory per run, full transcripts under
`evals/runs/<timestamp>/`. See [METRICS.md](METRICS.md).
## Safety
Three layers, in increasing order of how much they actually protect you.
**Path jail.** Every path a tool is given is resolved with
`filepath.EvalSymlinks` and refused if it lands outside the working root,
including through a symlink that points out of the tree. Cheap, and it catches
a lot.
**Approval prompts.** Read-only tools run unattended. `bash`, `edit_file` and
`write_file` ask, with "yes / no / always this exact one / always this tool" and
session memory. `/allowed` lists what you have agreed to.
Shell commands are parsed with `mvdan.cc/sh/syntax` so the prompt can tell you
what a command will actually run — `git status; rm -rf ~` is shown as running
both `git` and `rm`. **This is not a security boundary**, and the harness does
not pretend otherwise: a command containing `$(...)`, backticks, or an
interpreter with `-c` cannot be analysed at all, so it is marked opaque and
"always allow this exact command" is refused for it. String-matching
allowlists for shell commands do not work; this exists to make the prompt
specific, not to make the decision for you.
**A sandbox.** This is the layer that actually contains a mistake, and the
harness cannot provide it for you. The honest recipe on Linux:
```bash
podman run --rm -it \
-v "$PWD":/work:Z -w /work \
-e ANTHROPIC_API_KEY \
docker.io/library/golang:1.26 \
/work/harness -C /work -yolo
```
**Decide about egress explicitly.** The container above has network access,
because the harness needs to reach the API — which means the agent's `bash`
has it too. That is the line between "broke my repo" and "read my `.env` and
made an HTTP request". If that matters for the tree you are pointing it at:
- Treat every secret reachable from inside the container as exposed.
- Use a token scoped to nothing else, and a clone rather than your working tree.
- For real isolation, run the harness on the host and put only `bash` in a
network-less namespace. That is not implemented here.
`-yolo` skips every prompt. It is correct inside a container or against a
throwaway clone, and wrong everywhere else. The eval runner sets it, because it
is headless and operates on a temp directory it created.
## Layout
```
cmd/harness flag parsing, wiring, REPL
internal/llm Messages API: types, HTTP, SSE, pricing
internal/agent turn loop, conversation state, compaction, sessions
internal/tools the tool registry and the tools
internal/skills discovery and parsing of SKILL.md
internal/approval allow / deny / ask, with shell analysis
internal/metrics JSONL event log
internal/ui line-based terminal rendering
internal/tui full-screen Bubble Tea front end
evals task suite and runner
```
Two rules hold the shape: `agent` never imports `ui` or `tui`, so the eval
runner can drive the loop headless; and every tool returns `(string, error)`
where the error is text the model reads, not an exception the harness swallows.
A failed edit is a normal event in the transcript.
The TUI obeys the first rule through two one-way hatches. Events reach it via
`agent.Observe`, forwarded into the Bubble Tea loop with `Program.Send`, which
is safe from the agent's goroutine. Approvals go out the same way and then
block the agent on a reply channel until a key arrives. That channel is
buffered and the wait honours the turn's context, so neither side can strand
the other.