Vectrum
Sign In
CodeyPublic
Like0 likesFork0 forks

harness: discover and wire up skills, document them

Wires internal/skills into cmd/harness: discovers personal and project
skills at startup, registers the skill tool when any are found, widens
the read-only tools' path jail to the skill directories, and threads
Skills through to both agent.Options and tui.Options. Adds -no-skills
to turn discovery off entirely. Documents the feature in the README.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Browse Files
Jjonicommitted 1 parent 62ffa3ecommitdc52c6d

Showing 2 changed files with 230 additions and 13 deletions.

+139−7README.mdView File
@@ -12,7 +12,7 @@ The code is the byproduct. The deliverable is [METRICS.md](METRICS.md).
1212 export ANTHROPIC_API_KEY=sk-ant-...
1313 go build -o harness ./cmd/harness
1414
15- ./harness -C /path/to/a/throwaway/clone # REPL
15+ ./harness -C /path/to/a/throwaway/clone # full-screen TUI
1616 ./harness -C /path/to/repo -p "why does TestFoo fail?" # one prompt
1717 ```
1818
@@ -30,7 +30,89 @@ which is what OAuth tokens and most gateways want; `-api-key` sends
3030 prompt and every file the agent reads go wherever that URL points.
3131
3232 Useful flags: `-model`, `-effort low|medium|high|xhigh|max`, `-max-turns`,
33- `-resume latest`, `-no-stream`, `-no-cache`, `-v`, `-yolo`.
33+ `-resume latest`, `-no-stream`, `-no-cache`, `-v`, `-yolo`, `-no-tui`, `-no-skills`.
34+
35+ ## The TUI
36+
37+ On a terminal, the interactive mode is a full-screen [Bubble Tea](https://github.com/charmbracelet/bubbletea)
38+ app. It opens on a welcome block naming the model, the endpoint and the root,
39+ and settles into a transcript above a prompt:
40+
41+ ```
42+ ⏺ Let me see what the test expects before changing anything.
43+
44+ ⏺ ReadFile(path=internal/agent/loop_test.go)
45+ ⎿ 1: package agent
46+ …
47+
48+ ⏺ EditFile(path=internal/agent/compact.go)
49+ 4 additions, 1 removal
50+ @@ -50,7 +50,9 @@
51+ + if window == 0 {
52+
53+ ✢ Noodling… (14s · turn 3 · ↓ 1.3k tokens · esc to interrupt) ● high · /effort
54+ ╭──────────────────────────────────────────────────────────────────────────────╮
55+ │ › _ │
56+ ╰──────────────────────────────────────────────────────────────────────────────╯
57+ joni ✳ Opus 5 ◐ 62.2k (6%) ↑ 4.2k ⟳ 58.0k ↓ 1.3k $0.0835
58+ ⏵ approvals on (shift+tab to cycle) · /help for commands
59+ ```
60+
61+ Every event hangs off a dot in the left gutter, and everything belonging to it
62+ is indented under that dot, so a screen of tool output still reads as a
63+ sequence of things that happened. Diffs are shown in full and in colour with a
64+ count of what changed; other output is elided after six lines.
65+
66+ The status bar reports the context gauge — the last turn's input size against
67+ the window, which drops when the conversation is compacted — and separates
68+ fresh input tokens from cache reads, because the gap between the two is what
69+ [METRICS.md](METRICS.md) is about. The prompt box grows with what you type, up
70+ to six lines.
71+
72+ | Key | Does |
73+ | --- | --- |
74+ | `enter` | send |
75+ | `ctrl+j` | newline without sending |
76+ | `shift+tab` | cycle the approval mode |
77+ | `ctrl+r` | expand elided tool output |
78+ | `pgup`/`pgdn` | scroll the transcript |
79+ | `esc` | interrupt the turn |
80+ | `ctrl+c` | interrupt the turn; again when idle to exit |
81+
82+ Commands: `/allowed`, `/clear`, `/help`, `/exit`, `/skills`.
83+
84+ `shift+tab` cycles between asking before every mutating call and accepting them
85+ all, which is `-yolo` reached from the keyboard. The mode line says which is in
86+ force at all times, and the switch is recorded in the transcript, because a
87+ mode that silently lets the agent write to disk is not one to keep in a corner.
88+
89+ Approvals appear inline as a bordered prompt, answered by number or by the key
90+ in brackets — `y`, `n`, `a` for this exact command, `t` for the whole tool. The
91+ agent goroutine blocks until you answer, which is the point: the tool has not
92+ run yet. Interrupting while that prompt is open denies the call and cancels the
93+ turn rather than leaving the agent waiting on a channel nobody will read.
94+
95+ The TUI is skipped automatically when stdin or stdout is not a terminal, and by
96+ `-p` and `-no-tui`, which use the line-based renderer instead. Both front ends
97+ consume the same `agent.Event` stream, so neither can show something the other
98+ cannot.
99+
100+ ### Driving it without an API key
101+
102+ `internal/tui/testdata/mockapi` is a fake Messages API that streams a canned
103+ reply and asks to run a shell command, which is enough to exercise streaming,
104+ tool rendering, approve, deny and interrupt for nothing:
105+
106+ ```bash
107+ go run ./internal/tui/testdata/mockapi &
108+ mkdir -p /tmp/demo && git -C /tmp/demo init -q
109+ go build -o harness ./cmd/harness
110+ ./harness -C /tmp/demo -base-url http://127.0.0.1:8899 -api-key mock
111+ ```
112+
113+ Type anything, and the approval prompt appears. `-slow 150ms` stretches the
114+ stream out if you want to watch the spinner or test `ctrl+c` mid-turn. It sits
115+ under `testdata` so `./...` ignores it.
34116
35117 ## Tools
36118
@@ -43,6 +125,47 @@ Useful flags: `-model`, `-effort low|medium|high|xhigh|max`, `-max-turns`,
43125 | `write_file` | prompts | Whole files only; separate from `edit_file` so the metrics stay honest |
44126 | `bash` | prompts | Own process group, killed whole on timeout; output capped head and tail |
45127
128+ ## Skills
129+
130+ A skill is a directory of instructions the agent pulls in when it is relevant:
131+
132+ ```
133+ ~/.agents/skills/commit-helper/SKILL.md every repo
134+ ./.agents/skills/commit-helper/SKILL.md this one, checked in
135+ ```
136+
137+ ```markdown
138+ ---
139+ name: commit-helper
140+ description: Use when writing commit messages for this repo
141+ ---
142+
143+ Read the last twenty commits for tone before writing anything.
144+ Conventions are in references/conventions.md.
145+ ```
146+
147+ The directory name is the skill's name, and a `name:` that disagrees with it is
148+ a load error. Only the description reaches the model until the skill is
149+ invoked, so it is what decides whether the skill fires at all; the body can be
150+ as long as you like.
151+
152+ Two ways in, one way to load. The agent calls the `skill` tool when a
153+ description matches what it is doing; you can force one with `/commit-helper
154+ fix the flaky test`, which the model answers by making the same call. `/skills`
155+ lists what was found, project shadowing personal on a name collision.
156+
157+ Skill directories are readable by `read_file`, `glob` and `grep` even though
158+ they sit outside the working root, so a skill can ship reference files and
159+ point at them. They are not writable by anything: `edit_file`, `write_file` and
160+ `bash` keep the working root as their only root. A skill is input, not a work
161+ product.
162+
163+ Discovery happens at startup. The name-and-description listing goes in the
164+ cached prompt prefix, so re-reading the directory each turn would cost a cache
165+ miss on every turn for a file that never changes. Restart to pick up a new
166+ skill. `-no-skills` turns the whole thing off; the eval runner never discovers
167+ skills at all, so METRICS.md is unaffected by what is in your home directory.
168+
46169 ## Evals
47170
48171 ```bash
@@ -107,13 +230,22 @@ cmd/harness flag parsing, wiring, REPL
107230 internal/llm Messages API: types, HTTP, SSE, pricing
108231 internal/agent turn loop, conversation state, compaction, sessions
109232 internal/tools the tool registry and the tools
233+ internal/skills discovery and parsing of SKILL.md
110234 internal/approval allow / deny / ask, with shell analysis
111235 internal/metrics JSONL event log
112- internal/ui terminal rendering
236+ internal/ui line-based terminal rendering
237+ internal/tui full-screen Bubble Tea front end
113238 evals task suite and runner
114239 ```
115240
116- Two rules hold the shape: `agent` never imports `ui`, so the eval runner can
117- drive the loop headless; and every tool returns `(string, error)` where the
118- error is text the model reads, not an exception the harness swallows. A failed
119- edit is a normal event in the transcript.
241+ Two rules hold the shape: `agent` never imports `ui` or `tui`, so the eval
242+ runner can drive the loop headless; and every tool returns `(string, error)`
243+ where the error is text the model reads, not an exception the harness swallows.
244+ A failed edit is a normal event in the transcript.
245+
246+ The TUI obeys the first rule through two one-way hatches. Events reach it via
247+ `agent.Observe`, forwarded into the Bubble Tea loop with `Program.Send`, which
248+ is safe from the agent's goroutine. Approvals go out the same way and then
249+ block the agent on a reply channel until a key arrives. That channel is
250+ buffered and the wait honours the turn's context, so neither side can strand
251+ the other.
+91−6cmd/harness/main.goView File
@@ -18,10 +18,17 @@ import (
1818 "harness/internal/approval"
1919 "harness/internal/llm"
2020 "harness/internal/metrics"
21+ "harness/internal/skills"
2122 "harness/internal/tools"
23+ "harness/internal/tui"
2224 "harness/internal/ui"
2325 )
2426
27+ // version is what the TUI's welcome block reports. It is a constant rather
28+ // than build-stamped: the harness is built from this tree, and a stamp that
29+ // can disagree with the source is worse than no stamp.
30+ const version = "v0.6.0"
31+
2532 func main() {
2633 if err := run(); err != nil {
2734 fmt.Fprintln(os.Stderr, "harness:", err)
@@ -46,6 +53,8 @@ func run() error {
4653 resume := flag.String("resume", "", "resume a saved session by id, or \"latest\"")
4754 verbose := flag.Bool("v", false, "print full tool output")
4855 yolo := flag.Bool("yolo", false, "skip every approval prompt. Only against a throwaway clone or inside a container")
56+ noTUI := flag.Bool("no-tui", false, "use the plain line-based REPL instead of the full-screen TUI")
57+ noSkills := flag.Bool("no-skills", false, "skip discovering skills in ~/"+skills.Dirname+" and ./"+skills.Dirname)
4958 flag.Parse()
5059
5160 root, err := filepath.Abs(*workdir)
@@ -72,21 +81,59 @@ func run() error {
7281 log.Log(metrics.Event{Type: "session_start", Extra: map[string]any{
7382 "root": root, "model": client.Model, "base_url": client.BaseURL,
7483 }})
84+
85+ // The TUI takes over the screen, so anything worth saying before the loop
86+ // starts has to go into its scrollback rather than to stderr.
87+ useTUI := !*noTUI && *prompt == "" && interactive()
88+ var banner []string
89+ notice := func(format string, args ...any) {
90+ msg := fmt.Sprintf(format, args...)
91+ if useTUI {
92+ banner = append(banner, msg)
93+ return
94+ }
95+ fmt.Fprintln(os.Stderr, "harness: "+msg)
96+ }
97+
7598 if !client.IsDefaultEndpoint() {
7699 // Worth saying out loud: the prompt and every file the agent reads go
77100 // to this endpoint, not to Anthropic.
78- fmt.Fprintf(os.Stderr, "harness: sending requests to %s\n", client.BaseURL)
101+ notice("sending requests to %s", client.BaseURL)
102+ }
103+
104+ var found []skills.Skill
105+ if !*noSkills {
106+ home, err := os.UserHomeDir()
107+ if err != nil {
108+ // No home directory is not fatal: project skills still work.
109+ home = ""
110+ }
111+ var problems []error
112+ found, problems = skills.Discover(home, root)
113+ for _, p := range problems {
114+ notice("skill skipped: %v", p)
115+ }
116+ }
117+
118+ // Skill directories are readable by the search tools and by nothing else,
119+ // so a skill can read the files it ships with.
120+ var skillDirs []string
121+ for _, s := range found {
122+ skillDirs = append(skillDirs, s.Dir)
79123 }
80124
81125 state := agent.NewState()
82126 reg := tools.NewRegistry(log).Register(
83- tools.NewRead(root, state),
84- tools.NewGlob(root),
85- tools.NewGrep(root),
127+ tools.NewRead(root, state, skillDirs...),
128+ tools.NewGlob(root, skillDirs...),
129+ tools.NewGrep(root, skillDirs...),
86130 tools.NewEdit(root, state),
87131 tools.NewWrite(root, state),
88132 tools.NewBash(root),
89133 )
134+ if t := tools.NewSkill(found); t != nil {
135+ reg.Register(t)
136+ }
90137
91138 in := bufio.NewScanner(os.Stdin)
92139 in.Buffer(make([]byte, 0, 64*1024), 1024*1024)
@@ -95,7 +142,8 @@ func run() error {
95142 approver := &ui.Approver{In: in, Out: os.Stdout}
96143 policy := approval.New(approver.Prompt)
97144 policy.YOLO = *yolo
98- if *yolo {
145+ if *yolo && !useTUI {
146+ // The TUI says this itself, in the banner.
99147 fmt.Fprintln(os.Stderr, "harness: -yolo — no approval prompts. Use a throwaway clone or a container.")
100148 }
101149 a := agent.New(client, reg, state, log, agent.Options{
@@ -105,6 +153,7 @@ func run() error {
105153 CompactAt: *compactAt,
106154 NoCache: *noCache,
107155 Stream: !*noStream,
156+ Skills: found,
108157 })
109158 a.Observe = render.Handle
110159 a.Approve = policy
@@ -117,7 +166,30 @@ func run() error {
117166 }
118167 a.Restore(prev)
119168 session = prev.ID
120- fmt.Printf("resumed session %s (%d messages, %d turns)\n", prev.ID, len(prev.Messages), prev.Turns)
169+ notice("resumed session %s (%d messages, %d turns)", prev.ID, len(prev.Messages), prev.Turns)
170+ }
171+
172+ if useTUI {
173+ // No signal handler here: Bubble Tea puts the terminal in raw mode, so
174+ // ctrl-c arrives as a key and the model decides what it means.
175+ return tui.Run(context.Background(), tui.Options{
176+ Agent: a,
177+ Policy: policy,
178+ Client: client,
179+ Root: root,
180+ SessionDir: sessionDir,
181+ SessionID: session,
182+ ToolNames: toolNames(reg),
183+ Verbose: *verbose,
184+ YOLO: *yolo,
185+ Skills: found,
186+
187+ Version: version,
188+ Effort: *effort,
189+ ContextWindow: *contextWindow,
190+
191+ Banner: banner,
192+ })
121193 }
122194
123195 ctx, stopSignals := interruptible(context.Background())
@@ -227,6 +299,19 @@ var errNoCredentials = errors.New(`no credentials. Supply one of:
227299 -api-key / -auth-token (flags override the environment)
228300 To use a non-Anthropic endpoint, set ANTHROPIC_BASE_URL or -base-url as well.`)
229301
302+ // interactive reports whether both ends of the terminal are a real one. A
303+ // piped stdin or a redirected stdout means the full-screen UI would be wrong,
304+ // and the line-based REPL still works.
305+ func interactive() bool {
306+ for _, f := range []*os.File{os.Stdin, os.Stdout} {
307+ info, err := f.Stat()
308+ if err != nil || info.Mode()&os.ModeCharDevice == 0 {
309+ return false
310+ }
311+ }
312+ return true
313+ }
314+
230315 func toolNames(reg *tools.Registry) []string {
231316 var names []string
232317 for _, t := range reg.List() {