Status: ready to build. Written for an agent that has not seen the discussion behind it.
Repo: Vectrum Code (/home/joni/Documents/DATA/Work/Coding/rust/code), a GPUI desktop client
for Claude Code, Pi and jcode. Read README.md first for the app's shape.
Agents can put rich, live output into their answers, like Claude's artifacts or OpenUI: diagrams, charts, cards, tables, small forms and test UIs, and whole HTML mini apps. The agent writes them as fenced code blocks in its normal Markdown answer. The app renders them natively in the transcript (MetalUI look, no webview), and interactive ones can talk back to the agent.
Four block kinds, all plain fenced code blocks, so they degrade to readable code anywhere else:
| Fence | Renders as | Priority |
|---|---|---|
```mermaid | Diagram image (flowchart, sequence, class, state, ER, pie, gantt, mindmap...) | P0 |
```openui | Native interactive UI from an OpenUI Lang program (section 5) | P0 |
```html | Code block as today, plus a Preview button that opens it as a mini app | P1 |
```svg | The SVG drawn as an image, with a "Show source" toggle | P2 |
Every rendered block keeps a small header row with Copy source and Show source (toggles back to the plain code block), so a bad render never hides the content.
Non-goals: arbitrary JS in the transcript, OpenUI Query/Mutation (tool calls from UI),
React/Zod component libraries, the OpenUI Gateway or Cloud.
Each phase must compile (cargo check), pass cargo test, and be committed with a message
in the repo's style ("Render mermaid blocks in answers as diagrams"). Do not batch phases.
export PATH=$HOME/.cargo/bin:$PATH.cargo check takes about 7 minutes on this laptop (i7-8550U). Incremental checks are
much faster. Run long builds in the background and keep working.cargo run (debug) or ./build.sh (release). The desktop is Hyprland (Omarchy),
grim can screenshot for visual checks.test attribute. Don't glob-import into test modules, or use
#[::core::prelude::v1::test] as src/ui/workspace.rs does.gpui-kit = "=0.7.1" (re-exports gpui, gpui_kit::component = gpui-component,
gpui_kit::base = gpui-base). Sources are in ~/.cargo/registry/src/*/gpui-*-0.7.1/ and
gpui-pre-0.3.8 for core GPUI. Read them when unsure; don't guess APIs.pub(super) fn markdown(state, p) -> TextView in
src/ui/chat.rs (around line 1993). It already adds a copy button per code block with
.code_block_actions(...). Its only transcript caller is render_assistant in
src/ui/chat/timeline.rs (around line 576/598). Plans use TextView::markdown(...) at
timeline.rs ~917 and ~1165 (optional to hook).Entity<TextViewState> per answer, created in src/session.rs
(push_assistant_text ~3779, item_from_saved ~4210). Streaming text is pushed with
AssistantText::show.gpui_kit::component::text::TextView has the hooks needed:
.markdown_block_parser(|node: &markdown_ast::Node, cx: &MarkdownParseContext| -> Option<MarkdownNode>).markdown_block_renderer(name, |node: &MarkdownNode, &mut Window, &mut App| -> impl IntoElement)MarkdownPlugin impl with is_block() -> true applied via .plugin(...).gpui_kit::component::text: MarkdownNode,
MarkdownParseContext, MarkdownPlugin, MarkdownExtensions, markdown_ast (mdast).markdown_ast::Node::Code(Code { value, lang, meta, .. }).
Match on lang (mermaid, openui, svg). Build the node with
MarkdownNode::new("openui", data).text(source).markdown(source) where source is
cx.node_source(node), so selection and copy still yield the original fence. Read the
data back in the renderer with node.data::<T>(). node.source_range() gives the byte
range in the document.gpui-base-0.7.1/src/text/markdown_ext.rs):
Send + Sync + 'static. Parsers run on a background
thread with no Window/App: do pure parsing only (the OpenUI parse is ideal here,
Mermaid layout too).parser_revision). Keep the set of parsers and names constant.div().pb(margin).markdown()
to also take the session's EntityId (Copy, Send) and capture it in the renderer closure.
render_item in timeline.rs has the session: &Entity<Session>.Streaming::Paragraphs,
finished_prefix in session.rs) whole code blocks tend to arrive at once, but with token
streaming the fence arrives partially (an unclosed fence parses as code to the end).
Renderers must accept partial input: render what resolves, show skeletons for the rest,
never panic, and only show a parse error once the fence is closed.MessageScroller, src/ui/chat.rs ~1452). If a block
changes height after an interaction or an async render, call
scroller.remeasure(cx) (see timeline.rs ~542 for the pattern), or the next frame may
jump.window.use_keyed_state alone for values: virtual list rows that scroll
out of view drop their element state. Keep UI values in a GPUI Global, e.g.
InlineUiStore { blocks: HashMap<(EntityId, u64 /* hash of source range + source */), BlockState> }.
use_keyed_state is fine for widget entities (InputState, SliderState) as long as
they are re-seeded from the store when recreated.window.refresh() so the custom block
re-renders. If the TextViewState view is cached and does not repaint, also
cx.notify(text_view_state_entity_id).SavedItem in session.rs) so forms keep
values after restart. Not required for v1.&mut App. Add a Global hub, set up once in Workspace::new
(src/ui/workspace.rs), holding WeakEntity<Workspace>, and a map
EntityId -> WeakEntity<Session> (or look sessions up through the workspace).ChatView::send (src/ui/chat.rs ~520-550): if the session is
working, queue_message vs send_message per the follow_up setting, then scroll to end.
Session::send_message(text, Vec::new(), cx) is at session.rs ~1253,
queue_message ~1444.[UI] Submitted "signup": {"name":"Ada","email":"ada@x.io"} or the literal text of
@ToAssistant("...").Workspace::open_surface(Surface::Browser, window, cx)
(src/ui/workspace/surfaces.rs ~62), panels made by shown_browser (workspace.rs ~1786),
BrowserPanel (src/ui/browser.rs) owns a BrowserSession with open(url)
(src/browser.rs ~522). Needs agent-browser installed (browser::find_binary()).
Fallback: workspace::open_url(url) (xdg-open, workspace.rs ~2362).AgentProcess::spawn in src/claude.rs (~174-200) passes
--append-system-prompt <prompt> only for the browser prompt today. Build one combined
prompt (browser prompt if any + inline UI prompt if enabled) and pass the flag once.system_reminder, see src/session.rs ~1600 (orchestrator::INSTRUCTIONS and
browser_note are joined there). Add the inline UI prompt to that list.pub const INSTRUCTIONS: &str in the new module (pattern:
src/orchestrator.rs ~44). Keep it short, under ~60 lines: when to use each block,
the component signatures, two examples, and the rule "plain Markdown first, UI only when
it helps".metal::palette(cx) -> &'static Palette (src/ui/metal.rs): page, s_hi, s, s_lo, well_top, well_bot, ink, ink2, ink3, rule, primary, green_ink, red, amber, blue_ink, tints[6], dark, shadow recipes. Surface helpers via MetalStyled: .plate(p),
.raised(p), .well(p), .region(p), .engraved(p). Also metal::tag, metal::rule,
metal::led, metal::toggle, metal::segmented, MetalButton::new(id) .primary()/.ghost()/.danger()/.compact()/.mini()/.label()/.icon()/.on_click(), and
.dropdown_menu(|menu, _, _| menu.item(PopupMenuItem::new(..).checked(..).on_click(..))).metal::SANS ("Geist"), metal::MONO ("Geist Mono").markdown_style in chat.rs ~1973. UI blocks should sit on a
.raised(p) or .region(p) card with 14px radius so they read as objects, not prose.mermaid-rs-renderer = { version = "0.3.1", default-features = false }
(pure Rust, 23 diagram types; default features pull clap and a second resvg version, so
keep them off).mermaid_rs_renderer::render_with_options(src, RenderOptions { theme, layout }) -> anyhow::Result<String> (SVG text). Theme has public String color fields
(font_family, font_size, primary_color, primary_text_color, primary_border_color, line_color, secondary_color, tertiary_color, edge_label_background, cluster_background, cluster_border, background, text_color, sequence_*, pie_colors[12], ...). Start from
Theme::modern() (light) or Theme::dark() and override from the palette (convert
Hsla to hex via gpui::Rgba::from(hsla)). Make the background transparent or the
palette's surface color."DejaVu Sans, sans-serif"), not Geist, or labels overflow their boxes.cx.background_spawn), the first call loads the system font DB and is slow. Rasterize
with cx.svg_renderer().render_single_frame(svg_bytes, window.scale_factor())
-> Arc<RenderImage>, and draw with img(render_image). ImageFormat::Svg through
Image::from_bytes renders at scale 1.0 and looks blurry on HiDPI, avoid it.Global keyed by (hash(src), dark, scale). RenderImage
ids are unique per render, so never re-render per frame. Drop evicted images with
window.drop_image(...)/cx.drop_image(...).width/height attrs or mermaid_rs_renderer::measure),
max_w_full(), keep aspect ratio (.aspect_ratio(w / h)). Center small diagrams. Wide
diagrams: horizontal scroll, or click to open full size in an overlay (nice to have).p.red.Follow OpenUI Lang v0.5 (spec: https://www.openui.com/docs/openui-lang/specification-v05.mdx, syntax: https://www.openui.com/docs/openui-lang/syntax.mdx) so models that know OpenUI write valid blocks. Implement a subset.
Module layout (keep the language pure, it gets the unit tests):
src/inline_ui/lang.rs: lexer + parser -> Program { statements: Vec<(Ident, Expr)> }.src/inline_ui/eval.rs: evaluator -> Value tree, given state.src/ui/inline/mod.rs: markdown hooks, globals (store, hub, cache).src/ui/inline/mermaid.rs, src/ui/inline/openui.rs, src/ui/inline/html.rs.Grammar (one statement per line, identifier = Expression):
name = Expr, state $name = default.Type(args...) (positional only), builtin @Name(args),
string "..." (with \" \n escapes), number, true/false/null, array [..],
object {key: value}, reference ident, state ref $ident, member access a.b.c
(on arrays it plucks the field from every element), ternary c ? a : b, binary
+ - * / % == != > < >= <= && ||, unary ! -. Precedence as in C/JS.// line comments. Blank lines ignored. Forward references allowed.root if present, else the first statement.Value::Pending and renders as a skeleton.
Cycles evaluate to an error value, not a stack overflow (depth limit ~64).Values: Null, Bool, Number(f64), String, Array(Vec), Object(BTreeMap), Component { name, args: Vec<Value> }, Action(Vec<Step>), State(name), Pending, Error(msg).
Evaluate the whole program on every render. Programs are small, keep it simple.
Builtins: @Count @Sum @Avg @Min @Max @First @Last @Filter(array, field, op, value) @Sort(array, field, dir?) @Round(n, decimals?) @Abs @Floor @Ceil @Each(array, varName, template).
Actions: Action([steps]) where steps are @Set($var, value), @Reset($a, $b),
@ToAssistant("message"), @OpenUrl("url"). Also accept v0.1 string actions on
buttons: "submit:<form>" (send the form's values to the agent), "action:<name>"
(send [UI] action: <name> to the agent). Query, Mutation and @Run are not
supported: render a small "not supported here" notice in their place.
Components (positional args in this order, optional ones may be omitted from the end). Names and orders match OpenUI's default library where it has them:
| Component | Signature | Render |
|---|---|---|
Stack | (children, direction?="column"|"row", gap?="s"|"m"|"l", align?, justify?, wrap?) | flex |
Card | (children, variant?="raised"|"well"|"plain", direction?, gap?) | MetalUI card |
CardHeader | (title, subtitle?) | |
TextContent | (text, size?="small"|"default"|"large"|"large-heavy") | |
Markdown | (text) | nested TextView |
Callout | (variant="info"|"success"|"warning"|"error", title, description?) | |
Tag | (text, variant?) | metal::tag |
Separator | () | metal::rule |
Progress | (value 0..1 or 0..100, label?) | |
Stat | (label, value, delta?) | KPI tile |
Image | (src, alt?) | http(s) or data URI only |
CodeBlock | (language, code) | |
Table | (columns) with Col(label, data, type?="text"|"number"|"tag") | rows = longest column |
BarChart | (labels, series, variant?="grouped"|"stacked", xLabel?, yLabel?) with Series(name, values) | gpui chart |
LineChart / AreaChart | (labels, series, xLabel?, yLabel?) | |
PieChart | (labels, values) | |
Tabs | (items) with TabItem(value, trigger, content) | metal::segmented + content |
Form | (name, fields, buttons) | collects field values by name |
FormControl | (label, input, hint?) | |
Input | (name or $var, placeholder?, type?="text"|"email"|"number"|"password", rules?) | |
TextArea | (name or $var, placeholder?, rows?) | |
Select | (name or $var, items, placeholder?), items = strings or {label, value} | MetalButton + dropdown |
Checkbox / Switch | (name or $var, label) | |
Slider | (name or $var, min, max, step?) | |
Buttons | (buttons, direction?) | |
Button | (label, action, variant?="primary"|"secondary"|"danger"|"ghost") | MetalButton |
Binding: a $var in an input's first argument binds it two-way; a plain string name keeps the
value in the enclosing Form. Expressions referencing $vars re-evaluate on change.
Unknown component: render its name as a tag and its children, so new names degrade softly.
GPUI building blocks (verified to exist in 0.7.1):
gpui_kit::component::chart::{BarChart, LineChart, AreaChart, PieChart}.
BarChart::new(data).band(|d| label).value(|d| v), AreaChart takes several .y(..)
for multi-series, PieChart::new(data).value(..).color(..).label(..). PlotValue is
implemented for f32/f64. Charts default their ElementId to the call site, so
several charts share hover state: always call .id(...) with a per-block id.
Give charts a fixed height (e.g. 220px).gpui_kit::component::input::{Input, InputState, InputEvent, Textarea, TextareaState}.
InputState::new(window, cx).placeholder(..).default_value(..), .value(),
.set_value(v, window, cx), subscribe to InputEvent (PressEnter, change). The app
draws inputs inside its own well with Input::new(&state).appearance(false)
(src/ui/browser.rs ~393).gpui_kit::component::slider::{Slider, SliderState, SliderEvent, SliderValue}.
SliderState::new().min(..).max(..).step(..).default_value(..),
SliderEvent::Change(SliderValue::Single(f32)).gpui_kit::component::checkbox::Checkbox::new(id).label(..).checked(..) .on_click(|checked: &bool, window, cx| ..), or metal::toggle(...) for the app's look.Safety: a block can only compose these components, open http(s) URLs after a click, and send text to the agent after a click. No file or network access on render. Images from remote URLs load only on click ("Load image") to avoid tracking pixels. Cap: 500 statements, 2000 rendered nodes, 10k table cells; beyond that, render an error.
code_block_actions in markdown() (chat.rs): for lang == "html"
add a Preview button next to Copy.settings::data_dir()/artifacts/<sha-or-hash>.html
(settings::data_dir() is in src/settings.rs), then open file://... in the thread's
browser panel (section 4.3), falling back to open_url if agent-browser isn't installed.lang == "svg": rasterize with cx.svg_renderer().render_single_frame like Mermaid, cap
size at 1MB of source, show source on error.
inline_ui: bool, default true, in Settings (src/settings.rs, struct ~438, defaults
~577, the changed-settings labels list ~681). Device preference, not an environment key.src/ui/settings/general.rs ~384, use row(...)/switch(...)/reset(...) like it),
search entry in src/ui/settings/search.rs (~83).## Inline UI in Vectrum Code
Your Markdown answers are shown in Vectrum Code, which renders some fenced blocks live:
- ```mermaid: diagrams (flowchart, sequenceDiagram, classDiagram, stateDiagram-v2, erDiagram, gantt, pie, mindmap).
- ```openui: native UI written in OpenUI Lang. One statement per line, `name = Component(args)`, positional args, `root = ...` first. State: `$name = default`. Actions: `Button("Go", Action([@ToAssistant("...")]))`, `@Set($x, v)`, `@Reset($x)`, `@OpenUrl("https://...")`.
Components: Stack, Card, CardHeader, TextContent, Markdown, Callout, Tag, Separator, Progress, Stat, Image, CodeBlock, Table/Col, BarChart/LineChart/AreaChart(labels, [Series(name, values)]), PieChart(labels, values), Tabs/TabItem, Form, FormControl, Input, TextArea, Select, Checkbox, Switch, Slider, Buttons, Button.
- ```html: a self-contained page (inline CSS/JS); the user can open it with Preview.
Use them when a picture, table or control says it better than prose: architecture and flows, comparisons, results data, choices for the user to make. Keep prose for everything else. When the user acts on your UI you receive a message starting with [UI].
Example:
```openui
root = Card([header, form])
header = CardHeader("Deploy", "Pick a target")
form = Form("deploy", [FormControl("Environment", Select("env", ["staging", "production"]))], Buttons([Button("Deploy", "submit:deploy", "primary")]))
```
Run the app (cargo run), start a Claude thread, and send each prompt. Screenshot each
result (grim) in both Bone and Graphite themes and look at it.
[UI] Submitted "deploy": {...} message and the agent replies.Unit tests (pure, in src/inline_ui): parse every example in this file and in the OpenUI
syntax page; operator precedence; member pluck on arrays; builtins; forward refs; cycles;
partial last line; @Each; actions; error values.
All phases committed, cargo test green, the 9 checks pass with screenshots looked at,
README "Features" gains a bullet describing inline diagrams, UI blocks and HTML previews.