vectrumcodePublic

Tell agents about inline UI, with a setting to turn it off

Claude and Pi get how to write mermaid, svg, html and openui blocks in
their system prompt, joined with the browser's into one
--append-system-prompt; jcode gets it with the first message. Settings →
General → Inline diagrams and UI (on by default) turns off both the
instructions and the rendering, which leaves the blocks as code. A test
keeps the examples agents are shown parsing. The README describes the
feature.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Browse Files

JjohannSocommitted 4 days ago1 parent 4a52bd0commit5dfe90d

Showing 9 changed files with 97 additions and 12 deletions.

+1−1Cargo.lockView File

@@ -7480,8 +7480,8 @@ dependencies = [
7480	7480	"gpui-kit",
7481	7481	"image",
7482	7482	"libc",
7483		- "openh264",
7484	7483	"mermaid-rs-renderer",
	7484	+ "openh264",
7485	7485	"serde",
7486	7486	"serde_json",
7487	7487	"smol",

+7−0README.mdView File

@@ -79,6 +79,13 @@ Gruvbox, Rosé Pine, Solarized and Dracula.
79	79	switch Claude into plan mode and back.
80	80	- Finished turns fold their tool calls and interim messages into one Worked for 12s row
81	81	above the answer. Click it to unfold.
	82	+ - Answers can carry live diagrams and UI, written as plain fenced code blocks. ` ```mermaid `
	83	+ draws a diagram in the theme's colors. ` ```openui ` ([OpenUI Lang](https://www.openui.com/))
	84	+ draws native cards, stats, tables, charts, tabs and forms; their state stays put while
	85	+ you scroll, and buttons and form submissions reach the agent as `[UI]` messages.
	86	+ ` ```svg ` draws the image. ` ```html ` gets a Preview button that opens the page in the
	87	+ thread's browser panel. Every block keeps Copy and Show source. Agents are told
	88	+ how to write them; Settings → General → Inline diagrams and UI turns it all off.
82	89	- Under each turn that touched files, N changed files +a −r with a folder tree and
83	90	per-file counts. Click a file, or Open diff, to read the diff in the app. The app
84	91	snapshots the working tree through a private git index, so your staging area is never

+9−6src/claude.rsView File

@@ -198,12 +198,15 @@ impl AgentProcess {
198	198	if config.tools.devices {
199	199	crate::device::agent::prepare_command(&mut command);
200	200	}
201		- // Claude and Pi take extra system prompt with this flag.
202		- if config.instance.kind != Provider::Jcode
203		- && let Some(session) = &config.browser_session
204		- && let Some(prompt) = browser::prepare_agent(&mut command, session)
205		- {
206		- command.args(["--append-system-prompt", &prompt]);
	201	+ // Claude and Pi take extra system prompt with this flag, once: the
	202	+ // browser's, if the thread has one, and how to write inline UI.
	203	+ if config.instance.kind != Provider::Jcode {
	204	+ let browser = config.browser_session.as_ref().and_then(\|session\| browser::prepare_agent(&mut command, session));
	205	+ let inline = crate::settings::get(\|settings\| settings.inline_ui).then(\|\| crate::inline_ui::INSTRUCTIONS.to_string());
	206	+ let prompts: Vec<String> = browser.into_iter().chain(inline).collect();
	207	+ if !prompts.is_empty() {
	208	+ command.args(["--append-system-prompt", &prompts.join("\n\n")]);
	209	+ }
207	210	}
208	211	Self::spawn_command(&binary, command, events)
209	212	}

+51−0src/inline_ui/mod.rsView File

   pub mod eval;
   pub mod lang;
++ /// What agents are told about rich blocks, when the setting is on: in the
++ /// system prompt for Claude and Pi, and with jcode's first message.
++ pub const INSTRUCTIONS: &str = r##"## Inline UI in Vectrum Code
++ Vectrum Code shows your Markdown answers and renders some fenced code blocks live. Plain Markdown comes first; use these only when a picture, table or control says it better than prose: architecture and flows, comparisons, results data, choices for the user to make.
++ - ```mermaid: a diagram (flowchart, sequenceDiagram, classDiagram, stateDiagram-v2, erDiagram, gantt, pie, mindmap).
++ - ```svg: a small self-contained SVG image, with width and height set.
++ - ```html: a self-contained page with inline CSS and JS. It shows as code; the user can open it with Preview.
++ - ```openui: native UI written in OpenUI Lang, below.
++ OpenUI Lang: one statement per line, `name = Expression`, `root = ...` first; forward references are fine. Arguments are positional, and optional ones can be left off the end. Values: strings, numbers, true/false/null, [arrays], {key: value} objects, references to other statements, `a.b` (on an array it takes `b` from every element), `cond ? a : b`, arithmetic, comparisons, && || !. State: declare `$name = default` and read `$name`; an input given `$name` as its first argument edits it, and everything reading it updates.
++ Built-ins: @Count @Sum @Avg @Min @Max @First @Last @Round(n, decimals) @Abs @Floor @Ceil @Filter(array, field, op, value) @Sort(array, field, "asc"|"desc") @Each(array, "item", template).
++ Actions, a Button's second argument: Action([steps]) with @Set($x, value), @Reset($x), @ToAssistant("message"), @OpenUrl("https://..."); or "submit:formName" to send a form's values to you, or "action:name".
++ Components:
++ - Layout: Stack(children, direction "column"|"row", gap "s"|"m"|"l", align, justify, wrap), Card(children, variant "raised"|"well"|"plain"), CardHeader(title, subtitle), Tabs([TabItem(value, label, content)]), Separator().
++ - Content: TextContent(text, size "small"|"default"|"large"|"large-heavy"), Markdown(text), Callout(variant "info"|"success"|"warning"|"error", title, description), Tag(text, variant), Stat(label, value, delta), Progress(value 0..1 or 0..100, label), Image(url, alt), CodeBlock(language, code).
++ - Data: Table([Col(label, values, type "text"|"number"|"tag")]), BarChart(labels, [Series(name, values)], "grouped"|"stacked", xLabel, yLabel), LineChart and AreaChart(labels, [Series(name, values)], xLabel, yLabel), PieChart(labels, values).
++ - Input: Form(name, [fields], Buttons([...])), FormControl(label, input, hint), Input(name, placeholder, type "text"|"email"|"number"|"password", rules ["required", "email"]), TextArea(name, placeholder, rows), Select(name, options, placeholder), Checkbox(name, label), Switch(name, label), Slider(name, min, max, step), Buttons([buttons], direction), Button(label, action, variant "primary"|"secondary"|"danger"|"ghost").
++ Query, Mutation and @Run aren't available. When the user acts on your UI you get a message starting with [UI], such as `[UI] Submitted "deploy": {"env":"staging"}`. Answer it in a new message; earlier blocks keep their state.
++ Examples:
++ ```openui
++ root = Card([header, form])
++ header = CardHeader("Deploy", "Pick a target")
++ form = Form("deploy", [FormControl("Environment", Select("env", ["staging", "production"]))], Buttons([Button("Deploy", "submit:deploy", "primary")]))
++ ```
++ ```openui
++ $count = 0
++ root = Stack([TextContent("Count: " + $count, "large"), Buttons([Button("-", Action([@Set($count, $count - 1)])), Button("+", Action([@Set($count, $count + 1)]), "primary")], "row")])
++ ```"##;
++ #[cfg(test)]
++ mod tests {
++     use super::{INSTRUCTIONS, eval, lang};
++     /// The examples agents are shown must parse and render.
++     #[::core::prelude::v1::test]
++     fn instruction_examples_parse() {
++         let blocks: Vec<&str> = INSTRUCTIONS.split("```openui\n").skip(1).map(|block| block.split("```").next().unwrap_or_default()).collect();
++         assert_eq!(blocks.len(), 2);
++         for block in blocks {
++             let program = lang::parse(block);
++             assert!(program.errors.is_empty(), "{block}: {:?}", program.errors);
++             assert!(program.undefined().is_empty(), "{block}");
++             assert!(!eval::evaluate(&program, &eval::State::new()).is_pending());
++         }
++         assert!(INSTRUCTIONS.lines().count() < 60);
++     }
++ }

+7−3src/session.rsView File

@@ -1607,11 +1607,15 @@ impl Session {
1607	1607	let mut request = json!({"req": "send_message", "content": prompt, "images": images});
1608	1608	// jcode's system prompt can't be added to, so the app's
1609	1609	// instructions go along with the first message.
1610		- let instructions = (!self.jcode.instructions_sent)
	1610	+ let first = !self.jcode.instructions_sent;
	1611	+ self.jcode.instructions_sent = true;
	1612	+ let instructions = first
1611	1613	.then(\|\| orchestrator::instructions(self.tool_caps()))
1612	1614	.filter(\|instructions\| !instructions.is_empty());
1613		- self.jcode.instructions_sent = true;
1614		- let reminder: Vec<String> = instructions.into_iter().chain(self.jcode.browser_note.take()).collect();
	1615	+ let inline = (first && settings::get(\|settings\| settings.inline_ui))
	1616	+ .then(\|\| crate::inline_ui::INSTRUCTIONS.to_string());
	1617	+ let reminder: Vec<String> =
	1618	+ instructions.into_iter().chain(inline).chain(self.jcode.browser_note.take()).collect();
1615	1619	if !reminder.is_empty() {
1616	1620	request["system_reminder"] = reminder.join("\n\n").into();
1617	1621	}

+5−0src/settings.rsView File

@@ -449,6 +449,9 @@ pub struct Settings {
449	449	pub slash_skills: bool,
450	450	/// Highlight Markdown in the composer as you type.
451	451	pub rich_composer: bool,
	452	+ /// Draw diagrams, charts and small UIs from agents' answers, and tell
	453	+ /// agents they can write them.
	454	+ pub inline_ui: bool,
452	455	/// Rest the composer to one line while scrolling a conversation.
453	456	pub collapse_composer: bool,
454	457	pub time_format: TimeFormat,
@@ -600,6 +603,7 @@ impl Default for Settings {
600	603	streaming: Streaming::Paragraphs,
601	604	slash_skills: true,
602	605	rich_composer: true,
	606	+ inline_ui: true,
603	607	collapse_composer: true,
604	608	time_format: TimeFormat::System,
605	609	proactive_panels: false,
@@ -709,6 +713,7 @@ impl Settings {
709	713	("Show skills in slash menu", self.slash_skills != d.slash_skills),
710	714	("Collapse composer on scroll", self.collapse_composer != d.collapse_composer),
711	715	("Rich text composer", self.rich_composer != d.rich_composer),
	716	+ ("Inline diagrams and UI", self.inline_ui != d.inline_ui),
712	717	("Send shortcut", self.send_shortcut != d.send_shortcut),
713	718	("Follow-up behavior", self.follow_up != d.follow_up),
714	719	("Context window indicator", self.context_meter != d.context_meter),

+7−2src/ui/inline/mod.rsView File

@@ -98,9 +98,9 @@ struct Fence {
98	98	program: Option<Arc<Program>>,
99	99	}
100	100
101		- /// Whether answers render rich blocks at all.
	101	+ /// Whether answers render rich blocks at all (Settings → General).
102	102	pub fn enabled() -> bool {
103		- true
	103	+ crate::settings::get(\|settings\| settings.inline_ui)
104	104	}
105	105
106	106	/// The workspace, so a block's buttons can open previews in its panels.
@@ -131,6 +131,11 @@ pub fn preview_html(code: &str, window: &mut Window, cx: &mut App) {
131	131
132	132	/// Adds the rich block hooks to an answer's Markdown view.
133	133	pub fn extend(view: TextView, icx: InlineCx) -> TextView {
	134	+ // With the setting off the view has no hooks, and the change in their
	135	+ // number makes it parse the answer again as plain code blocks.
	136	+ if !enabled() {
	137	+ return view;
	138	+ }
134	139	view.markdown_block_parser(parse).markdown_block_renderer(NODE, move \|node, window, cx\| render(node, &icx, window, cx))
135	140	}
136	141

+9−0src/ui/settings/general.rsView File

@@ -388,6 +388,15 @@ impl SettingsView {
388	388	.reset(reset(&view, cx, \|s\| s.rich_composer, \|s, on\| s.rich_composer = on))
389	389	.control(switch("rich-composer", saved.rich_composer, &view, p, \|settings, on\| settings.rich_composer = on)),
390	390	)
	391	+ .row(
	392	+ row("Inline diagrams and UI")
	393	+ .id("inline-ui")
	394	+ .description(
	395	+ "Let agents draw diagrams, charts and small interactive UIs in their answers. Applies to the agent instructions when a thread's agent next starts.",
	396	+ )
	397	+ .reset(reset(&view, cx, \|s\| s.inline_ui, \|s, on\| s.inline_ui = on))
	398	+ .control(switch("inline-ui", saved.inline_ui, &view, p, \|settings, on\| settings.inline_ui = on)),
	399	+ )
391	400	.row(
392	401	row("Collapse composer on scroll")
393	402	.id("composer-collapse")

+1−0src/ui/settings/search.rsView File

@@ -81,6 +81,7 @@ pub const ITEMS: &[Item] = &[
81	81	item("proactive-panels", "Proactive panels", Page::General, "automatically open diff right panel agent completion"),
82	82	item("skills-in-slash-menu", "Show skills in slash menu", Page::General, "command menu dollar $ slash /"),
83	83	item("composer-rich-text", "Rich text composer", Page::General, "composer rich text bold italic markdown styled wysiwyg"),
	84	+ item("inline-ui", "Inline diagrams and UI", Page::General, "mermaid openui charts tables forms svg html preview artifacts render answers"),
84	85	item("composer-collapse", "Collapse composer on scroll", Page::General, "composer rest resting scroll wheel conversation shrink minimize"),
85	86	item("send-shortcut", "Send shortcut", Page::General, "enter return ctrl multiline prompt new line composer"),
86	87	item("follow-up-behavior", "Follow-up behavior", Page::General, "queue steer running turn send default behavior composer"),