Skip to main content

Draw it, don't describe it

The slowest, most lossy part of working with an agent on a UI is describing what you mean: "the card at the top left, no the other one." Sketch mode cuts the description out — draw directly on the running app, and the agent gets the drawing plus the element context underneath it.

Silent by design — 0:32. Every beat is legible with the sound off.

What happens

Open (0:04). Sketch mode opens over the live dashboard — full toolbar, style panel, dot grid. The app keeps running underneath.

Draw (0:10). A rectangle around the Revenue card; an arrow from Recent invoices up to it. Since 0.15.1 the default ink is theme-aware — on this dark dashboard sketch mode opens with light ink, no picker detour. (Before the fix, the first thing anyone drew on a dark app was invisible.)

Send (0:17). sketch.save() pushes the drawing over the real proxy channel. The agent's reply names what was drawn and where — no screenshot to attach, no prose to parse.

How it's made

Recorded by the scripted demo engine (docs-site/screenshots/engine/) against the real stack; the drawing is real mouse input on the actual sketch canvas, and the agent's toasts travel the real PROXY TOAST transport. make demo NAME=live-sketch re-records it.