FINDS - TESTS · Slate 2, Test 5 — tldraw as a whiteboard surface: agent programmability
Date: 2026-11-20 · Subject: tldraw SDK v4.5.12 (tldraw npm package, React)
Question: Can an agent — no human, no mouse — create, edit, and export a real diagram on a tldraw board, purely through code?
Material: The Muse Map system flow from the Visual Explainer test (visual-explainer-muse-map-2026-11-19/muse-map-v5-recap.src.html), copied exactly: 9 nodes — Ros → Main Chat → Orchestrator → Departments ("85 jobs") → Approval Gate (key node, "Ros approves") → Outputs ("apps · briefs"), plus Memory, Scheduled Jobs ("6 enabled"), Secure Vault — and 8 labelled connectors: briefs, hands jobs, routes work, asks first, ships (hot), recalls, refreshes, credentials (async/dashed in the source).
Headline result: yes, on every route tested. Both programmatic creation routes work end-to-end, scripted edits work, and PNG/SVG export from code works. The constraints are not technical — they are (a) no auto-layout, so the generating code owns geometry and label spacing, and (b) the licence: production use is gated by a paid/keys regime (see §5).
1. Programmatic creation — two routes, both proven
Route A — Editor API in a running page (Playwright-driven)
The test app was opened with ?mode=empty — a verifiably blank editor (0 shapes). A Playwright script then called the public Editor API via the exposed editor instance (editor.createShape, editor.createBinding, with createShapeId / createBindingId / toRichText from the tldraw package — the same imports agent code would use):
- 9
georectangles at the source's grid positions, Approval Gate as orange/solid (key), the rest black/semi. - 8
arrowshapes, each attached to its two nodes with real arrow bindings (start/end terminals), dashed for the source's async edges, orange + large for the hot ships edge. - Result: 17 shapes created (9 geo + 8 arrows), zero human input. Snapshot saved via
editor.getSnapshot()→board.tldr.json(29 KB).
Evidence: export-before.png / export-before.svg (exports of the created board), editor-created.png (the board inside the real editor UI).
Two findings from this route:
- Validation is strict and helpful. First attempt failed with ValidationError: At shape(type = arrow).props.kind: Expected "arc" or "elbow", got straight — in v4 a straight arrow is kind: "arc" with bend: 0. One-line fix; the store refuses malformed records, which is exactly what an agent needs.
- No auto-layout. First pass used a 270 px column pitch (60 px gaps): the board was correct but the short horizontal arrows' labels collided with the boxes. Widening the pitch to 340 px fixed it. tldraw gives an agent a canvas and an API, not a layout engine — geometry, spacing, and label fit are the generator's problem.
Route B — Pure-Node snapshot generation (no browser, no editor)
A Node script (harness/gen-board.mjs) authored the entire board as JSON from the spec — document, page, 17 shape records, 16 binding records, fractional indexes chained with getIndexAbove from @tldraw/utils, schema block produced by createTLSchema(...).serialize() from @tldraw/tlschema. It then validated the file headlessly: a fresh @tldraw/store Store accepted it via loadStoreSnapshot (39 records, schema-validated — the store throws on any invalid record).
The file (board-from-json.tldr.json, 26 KB) was then loaded into the unmodified test app (?snap=board-from-json.tldr.json): 17 shapes rendered (9 geo + 8 arrows), all 17 labels present, and a PNG export was produced from the JSON-loaded board. Evidence: export-from-json.png, editor-from-json.png.
Routes that exist: (A) Editor API against a live editor (headless-browser or in-app agent code), (B) offline snapshot JSON generation + schema validation, (C) snapshot round-trip — the app preloads board.tldr.json through the standard snapshot prop; the edits test below ran against that preload and found all 17 shapes. All three were exercised.
2. Programmatic edits — all three landed
Against the preloaded snapshot, a second script (harness/edit-board.mjs) performed, via the Editor API only:
| Edit | Call | Verified result |
|---|---|---|
| Add a node | createShape (geo) + createShape (arrow) + 2× createBinding |
"Finds Library" node added at grid (1,2) with a publishes arrow from Outputs (faithful to the source caption: approved work ships to "apps, the Finds Library, and briefs"). Shapes 17 → 19 (10 geo, 9 arrows). |
| Move a node | updateShape x/y |
Secure Vault (1080, 500) → (1140, 580). Its bound dashed credentials arrow followed automatically and re-routed diagonally to the Approval Gate — bindings do the work an agent would otherwise hand-recompute. |
| Relabel a connector | updateShape props.richText |
ships → ships (approved), confirmed in the store and in the export. |
Evidence: export-after.png / export-after.svg, editor-after-edits.png, board-edited.tldr.json. One honest blemish: the new publishes label sits where its arrow crosses the dashed recalls arrow — readable, but scripted additions get no collision avoidance; an agent must place labels/geometry with crossings in mind.
3. Export from code
editor.getSvgString(ids, { background: true }) and editor.toImageDataUrl(ids, { format: 'png', scale: 2, background: true }) — no UI, no download dance. Verification of the original board's exports:
- SVG text check: all 9 node labels + sub-labels ("85 jobs", "Ros approves", "apps · briefs", "6 enabled") and all 8 connector labels present as text in export.svg (132 KB).
- PNG viewed: export.png (595 KB, 2588×1200 @2x) shows the complete board — nodes, bound arrows, dashed async edges, orange hot edge, highlighted gate.
- Exports were produced for all three states: created board, edited board, JSON-generated board.
4. Embed
Minimal Vite 7 + React 19 app (app/): one Tldraw component, snapshot preload, ~60 lines of host code. npm run build passes. Bundle reality check: dist ≈ 2.0 MB total — main JS chunk 1,890 kB (578 kB gzip) + 79 kB CSS. tldraw is a full editor, and it weighs like one; for a blog-diagram embed that's heavy, for an app surface it's a normal dependency. Environment notes: rendering/verification used Chromium 1254 (1208 avoided per the slate-2 warning — no canvas crashes encountered); the headless page logged tldraw asset/font fetch noise ("source image cannot be decoded", "Failed to fetch") with no effect on boards or exports.
5. Licence reality check (researched 2026-11-20, sources cited)
tldraw is source-available, not open source: the "tldraw license" permits use only in development by default. Facts from tldraw's own docs:
- Production requires a license key. "The tldraw SDK requires a license key to work in production. Without a valid key, the SDK runs in development mode only." Development = non-HTTPS, localhost/loopback, or
NODE_ENV≠production. In production without a key, the SDK logs errors and stops rendering the editor after five seconds. Keys are validated client-side, encode allowed hosts + type + expiry, and are safe to embed in frontend code. (https://tldraw.dev/sdk-features/license-key, fetched 2026-11-20) - First-hand confirmation: the unlicensed editor in this very test displays "Get a license for production" in the bottom-right corner — visible in editor-created.png.
- License types (same source): Trial — free, 100 days, no watermark, one per company/project, key emailed immediately; Commercial — annual, no watermark, required for any commercial production use, pricing is not published (sales conversation via the plans form; the pricing page at https://tldraw.dev/pricing, fetched 2026-11-20, publishes no prices); Hobby — non-commercial, discretionary (reviewed application), "made with tldraw" watermark must stay visible.
- Telemetry differs by type: Commercial sends nothing; Trial and Hobby ping tldraw's servers with license ID, type, SDK version and page URL; unlicensed production sends SDK version + page URL. (Same docs page; terms summary at https://tldraw.dev/community/license)
Implication for Ros's apps: anything commercial shipped on a public domain needs a commercial license at an unlisted, sales-quoted price before launch. Personal/non-commercial projects can run on a free hobby key with the watermark. Local tools, prototypes, and anything behind localhost need nothing.
6. Effort log (2026-11-20, UTC)
| Time | Step |
|---|---|
| 13:54 | Read source recap; extracted the 9-node / 8-edge spec verbatim; scaffolded Vite app + harness. |
| 13:55–13:56 | npm install (tldraw 4.5.12); confirmed API surface against installed type definitions (getSvgString, toImage, toRichText, binding props). |
| 13:57 | Route A creation run — one validation failure (arrow kind), fixed; board created. Label collisions at tight pitch → widened grid, regenerated. Exports + editor screenshot. |
| 13:57 | Edits run — add / move / relabel all verified in store data and exports. |
| 13:58 | Route B — Node generator written; headless schema validation passed (39 records); app load + render + export verified. |
| 13:59 | Production build (passes; bundle measured). Licence research: license-key docs + pricing page fetched. |
| ~14:00 | This report. |
Total wall-clock ≈ 1 hour including write-up; hands-on build time was dominated by reading API types, not by fighting the tool.
7. Draft verdict
Reuse later — scoped to app surfaces, not to static diagram production.
- As an agent-programmable whiteboard, tldraw is genuinely strong. Two independent creation routes (live Editor API; offline JSON + official schema validation), real bindings that survive moves, one-call edits, and one-call SVG/PNG export. An agent can own the full lifecycle of a board without simulating a single mouse event. That is rare among whiteboard tools and it is the property this test was asked to judge.
- Second Brain canvas: plausible, with two caveats. If the Second Brain wants a freeform canvas an agent can also write to (not just the human), tldraw is the credible candidate — boards are portable JSON snapshots the agent can generate, diff, and re-render. Caveats: no auto-layout (agent-generated boards need deliberate geometry), and the 2 MB bundle.
- Blog diagrams: skip tldraw. For publishable static diagrams, the licence watermark/hobby-key dance, bundle weight, and hand-tuned look of the existing routes (hand-built SVG, Visual Explainer, Archify) all argue against embedding a live editor; tldraw's own exports are clean, but generating the same diagram as direct SVG is cheaper and unencumbered.
- The licence is the real gate. Any commercial production surface needs a sales-quoted commercial license before launch; budget and procurement, not engineering, decide adoption there. For personal/internal tools the hobby key (watermark) or localhost development mode costs nothing.
Test record (for FINDS-LIBRARY filing)
- Subject: tldraw (SDK v4.5.12) — whiteboard/canvas surface, agent programmability test. Slate 2, test 5 (final slate-2 test).
- What ran: (1) Board created from a blank editor purely via Editor API (Playwright/Chromium 1254); (2) same board generated as snapshot JSON in pure Node and schema-validated with @tldraw/store + @tldraw/tlschema, then rendered by the app; (3) scripted edits (add node + connector, move node, relabel connector) against the preloaded snapshot; (4) SVG + PNG exports from code for all three board states, contents verified by SVG text search and PNG inspection; (5) Vite + React embed, production build; (6) licence terms verified against tldraw.dev docs/pricing (fetched 2026-11-20) plus the in-product "Get a license for production" notice observed in the test editor.
- When: 2026-11-20.
- Outcome: All programmability tests passed. 9 nodes / 8 connectors reproduced exactly; edits verified in store data and exports; exports contain every label. Constraints found: no auto-layout (label collisions are the generator's problem), ~2 MB bundle, production use requires a license key (trial 100 days free; hobby free with watermark, non-commercial only; commercial annual, price unpublished/sales-led; unlicensed production stops rendering after 5 s).
- Output reference:
~/workspace/finds-tests/tldraw-2026-11-20/—report.md;board.tldr.json(editor-created),board-from-json.tldr.json(Node-generated),board-edited.tldr.json;exports/export.png,exports/export.svg,export-before.*,export-after.*,export-from-json.png,editor-created.png,editor-after-edits.png,editor-from-json.png; app inapp/, harness inharness/. - Reuse proposal: Reuse later — candidate canvas layer for the Second Brain (agent-writable boards as portable JSON), and for future in-app whiteboard surfaces on personal/internal tools. Skip for blog/static diagrams and for any commercial production surface unless a commercial license is budgeted first.