Chapter 15: Plugin Architecture & Registry

While sandboxed Lua scripts (Chapter 14) handle fast buffer edits and daemon queries inside AutoDoc's process, some workflows require persistent database connections, background HTTP servers, external toolchains, or rich custom panels. AutoDoc's plugin architecture (ADR 0209, ADR 1791268009, ADR 1791762474, ADR 1792078052, ADR 1792254331; see docs/plugins.md) runs every plugin as a separate OS process in its own process group, communicating over stdin and stdout via asynchronous msgpack-RPC notifications.

This chapter explains why out-of-process notification framing keeps the editor crash-proof, details the three plugin shapes (dialog, docked card, and headless service), traces the capability progression from Protocol 1 through Protocol 8, shows how the two UI kits (plugin/card and plugin/decl) draw a card for you, and shows how to build and install plugins from a Git URL (a plugin registry is coming; there is none yet).


15.1 Why Out-of-Process Msgpack-RPC Notifications?

In-process plugin hosts (shared libraries, unsandboxed interpreters, or synchronous RPC hooks) inevitably expose the editor to latency spikes, memory leaks, and thread deadlocks. AutoDoc avoids all three by enforcing three architectural rules:

  1. Process-Group Isolation: Each plugin starts in its own directory inside a dedicated OS process group with AUTODOC_PLUGIN_PROTOCOL, AUTODOC_PLUGIN_DIR, and AUTODOC_SOCKET in its environment. Standard output (stdout) carries the msgpack-RPC wire stream; standard error (stderr) is captured to the plugin's log and surfaced in a corner toast if the process exits unexpectedly.
  2. Bidirectional Notifications Only (Zero Blocking Requests): Every wire message in both directions is a msgpack-RPC notification carrying a single map payload. Neither AutoDoc nor the plugin ever blocks waiting for a synchronous RPC return frame on the wire. Where a correlation is required—such as invoking a host command in Protocol 3 (host.call / plugin.result) or paging a result cursor in Protocol 4 (plugin.table.fetch / host.rows)—the correlation is carried explicitly by a call number or table id.
  3. Bounded 1 MiB Envelope and 3-Second Shutdown: Every encoded notification is capped at 1 MiB. When a plugin closes (via Esc, host.close, Manage › Turn off, or TUI exit), AutoDoc sends plugin.close, waits up to 2 seconds for clean teardown, sends SIGTERM to the entire process group, and sends SIGKILL 1 second later if any child remains.

15.2 The Three Plugin Shapes: Dialogs, Cards, and Services

A plugin is a program AutoDoc runs in a dialog over the page, from the Plugins menu. The first one is AutoTetris, autodoc-tetris.

Every plugin directory contains a strictly decoded plugin.toml manifest at its root that declares its shape:

Cards. A card plugin (AutoDB's, say) is a docked panel of its own beside the page, at the edge its manifest names first (AutoDB's: the right), and moves, resizes and keeps its place as every panel does (Alt/Option-drag, or in --gui a plain drag of its title row or border, Go › Arrange panel). It takes the keys when clicked or when one of its commands runs, and then every key is the plugin's, q included. Esc, or Close (Esc) on its right-click menu, closes the panel and hands the keys back to the page; the plugin runs on, its results and session kept, and a command or its Plugins menu entry opens the panel again.


15.3 Progression of Protocols 1 Through 11

AutoDoc's plugin contract evolves through ten cumulative protocol versions. A manifest must declare the minimum protocol version required for the features it uses, and AutoDoc refuses any manifest that mixes newer keys with an older protocol number—ensuring a plugin never runs half-broken on an older host.

flowchart LR
  P1["Protocol 1 (Modal Dialog & Styled Frames)"] --> P2["Protocol 2 (Docked Cards, Services, Feed & Commands)"]
  P2 --> P3["Protocol 3 (Host Command Calls via calls = [...])"]
  P3 --> P4["Protocol 4 (Hosted Paged Table & Byte Offsets)"]
  P4 --> P5["Protocol 5 (Typed Cell Kinds: NULL, Number, Bool, JSON)"]
  P5 --> P6["Protocol 6 (Pointer: Left Press & Wheel in the Card)"]
  P6 --> P7["Protocol 7 (Session Slot, Close Reasons & Command Args)"]
  P7 --> P8["Protocol 8 (Notifications: host.notify)"]
  P8 --> P9["Protocol 9 (Opening Files: [open] extensions)"]
  P9 --> P10["Protocol 10 (Pixel Surfaces: [surface], the side channel)"]
  P10 --> P11["Protocol 11 (Editor Menu Rows: [[menu]], host.menu)"]

Protocol 1: Modal Dialogs and Styled Cell Frames

Protocol 1 establishes the handshake (plugin.open {protocol, width, height, theme} answered within 5 seconds by host.ready {protocol}), key forwarding (plugin.key), geometry/theme notifications (plugin.resize, plugin.theme), and host.frame {rows}. Each row in host.frame is a list of styled text runs {t, fg, bg, b, i, u} using theme semantic tokens, ANSI names, or #rrggbb hex colors. AutoDoc always paints the newest frame and drops intermediate frames if the plugin renders faster than the display refresh.

Protocol 2: Docked Cards, Services, Document Feed, and Commands

Protocol 2 introduces [dialog] modal = false (docked cards), kind = "service", the live document feed ([feed] document = true), and user commands ([[commands]]).

Protocol 3: Whitelisted Host Command Calls (calls)

Protocol 3 allows a plugin to invoke AutoDoc's own catalog commands—or user Lua commands (lua.<name>)—provided every target command ID is explicitly listed in top-level calls = ["file.save", "buffer.new"] in plugin.toml (wildcards like todos.* and cross-plugin plugin.* IDs are refused). The plugin sends host.call {call, id, args} with a unique in-flight integer call (up to 64 concurrent calls), and AutoDoc replies with plugin.result {call, ok, error}.

A flagship Protocol 3 capability is buffer.new: calling peer.Call("buffer.new", "query.sql", "SELECT * FROM users LIMIT 100;\n") opens a new unsaved tab up to 256 KiB with syntax highlighting inferred from the suggested filename, without touching the filesystem until the user presses Ctrl+S.

Protocol 4: Hosted Paged TableView and UTF-8 Byte Offsets

Protocol 4 ([table]) offloads tabular result rendering from the plugin to AutoDoc's native bottom Results panel (TableView) and closes the race between document feed debouncing and command execution:

sequenceDiagram
  participant Host as AutoDoc TUI Host
  participant Plugin as Plugin Process Group
  Host->>Plugin: plugin.open {protocol, width, height, theme}
  Plugin->>Host: host.ready {protocol}
  Host->>Plugin: plugin.document {path, workspace, text, cursor_byte, selection_bytes, version}
  Host->>Plugin: plugin.command {id, document}
  Plugin->>Host: host.table {id, title, fields, rows, kinds, more}
  Host->>Plugin: plugin.table.fetch {id, n}
  Plugin->>Host: host.rows {id, rows, kinds, more}
  Host->>Plugin: plugin.close {}
  Note over Host,Plugin: Bounded 3s shutdown: 2s wait -> SIGTERM -> 1s wait -> SIGKILL
  1. Exact Byte Offsets (cursor_byte, selection_bytes): Because cursor and selection count UTF-8 grapheme clusters, slicing text in a compiled language would otherwise require re-implementing Unicode grapheme segmentation. Protocol 4 adds cursor_byte and selection_bytes ([start, end) byte offsets) computed from the exact same cluster pass.
  2. Document Snapshot in plugin.command: Because plugin.document waits for a 300 ms typing pause, invoking a command immediately after moving the cursor or typing could otherwise run against a stale buffer. Under Protocol 4 + [feed] document = true, plugin.command {id, document} embeds the exact buffer state as it stood at the instant the command was triggered.
  3. Demand-Paged host.table and plugin.table.fetch: The plugin opens a result set with host.table {id, title, fields, rows, more}. As the user scrolls near the bottom of the loaded rows, AutoDoc sends plugin.table.fetch {id, n} (at most one fetch in flight at a time), and the plugin replies with host.rows {id, rows, more} or host.table_end {id, error} up to a 100,000-row safety cap.

The results panel. A plugin may show a table of rows (a query's result, say) in AutoDoc's own results panel, at the bottom (Go › Results): numbered rows, a cursor on cells with vim's keys (h j k l with counts, gg, G; v, V and Ctrl+V select cells, rows and a block; y copies), an ORDER BY bar and s to sort the loaded rows, and a right-click menu. The panel acts on the table in the view: its sort, copies (TSV, CSV, JSON, Markdown), results.markdown and exports (CSV or JSON, written whole or not at all) take the loaded rows in the order shown, and none of them fetches a row. An export writes the rows loaded in the view; scroll to load more, or change the query. Closing the panel lets the table go.

Illustrative Hosted Table Page Transfer Profile Within 1 MiB Envelope0246810121416Latency (ms)100 rows250 rows500 rows1000 rowsPage Batch Size (Rows per host.table / host.rows)Pipeline StageMsgpack TransferTotal Host Table Render

Protocol 5: Typed Cell kinds for NULLs, Numbers, Booleans, and JSON

In Protocol 4, every cell is a string, making SQL NULL indistinguishable from the four-letter string 'NULL'. Protocol 5 adds an optional kinds array to host.table and host.rows—one string per row with exactly one byte per column:

Protocol 6: The Pointer in a Card

Protocol 6 adds plugin.pointer {kind, button, x, y, count}: a left press ("press", "left") or a vertical wheel step ("wheel", "up"/"down") inside the plugin's frame, in frame coordinates from zero. count is bounded to 1–16. AutoDoc sends it only to a protocol-6 plugin, only for unmodified input inside the frame, and keeps the right press for the card's own context menu. A Go plugin takes it by implementing plugin.PointerHandler; a plugin can ignore it and keep its keyboard controls. AutoDB's card (Chapter 16) selects a connection or a tree row with it, and both UI kits below handle it for you.

Protocol 7: A Session That Outlives the Process, and a Lua Package

Protocol 7 is for a plugin that signs in to something. AutoDoc keeps a small session slot for each plugin in its memory, never on disk, until it quits. The plugin replaces the slot with host.session {values} (the SDK's Peer.SetSession), and every later plugin.open hands it back as Open.Session. A plugin that is turned off and on, updated, or restarted after a crash reads its token back, checks it, and does not ask the user to sign in again. A slot is bounded (32 keys, 8 KiB a value, 64 KiB in all), and nothing logs its values.

plugin.close now carries a reason, which the plugin reads with Peer.CloseReason() inside Close. On quit (AutoDoc is going, and the slot with it) and remove, it logs out. On off and update, it keeps the token for the next start. A card's toggle only hides the card, so the plugin keeps running and stays signed in. Plugins › Turn off is what stops it.

A plugin may also ship a Lua package in its folder (lua/package.toml, lua/init.lua). AutoDoc loads it gated as plugin:<name>, and its prompt names the plugin. It can define : commands, keys and menu rows. autodoc.plugin.run(id, args…) sends one of its own plugin's commands, with arguments (plugin.command {id, args}, taken by a plugin.CommandArgser), and it reaches no other plugin. AutoDB's :autodb query select 1 (Chapter 16) works this way. If you write a package of your own with the plugin's name, it replaces the plugin's.

Protocol 8: Notifications

Protocol 8 lets a plugin speak where the user is, whatever panel is open. host.notify {message} (the SDK's Peer.Notify) shows a toast in AutoDoc's corner, kept in the notification history (SPC h). A long task gives the toast an id and ongoing: each notice with that id updates the same toast, and one without ongoing ends it, its last message kept in the history. The ids are the plugin's own, so two plugins never touch each other's, and a plugin that stops has its ongoing toasts ended for it. A message is at most 1 KiB; at most 16 toasts of one plugin run at once.

Protocol 9: Opening Files

Protocol 9 lets a plugin open a kind of file AutoDoc does not read itself. Its manifest's [open] names the extensions (extensions = [".ipynb"]) and one of its commands (command = "import"). However the file is opened (the picker, File › Open file…, the explorer, a link, back and forward, a plugin's file.open_path), AutoDoc runs that command with the file's absolute path, starting the plugin first when it is not running; nothing opens as text. The plugin decides what opening means: autodoc-jupyter writes an .ipynb as a lab note beside it and opens that. A plugin never claims AutoDoc's own extensions (.md, .txt, .yaml, .csv, .html, …), and nothing on the wire changes.

Protocol 10: Pixel Surfaces

Protocol 10 ([surface]) lets a card or dialog draw pixels: in AutoDoc's GUI its area is a native view drawing a golib gui stream, and in the terminal the plugin draws cells. Frames cross a side channel, a Unix socket inherited as fd 3, never stdout:

The pointer arrives in dp. The host keeps IME composition, drawing the preedit at the caret the plugin reports and sending committed text as keys. A malformed side-channel message closes the plugin. Linux ships first; macOS and plugin-exported DMA-BUFs follow (ADR 1792797354).

Protocol 11: Editor Menu Rows

Protocol 11 ([[menu]]) puts a plugin's rows on the editor's right-click menu, under a submenu of its title. Each row runs one of its [[commands]], for the files its globs match:


15.4 Anatomy of a plugin.toml Manifest

AutoDoc validates plugin.toml with strict unknown-key rejection before starting a plugin. Below is a complete Protocol-5 manifest showing every top-level table and capability declaration:

# plugin.toml — Protocol 5 docked card with document feed, host calls, and paged table
name        = "autodb"
title       = "AutoDB Workbench"
version     = "0.4.0"
description = "Live SQL sessions, schema browser, and paged query results"
protocol    = 5
kind        = "dialog"
command     = ["./autodoc-autodb"]
calls       = ["buffer.new", "file.save"]

[install]
build = ["go", "build", "-o", "autodoc-autodb", "./cmd/autodoc-autodb"]

[dialog]
modal      = false
width      = 44
height     = 24
placements = ["right", "left"]
esc        = "hide"

[feed]
document = true

[table]
title = "Query Results"

[[commands]]
id    = "connect"
title = "Connect / Unlock Session"
key   = "c"

[[commands]]
id    = "run"
title = "Run Statement at Cursor"
key   = "r"

[[commands]]
id    = "script"
title = "Run Full SQL Script"
key   = "s"

15.5 Writing a Plugin with the Go SDK

Writing one: docs/plugins.md covers the manifest, the protocol, and the SDK in package plugin.

The github.com/yongjohnlee80/autodoc/plugin Go package implements Protocols 1–7 with concurrency-safe Peer methods and optional handler interfaces (DocumentReader, Commander, CommandArgser, CommandDocer, Focuser, TableFetcher, Hider, PointerHandler):

package main

import (
	"context"
	"os"

	"github.com/yongjohnlee80/autodoc/plugin"
)

type demoPlugin struct {
	plugin.Base
	peer *plugin.Peer
	doc  plugin.Document
}

func (d *demoPlugin) Open(p *plugin.Peer, o plugin.Open) { d.peer = p }
func (d *demoPlugin) Document(doc plugin.Document)      { d.doc = doc }

func (d *demoPlugin) Command(id string) {
	if id == "inspect" && !d.doc.TooLarge {
		_ = d.peer.Table(plugin.TableOpen{
			ID:     "doc-meta-1",
			Title:  "Document Offsets",
			Fields: []string{"path", "version", "cursor_byte", "nullable_note"},
			Rows:   [][]string{{d.doc.Path, "1", "42", "NULL"}},
			Kinds:  []string{"tnnz"}, // text, number, number, NULL
			More:   false,
		})
	}
}

func main() {
	if err := plugin.Serve(context.Background(), &demoPlugin{}); err != nil {
		os.Exit(1)
	}
}

Writing Plugins in Python or Any Other Language

Because the wire protocol is standard msgpack-RPC over stdin and stdout, a plugin can be written in Python, Rust, Zig, or Node.js without linking against Go. The repository includes a reference Python implementation (examples/python/echo_plugin.py) that reads [2, method, [params]] notification tuples from sys.stdin.buffer and writes host.ready and host.frame notifications to sys.stdout.buffer with flush=True.


15.6 Building the Card's UI: plugin/card and plugin/decl

A card or dialog can draw every cell itself into a plugin.Frame, as the example above could. AutoDB's first card did, in about 860 lines of layout, wrapping, focus, list and theme code before any database logic. The SDK's two UI kits (ADR 1792254331) do that work, so a card is its controls and its behaviour. Neither changes the wire: both end in the same host.frame.

flowchart LR
  subgraph Plugin["Plugin process"]
    Page["card.Page: blocks & controls"] --> Card["card.Card: fit, theme roles, keys, pointer"]
    QML["QML layout + Theme.* palette"] --> Prog["tuidecl.Program on a frame backend"]
    Card --> Frame["plugin.Frame (grapheme cells)"]
    Prog -->|"FlushSized: diff + size drawn at"| Frame
  end
  Frame -->|"host.frame (unchanged wire)"| Host["AutoDoc host: draws runs a cluster at a time"]
  Host -->|"plugin.key / resize / theme / focus / pointer"| Card
  Host -->|"same notifications, as terminal events"| Prog

Which to pick: the native kit starts drawing at once and adds little to the binary (about 3 MB for a minimal stripped plugin); the QML adapter parses its layout first and brings golib's widgets with it (about 7.5 MB), in exchange for declarative layouts and widgets such as TreeView and TableView. Both measure text exactly as AutoDoc does: a CJK character takes two cells, and an accent written as a combining mark, a ZWJ emoji or a flag stays one character. plugin.Frame itself places characters this way, and AutoDoc's renderer draws them a grapheme cluster at a time.

Each example's test drives its plugin over the protocol, as AutoDoc does: open, type, press a button, switch the theme. See docs/plugins.md § "Drawing a card" for the full API.


15.7 Installing from a Git URL, Security Boundaries, and Debugging

Adding one. Plugins › Add from a git URL… takes the plugin's repository, which holds a plugin.toml at its top. There is no hub: the URL is the source.

Before it builds or starts the plugin ([install] build = [...]), AutoDoc presents an explicit capability summary derived from plugin.toml—showing the build command, executable argv, surface type (dialog, card, or service), whether it reads the open note ([feed]), every [[commands]] shortcut on SPC p, every host command in calls, and whether it publishes to the [table] Results panel. You can also override any plugin command's SPC p key letter via the preference tui.plugin.<name>.key.<id>.

Managing them. Plugins › Manage plugins… lists each plugin with its source, commit and placement.

Where they are.

What a Plugin May Not Do

By architectural contract (docs/plugins.md), a plugin cannot:

  1. Mutate the open buffer directly: A plugin reads the document feed (plugin.document); it can only modify buffers by invoking a whitelisted command declared in calls (such as buffer.new or a user Lua command).
  2. Execute undeclared host commands or another plugin's commands: Any host.call targeting an ID outside calls or starting with plugin. is rejected immediately.
  3. Block the editor's 60fps render loop: Syntax highlighting and frame layout run synchronously in Go; a slow or wedged plugin process only delays its own card or table page and is terminated cleanly within 3 seconds on close.
DirectionNotification / Manifest KeyProtocolPayload / SchemaBehavior and Contract
Host -> Pluginplugin.open1{protocol, width, height, theme}Sent once at startup after initial layout; omitted width/height for kind = service
Plugin -> Hosthost.ready1{protocol}Required within 5s of plugin.open; confirms negotiated protocol version (1..5)
Host -> Pluginplugin.key1{key, text, ctrl, alt, shift}Forwards keystrokes while dialog is open or docked card is focused (except Esc)
Host -> Pluginplugin.resize / plugin.theme1{width, height} / {theme}Notifies plugin of panel geometry changes or live theme switches
Host -> Pluginplugin.hide / plugin.show1{}Sent when esc = hide hides a dialog or when a docked card panel closes/reopens
Plugin -> Hosthost.frame / host.title1{rows} / {title}Paints styled cell runs {t, fg, bg, b, i, u} or updates the surface title
Plugin -> Hosthost.close / plugin.close1{}Requests surface close or initiates bounded 3s shutdown (2s -> SIGTERM -> 1s -> SIGKILL)
Manifest[dialog] modal = false2boolean (default true)Turns a dialog into a non-modal docked card panel alongside the editor
Manifestkind = service2start = use | launchRuns headless without a UI surface; started on first command/feed or at TUI launch
Host -> Pluginplugin.focus2{focused}Notifies a docked card when it acquires or releases keyboard focus
Host -> Pluginplugin.document2{path, workspace, root, text, cursor, selection, version, too_large}Sent on start and 300ms after editing/cursor pause under [feed] document = true
Host -> Pluginplugin.command2{id} (or {id, document} in P4+)Fires when user invokes a [[commands]] entry from SPC p or the Plugins menu
Manifestcalls = [...]3list of exact command IDsWhitelists AutoDoc catalog commands (e.g. file.save, buffer.new) the plugin may call
Plugin -> Hosthost.call / plugin.result3{call, id, args} -> {call, ok, error}Executes a whitelisted host command on the UI loop and returns matched result by call number
Manifest[table]4title = stringEnables hosted virtualized TableView in AutoDoc's bottom Results panel
Host -> Pluginplugin.document (P4 offsets)4adds {cursor_byte, selection_bytes}Provides exact UTF-8 byte offsets matching grapheme cursor/selection coordinates
Plugin -> Hosthost.table / host.rows4{id, title, fields, rows, more}Opens a paged table in the Results panel or answers plugin.table.fetch {id, n}
Plugin -> Hosthost.table_end4{id, error}Ends a paged table stream early when a cursor errors or is cancelled
Host -> Pluginplugin.table.close4{id}Notifies plugin that its table was closed or replaced so it can free server cursors
Plugin -> Hosthost.table / host.rows (P5 kinds)5adds {kinds} (1 byte per field: t, z, n, b, j)Distinguishes SQL NULL (z) from 'NULL' (t), sorts numbers/booleans accurately, exports typed JSON
Manifest[[menu]]11{command, where = "editor", files?, title?}A row on the editor's right-click menu under the plugin's title, running one of its [[commands]] for the files its globs match; one row a command
Plugin -> Hosthost.menu11{id, title?, enabled?}Retitles or disables a [[menu]] row from the next menu opening; kept until the plugin stops; an undeclared id is refused

← Previous: Chapter 14: Lua Scripting & Custom Commands | Index | Next: Chapter 16: AutoDB Live SQL Workbench →