Writing an AutoDoc plugin

A plugin is a program. AutoDoc's TUI starts it and the two speak msgpack-RPC notifications, both ways, over the plugin's stdin and stdout. Nothing is a request: where a plugin needs an answer (a call, protocol 3), the answer is a notification of its own, matched by a number. A plugin is one of three things:

A protocol-2 plugin can also read the open note (the document feed) and have commands on the Plugins menu and the SPC p card. A protocol-3 plugin can also run AutoDoc's own commands, the ones its manifest declares (calls). A protocol-4 plugin can also show a table of rows in AutoDoc's own results panel ([table]), and its commands carry the note they were chosen on. A protocol-5 plugin can also say each cell's kind, so AutoDoc tells a NULL from the text NULL, sorts numbers as numbers and writes JSON with its nulls, numbers and booleans. ADR 0209, ADR 1791268009, ADR 1791762474 and ADR 1792078052 are the decisions behind it.

autodoc-tetris is a complete protocol-1 example. examples/python/echo_plugin.py speaks the protocol with no AutoDoc code at all.

Installing, and the registry

Users install a plugin from Plugins › Browse the registry…, or from Add from a git URL… by its URL or by its name in the registry. A Lua package installs the same way with :lua-install <name or git-url>. The registry (autodoc-plugin-registry, ADR 1792351195) lists plugins, Lua packages and KB types, each with the audits run on its commits.

Every install asks first. The card names the commit it would install, and says whether that exact commit was audited ("✓ audited by <model> on <date> at this commit"), or when the entry was last audited, or that it never was. The card's warning stays either way: an audit is a reading of one commit, not a guarantee. To be listed, add an entry to the registry repo by pull request.

Each install or update stages its own clone, pinned to that commit: the card, Audit… and the install all use it, and the installed folder is never built in. Audit… hands a fresh copy of the clone to one of the user's agents, with a task file beside it (docs/audit-prompt.md and the commit), and leaves a notification whose click brings the card back. The agent only reads; its report is advice.

Keep your plugin's state outside its folder. An update builds the new commit in its clone, then replaces the installed folder with it. The folder it replaced is kept in plugins/.previous/<name> until the next update or its removal, so files a plugin wrote into its own folder don't carry over.

The manifest

plugin.toml, at the top of the plugin's directory (and of its repository). A dialog, protocol 1:

name     = "tetris"                   # unique: lower-case letters, digits and -
title    = "AutoTetris"               # the menu entry and the dialog's title
kind     = "dialog"                   # "dialog", or "service" (protocol 2)
protocol = 1                          # the plugin protocol it speaks
command  = ["./bin/autodoc-tetris"]   # argv: a bare name on PATH, or a path from this directory

[dialog]
width      = 44                       # the columns inside the border (10 to 300; 40 when unset)
height     = "80%"                    # the rows inside the border (4 to 100; 20 when unset),
                                      # or a share of the screen's height ("10%" to "100%")
placements = ["right", "left"]        # where it is designed to sit, the first its default:
                                      # center, top, bottom, left, right, or a corner
                                      # (top-left …); the user picks among them (Manage › Place)
esc        = "hide"                   # "close" (the default), or "hide" for a plugin with its
                                      # own quit: Esc hides the dialog, the plugin runs on.
                                      # Not on a card, whose Esc closes its panel

[install]
build = ["go", "build", "-o", "bin/autodoc-tetris", "."]   # run here when it is added or updated

A card on the feed, protocol 2:

name     = "doc-stats"
title    = "Doc stats"
kind     = "dialog"
protocol = 2                 # it declares points of protocol 2
command  = ["autodoc-doc-stats"]

[dialog]
modal = false                # a card, a panel of its own; default true (a dialog that holds the keys)
width = 40                   # the panel's width at the right or left edge, height its height
placements = ["right", "left"]  # the panel docks at the right edge, or the left

[feed]
document = true              # the open note, as it is edited

[[commands]]
id    = "toggle"
title = "Show / hide the stats"
key   = "s"                  # its letter on the SPC p card; optional

A service on the feed that saves the note before it previews it, protocol 3. start and calls are top-level keys, so they come before the first table:

name     = "html-preview"
title    = "HTML preview"
kind     = "service"
protocol = 3                 # it declares calls, protocol 3's
start    = "use"             # "use" (default) or "launch"; services only
calls    = ["file.save"]     # AutoDoc's commands it may run (host.call), by id; optional
command  = ["autodoc-html-preview"]

[feed]
document = true

[[commands]]
id    = "open"
title = "Open in the browser"
key   = "h"

A card that runs SQL and shows the results, protocol 4. Its table is AutoDoc's to show; the plugin sends the rows:

name     = "sql"
title    = "SQL"
kind     = "dialog"
protocol = 4                 # it declares [table], protocol 4's
command  = ["autodoc-sql"]

[dialog]
modal = false

[feed]
document = true              # each command carries the note as it stood (protocol 4)

[table]
title = "Results"            # the results panel's heading when a table names none

[[commands]]
id    = "run"
title = "Run statement or selection"
key   = "r"

A manifest AutoDoc cannot run is listed in the Plugins menu disabled, with why. Among the reasons:

Adding a plugin from a git URL asks first. The question shows what its build runs, what it starts and the risk, then what it declares: "a card beside the page · reads the open note's text (feed) · commands: Show / hide the stats (SPC p s)", the commands of AutoDoc's it may run: "may run AutoDoc's commands: file.save", a table it shows: "shows a table of results (Results)", and the files it opens: "opens .ipynb files".

Versions

AutoDoc speaks plugin protocols 1 to 11, and speaks to each plugin the one its manifest says. A plugin of an earlier protocol runs as it always has, and is sent what it always was.

The run

The protocol

Each notification carries one map.

From Notification Since Its map
AutoDoc plugin.open 1 {protocol, width, height, theme}: the first, once. A service's has no width or height; from 7, {session} when its slot holds one
AutoDoc plugin.key 1 {key, text, ctrl, alt, shift}
AutoDoc plugin.resize 1 {width, height}: the dialog laid out at another size
AutoDoc plugin.theme 1 {theme}: the theme switched
AutoDoc plugin.hide, plugin.show 1 {}: under esc = "hide", Esc hid the dialog, and its menu entry showed it again; a card's panel closed, and opened again
AutoDoc plugin.focus 2 {focused}: a card took the keys, or gave them back
AutoDoc plugin.document 2 {path, workspace, text, cursor, selection, version, too_large}: the open note, under [feed] document = true; from 4 also {cursor_byte, selection_bytes}; root (the workspace's absolute folder) whenever the note is a workspace's
AutoDoc plugin.command 2 {id}: one of the manifest's [[commands]], chosen; from 4, under the feed, {id, document}; from 7, {args} too when it has arguments
AutoDoc plugin.result 3 {call, ok, error}: the answer to the host.call numbered call
AutoDoc plugin.table.fetch 4 {id, n}: up to n more rows of the table id
AutoDoc plugin.table.close 4 {id}: the table is gone: its panel closed, or a newer table replaced it
AutoDoc plugin.pointer 6 {kind, button, x, y, count}: an unmodified left press (press, left) or wheel step (wheel, up/down) inside the card or dialog; frame-local nonnegative coordinates and count 1–16
AutoDoc plugin.close 1 {}: the last; from 7, {reason} (below)
plugin host.ready 1 {protocol}: the answer to plugin.open, with the protocol it was opened at
plugin host.frame 1 {rows}: the whole dialog, at any time. A service's is dropped
plugin host.title 1 {title}
plugin host.call 3 {call, id, args}: run AutoDoc's command id, one the manifest's calls declares, with args (strings)
plugin host.table 4 {id, title, fields, rows, more}: a table for the results panel, under [table]; from 5 also {kinds}
plugin host.rows 4 {id, rows, more}: the table's next rows, answering plugin.table.fetch; from 5 also {kinds}
plugin host.table_end 4 {id, error}: the table's rows ended before the last; error, when set, says why
plugin host.session 7 {values}: the plugin's session slot replaced, a map of strings (below)
plugin host.notify 8 {message, id, ongoing}: a notification; with an id, a toast updated in place while ongoing, ended by one without it (below)
plugin host.close 1 {}: asks to close; a card asks its panel to close, and runs on

Cards ([dialog] modal = false)

The document feed ([feed] document = true)

Tables ([table], protocol 4)

A plugin with rows to show (a query's result, say) does not draw them: AutoDoc shows them in its results panel, docked at the bottom, a table with a cursor on cells, its columns scrolled and the loaded rows sortable. The plugin sends the rows; AutoDoc asks for more as the user reads down.

Sessions, close reasons and arguments (protocol 7)

Notifications (protocol 8)

A plugin's Lua package (lua/)

A plugin may ship a Lua package in its folder, lua/package.toml and lua/init.lua, as a package in your own lua/ folder is made (docs/lua/reference.md). AutoDoc loads it after yours, under the name plugin:<name>:

-- lua/init.lua: :autodb query select 1
autodoc.command{ name = "autodb", run = function(ed, args)
  local ok, err = autodoc.plugin.run(args[1] or "run", select(2, unpack(args)))
  if not ok then error(err, 0) end
end }

Services (kind = "service")

A service has no surface: plugin.open has no size, and its frames and titles are dropped. It runs until the TUI quits, or until it sends host.close. It may serve something of its own (a preview on localhost, say) and use AutoDoc's API through AUTODOC_SOCKET.

Commands ([[commands]])

Editor menu rows ([[menu]], protocol 11)

A plugin puts rows on the editor's right-click menu, under a submenu of its title. Each runs one of its [[commands]], starting the plugin when it isn't running, as the Plugins menu does:

name     = "db"
title    = "Database"
kind     = "dialog"
protocol = 11
command  = ["./bin/db"]

[[commands]]
id    = "execute"
title = "Execute"

[[menu]]
command = "execute"   # one of its [[commands]], and the row's id: one row a command
where   = "editor"    # the editor's right-click menu, the only place for now
files   = ["*.sql"]   # globs over the open file's base name; left out, every file
title   = "&Execute"  # & marks its key; left out, the command's title

Opening files ([open], protocol 9)

[open]
extensions = [".ipynb"]   # the files it opens
command    = "import"     # one of its [[commands]]: run with the file's absolute path

Pixel surfaces ([surface], protocol 10)

A card or dialog can draw pixels instead of cells: a picture of its own, a canvas, a web page (ADR 1792797354). In AutoDoc's GUI the surface fills the card's area; in the terminal the plugin draws cells, as any plugin does.

protocol = 10

[surface]
required     = false     # true: a host without pixels (the terminal) doesn't open the plugin
context_menu = "host"    # "host": a right press opens the card's menu; "plugin": it is the plugin's press
text_input   = true      # the surface takes keys and IME text, and reports a caret

In Go, a Handler that implements plugin.SurfaceHandler gets the surface:

func (s *spike) Surface(sf *plugin.Surface, info plugin.SurfaceInfo) {
	go func() {
		for {
			b, err := sf.Buffer(ctx) // a credited buffer of the current generation
			if err != nil {
				return
			}
			draw(b.RGBA())
			sf.Present(b, image.Rectangle{}, lastInputT)
		}
	}()
}
func (s *spike) SurfacePointer(p plugin.SurfacePointer) { /* p.X, p.Y in dp */ }

examples/surface-go is a complete example: an animated test pattern with every input event drawn over it, and its frame rate and latency in its log.

Calls (calls = [...], protocol 3)

A plugin runs AutoDoc's own commands by their ids: those of AutoDoc's command catalog, which F1 lists under COMMANDS with their ids (file.save, todos.open, search.open), and your Lua commands (lua.<name>). It may run only the ids its manifest's calls lists, so what a plugin can do in AutoDoc is there to read before it is installed.

file.open_path and plugins.update

view.redraw: what the Rendered view draws from changed

view.redraw lays the GUI's Rendered view out again, the note's text unchanged. A block that draws from a file beside the note is laid out once and kept until the note is edited, so a plugin that writes such a file says so: autodoc-jupyter runs it after it keeps a cell's outputs, and the run's outputs show under the cell. Without a Rendered view (the terminal) it does nothing. An AutoDoc without it answers unknown, and the outputs show once the note is drawn again.

buffer.new: a draft for the user to edit

buffer.new opens a new buffer, in a tab of its own, and shows it: an unsaved draft holding the plugin's text, the cursor at its start. A database plugin opens a query for the user to run this way, a SELECT … LIMIT 100; on the table chosen. Its manifest lists it, calls = ["buffer.new"], and the plugin calls:

ok, err := p.peer.Call("buffer.new", "public.users.sql", "SELECT *\nFROM public.users\nLIMIT 100;\n")

The SDK, in Go

package main

import (
	"context"
	"os"

	"github.com/yongjohnlee80/autodoc/plugin"
)

type hello struct{ peer *plugin.Peer }

func (h *hello) Open(p *plugin.Peer, o plugin.Open) {
	h.peer = p
	f := plugin.NewFrame(o.Width, o.Height)
	f.Text(1, 1, "hello from a plugin", plugin.Style{FG: "cyan", Bold: true})
	_ = p.Frame(f)
}
func (h *hello) Key(k plugin.Key) {
	if k.Key == "q" {
		_ = h.peer.Close()
	}
}
func (h *hello) Resize(w, hh int)     {}
func (h *hello) Theme(t plugin.Theme) {}
func (h *hello) Close()               {}

func main() {
	if err := plugin.Serve(context.Background(), &hello{}); err != nil {
		os.Exit(1)
	}
}

Protocol 2 adds optional interfaces, which Serve probes for as it does Hider. A notification no interface takes is dropped.

Interface Method Takes
plugin.DocumentReader Document(d plugin.Document) the feed. Only the newest waits for it: a newer document replaces one not yet handled
plugin.Commander Command(id string) a command
plugin.Focuser Focus(focused bool) a card's focus
plugin.TableFetcher FetchRows(f plugin.TableFetch), TableClosed(id string) protocol 4: a fetch of a table's rows, and the table gone
plugin.PointerHandler Pointer(p plugin.Pointer) protocol 6: a left press or wheel step in the displayed frame

plugin.Base is a Handler that does nothing. A service embeds it and adds what it needs:

type preview struct {
	plugin.Base
	peer *plugin.Peer
}

func (p *preview) Open(peer *plugin.Peer, o plugin.Open) { p.peer = peer }
func (p *preview) Document(d plugin.Document)           { /* render d.Text, serve it */ }
func (p *preview) Command(id string) {
	if ok, err := p.peer.Call("file.save"); !ok { // protocol 3: one of the manifest's calls
		log.Print(err)
	}
	/* "open": open the browser */
}

Protocol 3 adds Peer.Call(id, args…) (ok bool, err error): it sends host.call and waits for AutoDoc's answer.

Protocol 4 adds the table and the command's note.

type sql struct {
	plugin.Base
	peer *plugin.Peer
	doc  plugin.Document
	rows *cursor // what serves a query's rows
}

func (s *sql) Open(p *plugin.Peer, o plugin.Open) { s.peer = p }
func (s *sql) Document(d plugin.Document)        { s.doc = d }
func (s *sql) Command(id string) {
	if s.doc.TooLarge { // the note was too large to send: say so on the card, run nothing
		return
	}
	stmt := statementAt(s.doc.Text, s.doc.CursorByte) // the note as it stood when "run" was chosen
	s.rows = query(stmt)
	page, more := s.rows.next(500)
	_ = s.peer.Table(plugin.TableOpen{ID: s.rows.id, Fields: s.rows.columns, Rows: page, More: more})
}
func (s *sql) FetchRows(f plugin.TableFetch) {
	page, more, err := s.rows.nextChecked(f.N)
	if err != nil {
		_ = s.peer.TableEnd(f.ID, err)
		return
	}
	_ = s.peer.Rows(f.ID, page, more)
}
func (s *sql) TableClosed(id string) { s.rows.close() }

Protocol 7 adds the session slot, the close's reason and a command's arguments.

func (a *db) Open(p *plugin.Peer, o plugin.Open) {
	a.peer = p
	if tok := o.Session["token"]; tok != "" && a.valid(tok) { // signed in before this start
		a.token = tok
	}
}
func (a *db) signedIn(tok string) { _ = a.peer.SetSession(map[string]string{"token": tok}) }
func (a *db) Close() {
	if r := a.peer.CloseReason(); r == plugin.CloseQuit || r == plugin.CloseRemove {
		a.logout()
	}
}

Protocol 8 adds Peer.Notify(plugin.Notice{Message, ID, Ongoing}), which sends host.notify. It returns plugin.ErrNoNotify when AutoDoc opened the plugin at a protocol before 8, and an error wrapping plugin.ErrNotice for a notice past a bound, sending nothing.

func (a *db) run(sql string) {
	id := fmt.Sprintf("run-%d", a.next())
	_ = a.peer.Notify(plugin.Notice{Message: "running…", ID: id, Ongoing: true})
	res, err := a.query(sql)
	if err != nil {
		_ = a.peer.Notify(plugin.Notice{Message: "failed: " + err.Error(), ID: id})
		return
	}
	_ = a.peer.Notify(plugin.Notice{Message: fmt.Sprintf("%d rows", res.Rows), ID: id})
}

Protocol 11 adds Peer.MenuRow(id, title, enabled), which sends host.menu: a [[menu]] row's new title, or whether it's enabled, each nil to leave as it is. It returns plugin.ErrNoMenu when AutoDoc opened the plugin at a protocol before 11, and an error wrapping plugin.ErrMenu for an update past a bound, sending nothing.

title := "&Execute on " + a.connection()
_ = a.peer.MenuRow("execute", &title, nil)

Protocol 6 adds plugin.PointerHandler. AutoDoc delivers plugin.pointer only after the protocol-6 handshake and only for unmodified input inside the frame. A right press still opens the card's context menu. Pointer.X and Pointer.Y are zero-based frame coordinates; Count is bounded to 1–16. A plugin can ignore pointer events and keep its keyboard controls.

Drawing a card: the UI kits

A card or dialog can draw every cell itself with plugin.Frame. Most don't need to: the SDK has two kits that draw it for you, in AutoDoc's look, keys and theme. Neither changes what AutoDoc and the plugin say to each other; both send host.frame like any other plugin.

plugin/card (native) plugin/decl (QML)
You write a Page of blocks and controls, in Go a golib QML layout, plus Go handlers
Controls Input (and masked), Buttons, List, Tree, Text, Line golib's widgets: TextField, Button, ListView, TreeView, TableView, Flex…
Starts drawing at once by parsing its layout
A minimal plugin, stripped about 3 MB about 7.5 MB
Example examples/card-go examples/card-qml

Pick plugin/card for a form, a list or a tree beside the page; pick plugin/decl for a richer screen, or one you'd rather write the way AutoDoc's own panels are written.

Both measure text as AutoDoc does: a CJK character takes two cells, and an accent written as a combining mark, an emoji sequence or a flag stays one character. plugin.Frame does the same for a plugin that draws its own.

plugin/card

The plugin keeps its controls and builds a card.Page each time the card is drawn; the Card lays it out, draws it and handles the keys:

type people struct {
	plugin.Base
	peer *plugin.Peer
	card *card.Card
	name card.Input
	list card.List
}

func (p *people) page() card.Page {
	return card.Page{
		Title:  "People",
		Header: []card.Block{card.Line(card.Title, "People")},
		Body:   []card.Block{&p.name, &p.list},
		Keys:   "Tab: move · Enter: add · q: close",
	}
}

// the Handler's methods forward to the Card
func (p *people) Open(peer *plugin.Peer, o plugin.Open) { p.peer = peer; p.card.Open(peer, o) }
func (p *people) Resize(w, h int)                      { p.card.Resize(w, h) }
func (p *people) Theme(t plugin.Theme)                 { p.card.Theme(t) }
func (p *people) Focus(focused bool)                   { p.card.Focus(focused) }
func (p *people) Pointer(at plugin.Pointer)            { p.card.Pointer(at) }
func (p *people) Key(k plugin.Key) {
	if !p.card.Key(k) && k.Key == "q" {
		_ = p.peer.Close()
	}
}

plugin/decl

decl.New is a whole plugin.Handler: pass it to plugin.Serve, or embed *decl.Card in a type that adds Command, Document or tables.

c := decl.New(decl.Options{
	Program: []tuidecl.ProgramOption{
		tuidecl.LayoutSource("card.qml", layout),
		tuidecl.Singleton("people", "1.0", "App"),
		tuidecl.Sources(map[string]any{"App.status": ""}),
		tuidecl.Handlers(map[string]gdecl.HandlerFunc{"App.add": add}),
	},
	Setup: func(p *tuidecl.Program, _ *plugin.Peer) error { return p.Set("App.status", "ready") },
})
_ = plugin.Serve(ctx, c)

import tui 1.0
import autodoc.plugin 1.0   // Theme, kept to AutoDoc's theme
import people 1.0

Window {
    palette.window: Theme.app.window; palette.windowText: Theme.app.windowText
    Text { text: App.status }
}

Testing a card

Both kits run without AutoDoc. card.Card.Attach draws to any Frame/Title host, so a test can send keys and read frames. The examples go further, driving the plugin over the protocol as AutoDoc does (examples/internal/hosttest).

To try an example from a checkout, run go build -o bin/autodoc-card-go . in its directory. Installing from a git URL is for a repository whose top holds plugin.toml, as autodoc-doc-stats does; to install one of these, copy its directory into a repository of its own.

What a plugin may not do