Expose the internal API for external use

CHANGELOG.md

@@ -2,6 +2,72 @@
 
 All notable changes to `wiki` are documented here. This project follows [semantic versioning](https://semver.org); while pre-1.0, breaking changes bump the minor version.
 
+## v0.9.0 — 2026-08-09
+
+The release that makes `wiki` importable. Everything that makes the tool useful now lives in public packages, so a Go program can build a bundle index and query it directly instead of spawning the CLI and parsing its output. The CLI runs on those same packages, so there is no second implementation of anything.
+
+### Breaking
+
+1. **Package paths.** `bundle`, `index`, and `parse` moved out of `internal/` to the module root. Nothing could import them before, so this breaks no existing code — it is listed because the paths are now a commitment.
+2. **A malformed `wiki.toml` is an error** rather than being silently half-read. Input that was never valid TOML but happened to scan — most likely unquoted array items, `types = [note, concept]` — now fails with a line number.
+3. **Two json keys renamed**, for consistency across commands: `checkboxes` reports the entry as `entry` (was `file`), and `unresolved` reports a link's far end as `to` (was `target`). csv/tsv headers follow; text output is unchanged.
+4. **`Index.OutLinks(*Entry)` is now `Index.Links(path string)`**, and returns one ref per occurrence rather than unique targets.
+5. **`Entry.SetFields` takes `map[string]any`** (was `map[string]string`), accepting a `string` or a `[]string` per key.
+
+Two behaviour changes worth knowing that are not API breaks: writing into a read-only directory now fails where a plain write succeeded, and a hardlinked entry now diverges instead of sharing an inode. Both are consequences of atomic writes and both are argued below.
+
+### Changed
+
+- **The core packages are importable.** `bundle`, `index`, and `parse` moved out of `internal/` to the module root, so a Go program can build a bundle index and query it directly instead of spawning the CLI and parsing its output. `output` (CLI presentation) and `wikilink` (a compat shim `index` uses without exposing) stay internal. The CLI is unchanged and still runs on the same packages, so there is no second implementation of anything.
+
+  Shelling out is a fine contract for occasional whole-bundle questions and a poor one for a consumer asking many small ones, since every invocation re-reads and re-parses the whole tree: cost scales with interaction rather than with change. It also forced consumers to reimplement rules that have one correct home here, which is how a separate UI ended up carrying its own frontmatter writer and its own link resolver.
+
+- **`index.ParseFilter` reads a `key=value` / `key!=value` expression.** The spelling is part of the query contract rather than of the CLI's argument handling, so it moved out of `cmd/wiki`; the flag now calls it. Any consumer accepting the same syntax gets the same parse, including the details that are easy to miss (`!=` matched before `=`, so a value may contain `=`; the value unquoted the way frontmatter is).
+
+- **A write API on `Entry`.** `SetField`, `SetFields`, `UnsetField`, and `SetCheckbox` change frontmatter and checkboxes surgically: the matching lines are replaced and every other byte is left alone, never parsing the frontmatter into a map and re-serializing it, which would silently drop nested maps, anchors, comments, and quoting style. `SetFields` applies several keys in one pass, since two writes can leave an entry half-updated. Each refreshes the entry in place, because inserting or removing a line shifts the line numbers that links, checkboxes, and headings all carry. `SetCheckbox` is keyed by line, the only stable identity a checkbox has.
+
+  `SetFieldList` writes list-valued fields (`tags`, `blockers`, and anything else the bundle spells as a list). Separate from `SetField` because a list is not a string that happens to contain brackets: passing `"[a, b]"` to `SetField` writes `key: "[a, b]"`, correctly quoted for a scalar and a one-element list when read back. A key already written as a block list stays one, so the API does not reformat frontmatter it was only asked to change.
+
+  `SetFields` takes `map[string]any`, accepting a `string` or a `[]string` per key and rejecting anything else by name and type. That mirrors `Frontmatter`, which returns the same shape, so a consumer can read the frontmatter, edit it, and write it back. With scalars and lists in separate calls, setting a status and a tag list together took two writes — precisely the half-updated entry one pass exists to prevent. Frontmatter is genuinely heterogeneous, so `map[string]string` was never the honest type for it.
+
+- **json keys are consistent across commands.** There were five names for two concepts: `checkboxes` called the entry `file` while `check` called it `entry`, and `unresolved` called a link's far end `target` while `links` and `backlinks` called it `to`. Now three categories with one name each — `_path` for rows that merge your frontmatter and so need a reserved key (`list`, `orphans`), `entry` for rows *about* an entry (`check`, `checkboxes`), and `from`/`to` for link rows (`links`, `backlinks`, `unresolved`). csv/tsv headers follow, since they derive from the json fields; text output is unchanged. `wiki version` also no longer claims to accept `--format`.
+
+- **`Index.OutLinks(*Entry)` is now `Index.Links(path string)`**, the mirror of `Backlinks(path string)`: same shape, same return type, both yielding `nil` for an unknown path. It also stops de-duplicating by target and returns one `LinkRef` per occurrence, as `Backlinks` always has. De-duplicating while still reporting a `Line` made that line the first of several, silently. Which behaviour is right depends on how the result is shown — `wiki links` prints a bare target so it collapses repeats, `wiki backlinks` prints `file:line` so it shows each one — and that is a presentation choice, so it moved to `cmd/wiki`. CLI output is unchanged.
+
+- **Reading and resolving, for consumers.** `Entry.Field`, `Entry.FieldList`, and `Entry.Frontmatter` reach arbitrary frontmatter; `FieldList` applies the same scalar-as-one-element-list rule matching uses, so filtering by hand agrees with `--where` rather than being subtly different. `Index.ResolveLink` and `RelativeLink` expose both directions of link spelling.
+
+- **`--where` works on the vocabulary commands.** `tags`, `properties`, `property`, and `checkboxes` now take `--where key=value` / `key!=value` with the same semantics `list` has (repeatable, ANDed, list-match-any), composing with `--prefix`.
+
+  `--prefix` scoped every one of these and `--where` scoped none, so a subtree could be narrowed anywhere and a field could not. The gap showed the moment a folder held more than one kind of entry: a backlog that also holds notes reported *their* statuses (`published`, `retired`) beside the tasks', with no way to ask the narrower question short of post-processing `list --format json` through `jq`. Now there are two filters, available wherever a set of entries is narrowed: **`--prefix` for where, `--where` for what.** `checkboxes` is included because its unit is a `- [ ]` line but its *scope* is still a set of entries; a named `[file]` stays explicit and ignores both filters.
+
+  Library signatures gained the parameter: `TagCounts`, `PropertyKeyCounts`, and `PropertyValueCounts` each take `props []PropFilter` alongside the path prefix. `checkboxes` also dropped a second copy of prefix matching and now routes through `Index.Filter` like everything else.
+
+- **`wiki.toml` is parsed as TOML, and `[tool.*]` is reserved.** `bundle` now uses `BurntSushi/toml` (one dependency, no transitive ones) instead of a hand-rolled line scanner, and `[tool.<name>]` tables are space granted to other tools over the same bundle: never parsed by `wiki`, never validated, never warned about. `bundle.Bundle.Tool` carries them and `DecodeTool` unmarshals one into a caller's own struct, so no tool writes a second `wiki.toml` parser.
+
+  Without the namespace, a tool with an opinion about a bundle had to put it in a satellite config beside `wiki.toml`, and a second tool meant a third file. `pyproject.toml` is the precedent: one file describes the directory, tools namespace their own settings inside it. Reserving space adds no opinion to the format — `wiki` gains no field it interprets and no behaviour.
+
+  **Behaviour change:** a malformed `wiki.toml` is now an error instead of being silently half-read. The config decides what counts as an entry and which types are valid, so carrying on with a partial parse produced confidently wrong answers. This also means input that was never valid TOML but happened to work — most likely unquoted array items, `types = [note, concept]` — now fails with a line number instead of being accepted.
+
+### Fixed
+
+- **Docs described the old canonical link form.** The format spec still called root-absolute links canonical and said `tidy --links` rewrote *to* root-absolute; since relative became the canonical on-disk form (v0.7.0) both are backwards. The scaffolded `AGENTS.md` contradicted itself, describing relative links in one section and `tidy --all` producing absolute ones in another, and both READMEs told Obsidian users to write *Absolute path in vault*. The spec also still documented `skip`, a field renamed to `ignore` long ago — so a config copied from the spec silently did nothing and warned as an unknown key — and never documented `ignore_orphans` at all.
+
+- **A quoted list item containing a comma was read as two broken items.** `parse` split a frontmatter flow list on every comma without honouring quotes, so `tags: ["a,b", "c"]` came back as `["\"a", "b\"", "c"]` — valid YAML, silently mis-parsed, in any bundle that spelled a list that way. The split now tracks the open quote.
+
+- **A key inside any `wiki.toml` table silently overrode bundle config.** The line-based reader ignored table headers, so every key was treated as top-level: `[tool.wikiview] types = [...]` replaced the bundle's `types` vocabulary, last-one-wins, and entries with undeclared types then passed `check` clean. Any table containing a key named `spec`, `types`, `ignore`, or `ignore_orphans` reconfigured the bundle from inside a namespace that was supposed to be inert.
+
+- **A multi-line array in `wiki.toml` silently disabled the setting.** `types = [` parsed to an empty list, which means "no vocabulary declared", which allows every type — the exact opposite of what the author wrote, with nothing reported. The one-line spelling of the same vocabulary errored on an undeclared type as intended. Valid TOML that any other tool would read correctly, so nothing suggested it was being misread.
+
+- **Unknown-key warnings name the full path.** They reported the leaf key with its table stripped, so a nested `path` or `columns` was unfindable in a file with several tables. Now `nested.key`, and only the shallowest unrecognized key is reported, since flagging every key inside an unknown table is noise rather than information.
+
+- **Writes are atomic.** Every file the engine rewrites now goes through a temp file and a rename instead of `os.WriteFile`, which opens with `O_TRUNC` and so empties the file before the new content lands. Anything reading in that window — an editor, an agent, a watcher, another `wiki` run — could see an empty or partial entry, and a process killed mid-write left it truncated on disk. Measured against a concurrent reader, roughly 9% of reads saw a torn file before; none do now. Permissions are preserved across the rename, and a symlinked entry is written through rather than replaced. Atomicity is per file: a command rewriting several can still be interrupted between them, which is a separate concern. A hardlinked entry now diverges instead of sharing an inode, which is the point: two names are two entries at two paths, so each needs its own relative links, and the shared inode meant one of them ended up pointing nowhere. Writing into a read-only directory now fails where a plain write succeeded.
+
+  **Not every system allows a replace while a file is open**, and on those the write now fails where the plain write it replaced succeeded. The replace retries briefly, but only while the error is contention, so a genuine permission error still fails at once; that covers the common cause, which is transient and not the user's doing (a scanner or indexer opening a file it just saw change). Contention held longer still fails, and the durable fix is filed as debt rather than rushed. Where a replace does not care who is reading, there is no retry and no cost.
+
+- **`move --include-frontmatter` finds relative frontmatter refs.** The flag shipped when root-absolute was still the canonical link form, so it matched frontmatter values by exact string equality against the moved entry's root-absolute path. Once relative links became canonical for bodies (v0.7.0), the natural spelling was the broken one: `blockers: [./task-1.md]` never equalled `/active/task-1.md`, so the flag silently did nothing and the ref dangled. Frontmatter refs are now **resolved** and matched the way body links are, so relative and root-absolute are both found, and they are **normalized to root-absolute** on write. The moved file's own relative refs are normalized too, closing the cross-folder dangle that body links already handled. Anchors are preserved and out-of-bundle values are left as authored. Only values ending in `.md` are treated as references, which is what keeps the pass from rewriting ordinary metadata like `title: Some Note` (an arbitrary string resolves to a valid in-bundle path). The opt-in caveat is unchanged: the flag rewrites every matching value, snapshot fields included.
+
+  **Frontmatter stays root-absolute on purpose**, and does not follow bodies to relative. A root-absolute value is a *stable key*, so every entry referencing `/epics/x.md` spells it identically and `wiki list --where epic=/epics/x.md` finds them all; a relative value spells the same target differently from each directory, so no single `--where` query can match every referrer (matching is exact string equality, by design, so the tool never has to guess that a value is a path). The rendering argument that motivated relative bodies does not apply either, since frontmatter is never rendered as a link. The rule: **a body link is relative because it must navigate; a frontmatter ref is root-absolute because it must be a stable key.**
+
 ## v0.8.0
 
 ### Changed

README.md

@@ -61,7 +61,7 @@ wiki list                     # every entry
 That's the whole thing: a folder of plain Markdown. Open it in any editor, commit it to git, point Obsidian at it, point an agent at it. `wiki` simply makes it queryable and keeps it honest.
 
 > [!NOTE]
-> If you use Obsidian, set it to write standard markdown links: Files and links → turn off **Use [[Wikilinks]]**, set **New link format** to *Absolute path in vault*, turn on **Automatically update internal links**. (`wiki` recognizes `[[wikilinks]]` for compatibility but flags them in `wiki check`, and `wiki tidy --wikilinks` converts them to standard links). See the [format spec](https://github.com/agentic-wiki/spec#links).
+> If you use Obsidian, set it to write standard markdown links: Files and links → turn off **Use [[Wikilinks]]**, set **New link format** to *Relative path to file* (the canonical on-disk form, so links navigate in any renderer), turn on **Automatically update internal links**. (`wiki` recognizes `[[wikilinks]]` for compatibility but flags them in `wiki check`, and `wiki tidy --wikilinks` converts them to standard links). See the [format spec](https://github.com/agentic-wiki/spec#links).
 
 ## What you can ask it
 
@@ -79,6 +79,8 @@ wiki outline /tech/infra/hetzner.md                     # its headings
 wiki table /finance/expenses.md --format csv            # a dataset's table as rows (csv/json), for jq/duckdb
 ```
 
+Two filters narrow any set of entries: **`--prefix` for where, `--where` for what.** Both work on `list`, `search`, `checkboxes`, `tags`, `properties`, and `property` — so a folder that mixes kinds can still be asked a narrow question.
+
 **Follow the graph** (the part `grep` cannot do)
 
 ```sh
@@ -97,6 +99,7 @@ wiki checkboxes                    # every open - [ ] checkbox, across the whole
 wiki list --where type=task        # list task entries (detailed entries)
 wiki tags --counts --sort=count    # what you write about most
 wiki property status --counts      # how many open vs done, draft vs final
+wiki property status --counts --where type=task   # …only the tasks', if the folder mixes kinds
 ```
 
 **Reshape it safely**
@@ -211,7 +214,7 @@ wiki check                           # the backlog stays conformant
 ## Design
 
 - **Standalone first:** agents call `wiki` directly, no server in the way.
-- **Minimal on purpose:** zero external dependencies, a single static binary, native on macOS, Linux, and Windows. Git is recommended but entirely optional.
+- **Minimal on purpose:** one dependency (a TOML parser, for `wiki.toml`), a single static binary, native on macOS, Linux, and Windows. Git is recommended but entirely optional.
 - **Files are truth:** the index is derived from disk and fully disposable.
 
 ## Develop

backlog/2-query-surface/016-list-body.md

@@ -0,0 +1,20 @@
+---
+type: task
+title: "list --body: bodies in the same json call"
+status: todo
+priority: low
+tags: [feature, query]
+---
+
+`wiki list --format json` carries every entry's full frontmatter plus `_path`, which makes it a one-call snapshot of a whole bundle's metadata. It carries no **body**, so a consumer that wants to show content alongside metadata has to follow up with one `wiki read` per entry: N process spawns, each rebuilding the whole index, to fetch text the `list` pass already had open.
+
+Surfaced by `wikanban` (card excerpts on a board), but it applies to any renderer, exporter, or static-site generator over a bundle.
+
+**Proposal:** `wiki list --body`, json only, adding a `body` key per entry (frontmatter stripped, exactly what `read` returns).
+
+- **json only.** Bodies are multi-line; they do not belong in text/csv/tsv output. Reject the flag on other formats rather than emitting something unusable.
+- **Opt-in**, because it changes the cost profile: `list` reads bodies on demand today (only `SortTime` stats timestamp-less entries), so the default stays cheap.
+- **Reuses `Entry.Body()`**, so there is one definition of "the body" across `read` and `list`.
+- The key is `body`, matching `read --format json`'s existing `{_path, type, body}` shape. Note the collision risk with a user's own `body:` frontmatter field, which `MarshalJSON` emits verbatim; either accept that it wins (consistent with frontmatter being the user's namespace) or reserve `_body` like `_path`. **Leaning `_body`**, for the same reason `_path` earned its underscore.
+
+**Acceptance:** `list --body --format json` includes each entry's body; other formats error; the flag composes with `--where`/`--prefix`/`--sort`; one bundle read, no extra spawns. Consider the same flag on `search`.