diff --git a/docs/rfd/drafts/D52-event-annotations.md b/docs/rfd/drafts/D52-event-annotations.md new file mode 100644 index 000000000..1b94fbd93 --- /dev/null +++ b/docs/rfd/drafts/D52-event-annotations.md @@ -0,0 +1,357 @@ + + +# RFD D52: Event Annotations + +- **Status**: Draft +- **Category**: Design +- **Authors**: rgrant +- **Date**: 2026-08-10 +- **Requires**: [RFD D59](D59-text-addressing-within-events.md) + +## Summary + +A conversation records what happened. +This RFD adds a place to record what you concluded about it: annotations, JSON +records attached to a location in a conversation, kept in `annotations.json` +beside `events.json` and never sent to the model. +Each names the shape of its own data with a `schema_id`, so a tool that does not +recognize a format can skip it. + +## Terminology + +- **Annotation**: structured data about a location in a conversation. +- **Stream entry**: one element of the ordered sequence that makes up a + conversation. + Conversation events and overlays are both stream entries. +- **Overlay**: a stream entry that applies to the conversation regardless of its + position, governing how the events around it are interpreted. + Introduced by [RFD 064]. +- **Event ID**: the per-event identifier introduced by [RFD 097]. + +Terms this RFD borrows from the rest of the system: + +- **Address**: a reference naming a location in a conversation, defined by [RFD + D59]. +- **Thread**: the rendered form of a conversation sent to an LLM provider, + assembled at query time from the event stream. +- **Storage root**: either of the two directories a conversation is written to, + the user's data directory or the workspace. +- **User-local copy**: the copy of a conversation in the user's data directory, + and the source of truth for its contents. +- **Workspace projection**: the copy of a conversation written into the + workspace's `.jp/conversations/` directory so it can be committed alongside + the project. +- **Trash**: to move a conversation directory into `.trash/`, preserving its + contents, with a `TRASHED.md` recording the reason. + +## Motivation + +The event stream records what happened. +Nothing in it holds what you concluded about it: that a tool result was wrong, +that one paragraph of a response is the part worth keeping, that a claim needs +checking before anyone acts on it. + +Two workarounds exist and both cost something. +Editing `events.json` puts the note inside the conversation, so the model reads +it on the next turn and it changes what happens next. +Keeping notes outside the workspace preserves the conversation but loses the +link: the note says "the third tool call" and then the third tool call moves. + +Doing nothing leaves those as the only options. +Commentary either contaminates the thing it is about or drifts away from it. + +Annotations are a third place. +They live with the conversation, travel with it into version control, point at a +specific location, and stay out of the model's input. + +## Design + +### The record + +```json +{ + "id": "a7k2m9x", + "target": { + "entry": "k3m9x2a", + "pointer": "/content", + "quote": { "exact": "empty tables parse fine now" } + }, + "schema_id": ".jp/schemas/review-note/v1.json", + "data": { + "verdict": "wrong", + "note": "still panics on a table holding only a comment" + } +} +``` + +- `id` identifies the annotation, so one can be edited or removed on its own. + Two annotations may share a target and a schema, so the target is not an + identity. +- `target` is an address. + It names a stream entry, a run of text within one, or a range between two + points. + [RFD D59] defines the shape, including the `start` and `end` form a range + takes. + The quote is stored as written rather than encoded, so `jq` over this file + shows what was pointed at. +- `schema_id` names the shape of `data`. + Required. + Either an absolute URI or a path relative to the workspace root. +- `data` is JSON. + JP stores it and never looks inside. + +**JP does not validate `data`.** `schema_id` is a promise made by whoever wrote +the annotation, not a contract JP enforces. +JP never resolves it, fetches it, or parses anything it points at. +A reader that recognizes the `schema_id` knows what it is looking at, and one +that does not skips the annotation. +That is the whole mechanism. + +Anything a format needs to record about itself, including who wrote the +annotation, is a field in `data` that its schema declares. + +### Where annotations live + +`annotations.json`, in the conversation directory beside `metadata.json`, +`base_config.json` and `events.json`. + +Annotations travel with the conversation. +`Projection::Projected` writes them to the user-local copy and to the workspace +projection, so they are committed alongside the events they describe. + +The file is a flat array, matching the shape of `events.json`: + +```json +[ + { "id": "a7k2m9x", + "target": { "entry": "k3m9x2a", "pointer": "/content", + "quote": { "exact": "empty tables parse fine now" } }, + "schema_id": ".jp/schemas/review-note/v1.json", + "data": { "verdict": "wrong", + "note": "still panics on a table holding only a comment" } }, + + { "id": "b2p8w4t", + "target": { "start": { "entry": "m1v5r7d" }, + "end": { "entry": "q9t3k6b" } }, + "schema_id": ".jp/schemas/review-note/v1.json", + "data": { "verdict": "digression", + "note": "abandoned approach, skip this on a reread" } } +] +``` + +The second annotation covers a stretch of the conversation rather than a piece +of text, so both of its endpoints name entries and stop there. + +Flat rather than a map keyed by target, because a range address does not make a +usable key, and because removing one annotation should not mean finding it +inside a nested list. + +### One unit, two files + +A conversation's stream and its annotations load and save together. +`jp_storage::load` already states this about the two files it has: "The stream +(`base_config.json` + `events.json`) is one unit: load both." +Annotations join that unit. + +This follows from [RFD 097]. +The set of entry IDs that were duplicated at load is private, load-scoped state +on `ConversationStream`, and RFD 097 requires a consumer to resolve ambiguous +references within the same load cycle, before any save. +Loading annotations separately would put that signal out of scope, and an +annotation would silently rebind to the wrong copy of a duplicated entry. + +`ConversationStream::from_parts` and `to_parts` already map parts to files, so +annotations become a third part. +`PersistBackend::write` and `LoadBackend::load_conversation_stream` keep their +signatures and no backend gains a method. + +Two files rather than one, because `stream_mtime` is the newer of `events.json` +and `base_config.json` and it drives conversation ordering. +Annotations stay out of that calculation: writing a note is not conversation +activity and should not move a conversation to the top of `jp conversation ls`. + +### Writing + +`write_json` is atomic per file and no atomic write spans the three. +`persist_conversation_to` writes `annotations.json` last, after `events.json`. +An interrupted save then loses a note, rather than leaving an annotation that +points at an entry which was never persisted. + +### A missing or corrupt file + +A missing `annotations.json` means the conversation has no annotations. + +A malformed one is renamed to `annotations.json.invalid` and the conversation +loads with no annotations. +It is **not** trashed. +`validate_entry` trashes a conversation directory for a missing or corrupt +`events.json`, and a bad notes file must never cost someone their conversation. +The rename preserves the user's bytes, which the next save would otherwise +overwrite. +`cleanup_tmp_files` removes only paths ending in `.tmp`, so the renamed file +survives startup. + +### When an annotation stops pointing at anything + +[RFD D59] defines when an address resolves. +An annotation whose address does not resolve is dropped at load, and each drop +is logged. + +This is the same shape of rule `ConversationStream::retain` already applies to +compaction overlays, which are dropped when a removal could have invalidated +their anchors. +Annotations track their own validity, keyed by entry ID rather than by turn +range. + +### Forking + +`jp conversation fork` carries annotations to the new conversation. +`--last N` drops entries, and any annotation whose address no longer resolves +against the forked stream is dropped with them. + +### Schemas in the workspace + +A `schema_id` may be a path relative to the workspace root, for a format local +to one project: + +``` +.jp/schemas/review-note/v1.json +``` + +`.jp/schemas/` is a convention, so that two tools in the same workspace do not +invent two places to look. +JP does not create the directory, read from it, or validate against anything in +it. + +The version belongs inside the identifier, whichever form it takes. +"Skip what you do not recognize" fails the moment a format changes without its +identifier changing. + +### Library and CLI + +The write path is a library API in `jp_conversation`, with the CLI as a shell +over it. + +Addresses quote text from the stored stream, and RFD D59 compares those quotes +exactly, so the library captures the quote from the entry itself. +A hand-pasted quote may carry different bytes than the stored text and would +then fail to match text that looks identical. + +## Drawbacks + +- **Every whole-conversation operation gains a third artifact.** Fork, export, + and the plugin protocol each grow a file to remember. + Archive and unarchive are directory renames, so they are free; the rest are + not. +- **A malformed `data` is found only by whoever reads it.** JP validates + nothing, so a writer that gets its own format wrong produces annotations that + look fine until a reader chokes on them. +- **Annotations load with the stream**, so a large `data` value is paid on every + open of that conversation, whether or not anything reads it. +- **`schema_id` conventions are unenforced.** Nothing stops two tools using one + identifier for different shapes, or one tool changing its shape without + changing its identifier. + +## Alternatives + +**A fifth `EventPayload` variant in `events.json`.** RFD 097 makes every stream +entry ID-addressable, so a new payload kind would fit. +Rejected on two counts: annotating would change `stream_mtime` and reorder `jp +conversation ls`, and it would put a concern the model never sees into the file +the model's input is built from. + +**A payload structured by JP.** Typed annotations with known fields would let JP +render, search and validate them. +Rejected because JP cannot anticipate what people annotate with, and every fixed +field would be wrong for someone. +`schema_id` gives any tool that wants structure the same benefit, without JP +owning the vocabulary. + +**Conversation-level notes.** Simpler, no addressing, no dependency on RFD D59. +Rejected because the value is in the link. +"This tool result is wrong" is worth recording; "something in this conversation +was wrong" is not. + +**Independently loadable annotations.** Loading and saving annotations on their +own would let a long-running process write them without touching the stream. +Rejected for now: it puts RFD 097's ambiguity signal out of scope and creates +two writers over data that reference each other with no atomic write between +them. +Worth revisiting if plugin or background writes arrive. + +## Non-Goals + +- **Sending annotations to the model.** They never enter the Thread. +- **Validating `data`.** `schema_id` is a promise. +- **Assistant or plugin authorship.** The write path is the library and the CLI. + A protocol message for plugins is a later RFD, and [RFD 077] owns the trust + policy it would need. +- **Searching inside `data`.** `jp conversation grep` searches the stream. +- **Rendering.** How and where annotations display is left open. + +## Risks and Open Questions + +- **No atomic write spans the files.** The write order makes the worst case a + lost note rather than an annotation pointing at nothing, but the window + exists. +- **The annotation ID form is unsettled.** Following `EventId` (short, opaque, + unique within its file, assigned on insertion) is the obvious answer and costs + nothing to adopt, but it has not been decided. +- **`.jp/schemas/` is a convention JP does not enforce**, so nothing stops it + being ignored. + +## Implementation Plan + +### Phase 1: the record type + +`Annotation` in `jp_conversation`, carrying its ID, address, `schema_id` and +`data`, with serde. +Depends on RFD D59 for the address type. +Mergeable on its own. + +### Phase 2: storage + +Extend `from_parts` and `to_parts` to carry annotations. +Write `annotations.json` after `events.json`. +Add the missing-file and malformed-file behavior, and drop annotations whose +address does not resolve at load. + +Tests: round-trip through both files; a malformed file is renamed and the +conversation still loads; removing an entry drops the annotations that named it. + +Depends on Phase 1. + +### Phase 3: library and CLI + +The write, edit and remove API, with the CLI over it. +Quote capture reads from the stored entry rather than accepting pasted text. +Depends on Phase 2. + +### Phase 4: fork and documentation + +Carry annotations through `fork`, dropping any that no longer resolve. +Document `.jp/schemas/`, and add "Annotation" to +`docs/architecture/ubiquitous-language.md`. +Depends on Phase 3. + +## References + +- [RFD 064], compaction overlays, whose drop-on-mutation rule annotations follow +- [RFD 072], the plugin protocol a future write path would extend +- [RFD 077], plugin trust policy +- [RFD 097], stable entry identifiers +- [RFD D59], the addressing scheme `target` uses + +[RFD 064]: ../064-non-destructive-conversation-compaction.md +[RFD 072]: ../072-command-plugin-system.md +[RFD 077]: ../077-plugin-configuration-and-trust-policy.md +[RFD 097]: ../097-stable-event-identifiers.md +[RFD D59]: D59-text-addressing-within-events.md diff --git a/docs/rfd/drafts/D59-text-addressing-within-events.md b/docs/rfd/drafts/D59-text-addressing-within-events.md new file mode 100644 index 000000000..d14b85fa0 --- /dev/null +++ b/docs/rfd/drafts/D59-text-addressing-within-events.md @@ -0,0 +1,435 @@ + + +# RFD D59: Text Addressing Within Events + +- **Status**: Draft +- **Category**: Design +- **Authors**: rgrant +- **Date**: 2026-08-10 +- **Extends**: [RFD 097](../097-stable-event-identifiers.md) +- **Required by**: [RFD D52](D52-event-annotations.md) + +## Summary + +A `jp://` URL names a conversation. +This RFD extends it with a fragment that names a location inside one: a stream +entry, a run of text within one, or a range between two points. +Locations are anchored by quoted text, with an optional position that breaks +ties between repeated matches, so an address either resolves to exactly one +place or fails visibly. + +## Terminology + +- **Address**: a reference that names a location in a conversation: a stream + entry, a run of text within one, or a range between two points. +- **Text quote selector**: naming a location by quoting the text there, along + with the text immediately before and after. +- **Text position selector**: naming a location by its position, counted in + Unicode code points from the start of the text. +- **Fragment**: the part of a URI following `#`, naming a part of the resource + that the rest of the URI identifies. +- **Grapheme cluster**: what a reader perceives as one letter, which may be + several Unicode code points. + +The two selector names come from the Web Annotation Data Model ([WADM]). +This RFD takes the names and sets its own rules, which are stricter: each point +that specification leaves to the implementation is fixed here. + +Terms this RFD borrows from the rest of the system: + +- **Event ID**: the per-event identifier introduced by [RFD 097]. +- **`jp://` scheme**: the URI scheme for JP-internal resources, of the form + `jp://[?]`, resolved by `jp_attachment_internal`. + +## Motivation + +`jp://jp-c17013123456?select=a:-1` is as precise as JP gets today: a +conversation, filtered by role and turn. +There is no way to point at a sentence. + +Three things want one. +An annotation about a specific claim in a response ([RFD D52]) has nothing to +attach to. +Quoting a passage from another conversation means attaching the whole message. +A link into a rendered conversation cannot point at what it is about. + +In addition to addressing any quote, it should be possible to address objects +like entire turns, or a compaction overlay with the same URI scheme. + +[RFD 097] supplies half the answer by giving every stream entry a stable ID. +What remains is naming a location inside an entry, and doing it in a way that +survives the hand-editing RFD 097 was written for. +A character offset alone does not: insert a word earlier in a message and every +offset after it points somewhere else, with no signal that anything moved. + +## Design + +### The URL + +An address is a `jp://` URL carrying a fragment: + +``` +jp://jp-c17013123456#k3m9x2a@/content:t=Y2hva2Vz +``` + +The part before `#` identifies the conversation, unchanged. +The fragment names the location: an entry, a run of text within one, or a range +between two points. +RFC 3986 puts resource identification in the query and part-of-resource +identification in the fragment, which is exactly this split. + +The fragment is discarded today, asserted by +`uri_to_entry_ignores_path_and_fragment`, so this gives meaning to a component +that is currently free. + +### Grammar + +**Proposed here, not settled.** + +```abnf +fragment = endpoint [ ".." endpoint ] +endpoint = entry-id [ "@" pointer [ ":" anchor ] ] +anchor = "t=" exact [ ";p=" prefix ] [ ";s=" suffix ] + / "o=" offset +``` + +``` +jp://jp-c17013123456#k3m9x2a +jp://jp-c17013123456#k3m9x2a@/arguments/query +jp://jp-c17013123456#k3m9x2a@/content:t=Y2hva2Vz +jp://jp-c17013123456#k3m9x2a@/content:o=42..z4n8v6c@/content:t=ZW1wdHkgbWFw +``` + +An address stops wherever it wants to. +Name an entry and you have named the entry; add a field and you have named the +whole of that field; add an anchor and you have named a run of text inside it. + +That matters because `TurnStart` holds no text at all, so naming the entry is +the only way to reach it. +It is also how you address a whole tool call without singling out a field, and a +whole turn by naming its `TurnStart`. + +Entries that are not conversation events are reached the same way as any other. +A compaction's summary is generated prose, at `/summary/summary`, and a config +delta's values are strings. + +None of the delimiters (`@`, `:`, `;`, `=`, `.`) appear in the base64url +alphabet, so an encoded quote can never be mistaken for structure. +`..` is the range separator because JP already writes ranges that way, inclusive +on both ends, in `--turn A..B`, the compaction DSL and timeline output. + +### JSON form + +An address is written two ways: as a fragment in a URL, and as JSON wherever one +is stored. +They carry the same information and convert both ways. + +```json +{ "entry": "k3m9x2a" } + +{ "entry": "k3m9x2a", "pointer": "/arguments/query" } + +{ "entry": "k3m9x2a", "pointer": "/content", + "quote": { "exact": "empty tables parse fine now" } } + +{ "entry": "k3m9x2a", "pointer": "/content", "offset": 42 } + +{ "start": { "entry": "k3m9x2a", "pointer": "/content", "offset": 42 }, + "end": { "entry": "z4n8v6c", "pointer": "/content", + "quote": { "exact": "empty map" } } } +``` + +`prefix` and `suffix` sit beside `exact` when present. +`quote` and `offset` are mutually exclusive. + +One location is written flat and a range nests under `start` and `end`. +The key that is present says which: `entry` means one location, `start` means a +range. +A range is a different thing from a location, so it gets a different shape +rather than a `start` with nothing after it. + +Text is written plainly here, not base64url encoded. +Encoding exists to stop a quote colliding with the fragment's delimiters, and +JSON has no such problem, so anyone running `jq` over a stored address sees what +it points at. + +### Naming a string inside an entry + +The pointer is a JSON Pointer ([RFC 6901]) into the entry's decoded JSON. + +Any string value is addressable. +There is no list of event kinds and no per-kind default, so a chat request's +text, a tool call's `arguments`, a tool response's `content` and the thinking +signatures in an entry's `metadata` are all reached the same way. +Adding a new event kind requires no change here. + +"Decoded" matters: `jp_conversation::storage` base64-encodes tool arguments, +tool response content and metadata at rest. +Pointers and offsets address the decoded values, never the stored encoding. + +This RFD addresses **any** stream entry, not only conversation events. +RFD 097 gives config deltas, compactions and unrecognized entries stable IDs but +exposes only conversation events programmatically, leaving the rest "until a +consumer RFD defines the raw-entry view it needs." +This is that RFD, and it defines that view. + +### Anchors + +An anchor is either a quote or an offset. + +A **quote** carries the exact text, and optionally the text immediately before +and after it. +Prefix and suffix are never required. +They raise the chance that a quote is unique without bounding it, so mandating +them would buy ceremony rather than a guarantee. + +A quote is a predicate over the text, so it matches zero, one or many places. +That is its nature, not a defect. + +An **offset** is a single number, counted in Unicode code points, starting at +zero, sitting *between* characters. +One number is enough because an anchor names a point, not a span. +Position zero is before the first character. + +Offsets count code points rather than bytes, which differs from `jp conversation +grep --json`, whose `submatches` are byte offsets mirroring `rg --json`. +Those coordinates index a string emitted in the same document; these are durable +addresses that people write by hand, and a byte offset can land mid-character. +The library converts, so "grep, then annotate what I found" never asks a user to +do the arithmetic. + +An address that would start or end inside a grapheme cluster is rejected when it +is created, so every stored address is well formed and resolution never handles +half a character. + +### Ranges + +Two endpoints, separated by `..`, name a range. +It runs from the **begin** of the first anchor's match to the **end** of the +second anchor's match, inclusive. +The endpoints need not use the same kind of anchor: a quote at one end and an +offset at the other is allowed. + +When both endpoints name the same entry, the result is an ordinary span, with no +special case. + +A range denotes a region of the stream, not a string. +Config deltas and compactions may lie between its endpoints, so anything +extracting content from a range receives a sequence of entries with partial +ends, not a single run of text. + +The extent of a range is the entries lying between its endpoints in array order. +That part is not content-anchored, and RFD 097 forbids using entry IDs for +ordering, so reordering entries by hand changes what a range covers. +Quoting cannot fix this, because the middle is not quoted. +Two rules keep the failure visible: a range is unresolved if either endpoint +entry is gone, and unresolved if the end entry does not follow the start entry. + +### Resolution + +An address naming only an entry resolves if that entry is present. +One naming a field resolves if the pointer finds a string there. + +Within a string, the quote is the anchor of record and the offset breaks ties. + +``` +1. Match the quote against the addressed string. -> zero, one, or many +2. Exactly one candidate -> resolved +3. Several candidates, offset present -> the candidate nearest it +4. Several candidates, no offset -> unresolved +5. No candidates -> unresolved +6. Offset with no quote -> resolved at that point +``` + +Comparison is exact. +No Unicode normalization is applied to either side, so a single coordinate +system serves both quotes and offsets and nothing sits between the bytes on disk +and what an address means. +The cost is that a quote captured from a differently normalized source fails to +match text that looks identical, which is why RFD D52 has the library capture +quotes from the stored entry. + +An offset must not win over a quote. +An offset always resolves, and after an edit it resolves to the wrong place +silently, which is the hazard RFD 097 exists to remove: a reference should +become "a *detectable* mismatch, never a silent positional aliasing." +Line 6 above is a deliberate exception with a stated property: an address with +no quote is fragile by construction, useful for pointing at something now, wrong +for anything durable. + +Used as a tie-break rather than an authority, an offset also degrades +gracefully. +Small edits shift it, but the quote has already narrowed the field to real +occurrences, so the nearest candidate is still the right one. + +### Encoding + +Quote text is base64url encoded without padding, using +`base64::engine::general_purpose::URL_SAFE_NO_PAD`. +Its alphabet is `A-Za-z0-9-_`, all unreserved, so a fragment never needs +percent-encoding and a quote can hold `#`, `&`, newlines or anything else. + +This differs deliberately from `jp_conversation::storage`, which uses +`STANDARD`. +That alphabet contains `+`, `/` and `=`, and `=` would collide with the grammar +above. + +`exact`, `prefix` and `suffix` are encoded separately, so the delimiters between +them stay visible. +Offsets are plain integers: there is no Unicode in `42`, and encoding it would +hide the one part of an address a person can check by eye. + +The encoding belongs to the URL. +Where an address is stored, as in RFD D52's `annotations.json`, the fields are +kept decoded, so `jq` over the file shows what was quoted. + +### Interaction with `select=` + +`select=` filters which parts of a conversation an attachment pulls in, by role +and turn. +Applied to an address, it intersects with the range: + +- The fragment gives an ordered set of entries with a partial span at each end. +- `select=` filters that set. + Absent means no filtering. +- It is an error for the filter to exclude an entry that an endpoint names. + The caller's own filter contradicting the location they asked for is a bad + request, distinct from an unresolved address, and the message should say + which: "select=a excludes k3m9x2a, where the range starts." + +### Where this differs from the Web Annotation Data Model + +WADM is an interchange format for independent clients annotating documents they +do not control, so its matching rules are mostly advisory. +JP defines its own storage, so each of those points is fixed here. +JP addresses are a strict subset of conformant WADM selectors: emitting valid +WADM is possible, accepting everything WADM permits is not. + +| WADM | Here | +| --- | --- | +| Range runs to the *beginning* of the end selector, excluding it | Runs to the end of it, inclusive | +| Start and end selectors SHOULD be the same class | Either end may be a quote or an offset | +| Multiple matches SHOULD select all of them | Nearest to the offset, or unresolved | +| Text MUST be normalized (markup stripped) before recording | No normalization; the text is the decoded stored value | +| A `State` is RECOMMENDED alongside a position selector | No `State`; the quote is the anchor of record | +| Selections SHOULD NOT split a grapheme cluster | Rejected at creation | + +## Drawbacks + +- **This modifies shipped behavior.** `jp_attachment_internal` currently + discards the fragment, with a test asserting it. + D59 stops being purely additive the moment that changes. +- **A range's extent is position-dependent** and no anchoring fixes it, so + reordering entries changes what a range covers. +- **Two offset conventions now exist** in the user-facing surface: `grep --json` + counts bytes, addresses count code points. +- **Encoded quotes are not readable in a URL.** The information survives, but + eyeballing an address does not tell you what it points at. + +## Alternatives + +**Offsets only, no quotes.** Short, deterministic, always resolves. +Rejected because it always resolves *to something*, and after an edit that +something is wrong with no signal. +Determinism without correctness is worse than a visible failure. + +**Quotes only, no offsets.** Removes the fragile form entirely. +Rejected because a repeated quote then has no tie-break short of extending +prefix and suffix (which is also not guaranteed to resolve the problem), and +because a short hand-written address has real value. + +**Content-addressed anchors, hashing the target text.** Precise and tamper +evident, but any edit invalidates the anchor, which defeats the editing workflow +which was one of the motivations for the design. + +**A separate address scheme rather than a `jp://` fragment.** Rejected because +`jp://` already identifies conversations and RFC 3986 already reserves the +fragment for addressing within a resource. + +## Non-Goals + +- **Validating what an address points at.** Resolution says where; it does not + interpret. +- **Byte-level addressing.** WADM calls that a `DataPositionSelector`. + Offsets here count code points. +- **Addressing rendered output.** Addresses name the stored text, not what a + terminal or the web view drew. +- **Stability across conversations.** Entry IDs are unique within one stream, + per RFD 097. +- **Rewriting addresses when content changes.** An address resolves or it does + not. + +## Risks and Open Questions + +- **The grammar is proposed, not settled.** Everything else here is decided; the + syntax is the part that most needs review. +- **A same-entry offset span has no short form.** `#e@/p:o=10..e@/p:o=20` + repeats the entry and pointer. + A shorthand such as `o=10..20` would help, and is not specified. +- **A lone offset anchor names a zero-width point.** Whether that is useful, or + should be an error outside a range, is open. +- **JSON Pointer is proposed for the path.** It is standard and already + understood, but it is one more syntax inside the fragment. +- **The raw-entry view has no shape yet.** RFD 097 deferred it; this RFD claims + it but the accessor is designed in implementation. + +## Implementation Plan + +### Phase 1: the address type + +The address, its endpoints and anchors, in `jp_conversation`, with serde and the +encode and decode helpers. +Grapheme-boundary rejection at construction. +Independent, mergeable on its own. + +### Phase 2: parsing and formatting + +Fragment grammar to and from the address type, with base64url on the text +fields. +Tests: round-trip; quotes containing `#`, `&` and newlines survive; malformed +fragments are rejected with a useful message. +Depends on Phase 1. + +### Phase 3: resolution + +The resolution ladder against a loaded stream, including the raw-entry view over +all stream entries, JSON Pointer lookup, and the two range rules. +Tests: each line of the ladder, including that a repeated quote with no offset +is unresolved and that a reordered range is unresolved. +Depends on Phase 2. + +### Phase 4: `jp://` integration + +`jp_attachment_internal` reads the fragment instead of discarding it, and +intersects it with `select=`, erroring when the filter excludes an endpoint. +Replaces `uri_to_entry_ignores_path_and_fragment`. +Depends on Phase 3. + +### Phase 5: documentation + +Add a character-offset section to `docs/architecture/indexing-conventions.md`, +which currently covers turn positions only, recording that offsets are 0-based +boundaries in code points and why that differs from `grep --json`. +Update the `jp_attachment_internal` README. +Depends on Phase 4. + +## References + +- [RFD 097], stable entry identifiers, which this RFD extends +- [RFD D52], the first consumer of these addresses +- [WADM], the Web Annotation Data Model, source of the selector vocabulary +- [RFC 6901], JSON Pointer +- `docs/architecture/indexing-conventions.md`, turn positions and counts + +[RFD 097]: ../097-stable-event-identifiers.md +[RFD D52]: D52-event-annotations.md +[RFC 6901]: https://www.rfc-editor.org/rfc/rfc6901 +[WADM]: https://www.w3.org/TR/annotation-model/