feat(config, conversation, cli): Compact only oversized items - #994
Open
JeanMertz wants to merge 5 commits into
Open
feat(config, conversation, cli): Compact only oversized items#994JeanMertz wants to merge 5 commits into
JeanMertz wants to merge 5 commits into
Conversation
Two building blocks for narrowing a compaction policy to the large items
in its range, neither of which has a consumer yet.
`ByteSize` is a size in bytes written as a human-readable string (`1MB`,
`512 KB`, `4GiB`) or a bare byte count. Unit suffixes are binary, so
`1KB` is 1024 bytes. Its `Display` picks the largest unit that divides
the value evenly and falls back to the raw count otherwise, which makes
the rendered form always parse back to the same value and therefore safe
to use as the serialized form. A separate `human()` gives the lossy
one-decimal rendering for terminal output.
`PolicySpec<P>` pairs a policy with the options qualifying which items it
reaches, today just an `over` size threshold. Declaring it once as a
generic wrapper keeps the option off each individual policy enum, so a
future policy cannot forget to honor it.
The serialized shape is the part worth attention: a spec without options
serializes as the bare policy, so adding this changes no existing config
file or event stream on disk. Only a spec that actually carries a
threshold takes the table form, where a map-shaped policy gains `over`
as a sibling key and a string-shaped one is promoted to
`{"policy": "...", "over": "..."}`.
Deserialization tries the whole map against `P` before falling back to
the promoted string. The order is load-bearing: a tagged unit variant
such as `{"policy": "omit"}` is indistinguishable from a promotion by
shape alone, and only the map reading is correct for it.
`FromStr` accepts `POLICY,option=value` so a `--cfg` assignment and the
inline compaction DSL can share one option syntax.
Signed-off-by: Jean Mertz <git@jeanmertz.com>
A compaction rule reaches every item in its turn range. That is the
right default for old turns, but it is the wrong tool for the case that
actually breaks a conversation: one `fs_read_file` that returned a 10 MB
log and now dominates the context window. Cutting it today means either
compacting a whole range of turns you wanted to keep, or editing
`events.json` by hand.
The reasoning and tool-call policies now accept an `over` size
threshold, so a rule reaches only the items large enough to matter:
jp conversation compact --tools=sres --over 1mb
jp query -k 'r,over=16kb+t=sres,over=1mb:..-3'
[[conversation.compaction.rules]]
tool_calls = { policy = "strip-responses", over = "1MB" }
Both surfaces compose with the existing range flags, so a threshold
narrows what a rule touches rather than replacing how it is scoped. This
also enables a tiered rule set: drop anything genuinely huge wherever it
sits, and strip everything older than the last few turns regardless of
size.
The threshold is stored in the compaction event and evaluated during
projection rather than resolved into a fixed set of matched items at
creation. Size comparison is cheap and deterministic, so it is a lazy
policy in RFD 064's terms, and projection stays a pure function of the
stored stream.
Sizes are judged per half for `Strip`, so a call with a short request
and a huge response loses only the response. `Omit` removes whole pairs,
so it is judged on the two halves combined and can never leave an
orphaned request behind. Measurement uses decoded content (tool
arguments are base64 at rest) against the raw stream, so re-running a
rule reaches the same items. `summary` takes no threshold: it replaces
its whole range rather than selecting from it.
Because a threshold reaches an unpredictable subset of its range, the
timeline names what it caught instead of leaving the range line to imply
everything was compacted:
Would have compacted turns 2..14 (13 total, tool responses over 1MB).
turn 4 fs_read_file (response) 10.3 MB
turn 9 cargo_test (response) 2.1 MB
A rule without a threshold does not itemize, and a threshold that
matched nothing says so.
Nothing changes for a rule that sets no threshold, on disk or in
behavior: a policy without one serializes as the bare string it always
was, and the built-in defaults carry no threshold.
BREAKING CHANGE: `reasoning` in `jp conversation show -F json`
The `reasoning` field of a compaction entry was a boolean; it is now the
serialized policy (`null`, `"strip"`, or `{"policy": "strip", "over":
"16KB"}`), matching how `tool_calls` already reported itself. Without
this a consumer could read `tool_calls.over` but had no way to see a
reasoning threshold.
Scripts testing truthiness need updating: `.reasoning == true` becomes
`.reasoning != null`.
Signed-off-by: Jean Mertz <git@jeanmertz.com>
Review follow-ups on the compaction size threshold, the first two of
which are real defects.
A tool call ID is not unique. `TurnMut::build` documents that providers
like Google reuse one synthetic ID across streaming cycles within a
turn, and validates responses by count for exactly that reason. Keying
the size map by ID gave every occurrence of an ID the last one's sizes,
so `--tools=sres --over 1mb` on a turn holding a 4 MB `tc1` response
followed by a 2-byte `tc1` response kept the 4 MB one and reported no
match. The oversized payload the threshold exists to remove then reached
every later query, silently.
Sizes are now keyed by stream position, and a response pairs with the
oldest request in its turn carrying the same ID that has no response
yet, mirroring the rule `TurnMut::build` already applies. That also
fixes the tool name on a stripped response for a reused ID, which had
the same weakness.
A misspelled option key was silently dropped. `tool_calls = { policy =
"strip-responses", oer = "1MB" }` deserialized to an unthresholded rule
and stripped every response in range, the opposite of what the line
asks for. Schematic emits `deny_unknown_fields` for the surrounding
config types, so `PolicySpec` was the one place in this tree accepting
junk. The promoted-string form now rejects any leftover key, naming it.
A policy that legitimately serializes as a map with fields of its own is
untouched, since it consumes the map itself.
A summary rule no longer itemizes. `-k 's+r,over=1kb'` parses, and the
threshold rides along on the rule, but a summary replaces its whole
range and projection never consults the threshold. The timeline listed
items nothing had selected, under a label that read `summary`.
The generated JSON Schema describes the table form field by field rather
than as an unconstrained value, so a schema consumer still validates
`policy` and still rejects an unknown key. The `tool_calls` and
`reasoning` doc comments list the modes they accept and say what
omitting them does, which is what a reader scanning a generated
`config.toml` needs.
A test pins the stacking interaction the threshold makes visible: a
later thresholded overlay replaces an earlier policy for its whole turn,
so an item below the threshold is left raw rather than falling back to
the earlier, broader policy. That follows from latest-wins resolution
and is unchanged by this feature, but it was untested at item
granularity.
Signed-off-by: Jean Mertz <git@jeanmertz.com>
Signed-off-by: Jean Mertz <git@jeanmertz.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A compaction rule reaches every item in its turn range. That is the
right default for old turns, but it is the wrong tool for the case that
actually breaks a conversation: one
fs_read_filethat returned a 10 MBlog and now dominates the context window. Cutting it today means either
compacting a whole range of turns you wanted to keep, or editing
events.jsonby hand.The reasoning and tool-call policies now accept an
oversizethreshold, so a rule reaches only the items large enough to matter:
Both surfaces compose with the existing range flags, so a threshold
narrows what a rule touches rather than replacing how it is scoped. This
also enables a tiered rule set: drop anything genuinely huge wherever it
sits, and strip everything older than the last few turns regardless of
size.
The threshold is stored in the compaction event and evaluated during
projection rather than resolved into a fixed set of matched items at
creation. Size comparison is cheap and deterministic, so it is a lazy
policy in RFD 064's terms, and projection stays a pure function of the
stored stream.
Sizes are judged per half for
Strip, so a call with a short requestand a huge response loses only the response.
Omitremoves whole pairs,so it is judged on the two halves combined and can never leave an
orphaned request behind. Measurement uses decoded content (tool
arguments are base64 at rest) against the raw stream, so re-running a
rule reaches the same items.
summarytakes no threshold: it replacesits whole range rather than selecting from it.
Because a threshold reaches an unpredictable subset of its range, the
timeline names what it caught instead of leaving the range line to imply
everything was compacted:
A rule without a threshold does not itemize, and a threshold that
matched nothing says so.
Nothing changes for a rule that sets no threshold, on disk or in
behavior: a policy without one serializes as the bare string it always
was, and the built-in defaults carry no threshold.
BREAKING CHANGE:
reasoninginjp conversation show -F jsonThe
reasoningfield of a compaction entry was a boolean; it is now theserialized policy (
null,"strip", or{"policy": "strip", "over": "16KB"}), matching howtool_callsalready reported itself. Withoutthis a consumer could read
tool_calls.overbut had no way to see areasoning threshold.
Scripts testing truthiness need updating:
.reasoning == truebecomes.reasoning != null.