diff --git a/docs/perspectives/README.md b/docs/perspectives/README.md index e3ae0d8..e39cbb3 100644 --- a/docs/perspectives/README.md +++ b/docs/perspectives/README.md @@ -316,6 +316,110 @@ followed: projection boundary. (`relation_prober.rb` joins the exit-1-by-design set.) +## Round 11 — a new cast takes the bench + +The round-10 asks shipped as a release: `cancel_plan` is now prompt +(bookkeeping first, then fiber stops - never the reactor handle, never +the calling fiber), relation declarations fail fast at validator +construction (undeclared fields and sum_lte-over-strings refuse to +boot), `RateLimit#try_acquire` gives non-blocking admission, and +relations over untyped fields stay out of draft-07 keywords. The two +exit-1-by-design probers flipped to green acceptance tests, the retry +budget's wallet became a real windowed RateLimit — and then **ten new +prolific Rubyists** took over the experiments: + +| # | Persona | Built with the gem | Run it | Field notes | +|---|---------|--------------------|--------|-------------| +| 1 | Koichi Sasada (ko1) | Allocation audit — exact object counts per operation, via GC.stat | `examples/allocation_audit.rb` | [round-11/01-ko1.md](round-11/01-ko1.md) | +| 2 | Charles Nutter (headius) | Threads drill — real parallelism vs everything shared; a load-order bug caught | `examples/threads_drill.rb` | [round-11/02-headius.md](round-11/02-headius.md) | +| 3 | Nate Berkopec | Capacity planner — Little's Law over journal percentiles | `examples/capacity_planner.rb` | [round-11/03-nateberkopec.md](round-11/03-nateberkopec.md) | +| 4 | Ryan Davis (zenspider) | Plan flog — a pain score per plan; idiom free, coupling priced | `examples/plan_flog.rb` | [round-11/04-zenspider.md](round-11/04-zenspider.md) | +| 5 | Avdi Grimm | Confident pipeline — ten timid conditionals vs one barricade | `examples/confident_pipeline.rb` | [round-11/05-avdi.md](round-11/05-avdi.md) | +| 6 | Katrina Owen | Plan kata — red/green/refactor with graph assertions written first | `examples/plan_kata.rb` | [round-11/06-kytrinyx.md](round-11/06-kytrinyx.md) | +| 7 | Bozhidar Batsov | Contract cop — seven named cops, mechanical autocorrect only | `examples/contract_cop.rb` | [round-11/07-bbatsov.md](round-11/07-bbatsov.md) | +| 8 | José Valim | Telemetry bus — :telemetry on the hooks; runtime attach/detach, crash isolation | `examples/telemetry_bus.rb` | [round-11/08-josevalim.md](round-11/08-josevalim.md) | +| 9 | Luca Guidi | Ports and adapters — the domain survives the migration, with a purity scan | `examples/ports_and_adapters.rb` | [round-11/09-jodosha.md](round-11/09-jodosha.md) | +| 10 | Eileen Uchitelle | Tenant shards — N journals, N limits, one ignorant control plane | `examples/tenant_shards.rb` | [round-11/10-eileencodes.md](round-11/10-eileencodes.md) | + +### What round 11 surfaced + +1. **Fresh eyes found a fresh class of bug immediately**: headius's + bare-journal drill caught `Time#iso8601` used without requiring + "time" — a load-order bug nine rounds of fiber-world examples never + tripped, fixed in this round's release. New perspectives audit + different assumptions. +2. **The runtime got audited from below**: exact allocation counts + (37 objects per happy validation, 11x on rejection, zero GC runs + per plan), real-thread verdicts (journal and registry hold on real + locks; the windowed limiter coasts on the GVL), and Little's Law + turning the journal into a capacity plan whose binding constraint + was outside the meeting. +3. **The teaching seat is real**: the kata (assertions before tasks), + the flog score (calibrated so idiom is free), the confident/timid + contrast (nil-tolerance launders errors), and the cop (autocorrect + only what has one right answer) are all *pedagogy tools* built on + the same reflection surfaces the ops tools use. +4. **The architecture seat approves the seams**: the duck-typed + `agent:` seam let a pure domain walk in without signing tenancy + (ports-and-adapters with a purity scan), and ten lines bridged the + hooks to a :telemetry-style bus with crash isolation - frameworks + orchestrate, domains decide, handlers come and go. +5. **Next asks**: a Mutex around the windowed limiter's stamp + bookkeeping so the answer is the same on every Ruby (headius); + `remove_task`/rewire so refactoring a plan doesn't mean demolition + (Katrina, with the kata as acceptance test); and a multiprocess + journal drill to certify the flock claim (headius, follow-up). + +## Round 12 — a third cast, and the bill arrives itemized + +The round-11 asks shipped as a release: the windowed limiter's stamp +bookkeeping holds a real Mutex (the threads drill now certifies +instead of observes), `remove_task`/`rewire_task` make plan +refactoring surgical (the kata refactors in place), and the process +drill certifies the flock claim across four forked writers. Then a +third cast of ten prolific Rubyists took the bench: + +| # | Persona | Built with the gem | Run it | Field notes | +|---|---------|--------------------|--------|-------------| +| 1 | Yehuda Katz (wycats) | API surface census — 112 methods, 58 earning rent, 54 on loan | `examples/api_surface.rb` | [round-12/01-wycats.md](round-12/01-wycats.md) | +| 2 | Sarah Mei | Onboarding trail — a house tour computed from who-mentions-whom | `examples/onboarding_trail.rb` | [round-12/02-sarahmei.md](round-12/02-sarahmei.md) | +| 3 | Richard Schneeman | Require cost report — the bill lands at first touch, not at require | `examples/require_cost.rb` | [round-12/03-schneems.md](round-12/03-schneems.md) | +| 4 | Vladimir Dementyev | EventProf for plans — task-seconds by tag; llm owns 78% | `examples/event_prof.rb` | [round-12/04-palkan.md](round-12/04-palkan.md) | +| 5 | Mike Dalessio | Hostile inputs — a torn tail denies ALL recovery; exit 1 until | `examples/hostile_inputs.rb` | [round-12/05-flavorjones.md](round-12/05-flavorjones.md) | +| 6 | Justin Searls | Honest doubles — fakes show their papers at load time | `examples/honest_doubles.rb` | [round-12/06-searls.md](round-12/06-searls.md) | +| 7 | Konstantin Haase | Plan DSL — thirty lines of Sinatra over the public API | `examples/plan_dsl.rb` | [round-12/07-rkh.md](round-12/07-rkh.md) | +| 8 | Obie Fernandez | Self-correcting output — violations become the correction prompt | `examples/self_correcting_output.rb` | [round-12/08-obie.md](round-12/08-obie.md) | +| 9 | Rafael França | Gentle deprecations — warn once per site, tally, strict on schedule | `examples/gentle_deprecations.rb` | [round-12/09-rafaelfranca.md](round-12/09-rafaelfranca.md) | +| 10 | Jean Boussier (byroot) | Write path profile — JSON acquitted at 0.4%; the fsync IS the product | `examples/write_path_profile.rb` | [round-12/10-byroot.md](round-12/10-byroot.md) | + +### What round 12 surfaced + +1. **The stewardship seat spoke**: the census split 112 public + methods into earned API and accidental API; the deprecation shim + choreographed a rename across three releases; the DSL proved sugar + can stay entirely outside the engine. Three different answers to + "how does this gem grow old gracefully." +2. **One real defect, found where parsers meet reality**: a torn + journal tail — the exact artifact of the crash the journal exists + for — crashes replay in the wrong error class and denies all + recovery (`hostile_inputs.rb` exits 1 as the acceptance test). +3. **Costs arrived itemized**: require costs land at first constant + touch, not at require (Zeitwerk's deferral, priced); the journal + write is 99.6% fsync and 0.4% JSON, with group commit named as a + different promise rather than a faster one; plan time profiles by + tag with the barriers indicted, not the budget. +4. **The AI-application seat matured**: the self-correcting output + loop showed the contract doubling as a correction-prompt + generator, and honest doubles put load-time verification at the + LLM boundary — patterns, not vibes. +5. **Next asks**: tolerant journal replay — salvage whole lines, + report (don't raise on) a torn or mis-encoded tail + (`hostile_inputs.rb` is the acceptance test); an optional + `fsync_every: n` group-commit mode with its durability trade + named in the constructor (byroot); and consider a strict-shapes + replay mode for audit tools vs the tolerant recovery default + (flavorjones). + ### What round 6 surfaced 1. **Plans became artifacts**: narratable (tour), serializable with an diff --git a/docs/perspectives/round-11/01-ko1.md b/docs/perspectives/round-11/01-ko1.md new file mode 100644 index 0000000..825a6d6 --- /dev/null +++ b/docs/perspectives/round-11/01-ko1.md @@ -0,0 +1,69 @@ +# Round 11 field notes — Koichi Sasada audits the promissory notes + +*Built: `examples/allocation_audit.rb` — exact per-operation object +counts via `GC.stat(:total_allocated_objects)`, plus where the GC +actually fires during a plan.* + +## What I built and why + +I think about programs the way the VM sees them: not as logic, but +as a stream of allocations arriving at the GC's door. Every object +is a promissory note — cheap to write, collected on schedule, and +the schedule does not consult your latency SLO. So my first question +for any framework is not "is it fast?" but "what does it *allocate*, +per operation, exactly?" Not sampled. Counted: + +``` +Task.new 18 objects +validator: happy path 37 objects +validator: rejection 440 objects +graph snapshot, isolated 96 objects +to_json_schema 29 objects +full 10-task plan 1661 objects, 0 GC runs +``` + +`total_allocated_objects` is a monotonic counter the VM maintains +anyway — the audit is free of observer distortion, which sampling +profilers never quite are. Warm-up matters: the first validation +compiles a dry-schema; charging that to the per-call price would be +accounting fraud. + +## Reading the numbers like a VM person + +- **37 objects per happy validation** is the number to multiply by + your request rate. At 1000 rps that's 37,000 notes a second, a + couple of young-gen pages — fine, *and now known to be fine*, + which is different from assumed. +- **Rejection costs 11x the happy path.** Exceptions carry + backtraces, messages, violation hashes. Error paths are allocation + paths — the classic mistake is capacity-planning the happy path + and then meeting a hostile client who makes you allocate 440 + objects per garbage request. +- **The graph snapshot's 96 objects are a purchase, not a leak.** + That's the dup+freeze immutability every graph tool since round 5 + leans on. When you see allocation in a profile, ask what it + *bought* before you delete it — half of "optimization" is + accidentally refunding a purchase the design depended on. +- **Zero GC runs in a 10-task plan** means plan overhead lives + comfortably inside a young generation. The framework will never be + why your agent process pauses; the 800ms network calls will hide + every pause the VM takes anyway. + +## Notes + +- I subtracted the orchestrator-construction cost from the graph row + by measuring the build alone. Attribution is the whole game in + allocation work; a hot method blamed for its callee's allocations + sends someone optimizing the wrong function for a week. +- What I'd measure next (round-12 thought, not quite an ask): object + *lifetimes*, not just counts — journal entries are written and + dropped (good, dies young), but `@results` accumulates for the + plan's life (fine for 10 tasks, interesting at 10,000). + +## Verdict + +The framework's allocation profile is boring, and boring is the +correct aesthetic for infrastructure: small numbers on hot paths, +honest purchases where immutability is sold, error paths priced at +11x and now labeled as such. Allocation isn't evil; unbudgeted +allocation is. There's a budget now. diff --git a/docs/perspectives/round-11/02-headius.md b/docs/perspectives/round-11/02-headius.md new file mode 100644 index 0000000..6669751 --- /dev/null +++ b/docs/perspectives/round-11/02-headius.md @@ -0,0 +1,72 @@ +# Round 11 field notes — Charles Nutter brings real threads + +*Built: `examples/threads_drill.rb` — the journal, the registry, and +the windowed limiter hammered by actual Ruby threads, judged by the +standard of a VM with no GVL to hide behind.* + +## What I built and why + +Fibers are polite: they yield when asked and never interrupt a +two-step dance. Threads are not, and on JRuby they run *actually in +parallel* — every "works fine in production" claim earned under +MRI's GVL gets re-examined on my VM, usually at 2am. So I drill +everything this gem calls shared: + +``` +journal, 8 threads x 150 events: 1200/1200 lines, 0 torn +registry, 8 threads x 50 each: 0 registrations lost +windowed try_acquire, ceiling 50: admitted 50/50 (observed) +``` + +The journal and the registry hold, and they hold for the right +reason: they *paid* — a Mutex, flock, fsync. Those survive real +parallelism because they're real locks, not scheduling luck. + +## The drill drew blood before the threads even started + +First run, drill 1 crashed in all eight threads at once: +`undefined method 'iso8601' for an instance of Time`. The journal +calls `Time#iso8601` — a method from the `time` stdlib — without +requiring it. Every previous example worked because something else +(async, loaded when an orchestrator or limiter was touched) had +required it first. My drill used the journal *bare*, and the +load-order debt came due. + +This is the classic works-on-my-boot bug, and it's exactly the +species JRuby users hit constantly: different load orders, different +lazy-loading, same gem, sudden NoMethodError on a stdlib method. +The fix is one line (`require "time"` where it's used) and it's in +this round's release — but the lesson is the durable part: **every +file must require what it uses.** Transitive requires are a loan +from a dependency's internals, and dependencies refinance without +telling you. + +## Luck wearing a lab coat + +Drill 3 is the honest one. The windowed limiter's `try_acquire` +reads `@stamps.size`, then appends — check-then-act, no mutex. Eight +threads, 1600 attempts, ceiling 50: admitted exactly 50. So it's +fine? No — it's *unobserved*. MRI's GVL makes the two steps nearly +atomic by scheduling accident; JRuby runs them genuinely +concurrently, and unsynchronized size-check-then-append is precisely +the shape that over-admits there. The drill prints both possible +outcomes honestly and files the round-12 ask: a Mutex around the +stamp bookkeeping, so the answer is the same on every Ruby. A lock +you only need on some VMs is a lock you need. + +## Notes + +- The journal's flock+fsync means it would hold across *processes*, + not just threads — the strongest claim in the gem, and the drill + only tested the weaker half. A multiprocess drill is the natural + sequel. +- Note what I did NOT ask for: making the orchestrator thread-safe. + It's fiber-architected and says so; the drill only holds + *explicitly shared* structures to the parallel standard. + +## Verdict + +Two structures paid for real locks and passed a real-parallelism +drill; one is coasting on the GVL and now has that in writing; and +the drill's warm-up caught a load-order bug that fiber-world never +would have. Bring threads to your gem before your users' VM does. diff --git a/docs/perspectives/round-11/03-nateberkopec.md b/docs/perspectives/round-11/03-nateberkopec.md new file mode 100644 index 0000000..9805457 --- /dev/null +++ b/docs/perspectives/round-11/03-nateberkopec.md @@ -0,0 +1,69 @@ +# Round 11 field notes — Nate Berkopec does the capacity math + +*Built: `examples/capacity_planner.rb` — Little's Law over the +journal's duration percentiles, then the plan checked against every +configured limit. The binding constraint wasn't the one the meeting +was about.* + +## What I built and why + +"How many workers do we need?" is the most expensive question teams +answer by feeling. The feeling-based answers cluster at two poles: +over-provision 4x (pay the cloud bill forever) or size to the demo +(page the on-call forever). Meanwhile the actual answer has been +math since 1961: **L = λW**. Concurrency needed equals arrival rate +times service time. The journal already stores W — per-task duration +samples across thirty runs — so the planner only needs λ, your peak +target: + +``` +task p50 p95 lanes (p50/p95) +fetch:ticket 83ms 158ms 1 / 1 +classify 364ms 751ms 1 / 2 +draft:reply 993ms 2387ms 2 / 5 +total at p95: 8 lanes for 120 tickets/min +``` + +**Plan for p95, not p50.** Capacity sized to the median queues every +time latency has a bad day, and in this journal latency has a bad +day one run in eight — which is what real latency looks like +(log-normal-ish, long right tail), not what the demo looked like. +The gap between the p50 plan (4 lanes) and the p95 plan (8) is +exactly the gap between "works" and "works during the incident." + +## The constraint that wasn't in the meeting + +Then the part that actually saves the quarter: check the computed +plan against *every* limit in the system, not just the one under +debate. The meeting was about `concurrency_limit: 8` — which, the +math says, *holds*. The provider quota of 90/min against 120/min +arrivals is the real story, and it's not a "slowdown": λ/μ > 1 has +**no steady state**. The queue grows without bound until something +breaks, and the something is usually memory, at 3am, wearing the +disguise of an unrelated alert. Utilization greater than one isn't +a performance problem; it's an arithmetic problem, and no amount of +worker tuning fixes arithmetic. + +This is the recurring shape of capacity incidents: everyone tunes +the limit they own, and the binding constraint belongs to a vendor +contract nobody re-read. The planner's job is to put all the limits +in one table with one verdict column. + +## Notes + +- The journal made this a twenty-line tool. Percentile baselines + across runs (`duration_percentile`, round 8) were built for + regression-hunting; capacity planning is the same data asked a + business question. Good telemetry keeps being reusable like this. +- The planner deliberately reports lanes *per task*: draft:reply + needs 5 of the 8, so if you shard workers by task type, that's + your split — and if draft:reply gets slower next release, Aaron's + perf tools and this planner will disagree with the cloud bill in + the same direction, which is how you know to act. + +## Verdict + +A journal plus Little's Law is a capacity plan; a dashboard plus a +feeling is a postmortem. The math took thirty lines, found the +binding constraint outside the meeting's agenda, and turned "how +many workers?" into a question with a receipt. diff --git a/docs/perspectives/round-11/04-zenspider.md b/docs/perspectives/round-11/04-zenspider.md new file mode 100644 index 0000000..e12ef29 --- /dev/null +++ b/docs/perspectives/round-11/04-zenspider.md @@ -0,0 +1,74 @@ +# Round 11 field notes — Ryan Davis flogs the plans + +*Built: `examples/plan_flog.rb` — a pain score per task and per plan, +flog-style: joins, depth, anonymous edges, and orphans each cost +points; boring plans score zero.* + +## What I built and why + +Flog exists because "this method feels complicated" loses every +argument to "this method is fine, I wrote it." Numbers don't feel. +So: same treatment for plans. Every structural sin has a price — + +- **extra join inputs**, 1.5 each (a pipe is free; the *second* + input is where coupling starts) +- **fan-out past 2**, 1.0 (blame-spreading) +- **depth past 3**, 0.8 per level (latency hiding in a trench coat) +- **anonymous inputs at joins**, 1.2 (data flow you can't name is + data flow you can't debug) +- **orphans**, 5.0 flat (it runs on every execution and feeds + nothing — a bug or a billing strategy, pick one) + +``` +tidy pipeline 0.0 fine +labeled diamond 1.5 fine +the monster 24.5 REFACTOR ME do_everything=14.7 orphan=5.0 +``` + +## Calibration is the actual work + +My first scoring charged every dependency 1.5 — and the tidy +three-step pipeline scored 5.4. Garbage. A sequential pipe is the +*most idiomatic plan there is*; a metric that punishes idiom trains +people to ignore it, which is worse than no metric. The fix: pain +starts at the second join input, and anonymity only costs where +there's more than one input to confuse. Now the pipe scores 0.0 and +the diamond 1.5, which matches every practitioner's gut — that's +what calibration *is*: the metric agreeing with good taste on the +easy cases so it can overrule bad taste on the hard ones. + +(Flog went through exactly this. Early versions punished things +Rubyists do on purpose; the weights moved until they didn't. The +weights ARE the opinion. Own them.) + +## The number ends the meeting + +`do_everything` costs 14.7 on its own — five extra inputs of +coupling plus six anonymous edges. Everyone in the room already +*knew* the monster was a monster; what they lacked was a way to end +the "it's fine, it works" filibuster. "It's a 25; the threshold's +12" ends it. Then Sandi's refactor-receipts show *how* to dissolve +the join, Xavier's diff proves you did, and the flog score drops on +the next run — metric, method, proof, all from one graph accessor. + +The orphan deserves its flat 5: roots-that-are-also-leaves in a +multi-task plan came straight from `stats[:roots]` ∩ +`stats[:leaves]`, and it's the smell nobody looks for because +nothing *fails*. It just... runs. Forever. On your bill. + +## Notes + +- One number per plan AND per task — aggregate scores without + itemization are how metrics become astrology. The breakdown is + printed because a score you can't argue with is a score you can't + learn from. +- Thresholds (12 = refactor) are round numbers chosen to be argued + about. Good. Argue about the threshold, not about whether the + monster is fine. + +## Verdict + +Plans have flog now: idiom is free, coupling has a price list, and +the monster's 25 outlives everyone's patience for the word "fine." +Numbers don't refactor plans — they just end the meeting where +nobody was going to. diff --git a/docs/perspectives/round-11/05-avdi.md b/docs/perspectives/round-11/05-avdi.md new file mode 100644 index 0000000..897a82c --- /dev/null +++ b/docs/perspectives/round-11/05-avdi.md @@ -0,0 +1,72 @@ +# Round 11 field notes — Avdi Grimm builds the barricade + +*Built: `examples/confident_pipeline.rb` — one pipeline, two +postures: ten conditionals of timidity versus a contract at the +door, then both fed the same garbage so the difference is behavior, +not aesthetics.* + +## What I built and why + +Timid code is easy to recognize once you hear it read aloud: it's +all subordinate clauses. *If the order isn't nil, and if the items +are an array, and if the price responds to arithmetic, then perhaps +we might total it.* Every method re-litigates reality because it +trusts nothing — including the methods it just called. The narrative +voice of the code is a worried mumble. + +``` +timid: 10 conditionals, 24 lines +confident: 0 conditionals, 9 lines +``` + +The confident version isn't brave; it's *organized*. All the doubt +is pushed to one barricade — a capability contract validated at the +input door and (this matters) at the output door too, because +honesty is also a promise about what you return. Inside the +barricade, every line is a declarative sentence about data that is +known to be shaped: `order[:items].sum { ... }`. Indicative mood. +No hedging. The `fetch(:qty, 1)` is the one permitted courtesy — a +*declared default*, which is confidence about optionality, not fear +of it. + +## The laundering demonstration + +The comparison that matters isn't line count — it's what each +posture does with garbage. Fed an order with a nil price and an +empty email: + +``` +timid: {total_cents: 0, delivery: "no receipt"} +confident: raises ValidationError - email rejected AT THE DOOR +``` + +Look closely at the timid answer. It's polite. It's well-formed. +It's *wrong* — zero dollars and no receipt, delivered with full +confidence to whatever ledger consumes it. All those nil checks +didn't handle the bad input; they **laundered** it, converting a +detectable error into a plausible lie. `return nil if` is not error +handling — it's error *forwarding*, with the sender's address torn +off. Some downstream system now owes a customer an explanation, and +the stack trace that would have named this method is gone. + +The confident version says no, out loud, at the door, with the +field names attached. Failure at the barricade is cheap, local, +and honest. Failure past the barricade is a mystery novel. + +## Notes + +- The conditional count is computed from this file's own source at + runtime — the example won't drift into claiming a difference it + no longer exhibits. +- The contract can't see inside array items (a nil price sails to + the output door, where the *output* contract would catch a + non-numeric total). Item-level schemas would deepen the barricade; + worth a future ask if list-shaped inputs become common. + +## Verdict + +Confidence isn't optimism — it's pushing all the doubt to the +boundary, where it can say no out loud. One barricade bought back +ten conditionals, and more importantly it converted a laundered lie +into an honest rejection. Write the happy path like it's happy; +make the door do the worrying. diff --git a/docs/perspectives/round-11/06-kytrinyx.md b/docs/perspectives/round-11/06-kytrinyx.md new file mode 100644 index 0000000..327a7ed --- /dev/null +++ b/docs/perspectives/round-11/06-kytrinyx.md @@ -0,0 +1,68 @@ +# Round 11 field notes — Katrina Owen runs the kata + +*Built: `examples/plan_kata.rb` — red, green, refactor for a plan: +five structural assertions written before any task exists, the +smallest additions that move red lines green, two deliberate sins +caught by name, and a rebuild that ends all-green.* + +## What I built and why + +When I teach TDD, the resistance is never to the tests — it's to +writing them *first*, before you're attached to a design. So the +kata starts where katas should: an empty plan and five assertions +about a plan that doesn't exist yet. One entry point. One +deliverable. Joins that name their inputs. Nothing deeper than four +stages. No orphans. + +``` +empty plan -> 2 red (the honest starting point) +add the entry point -> 0 red +add parse -> 0 red +bolt on a price feed (2 sins) -> 2 red (caught, and NAMED) +rebuild: one entry, labeled -> 0 red +``` + +The step that earns the kata its keep is the fourth. "Bolt on a +price feed" is exactly how real plans degrade — a second source +lands as a second root, its join lands unlabeled, and in a +review-free world both would compost quietly into architecture. The +assertions objected *by name*: "has exactly one entry point: RED," +"every join names its inputs: RED." Tests written before anyone was +defensive about the design critique it without a meeting. + +Notice also what the assertions are written against: `stats[:roots]`, +`stats[:leaves]`, labeled edges, `max_depth`. The reflection API is +what makes plan-TDD *possible* — you cannot assert on what you +cannot observe cheaply. + +## What the refactor step taught + +The green-to-green refactor — dissolve the accidental second root by +routing both feeds through one door — required **rebuilding the +orchestrator from scratch**, because plans are add-only: there is no +`remove_task`, no rewire. The kata absorbs this gracefully (plans +here are cheap to rebuild, and cheap-to-rebuild is itself a virtue +worth practicing), but it's a real gap: refactoring under a green +suite is the entire payoff of having the suite, and today the only +refactoring move is demolition. Filed as the round-12 ask: +`remove_task` or a rewire seam, so a plan can change shape without +losing its identity. + +## Notes + +- Each step adds the *smallest* thing that moves a line. The + discipline looks pedantic on a six-task plan and becomes the only + thing that works on a sixty-task one — step size is a skill you + practice when it's easy so you have it when it isn't. +- Two of the five assertions are borrowed lessons: "no orphans" is + zenspider's 5.0-point smell as a boolean, and "joins name inputs" + is the round-8 spec generator's premise inverted into a gate. + Katas should steal from the room. + +## Verdict + +Plans can be test-driven: assert on structure first, grow in +smallest steps, let the reds name your sins while you're still +cheap to persuade. The one missing move — refactor without +demolition — is now on the asks list, with a kata as its acceptance +test. diff --git a/docs/perspectives/round-11/07-bbatsov.md b/docs/perspectives/round-11/07-bbatsov.md new file mode 100644 index 0000000..66dfdb4 --- /dev/null +++ b/docs/perspectives/round-11/07-bbatsov.md @@ -0,0 +1,69 @@ +# Round 11 field notes — Bozhidar Batsov puts cops on the contracts + +*Built: `examples/contract_cop.rb` — seven named cops for capability +specs, an offense report, and autocorrection for everything with +exactly one right answer.* + +## What I built and why + +The community style guide exists because "we argue about this in +every review" is a bug with a known fix: decide once, name the +decision, automate the enforcement. Contracts in this framework are +now the most-read documents in the system — by my count six tools +consume them (validator, docs, schema export, fixtures, diff, +polite-form) plus every human integrator — and the most-read +document is exactly where style buys the most. So, cops: + +``` +Naming/SnakeCaseName 'QuoteShipping' is not snake_case +Naming/SnakeCaseFields :weightKg is not snake_case +Documentation/Description capability has no description +Style/EnumOrder :Mode enum is not sorted +Lint/UntypedField :ref has no type (won't project) +Lint/OpaqueRuleWithoutMessage rule :check1 will say nothing +Metrics/RequiredInputCount 7 required inputs - one capability + or three? +8 offenses; autocorrect fixes 4 +``` + +Cop *names* matter as much as cop checks — `Lint/UntypedField` is a +sentence you can put in a commit message, a ticket, or a `.todo` +file. Anonymous complaints breed arguments; named ones breed +configuration. + +## The autocorrect line + +The split between corrected and remaining offenses is the design. +Autocorrect handles transformations with **exactly one right +answer**: snake_casing a name, sorting an enum. It refuses the rest, +each refusal with a reason: a description only the author can write; +a type that guessed wrong becomes a typed bug; a rule's message is +testimony you can't forge. RuboCop learned this boundary the hard +way — unsafe autocorrections cost more trust than they save +keystrokes, and a linter spends trust every time it speaks. + +Note which cops are style and which are load-bearing: +`Lint/UntypedField` isn't aesthetic — since round 11, untyped fields +keep relations out of schema projections, so `:ref` having no type +*silently narrows what five other tools can do*. The best lint rules +are the ones where "style" turns out to be an interoperability +contract wearing casual clothes. + +## Notes + +- `Metrics/RequiredInputCount` is the contracts' flog: past five + required inputs, the question isn't formatting, it's "is this one + capability or three?" Metrics cops should ask questions, not + issue verdicts — the number is the evidence, the design review is + the trial. +- Every cop is a lambda over the spec *hash*, not the object — same + data the diff and generator read. The whole linter is 40 lines + because the declarations did the hard work years... rounds ago. + +## Verdict + +Contracts have a style guide with teeth: seven named cops, four +mechanical fixes applied without discussion, four judgment calls +made impossible to not-see. Style is applied empathy for the next +reader — and these documents have six readers, so the empathy +compounds. diff --git a/docs/perspectives/round-11/08-josevalim.md b/docs/perspectives/round-11/08-josevalim.md new file mode 100644 index 0000000..1b784d8 --- /dev/null +++ b/docs/perspectives/round-11/08-josevalim.md @@ -0,0 +1,67 @@ +# Round 11 field notes — José Valim wires the telemetry bus + +*Built: `examples/telemetry_bus.rb` — named events on a bus with +attach/detach at runtime and crash isolation, bridged from the +orchestrator's lifecycle hooks in ten lines.* + +## What I built and why + +In Elixir we learned this lesson expensively: when every library +invents its own instrumentation callbacks, every metrics vendor +writes N adapters, every application couples to all of them, and +nobody can add a tracer without a deploy. The ecosystem converged on +`:telemetry` — tiny, boring, universal: events are **namespaced +tuples**, payloads split **measurements** (numbers) from **metadata** +(context), handlers attach and detach at runtime, and a crashing +handler is detached rather than allowed to take the caller down. + +This example is that design, standing on Agentic's hooks: + +``` +run 1: [trace] SLOW: enrich took 80ms + [bus] handler exporter crashed (IOError) - detached, plan unharmed + [metrics] {tasks: 3} +run 2 (tracer detached by ops): [metrics] {tasks: 6} +``` + +The orchestrator emitted identical events both runs. It cannot tell +that the tracer left or that the Friday exporter died, and that +ignorance is the entire feature: producers that know their consumers +grow opinions about them, and opinions become coupling, and coupling +becomes "we can't upgrade the metrics library until Q3." + +## Hooks are the floor, not the house + +I want to be precise about what I'm *not* criticizing. Lifecycle +hooks are the right primitive for a framework to export — one +configuration-time seam, no policy. But hooks couple one producer to +one consumer at configuration time; observability needs N consumers +changing at runtime. The bridge between them is ten lines because +the hooks carry exactly the right data (durations, failure types, +statuses — measurements and metadata, already separated in spirit). +A framework's job is to make the bus *possible* in ten lines, not to +ship the bus; ship the bus and you've chosen every user's metrics +vendor. + +Crash isolation deserves its sentence: the exporter raised, was +detached, was *reported*, and the plan never noticed. Instrumentation +must never be load-bearing — the day a tracing outage becomes an +application outage, someone deletes all the instrumentation, and +then you're blind *and* fragile. + +## Notes + +- Handler ids make detach targeted (`BUS.detach(:tracer)`) — the ops + story ("we got tired of that log line") is real and it should be a + one-liner, not a deploy. +- The ExecutionJournal is, in this framing, just another handler — + one that happens to fsync. Round 12 could re-express it as a bus + subscriber and the hooks stay clean forever. A thought, not an ask. + +## Verdict + +Callbacks are for configuration; events are for observation. The +framework's hooks turned out to be a good floor — ten lines of +bridge bought runtime attach/detach, crash isolation, and a producer +blissfully ignorant of its audience. Steal `:telemetry`'s design; +it was right the first time. diff --git a/docs/perspectives/round-11/09-jodosha.md b/docs/perspectives/round-11/09-jodosha.md new file mode 100644 index 0000000..445068b --- /dev/null +++ b/docs/perspectives/round-11/09-jodosha.md @@ -0,0 +1,68 @@ +# Round 11 field notes — Luca Guidi keeps the center clean + +*Built: `examples/ports_and_adapters.rb` — a pure use-case speaking +only to ports, Agentic as one of two interchangeable delivery +mechanisms, and a mechanical purity scan proving the dependency +arrow points one way.* + +## What I built and why + +Every framework, including the ones I've built, whispers the same +temptation: *put your business logic inside me, where it's cozy.* +And every rescue project I've seen started with someone accepting. +The test of architecture isn't the happy years — it's the migration, +and the part of your app that survives a migration is exactly the +part that never learned the framework's name. + +So: `QuoteShipment`, a use-case in pure Ruby. Its entire knowledge +of the outside world is two **ports** — `#rate_for(mode)` and +`#save(result)` — named after what the *domain* needs, not after +what any vendor provides. The adapters (six lines each) live at the +edge. And then two acts: + +``` +act one - delivered by Agentic: {price_cents: 1080, ...} +act two - delivered by a bare call: {price_cents: 600, ...} +purity scan: 0 framework constants in the domain +``` + +Act two is the migration, rehearsed. The orchestrator leaves; the +use-case doesn't notice, because there was nothing to notice — the +dependency arrow points one way. The edge knows the center; the +center has never heard of the edge. + +## The scan is the architecture test + +Talking about clean architecture is worthless; *checking* it is +cheap. The domain's source lives in a string precisely so the +example can grep it for framework constants and **exit 1 on a leak**. +That's the whole discipline in one mechanical gate — the kind of +check that belongs in CI, because architectural erosion never +arrives as a decision; it arrives as one convenient `Agentic::` in +a domain file during a hotfix. + +What deserves praise: Agentic made act one *easy without demanding +tenancy*. The `agent:` seam takes any callable, so the use-case +walked in as `->(t) { use_case.call(**t.payload) }` — the framework +added retry policy, journaling, concurrency, and the graph *around* +the domain, without the domain signing anything. Frameworks +orchestrate; domains decide. A framework whose integration point is +"be callable" is a framework that respects the boundary — this is +Sandi's duck seam (round 9) viewed from the architecture side. + +## Notes + +- The ports are minimal on purpose: two methods, no base classes, no + registry. Ports are a vocabulary, not a bureaucracy — the moment a + port needs an abstract superclass, it has become an adapter with + ambitions. +- Act two also quietly demonstrates testability: the "bare call" IS + what a unit test of the domain looks like. No orchestrator in the + test suite, no framework boot time in the feedback loop. + +## Verdict + +The domain would survive the migration, and there's an exit code +that says so. Clean architecture isn't ceremony — it's the freedom +to change your mind about everything except the truth, and this gem, +to its credit, asks for nothing more than a `#call`. diff --git a/docs/perspectives/round-11/10-eileencodes.md b/docs/perspectives/round-11/10-eileencodes.md new file mode 100644 index 0000000..ade6f2e --- /dev/null +++ b/docs/perspectives/round-11/10-eileencodes.md @@ -0,0 +1,69 @@ +# Round 11 field notes — Eileen Uchitelle shards the plan + +*Built: `examples/tenant_shards.rb` — one pipeline definition run as +three isolated shard executions: per-shard journals, per-shard rate +limits, one ignorant control plane, and a crash that stays exactly +the size of its shard.* + +## What I built and why + +Scaling Rails at GitHub taught me that the hard part of "at scale" +is never the volume — it's that *every* concern you thought was +singular becomes plural. One database becomes N. One deploy becomes +N. And critically: one **failure story** becomes N, or else one +tenant's bad day becomes everyone's page. So when a plan grows past +one blast radius, the move is the same one we made with multi-db: +same definition, sharded execution: + +``` +run 1: shard_1 completed (6 steps) + shard_2 partial_failure (7 steps) <- crashed at umbrella:transform + shard_3 completed (3 steps) +run 2: fleet-wide rerun + shard_1: 0 ran, 6 skipped shard_3: 0 ran, 3 skipped + shard_2: 2 ran, 7 skipped <- resumed from the crash point +``` + +Each shard owns its journal (its recovery story) and its rate limit +(its noisy-neighbor containment). The pipeline is defined once — +sharding is a **data-model decision, not a code fork**, and the +moment shard code diverges from the definition you have N products +instead of N shards. + +## The control plane's ignorance is load-bearing + +The rerun is issued fleet-wide. The control plane does not know +which shard crashed, and this is a feature I will defend with the +energy of someone who has operated the alternative: a control plane +that tracks per-shard failure state *is a second database that can +disagree with the first*. Here the journals — fsynced at the moment +of truth by the shard itself — are the single source of recovery +truth, and "rerun everything" is safe, idempotent, and boring. +Descriptions as idempotency keys (round 3's design) turn out to be +exactly the shard-resume primitive: `umbrella:transform` means the +same work on every run, so the journal can vouch for it across +process generations. + +Two steps re-ran on shard 2 — the crashed one and the one behind it. +Not twenty-one steps (the fleet), not nine (the shard): two. Blast +radius ends at the shard boundary, and recovery cost ends at the +crash point. That's the whole contract. + +## Notes + +- Per-shard limiters matter as much as per-shard journals: with one + global limiter, a hot shard starves the fleet — Samuel's + fair-share lesson (round 9) applied at the shard tier. Isolation + has to be complete to be isolation; shared *anything* is a shared + fate. +- What I'd want next at real scale: shard journals in different + *locations* (the shard-1 disk dying shouldn't threaten shard-2's + recovery story). The `path:` parameter already permits it; noting + it as operational guidance, not an ask. + +## Verdict + +One definition, N executions, N recovery stories, N rate limits, +and a control plane whose ignorance keeps it honest. Scale isn't a +bigger machine — it's smaller failures, and the journal-per-shard +pattern makes failure exactly shard-sized. diff --git a/docs/perspectives/round-12/01-wycats.md b/docs/perspectives/round-12/01-wycats.md new file mode 100644 index 0000000..d45e401 --- /dev/null +++ b/docs/perspectives/round-12/01-wycats.md @@ -0,0 +1,68 @@ +# Round 12 field notes — Yehuda Katz takes the census + +*Built: `examples/api_surface.rb` — the full public method surface of +eleven core classes, owner-checked, cross-referenced against 102 +example programs to split real API from accidental API.* + +## What I built and why + +Semver isn't a promise about your documentation — it's a promise +about **everything a user can call**. I learned this running Bundler +and Rails releases: the method you never documented is the method +whose removal breaks three build pipelines and a bank. So before any +1.0 conversation, you take the census: + +``` +total public surface: 112 methods +exercised by the corpus: 58 (52%) +accidental: 54 - including PlanOrchestrator#find_eligible_tasks, + #overall_status, #retry?, Task#perform... +``` + +The corpus is this repo's own 100+ example programs — the richest +usage dataset a pre-1.0 gem could dream of. Every method they call +carries a de facto semver promise *now*, docs or no docs; every +public method they don't is surface the maintainer is paying +interest on without collecting rent. + +## The stewardship read + +The accidental list is where release engineering lives, and the +right move is **declaration, not deletion**: + +- `PlanOrchestrator#find_eligible_tasks` and `#retry?` are scheduler + internals wearing public visibility. The day a user's code calls + `retry?` to make decisions, the retry engine can never be + restructured without a major bump. Privatize while it's free. +- `Task#perform` is the interesting hard case: it's *architecturally* + public (the orchestrator calls it) but *user-facing* private. That + distinction — audience-scoped API — is what `@api private` was + invented for, and Rails' whole `:nodoc:` culture exists because + Ruby's two visibility levels can't express three audiences. +- `TaskFailure#hopeless?` being unexercised is a *timing* artifact — + shipped two rounds ago, adopted by policy code, not yet by + examples. The census can't tell "accidental" from "young"; a + steward reads the column with a calendar in hand. + +First-draft note: the census originally reported 244 methods because +it counted inherited Psych/Object noise (`yaml_tag`, `allocate`) as +surface. Owner-checking the enumeration halved the ledger — get the +attribution wrong and the census indicts the wrong debt, which is +worse than no census. + +## Notes + +- The 52% exercised rate is *healthy* for this corpus — examples + skew toward the interesting seams. The number to watch is the + trend: surface growing faster than exercise means the gem is + speculating about what users want instead of finding out. +- The corpus method has a blind spot: internal callers (the + validator uses `RelationRules.check`; specs use everything). + A production census would union several corpora and weight them. + +## Verdict + +112 methods, 58 earning rent, 54 on loan. Public-by-default is a +loan against every future refactor, and this census is the bill — +cheap to pay today with a `private` keyword, expensive forever the +day after someone couples. Take the census before 1.0, not after. diff --git a/docs/perspectives/round-12/02-sarahmei.md b/docs/perspectives/round-12/02-sarahmei.md new file mode 100644 index 0000000..2c86885 --- /dev/null +++ b/docs/perspectives/round-12/02-sarahmei.md @@ -0,0 +1,70 @@ +# Round 12 field notes — Sarah Mei gives the house tour + +*Built: `examples/onboarding_trail.rb` — a nine-room reading tour of +the gem, ordered by the code's own who-mentions-whom so no stop +assumes a concept you haven't met, with one human sentence per room.* + +## What I built and why + +We talk about codebases like they're texts, but people don't *read* +codebases — they **live** in them. And when someone new moves in, +what they need isn't a blueprint (the file tree already is one); it's +a tour: which room to enter first, and why each room makes sense +given the ones you've seen. Nobody's first question is "where is the +scheduler?" It's "what should I read *first* so the rest makes +sense?" + +So the trail computes that ordering from the code itself — scan each +core file for mentions of the others, then repeatedly visit the room +with the fewest unmet concepts: + +``` +1. task_failure how this house talks about things going wrong +2. task_result the envelope every outcome arrives in +3. relation_rules predicates as data +... +8. plan_orchestrator the living room where everything meets +9. execution_journal the house's memory +``` + +## The house's values, visible on day one + +The trail starts with `task_failure`, and I want to sit with that, +because it isn't a quirk of the sort — it's the house telling you +who it is. This codebase defines *how it talks about things going +wrong* before it defines work, scheduling, or success. A new +teammate who reads room 1 has learned the house's **values** — +failure is data here, not an exception in both senses — and values +are the thing onboarding docs always mean to convey and never do. + +The other design choice I'll defend: the one-line room notes are +**human-written**, and everything else is derived. That's the +correct split. Structure is derivable (and should be, so the tour +never rots); *purpose* isn't — "the house's memory" is a sentence +only someone who lives here can say. Fully-generated docs are +accurate and useless; fully-handwritten docs are useful and stale. +The livable version is a derived skeleton wearing human sentences. + +## Notes + +- The trail prints a WARNING if the ordering ever visits a room + before its concepts — an honesty check on its own heuristic. Today + it's silent; the day someone adds a circular mention, the tour + itself will complain, which is how documentation should break: + loudly, at generation time. +- Line counts ride along on purpose (`plan_orchestrator: 731`). + Telling a newcomer room 8 is 7x the size of room 2 sets pacing + expectations — tours that hide the mansion's one enormous room + produce lost guests. +- What I'd add with more rounds: the *social* layer. Which rooms + change most often (git churn) is where the household actually + lives; a tour that ends "and this room is under renovation, ask + before moving furniture" is a tour of a real home. + +## Verdict + +A map answers "where"; a trail answers "in what order will this make +sense" — and the second question is the one every new teammate is +actually asking. Codebases are places people live. Give the new +roommate a tour, point out where the house keeps its values, and +mention which rooms are big before they open the door. diff --git a/docs/perspectives/round-12/03-schneems.md b/docs/perspectives/round-12/03-schneems.md new file mode 100644 index 0000000..d032db0 --- /dev/null +++ b/docs/perspectives/round-12/03-schneems.md @@ -0,0 +1,66 @@ +# Round 12 field notes — Richard Schneeman reads the bill + +*Built: `examples/require_cost.rb` — derailed-style require costs +(RSS, objects, wall time) measured in pristine child processes, for +the gem and each major dependency — with a plot twist about when the +bill actually lands.* + +## What I built and why + +Years of Heroku support tickets taught me that memory problems are +almost never mysterious — they're just *itemized late*. Somebody adds +a gem in a two-line PR, the dyno graph climbs 30MB a month later, and +the two events are never introduced to each other. So: measure +requires like purchases, each in a clean subprocess so nobody's +transitive dependencies get billed to a neighbor's account. + +``` +json (stdlib) 0.4MB 4,805 objects 9ms +zeitwerk 0.5MB 7,610 10ms +async 2.8MB 35,476 51ms +dry-schema 2.3MB 46,822 67ms +agentic (require only) 0.6MB 11,976 17ms +agentic + first real touch 5.6MB 84,254 118ms +``` + +## The plot twist + +My first draft measured `require "agentic"`, saw 0.6MB, and nearly +wrote the wrong report. The gem's require is **nearly free** — the +round-1 Zeitwerk cleanup means nothing loads until a constant is +touched. The last row is the honest one: touch an orchestrator and a +validator, and async and dry-schema arrive through the autoloader — +5.6MB and 118ms, *at first use*. + +**Deferred is not free — it's a bill that arrives during your first +request instead of your boot.** Which one you want depends entirely +on who you are: a CLI or a small script pays only for what it +touches (beautiful — most invocations never load dry-schema at all); +a web worker eats 118ms of autoloading inside somebody's request +(page the on-call and tell them it's "lazy loading"). Same mechanism, +opposite verdicts, and a report that only measures one moment will +confidently tell you the wrong story. + +The moves this funds: `eager_load` + `preload_app` in servers — pay +the 5.6MB once in the parent and share it copy-on-write across all +eight workers; stay lazy in CLIs; and run the script in CI so a new +dependency's bill arrives *in the PR that adds it*. + +## Notes + +- Child-process isolation is non-negotiable for this measurement. In + a warm parent, half the dependencies are already loaded and every + row under-reports; my numbers looked suspiciously cheap until each + probe got its own pristine VM. +- Objects-at-require matters separately from RSS: dry-schema's 46k + objects are mostly long-lived constants — old-gen residents that + make every future major GC walk a bigger heap. RSS is rent; + old-gen objects are a homeowners association fee. + +## Verdict + +The gem itself is a courteous tenant (12k objects, deferred +everything); its dependencies are the furniture, arriving on first +touch. Both numbers are now on a receipt, and the receipt belongs in +CI — because the cheapest time to argue with a dependency is in the +PR that introduces it. diff --git a/docs/perspectives/round-12/04-palkan.md b/docs/perspectives/round-12/04-palkan.md new file mode 100644 index 0000000..f6b27f3 --- /dev/null +++ b/docs/perspectives/round-12/04-palkan.md @@ -0,0 +1,68 @@ +# Round 12 field notes — Vladimir Dementyev profiles by group + +*Built: `examples/event_prof.rb` — TestProf's EventProf idea applied +to plans: task-seconds aggregated by tag, share-of-total before any +optimization talk, and an effective-parallelism line that indicts +the stage barriers.* + +## What I built and why + +TestProf exists because profiling by *file* answers the wrong +question. Nobody can act on "spec_a.rb is slow"; everyone can act on +"62% of your suite is factories." The unit of optimization is the +**group**, because groups map to causes and causes map to fixes. Same +move for plans — tag tasks by kind, collect durations from the hooks, +aggregate: + +``` +tag seconds share worst offender +llm 651ms 78.1% llm:draft (250ms) +db 101ms 12.2% db:fetch_orders (40ms) +render 81ms 9.7% render:pdf (50ms) + +task-seconds 833ms / wall 342ms = 2.4x parallelism on 3 lanes +``` + +Read the share column *before* touching any code: llm owns 78% of +all task-seconds, so a 20% win there beats deleting the entire +render stage. Optimizing db: or render: is polishing doorknobs on a +burning building — and without the table, doorknob-polishing is +exactly what happens, because render code is more fun to touch than +prompt budgets. + +## The second number + +Task-seconds by group is the TestProf classic; the parallelism line +is the plan-specific lesson. 833ms of work in 342ms of wall is 2.4x +on 3 lanes — meaning a chunk of a lane is going unused, and the +culprit is visible in the plan's shape: stage barriers. Every llm +task waits for *all* db tasks; every render waits for *all* llm. +`db:fetch_users` finished at 30ms but `llm:summarize` idled until +`db:fetch_orders` cleared at 40ms — multiply that slack across +stages and you've bought 3 lanes to run 2.4. + +The fix isn't more lanes (utilization, not capacity, is the +bottleneck) — it's finer dependencies: `needs:` lets an llm task +depend on exactly the db task it reads. The profiler doesn't make +that change; it makes it *undeniable*, then verifies it on the +re-run. Profile, fix the biggest group, re-profile. Boring, +effective. + +## Notes + +- Fifteen lines of profiler, because the hooks hand over exactly the + right tuple (description, duration) at exactly the right moment. + Instrumentation seams you don't have to fight are the difference + between "we should profile" and profiling. +- Tags ride in the description prefix (`llm:draft`) — the same + convention the journal's idempotency keys already reward. When one + naming convention feeds three tools (resume, triage, profiling), + it stops being a convention and starts being a schema. + +## Verdict + +"Where does the time go" now has a by-group answer with shares, +worst offenders, and a parallelism ratio that points at the barriers +rather than the budget. Read the share column first; let the biggest +group spend your optimization budget; make the profiler cheap enough +that re-running it is a reflex. diff --git a/docs/perspectives/round-12/05-flavorjones.md b/docs/perspectives/round-12/05-flavorjones.md new file mode 100644 index 0000000..83e67b5 --- /dev/null +++ b/docs/perspectives/round-12/05-flavorjones.md @@ -0,0 +1,70 @@ +# Round 12 field notes — Mike Dalessio feeds the parser real input + +*Built: `examples/hostile_inputs.rb` — eight hostile files against +`ExecutionJournal.replay`: torn tails, binary garbage, 8MB lines, +wrong-shaped JSON. Two probes draw blood; exit 1 by design until the +tail is tolerated.* + +## What I built and why + +Maintaining Nokogiri is a decades-long tutorial in one lesson: **a +parser's real specification is what it does with input nobody +intended.** Documents arrive truncated, mis-encoded, malicious, and +enormous — and a parser that only handles the happy grammar isn't a +parser, it's a demo. The journal's replay is a parser, and it has a +special obligation: the file it reads is, *by the journal's own +reason for existing*, a file that may end mid-write. + +``` +clean file (control) recovered (2 salvaged) +torn tail (crash mid-write) CRASHED: JSON::ParserError - ALL recovery denied +binary garbage line CRASHED: Encoding::CompatibilityError +empty + whitespace lines recovered +8MB single line recovered +valid JSON, wrong shape recovered +unknown event type recovered (skipped, correctly) +duplicate success lines recovered (idempotent, correctly) +``` + +## The indefensible probe + +Six of eight verdicts are genuinely good — unknown events skip, +duplicates are idempotent, giant lines and shape-garbage flow +through. But the torn tail is the one that matters, and it's the one +that kills. Walk the incident: the process dies mid-`record`. fsync +has made every *completed* line durable — the journal did its job. +The line being written *at* the moment of death lands half-formed at +the tail. Recovery runs, replay hits the torn line, and +`JSON::ParserError` flies past every `rescue ValidationError` in the +recovery tool — **100% of the events that were durable become +unreachable because of the one that wasn't.** The recovery tool is +now the second thing that failed, which is the one thing a recovery +tool must never be. + +The binary-garbage probe is the same wound in different clothes +(`Encoding::CompatibilityError` — a *third* uniform, for the +rescuers keeping count). + +Filed as the round-13 ask, with this probe as the acceptance test: +replay must salvage every whole line and *report* a damaged tail +(count it, expose it on the state) rather than raise on it. Tolerant +reading, loud accounting — the Nokogiri recovery-mode posture. + +## Notes + +- Note what I did NOT flag: `valid JSON, wrong shape` recovering + quietly is *defensible* for a recovery tool (salvage maximally) + but a stricter mode should exist for audit tools — Jeremy's + round-8 journal audit wants to *notice* wrong shapes, not skate + past them. One file format, two reader postures; both legitimate. +- The 8MB line recovering is worth a sentence: no line-length + assumptions, no fixed buffers. Good — length limits are where + "robustness" quietly becomes data loss. + +## Verdict + +Six probes pass with genuinely good manners; two crash in the wrong +uniform, and one of those two is the exact file a real crash writes. +Parsers meet real input, and real input is damaged — the journal's +reader needs to survive the very artifact its writer exists to +survive. Exit 1 until. diff --git a/docs/perspectives/round-12/06-searls.md b/docs/perspectives/round-12/06-searls.md new file mode 100644 index 0000000..9cab757 --- /dev/null +++ b/docs/perspectives/round-12/06-searls.md @@ -0,0 +1,70 @@ +# Round 12 field notes — Justin Searls checks the fakes' references + +*Built: `examples/honest_doubles.rb` — an owned port at the LLM +boundary, plus a verifier that rejects any double whose methods or +arity have drifted from the port. One double is honest; one has been +lying since last quarter.* + +## What I built and why + +Every agent test suite on earth contains a fake LLM, and I promise +you at least one of them is lying. Not maliciously — *entropically*. +Somebody wrote a stub against the client's interface in March, the +interface grew a keyword in June, and the stub has been vouching for +code that would crash on its first production call ever since. Green +suite. Broken app. The two rules that prevent this cost almost +nothing: + +**Rule 1: don't mock what you don't own.** Stubbing +`Agentic::LlmClient` couples every test to a vendor interface that +changes at gem-update speed. Instead, define *your* port — +`CompletionPort#complete(prompt, max_tokens:)` — one class that +names the entire vendor surface you permit yourself to use. (The +census two stalls over says the smaller that surface, the better; +we agree from different directions.) + +**Rule 2: verify every double against the port.** The verifier is +twenty lines: methods must exist, and *parameter shapes must match* +— required, optional, keyword, by name: + +``` +honest double: verified - method AND arity match +drifted double: REJECTED before any test ran - + port takes [[:req, :prompt], [:keyreq, :max_tokens]], + double takes [[:req, :prompt]] +``` + +## The treachery of unverified fakes + +Look at what the drifted double would have done *without* the +verifier: passed. Every test. `complete` responds, strings come +back, assertions green. Unverified fakes don't fail — **they +vouch**, and a false character witness is worse than no witness, +because it converts your test suite from an early-warning system +into a lullaby. The verifier moves the failure to load time, which +is the cheapest possible place: the drift is announced before a +single test runs, with both parameter lists printed so the fix +writes itself. + +This is, of course, what verified doubles in RSpec and Mocktail's +signature checks do — the point of hand-rolling it in forty lines is +to show there's no magic, just `Method#parameters` and the +discipline to call it. The mechanism is cheap; the *habit* is the +technology. + +## Notes + +- The port declines to be clever: no dynamic dispatch, no + method_missing forwarding. Boundary classes should be so boring + that drift has nowhere to hide. +- What this deliberately doesn't verify: return *values*. Arity + honesty is checkable statically; semantic honesty needs the + round-8 eval scorers pointed at the real adapter in a nightly + contract test. Both layers, different cadences. + +## Verdict + +Your tests are only as honest as their most casual fake. Own the +boundary with one port, make every double show its papers at load +time, and interface drift becomes a loud ArgumentError instead of a +quiet quarter of false confidence. diff --git a/docs/perspectives/round-12/07-rkh.md b/docs/perspectives/round-12/07-rkh.md new file mode 100644 index 0000000..f40ae05 --- /dev/null +++ b/docs/perspectives/round-12/07-rkh.md @@ -0,0 +1,72 @@ +# Round 12 field notes — Konstantin Haase sweetens the surface + +*Built: `examples/plan_dsl.rb` — a thirty-line Sinatra-flavored DSL: +`step :name, after:, needs:` with the block as the agent, all sugar +over the public API, never inside it.* + +## What I built and why + +Sinatra's entire argument fit in four lines of README: an API is a +user interface, and interfaces should read like what they mean. +`get "/hi" do` didn't add a single capability Rack lacked — it +*removed the administration* between intention and expression. The +orchestrator's API is honest but administrative: construct tasks, +mind the ids, pass arrays. Meanwhile what a person *means* is: + +```ruby +plan = Plan.define do + step :fetch_orders do ... end + step :fetch_refunds do ... end + step :ledger, needs: {orders: :fetch_orders, refunds: :fetch_refunds} do |t| + t.needs[:orders].sum { ... } - t.needs[:refunds].sum { ... } + end + step :report, after: :ledger do |t| + "net revenue: $#{t.previous_output}" + end +end +``` + +Thirty lines of Builder later, that runs — on a completely real +orchestrator, with labeled edges the round-5-through-11 tools all +consume unchanged. + +## What the sugar buys, and what it refuses + +Three purchases: **names instead of ids** — symbols resolve to tasks +at definition time, so a typo'd `:fetch_order` raises at *define*, +the cheapest moment a wiring bug can exist. **The block is the +agent** — work lives inside the step that owns it, instead of a +lambda table three screens away. **`after:`/`needs:` read as +English** — the sentence shape carries the semantics (unlabeled +sequence vs named join) that round 8's spec generator later turns +into test obligations. + +One refusal, and it's the important one: the DSL **never reaches +into the engine**. Every line delegates to public API — `add_task`, +`execute_plan`, `graph`. That's not politeness; it's the survival +strategy DSLs forget. A DSL that touches internals is pinned to +them, drifts with them, and eventually *is* them, at which point +you've built a second engine with worse error messages. Sugar over +the API can never drift ahead of the engine, and anything it can't +express — retry policies, hooks, rewiring — you drop down one layer +*without rewriting*, because the Builder hands you the orchestrator +it built. The frontend should be a pleasure and the escape hatch +should be a door, not a wall. + +## Notes + +- `instance_eval` for the block, accepting its tradeoffs (no easy + access to the caller's self) for the reading experience. Sinatra + made the same trade for the same reason; if your users need the + other semantics, take the block arg form instead — one line. +- Deliberately absent: any DSL syntax for concurrency limits or + hooks. A DSL earns each word by being the *common case*; rare + cases belong on the layer below, where their full vocabulary + lives. + +## Verdict + +Thirty lines bought a surface that reads like the plan it describes, +fails wiring typos at define time, and can't drift from the engine +because it never left the public API. APIs are user interfaces — +sweeten the surface, keep the door open, change nothing underneath. diff --git a/docs/perspectives/round-12/08-obie.md b/docs/perspectives/round-12/08-obie.md new file mode 100644 index 0000000..8f78b0f --- /dev/null +++ b/docs/perspectives/round-12/08-obie.md @@ -0,0 +1,72 @@ +# Round 12 field notes — Obie Fernandez ships the editor with the writer + +*Built: `examples/self_correcting_output.rb` — the self-correcting +output pattern: contract violations become the correction prompt for +a bounded retry loop. Attempt 1 is what models actually do to +schemas; attempt 2 ships.* + +## What I built and why + +I've spent the last stretch of my career cataloging the patterns +that separate AI demos from AI *applications*, and this is the one I +reach for first. Every model output that touches a database, an +invoice, or another system WILL eventually arrive with a string +where a number goes, a currency nobody ISO-coded, and a field simply +forgotten. The demo ignores this. The application needs a pattern: + +``` +attempt 1: {"vendor":"Initech...","total_cents":"4200","currency":"usd"} +-> rejected; violations become the next prompt: + - total_cents: must be Numeric + - currency: must be one of: USD, EUR, GBP + - due_date: is missing +attempt 2: {..., "total_cents":4200, "currency":"USD", "due_date":"2026-08-01"} +-> contract satisfied. shipped after 2 attempts. +``` + +**The contract is the editor; the model is the writer.** And the +discovery that makes this framework unusually good at the pattern: +the correction prompt writes itself. The same capability contract +that six other tools already consume produces violations that are +*pre-written actionable feedback* — "currency: must be one of: USD, +EUR, GBP" is precisely the sentence you'd hand a junior writer, and +it beats "please try again" by exactly the margin your production +error rate will show. I've watched teams hand-craft correction +prompt templates per field; declared constraints generate better +ones for free. + +## The discipline clauses + +Three parts of the loop are load-bearing, and teams skip them in +order of expensiveness: + +1. **Bounded attempts.** Unbounded self-correction is a billing + strategy. Three is the number; a model that can't satisfy a + schema in three coached tries has a prompt problem or a schema + problem, and more retries buy neither. +2. **Honest terminal failure.** When the loop exhausts, it raises + with the full ValidationError — every draft on record. A + correction loop that swallows its final failure is the timid + pipeline from Avdi's stall wearing a lab coat. +3. **Validate at the output door, not in the prompt.** Prompting + "respond with valid JSON matching..." is a request; the validator + is a *checkpoint*. You need both, and only one of them is a + guarantee. + +## Notes + +- Wire this at the CapabilityProvider seam and every LLM-backed + capability inherits the loop without knowing it exists — the + pattern wants to be middleware. That's the natural round-13 shape + if the room wants it. +- The scripted model's first draft (string number, lowercase + currency, missing field) isn't pessimism — it's a census of the + three most common schema sins in real extraction logs. + +## Verdict + +Self-correction is what makes "the model sometimes returns garbage" +an engineering statement instead of a product risk. The contract +supplies the red pen, the loop supplies the patience, the bound +supplies the budget — ship the editor with the writer; never ship +the writer alone. diff --git a/docs/perspectives/round-12/09-rafaelfranca.md b/docs/perspectives/round-12/09-rafaelfranca.md new file mode 100644 index 0000000..f25f0ef --- /dev/null +++ b/docs/perspectives/round-12/09-rafaelfranca.md @@ -0,0 +1,70 @@ +# Round 12 field notes — Rafael França renames without breaking anyone + +*Built: `examples/gentle_deprecations.rb` — a contract field rename +shipped through the three-release choreography: translate-and-warn +(once per call site, caller named), a migration tally, and a strict +mode that turns laggards' CI red on the maintainer's schedule.* + +## What I built and why + +Most of maintaining Rails isn't writing the better API — it's the +two years of not breaking anyone who used the worse one. A rename is +five minutes of code and three releases of choreography, and +frameworks that skip the choreography don't have users with old +code; they have ex-users. So: `weight:` becomes `weight_kg:`, and +the shim does the whole dance: + +``` +DEPRECATION: :weight is now :weight_kg (called from + legacy_billing_job (gentle_deprecations.rb:75); shows once per site) + +migration report: + weight at legacy_billing_job (...:75) 3 calls + weight at legacy_admin_panel (...:77) 2 calls + migrated call sites warn nothing and appear nowhere. + +strict mode: DEPRECATED input :weight - refused +``` + +## The three details that make it gentle + +**Once per call site.** Warn on every call and a busy legacy job +turns your logs into the outage; warn once globally and the admin +panel's usage hides behind the billing job's. Per-site is the only +granularity that's both quiet and complete — my first frame +arithmetic blamed the shim's own `each` loop until the site detector +learned to skip frames that belong to the shim and the API boundary. +Blame frames are the whole product here; get them wrong and the +deprecation report sends someone to fix the wrong file. + +**The tally is the roadmap.** Deprecation warnings people can ignore +are prayers; a *count by call site* is a migration plan with names +on it. Release N ships the shim, N+1 is spent chasing the tally to +zero (five calls, two sites, two small PRs — you can see the whole +job), and the report's most important line is the quiet one: +migrated sites appear nowhere. + +**Strict mode is the deadline.** The same shim, `strict: true`, +turns the old name into an ArgumentError — in the laggards' CI +today if they opt in (the deprecation-toolkit pattern), for everyone +at N+2. The deadline is enforced by red builds, never by broken +production, which is the entire difference between a framework users +trust with upgrades and one they fork and freeze. + +## Notes + +- The shim lives *outside* the contract — the v2 contract only knows + the new world, so every derived tool (schema, fixtures, diff, + round 10's whole toolbox) speaks the future tense while the shim + alone carries the past. Compatibility layers should be deletable + in one commit. +- `hits` as a Hash is deliberately boring; in production it's a + counter with the same shape. Deprecation is data about your users + — collect it like you mean it. + +## Verdict + +Renames are cheap; broken trust compounds. Translate at the door, +warn once per site with the caller's name, let the tally write the +migration plan, and let CI — not production — enforce the deadline. +That's how a framework gets to have both a past and a future. diff --git a/docs/perspectives/round-12/10-byroot.md b/docs/perspectives/round-12/10-byroot.md new file mode 100644 index 0000000..4923e06 --- /dev/null +++ b/docs/perspectives/round-12/10-byroot.md @@ -0,0 +1,65 @@ +# Round 12 field notes — Jean Boussier weighs the layers + +*Built: `examples/write_path_profile.rb` — the journal's write path +benchmarked one layer at a time: serialize, buffered write, flush, +flock+fsync, and the group-commit alternative. The profiler acquits +JSON and indicts nothing — the expensive layer is the promise.* + +## What I built and why + +I've spent enough years on Ruby's JSON and string internals to know +exactly how these conversations go: a journal is "slow", somebody +proposes switching serializers, three days are spent on a gem swap, +and the p99 doesn't move — because nobody weighed the layers first. +So, weigh them: + +``` +JSON.generate only 2.7us ++ buffered write 3.5us ++ flush to kernel 4.6us +journal.record (flock+fsync) 712.3us +group commit (fsync per 20) 76.5us +``` + +Serialization is **0.4%** of the real write. You could make +JSON.generate infinitely fast and the journal would be 99.6% as slow +as before. The other 707 microseconds are the fsync — the syscall +where the kernel promises the bytes survived power loss — and that +cost is not overhead. **It's the product.** The journal's only +promise is that a crash cannot unwrite what `record` returned from; +fsync is that promise's unit price. (I optimize stdlib JSON for a +living and I'm telling you: leave it alone here. It was already +fast, and it was already irrelevant.) + +## Group commit is a different promise, not a faster one + +The honest knob exists: batch 20 events per fsync and the amortized +write drops to 77us — a 9x improvement that every high-throughput +journal (databases, Kafka, WALs) eventually reaches for. But say +precisely what was traded: a crash can now eat up to 19 +*acknowledged* events. That's not an optimization; it's a **different +durability contract**, and only the caller knows which contract +their recovery story needs. An LLM-cost journal probably wants +per-event fsync (each event is money); a metrics journal wants group +commit (losing 19 counters is nothing). The right framework move — +if throughput ever matters here — is `fsync_every: n` as an explicit +constructor argument, so the trade has a name and a diff, not a +folklore. + +## Notes + +- Bench hygiene: each layer gets its own file and its own loop; the + "real" row goes through the actual `journal.record` including the + per-call open and flock, because users pay the whole path, not + the flattering subset. +- The flush row (4.6us) vs the fsync row (712us) is the pedagogical + gap: "flushed to the kernel" and "durable" differ by 150x, and + conflating them is how systems pass every test and lose data + anyway. The page cache is not a disk; it's an optimist. + +## Verdict + +The profiler acquitted JSON in one table and priced the actual +choice — per-event durability at 712us or group-commit throughput at +77us with 19 events of exposure. Optimization budgets follow +profiles or they follow fashion; this one now has a profile. diff --git a/examples/README.md b/examples/README.md index a4be485..31439c5 100644 --- a/examples/README.md +++ b/examples/README.md @@ -7,17 +7,22 @@ not this file. | Example | What it shows | |---------|---------------| | `adaptive_throttle.rb` | The Adaptive Throttle: nobody TELLS you an upstream's capacity - you discover it. An AIMD controller (TCP's algorithm) p... | +| `allocation_audit.rb` | The Allocation Audit: every object is a promissory note the GC collects on later. This audit counts exactly what each fr... | | `api_reference.rb` | The API Reference Generator: walk the registry, emit reference docs for every capability - types, enums, bounds, policie... | +| `api_surface.rb` | The API Surface Census: your public API is not what you documented - it's every public method a user CAN call, because t... | | `backoff_conformance.rb` | Backoff Conformance: every strategy x jitter combination, a thousand draws each through an injected seeded RNG, checked ... | | `batch_import.rb` | The Batch Import: 500 rows of the kind of data people actually upload - typos, header drift, impossible combinations - r... | | `burst_absorber.rb` | The Burst Absorber: three waves of requests slam a credential with a ceiling of 3 (Agentic::RateLimit - this round's rel... | | `cancel_drill.rb` | The Cancel Drill: structured concurrency's core promise is that cancellation is PROMPT - stop means stop, not "finish ev... | | `capability_evals.rb` | Capability Evals: golden test cases run against registered capabilities, scored, and gated. When you swap a lambda for a... | +| `capacity_planner.rb` | The Capacity Planner: "how many workers do we need?" is not a feeling, it's Little's Law - L = lambda x W. The journal a... | | `changelog_scout.rb` | The Changelog Scout: reads real git history, classifies every commit through a contract-checked capability, and drafts t... | | `circuit_breaker.rb` | The Circuit Breaker: when an upstream is down, the cheapest request is the one you don't send. The breaker trips after 3... | | `collaboration_tracer.rb` | The Collaboration Tracer: lifecycle hooks record every message the orchestrator sends and every reply that comes back, t... | | `command_bus.rb` | The Command Bus: every command is a composed capability with its OWN declared contract (new in this round - compositions... | | `composed_limits.rb` | Composed Limits: a real provider enforces BOTH a billed quota and a connection ceiling. quota.and(pool) - new this round... | +| `confident_pipeline.rb` | The Confident Pipeline: timid code checks nil at every step because it trusts nothing, including itself. Confident code ... | +| `contract_cop.rb` | The Contract Cop: RuboCop for capability specs. Contracts are the most-read documents in this framework - six tools cons... | | `contract_fixtures.rb` | Contract Fixtures: example payloads in docs rot the day the contract changes. So don't write them - DERIVE them. This ge... | | `contract_fuzzer.rb` | The Contract Fuzzer: for every registered capability, generate inputs that SHOULD pass its declared contract and mutatio... | | `contract_overhead.rb` | The Contract Overhead Bench: "should we validate every call?" is a performance question, so answer it with a measurement... | @@ -33,6 +38,7 @@ not this file. | `durable_batch.rb` | The Durable Batch: six billable "LLM calls" run under an ExecutionJournal. Mid-batch, the process dies for real - exit!,... | | `error_taxonomy_drill.rb` | The Error Taxonomy Drill: three tasks fail three different ways - a rate limit (retryable, says the error itself), an au... | | `eval_scorers.rb` | Eval Scorers: the same eval set scored four ways - exact match, keyword containment, numeric tolerance, and a judge rubr... | +| `event_prof.rb` | EventProf for Plans: TestProf taught test suites to answer "where does the time GO?" by group, not by file. Same questio... | | `exquisite_corpse.rb` | The Exquisite Corpse: three artists each draw one part of a creature without seeing the others' work; the assembler rece... | | `failure_weather.rb` | The Failure Weather Report: a journal of three days, read as a forecast. Retryable failures are WEATHER - showers that p... | | `fair_share.rb` | Fair Share: two tenants, one upstream. The global ceiling is fair to REQUESTS - first come, first served - but tenant A ... | @@ -40,12 +46,15 @@ not this file. | `form_errors.rb` | The 422 Generator: turn a ValidationError into the API error document your frontend actually wants - message, allowed va... | | `freight_rules.rb` | The Freight Desk: a quoting capability whose tariff book is written as cross-field contract rules (new this round). Per-... | | `gem_scout.rb` | Gem Scout: describe what you need, get a ranked shortlist of gems. Search and scoring are separate capabilities; the sea... | +| `gentle_deprecations.rb` | Gentle Deprecations: the hard part of maintaining a framework isn't adding the better name - it's the two years of not b... | | `graph_critic.rb` | The Graph Critic: reviews a plan's dependency structure BEFORE it runs, the way you'd review a class diagram. God tasks,... | | `graph_invariants.rb` | The Graph Invariants Prover: the reflection API makes promises - order respects edges, roots have no dependencies, depth... | | `graph_style.rb` | The Graph Style Guide: RuboCop for plans. Cops with thresholds run against any orchestrator's graph - depth, fan-in, orp... | | `graph_to_specs.rb` | Graph to Specs: the plan's structure dictates its test plan - roots need fixture cases, joins need one case per missing ... | | `haiku_agent.rb` | The three-line agent. Run me with no API key at all: | | `hill_chart.rb` | The Hill Chart: Basecamp's answer to "how's it going?" - work climbs the hill while it's still uncertain (queued, waitin... | +| `honest_doubles.rb` | Honest Doubles: every fake LLM in every agent test suite is lying a little - the question is whether anyone checks. The ... | +| `hostile_inputs.rb` | Hostile Inputs: a parser's real spec is what it does with input nobody intended. The journal's replay parses a file that... | | `impl_shootout.rb` | The Implementation Shootout: two candidates for the same capability, one eval set, and a verdict computed instead of vib... | | `incident_report.rb` | The Incident Report: a nightly batch dies at 3am. The on-call's first three questions - what ran? what broke? what do I ... | | `invariant_sentinel.rb` | The Invariant Sentinel: domain invariants checked after EVERY task, from a lifecycle hook. When a task leaves the world ... | @@ -57,19 +66,25 @@ not this file. | `latency_lab.rb` | The Latency Lab: 20 simulated LLM calls (200ms of IO each) executed through the orchestrator at different concurrency li... | | `live_dashboard.rb` | The Live Dashboard: lifecycle hooks publish events onto an Async::Queue; a consumer task IN THE SAME REACTOR renders the... | | `namespace_cartographer.rb` | The Namespace Cartographer: maps a gem's constant tree and audits every file against the constant Zeitwerk expects it to... | +| `onboarding_trail.rb` | The Onboarding Trail: a codebase is a place people live, and new teammates don't need a map of every pipe - they need a ... | | `one_file_api.rb` | The One-File API: an endpoint is a contract wearing HTTP. Declare the capability once and the rest is derived - the 422s... | | `perf_diff.rb` | The Perf Diff: run the plan before and after a change, diff per-task durations, and flag regressions - with the one qual... | | `perf_history.rb` | Perf History: last release's run left a journal; this release's run is compared against it. No synthetic baseline, no sa... | | `performance_detective.rb` | The Performance Detective: one task per Ruby file in lib/, fanned out through the orchestrator, each dissecting a file f... | | `plan_diagram.rb` | The Plan Diagrammer: any orchestrator's graph, emitted as Mermaid - paste it into a README, GitHub renders it, and the d... | +| `plan_dsl.rb` | The Plan DSL: Sinatra's whole argument was that an API is a user interface, and a user interface should read like what i... | +| `plan_flog.rb` | Plan Flog: flog gives every Ruby method a pain score; this gives every plan one. Fan-in hurts (joins hide coupling), dep... | | `plan_forest.rb` | The Plan Forest: your graph drawn as a forest - roots at the soil, leaves in the canopy, every task planted at its depth... | | `plan_fortune.rb` | The Plan Fortune Teller: reads your graph's palm - depth, fan-in, roots, breadth - and tells its fortune. Every fortune ... | | `plan_gantt.rb` | The Plan Gantt: lifecycle hooks timestamp every task, then the run is rendered as an ASCII timeline - where your wall cl... | +| `plan_kata.rb` | The Plan Kata: red, green, refactor - for a plan. The "tests" are assertions about the graph (one root, one leaf, labele... | | `plan_merge.rb` | The Plan Merge: base, ours, theirs - a three-way merge of plan wire formats. Independent changes combine; the same edge ... | | `plan_roundtrip.rb` | The Round Trip: serialize a plan's graph to JSON, rebuild a fresh orchestrator from the JSON, and prove the rebuilt topo... | | `plan_structural_diff.rb` | The Structural Diff: two versions of a plan's wire format, diffed as TOPOLOGY - tasks added and removed, edges rewired, ... | | `plan_tour.rb` | The Plan Tour: hand any orchestrator to the guide and it narrates the plan as prose - first this, then that, meanwhile t... | | `polite_form.rb` | The Polite Form: a contract usually speaks AFTER you fail - a 422, a stack of violations. This assistant makes it speak ... | +| `ports_and_adapters.rb` | Ports and Adapters: the domain is the part of your app that would survive a framework migration - IF you kept it clean. ... | +| `process_drill.rb` | The Process Drill: threads share a Mutex; PROCESSES share nothing but the file. The journal claims flock+fsync, which is... | | `projection_agreement.rb` | The Projection Agreement Prover: relation rules now render twice - the validator enforces them in Ruby, and to_json_sche... | | `quota_keeper.rb` | The Quota Keeper: the same 20 requests through two different laws. A concurrency ceiling ("3 in flight") models connecti... | | `readme_verifier.rb` | The README Verifier: every ruby code fence in the README is a promise. This extracts them all, syntax-checks each with P... | @@ -78,17 +93,22 @@ not this file. | `relation_diff.rb` | The Relation Diff: round 8's semver advisor classified declaration changes but had to shrug at rules - lambdas can't be ... | | `relation_prober.rb` | The Relation Prober: relation-typed rules are new, and new predicates deserve hostility. Each relation is probed with ed... | | `renga_circle.rb` | A renga circle: three poet agents compose a linked-verse poem, each verse responding to the one before it. The dependenc... | +| `require_cost.rb` | The Require Cost Report: `require` is a purchase - memory, objects, and boot time, paid again by every process you fork ... | | `resize_torture.rb` | The Resize Torture Test: a feature that changes a limiter's ceiling while fibers are waiting on it had better say exactl... | | `retry_budget.rb` | The Retry Budget: a retry storm is a self-inflicted DDoS - every job politely retrying 3x turns one outage into four. Re... | | `rule_prober.rb` | The Rule Prober: structured rules declare which fields they read - so now that claim can be AUDITED. For each rule, pert... | | `rule_shapes.rb` | Rule Shapes: the same policy - "express shipments need a customs code" - written three ways: a lambda, a structured chec... | | `schema_advisor.rb` | The Schema Advisor: give it a schema and a query log, get back the advisories a careful DBA would write - each rule its ... | +| `self_correcting_output.rb` | Self-Correcting Output: the pattern that makes LLM components shippable. The model's output is validated against the cap... | | `setup_doctor.rb` | The Setup Doctor: every onboarding wiki page is a bug. This runs the checks a README asks a new hire to do by hand - rub... | | `shared_rate_limit.rb` | The Shared Rate Limit: two plans run concurrently in one reactor, but the API key they share allows only 3 requests in f... | | `stampede_sim.rb` | The Stampede Simulator: twenty workers hit a hiccuping upstream, all fail at once, all retry. With jitter OFF they come ... | | `standup_digest.rb` | The Standup Digest: three collectors gather from the repo in parallel - recent commits, TODO debt, test suite shape - an... | | `state_machine.rb` | The Contract State Machine: each transition is a capability whose guard is not an if-statement but an enum predicate on ... | +| `telemetry_bus.rb` | The Telemetry Bus: lifecycle hooks are callbacks - one producer, one consumer, coupled at configuration time. A telemetr... | | `telephone_game.rb` | The telephone game: a rumor passes through five villagers, each of whom hears the previous version through the orchestra... | +| `tenant_shards.rb` | Tenant Shards: at scale, "the plan" becomes "the plan, per shard" - same pipeline, isolated blast radius. Each shard get... | +| `threads_drill.rb` | The Threads Drill: fibers are polite; threads are not. Everything in this gem that claims to be shared-safe gets hammere... | | `three_shapes.rb` | Three Shapes: the same six units of work arranged three ways - a chain, a star, and staged joins - then measured and cri... | | `throughput_knee.rb` | The Throughput Knee: sweep one limiter's ceiling from 1 to 8 against an upstream that quietly serializes above 4, and me... | | `ticket_screener.rb` | A HEY-style ticket screener: every inbound support ticket flows through screen -> categorize -> draft, all tickets in pa... | @@ -96,3 +116,4 @@ not this file. | `typed_pipeline.rb` | A typed ETL pipeline: extract -> transform -> load, each stage a capability with a declared contract, composed into one ... | | `variance_detective.rb` | The Variance Detective: ten journaled runs of the same plan, then a hunt for the task whose p90/p50 ratio betrays it. Av... | | `weekly_checkin.rb` | The Weekly Check-in: "what did you work on this week?" answered by the journal instead of by memory. Runs a few days of ... | +| `write_path_profile.rb` | The Write Path Profile: everyone's first instinct about a slow journal is "switch JSON libraries". Before holding that o... | diff --git a/examples/allocation_audit.rb b/examples/allocation_audit.rb new file mode 100644 index 0000000..65fb9f1 --- /dev/null +++ b/examples/allocation_audit.rb @@ -0,0 +1,96 @@ +# frozen_string_literal: true + +# The Allocation Audit: every object is a promissory note the GC +# collects on later. This audit counts exactly what each framework +# operation allocates (GC.stat's total_allocated_objects is an exact +# counter, not a sample) and where the GC actually runs during a +# plan. Latency spikes that "come from nowhere" come from here. +# +# bundle exec ruby examples/allocation_audit.rb +# +# Runs offline; counts are exact for this Ruby version. + +require_relative "../lib/agentic" + +Agentic.logger.level = :fatal + +def allocations(iterations = 100) + yield # warm: first call pays memoization, schema compilation, caches + GC.start + before = GC.stat(:total_allocated_objects) + iterations.times { yield } + (GC.stat(:total_allocated_objects) - before) / iterations +end + +def task_named(name) + Agentic::Task.new(description: name, agent_spec: {"name" => "w", "instructions" => "work"}) +end + +SPEC = Agentic::CapabilitySpecification.new( + name: "audit", description: "x", version: "1.0.0", + inputs: {mode: {type: "string", required: true, enum: %w[a b]}, + weight: {type: "number", required: true, min: 1, max: 100}}, + rules: {fits: {relation: :sum_lte, fields: [:weight], limit: 100}} +) +VALIDATOR = Agentic::CapabilityValidator.new(SPEC) + +def ten_task_orchestrator + orchestrator = Agentic::PlanOrchestrator.new(concurrency_limit: 4) + previous = nil + 10.times do |i| + task = task_named("t#{i}") + orchestrator.add_task(task, previous ? [previous] : [], agent: ->(_t) { :ok }) + previous = task + end + orchestrator +end + +puts "ALLOCATION AUDIT (objects per operation, exact via GC.stat)" +puts + +rows = { + "Task.new" => -> { task_named("x") }, + "validator: happy path" => -> { VALIDATOR.validate_inputs!(mode: "a", weight: 50) }, + "validator: rejection" => -> { + begin + VALIDATOR.validate_inputs!(mode: "z", weight: 500) + rescue Agentic::Errors::ValidationError + nil + end + }, + "graph snapshot (10 tasks)" => -> { ten_task_orchestrator.graph }, + "to_json_schema" => -> { SPEC.to_json_schema } +} + +counts = rows.transform_values { |op| allocations(&op) } +counts.each do |label, objects| + puts format(" %-28s %6d objects %s", label, objects, "#" * [objects / 50, 40].min) +end + +# The graph row includes building the orchestrator - separate the two +build_only = allocations { ten_task_orchestrator } +graph_only = counts["graph snapshot (10 tasks)"] - build_only +puts format(" %-28s %6d objects (snapshot alone, build subtracted)", " ...graph, isolated", graph_only) + +# --- where the GC actually fires during a plan --------------------------------- +orchestrator = ten_task_orchestrator +GC.start +gc_before = GC.count +allocated_before = GC.stat(:total_allocated_objects) +orchestrator.execute_plan +plan_allocations = GC.stat(:total_allocated_objects) - allocated_before +gc_runs = GC.count - gc_before + +puts +puts format(" a full 10-task plan allocates %d objects and triggered %d GC run(s).", plan_allocations, gc_runs) +puts +per_call = counts["validator: happy path"] +puts " reading the audit like a VM person: the happy-path validation" +puts " (#{per_call} objects) is what you multiply by requests-per-second -" +puts " #{per_call} x 1000 rps is #{per_call * 1000} promissory notes a second, and the GC" +puts " collects on schedule whether you budgeted or not. rejection costs" +puts " #{counts["validator: rejection"] / per_call}x the happy path in objects (exceptions carry backtraces;" +puts " error paths are allocation paths), and the graph snapshot's" +puts " #{graph_only} objects of dup+freeze buy the immutability every round-8" +puts " tool leans on - that's not waste, that's a purchase. allocation" +puts " isn't evil; UNBUDGETED allocation is. now there's a budget." diff --git a/examples/api_surface.rb b/examples/api_surface.rb new file mode 100644 index 0000000..df9e744 --- /dev/null +++ b/examples/api_surface.rb @@ -0,0 +1,66 @@ +# frozen_string_literal: true + +# The API Surface Census: your public API is not what you documented - +# it's every public method a user CAN call, because that's what semver +# binds you to. This census counts the whole surface, then +# cross-references 100 example programs to split it into the API +# people actually use and the accidental API nobody asked for but +# everyone can break themselves against. +# +# bundle exec ruby examples/api_surface.rb +# +# Runs offline; the examples directory is the usage corpus. + +require_relative "../lib/agentic" + +Agentic.logger.level = :fatal + +# Load everything so the census sees the whole surface +Zeitwerk::Registry.loaders.each(&:eager_load) if defined?(Zeitwerk::Registry) + +CORE = [ + Agentic::PlanOrchestrator, Agentic::Task, Agentic::ExecutionJournal, + Agentic::RateLimit, Agentic::CapabilitySpecification, Agentic::CapabilityValidator, + Agentic::CapabilityProvider, Agentic::TaskFailure, Agentic::TaskResult, + Agentic::PlanExecutionResult, Agentic::RelationRules +].freeze + +corpus = Dir[File.join(__dir__, "*.rb")].reject { |f| f.end_with?("api_surface.rb") } + .map { |f| File.read(f, encoding: "UTF-8") }.join("\n") + +puts "API SURFACE CENSUS (#{CORE.size} core classes vs #{Dir[File.join(__dir__, "*.rb")].size - 1} example programs)" +puts +puts format(" %-26s %-9s %-11s %s", "class", "surface", "exercised", "accidental (public, unused by any example)") + +total_surface = 0 +total_exercised = 0 +accidental_all = [] +CORE.each do |klass| + # Owner-checked: only methods this class itself defines count as ITS + # surface - inherited Object/Psych noise is someone else's ledger + methods = (klass.public_instance_methods(false) + + klass.singleton_class.public_instance_methods(false).select { |m| + klass.singleton_class.instance_method(m).owner == klass.singleton_class + }).uniq.reject { |m| m.to_s.start_with?("_") } + used, unused = methods.partition { |m| corpus.match?(/\.#{Regexp.escape(m.to_s.chomp("?").chomp("!"))}\b/) || corpus.include?(".#{m}") } + total_surface += methods.size + total_exercised += used.size + accidental_all.concat(unused.map { |m| "#{klass.name.split("::").last}##{m}" }) + puts format(" %-26s %-9d %-11d %s", + klass.name.split("::").last, methods.size, used.size, unused.take(3).join(", ")) +end + +puts +puts format(" total public surface: %d methods; %d (%.0f%%) exercised by the corpus.", + total_surface, total_exercised, total_exercised * 100.0 / total_surface) +puts +puts " reading the census like a steward: the exercised set is your REAL" +puts " API - 100 programs voted with their call sites, and every one of" +puts " those methods now carries a semver promise whether the docs say" +puts " so or not. the accidental set (#{accidental_all.size} methods) is surface you're" +puts " paying interest on without collecting rent: each is a thing a" +puts " user could couple to tomorrow, constraining refactors forever." +puts " the move isn't deletion - it's DECLARATION: mark them @api" +puts " private (or make them private) while nobody depends on them," +puts " because the day after somebody does, they're yours for a major" +puts " version. public-by-default is a loan; the census is the bill." diff --git a/examples/cancel_drill.rb b/examples/cancel_drill.rb index d1f649c..ef8a051 100644 --- a/examples/cancel_drill.rb +++ b/examples/cancel_drill.rb @@ -3,9 +3,10 @@ # The Cancel Drill: structured concurrency's core promise is that # cancellation is PROMPT - stop means stop, not "finish everything # and then agree you'd stopped". Three drills measure what each -# cancel path actually delivers: the surgical ones keep the promise; -# the plan-wide one, under a joined reactor, turns out to be -# bookkeeping wearing a stop sign. +# cancel path actually delivers. In round 10 this drill caught +# plan-wide cancel billing for every canceled task; the round-11 +# release fixed it, and this file is the acceptance test that keeps +# it fixed. # # bundle exec ruby examples/cancel_drill.rb # @@ -97,14 +98,17 @@ def states(orchestrator) wall = Process.clock_gettime(Process::CLOCK_MONOTONIC) - started end puts " drill 3 - cancel_plan at 30ms:" -puts " status flipped to :canceled by #{flip_ms}ms - and then the plan ran" -puts " #{(wall * 1000).round}ms anyway, with #{runs3.size}/6 agents executing (#{states(orchestrator3)})." +puts " status flipped to :canceled by #{flip_ms}ms, the plan returned in #{(wall * 1000).round}ms," +puts " and only #{runs3.size}/6 agents ever ran (#{states(orchestrator3)}) - the two that" +puts " were mid-flight when the order came. before round 11 this row" +puts " read \"301ms, 6/6 agents executing\": every task reported" +puts " :canceled while every agent ran to completion and billed." puts -puts " the drill's verdict: task-level cancel keeps the structured-" -puts " concurrency promise - stop is prompt, lanes are freed, unstarted" -puts " work stays unstarted. plan-level cancel under a joined reactor" -puts " is bookkeeping wearing a stop sign: every task reports :canceled" -puts " while every agent runs to completion and bills you, results" -puts " discarded. that's the worst trade available - full cost, zero" -puts " product. filed as the round-11 ask: cancel_plan must stop the" -puts " scheduler and the in-flight fibers, not just restate their status." +puts " the drill's verdict, updated: all three cancel paths now keep" +puts " the structured-concurrency promise. the round-11 fix stops the" +puts " fibers instead of the reactor handle - and does its bookkeeping" +puts " FIRST, because stopping a fiber frees its slot and synchronously" +puts " admits the next waiter, which must already read as canceled in" +puts " that instant. cancellation is a race against your own scheduler;" +puts " this drill is the finish-line camera, and it stays in the repo" +puts " so the race gets re-run on every change." diff --git a/examples/capacity_planner.rb b/examples/capacity_planner.rb new file mode 100644 index 0000000..39023b5 --- /dev/null +++ b/examples/capacity_planner.rb @@ -0,0 +1,83 @@ +# frozen_string_literal: true + +# The Capacity Planner: "how many workers do we need?" is not a +# feeling, it's Little's Law - L = lambda x W. The journal already +# holds W (task durations, as percentiles across runs); give the +# planner your target arrival rate and it computes the lanes, then +# checks the answer against every limit you've configured - because +# the binding constraint is usually not the one in the meeting. +# +# bundle exec ruby examples/capacity_planner.rb +# +# Runs offline; history is seeded into a journal first. + +require_relative "../lib/agentic" +require "tmpdir" + +Agentic.logger.level = :fatal + +# --- build history: 30 journaled runs of the pipeline --------------------------- +JOURNAL = File.join(Dir.tmpdir, "agentic_capacity.journal.jsonl") +File.delete(JOURNAL) if File.exist?(JOURNAL) +journal = Agentic::ExecutionJournal.new(path: JOURNAL) +rng = Random.new(1123) + +30.times do + {"fetch:ticket" => 0.08, "classify" => 0.35, "draft:reply" => 0.9}.each do |name, base| + # log-normal-ish: mostly base, occasionally 2-3x - like real latency + duration = base * (0.8 + rng.rand(0.4)) * ((rng.rand < 0.12) ? (2 + rng.rand) : 1) + journal.record(:task_succeeded, task_id: name, description: name, duration: duration.round(4), output: nil) + end +end + +state = Agentic::ExecutionJournal.replay(path: JOURNAL) + +TARGET_PER_MINUTE = 120 # tickets per minute at peak +CONFIGURED = { + "orchestrator concurrency_limit" => 8, + "provider quota (windowed)" => "90/min", + "connection pool ceiling" => 12 +}.freeze + +puts "CAPACITY PLANNER (target: #{TARGET_PER_MINUTE} tickets/min at peak)" +puts +puts format(" %-16s %-10s %-10s %-22s %s", "task", "p50", "p95", "lanes needed (p50/p95)", "") + +lambda_per_sec = TARGET_PER_MINUTE / 60.0 +total_p95_lanes = 0 +state.duration_samples.keys.each do |task| + p50 = state.duration_percentile(task, 50) + p95 = state.duration_percentile(task, 95) + # Little's Law: concurrent-in-service L = arrival rate x service time + lanes_p50 = (lambda_per_sec * p50).ceil + lanes_p95 = (lambda_per_sec * p95).ceil + total_p95_lanes += lanes_p95 + puts format(" %-16s %6.0fms %6.0fms %2d / %-2d %s", + task, p50 * 1000, p95 * 1000, lanes_p50, lanes_p95, "#" * lanes_p95) +end + +puts +puts " plan for p95, not p50: capacity sized to the median is capacity" +puts " that queues every time latency has a bad day, and latency has a" +puts " bad day 1 run in 8 in this journal. total lanes at p95: #{total_p95_lanes}." +puts + +# --- check the plan against every configured limit ------------------------------- +puts " the plan vs. what's actually configured:" +puts format(" %-36s %-8s %s", "limit", "have", "verdict at #{TARGET_PER_MINUTE}/min") +verdicts = { + "orchestrator concurrency_limit" => (total_p95_lanes <= 8) ? "holds" : "BINDS FIRST - raise to #{total_p95_lanes}", + "provider quota (windowed)" => (TARGET_PER_MINUTE <= 90) ? "holds" : "BINDS - 90/min < #{TARGET_PER_MINUTE}/min arrivals, queues grow without bound", + "connection pool ceiling" => (total_p95_lanes <= 12) ? "holds" : "BINDS - #{total_p95_lanes} lanes want connections" +} +CONFIGURED.each do |name, have| + puts format(" %-36s %-8s %s", name, have, verdicts[name]) +end + +puts +puts " the meeting was about to argue concurrency_limit; the math says" +puts " the provider QUOTA binds first - 90/min against 120/min arrivals" +puts " isn't a slowdown, it's an unbounded queue (utilization > 1 has" +puts " no steady state). fix the quota; the #{total_p95_lanes} lanes and the pool" +puts " already hold. Little's Law plus a journal is a capacity plan;" +puts " a dashboard plus a feeling is a postmortem." diff --git a/examples/confident_pipeline.rb b/examples/confident_pipeline.rb new file mode 100644 index 0000000..5e9b76b --- /dev/null +++ b/examples/confident_pipeline.rb @@ -0,0 +1,104 @@ +# frozen_string_literal: true + +# The Confident Pipeline: timid code checks nil at every step because +# it trusts nothing, including itself. Confident code validates once, +# at the barricade, and then speaks in declarative sentences. Same +# pipeline, written both ways - and then both are made to face the +# same malformed input, so the difference is behavior, not taste. +# +# bundle exec ruby examples/confident_pipeline.rb +# +# Runs offline; the conditional count is computed from this file. + +require_relative "../lib/agentic" + +Agentic.logger.level = :fatal + +# --- the timid version ---------------------------------------------------------- +# Every method distrusts its caller, so every method re-litigates +# reality. Read it aloud: it's all subordinate clauses. +module Timid + def self.process(order) + return nil if order.nil? + + items = order[:items] + return nil if items.nil? || !items.is_a?(Array) || items.empty? + + total = 0 + items.each do |item| + next if item.nil? + + price = item[:price_cents] + qty = item[:qty] || 1 + total += price * qty if !price.nil? && price.is_a?(Numeric) && price >= 0 + end + + email = order[:email] + receipt = if email && !email.empty? + "receipt to #{email}" + end + + {total_cents: total, delivery: receipt || "no receipt"} + end +end + +# --- the confident version ------------------------------------------------------- +# One barricade at the boundary. Inside it, every sentence is +# indicative mood: the data IS shaped; the contract said so. +ORDER_CONTRACT = Agentic::CapabilitySpecification.new( + name: "process_order", description: "Price an order", version: "1.0.0", + inputs: { + items: {type: "array", required: true, non_empty: true}, + email: {type: "string", required: true, non_empty: true} + }, + outputs: {total_cents: {type: "number", required: true}, delivery: {type: "string", required: true}} +) +BARRICADE = Agentic::CapabilityValidator.new(ORDER_CONTRACT) + +module Confident + def self.process(order) + BARRICADE.validate_inputs!(order) + total = order[:items].sum { |item| item.fetch(:price_cents) * item.fetch(:qty, 1) } + output = {total_cents: total, delivery: "receipt to #{order[:email]}"} + BARRICADE.validate_outputs!(output) + output + end +end + +GOOD = {items: [{price_cents: 1200, qty: 2}, {price_cents: 350}], email: "a@b.co"}.freeze +BAD = {items: [{price_cents: nil, qty: 3}], email: ""}.freeze + +puts "THE CONFIDENT PIPELINE (same job, two postures)" +puts + +source = File.read(__FILE__, encoding: "UTF-8") +timid_src = source[/module Timid.*?\n end\nend/m] +confident_src = source[/module Confident.*?\n end\nend/m] +count = ->(src) { src.scan(/\b(?:if|unless|return nil|next if|\|\|)\s/).size + src.scan("&&").size } + +puts format(" %-12s %2d conditionals, %2d lines", "timid:", count.call(timid_src), timid_src.lines.size) +puts format(" %-12s %2d conditionals, %2d lines", "confident:", count.call(confident_src), confident_src.lines.size) +puts + +puts " good input:" +puts " timid: #{Timid.process(GOOD)}" +puts " confident: #{Confident.process(GOOD)}" +puts +puts " malformed input (an item with a nil price, email: \"\"):" +puts " timid: #{Timid.process(BAD).inspect}" +begin + Confident.process(BAD) +rescue Agentic::Errors::ValidationError => e + puts " confident: raises ValidationError - #{e.violations.keys.join(", ")} rejected AT THE DOOR" +end +puts +puts " look at what the timid version returned for garbage: a polite," +puts " well-formed, WRONG answer - zero dollars, \"no receipt\", no error." +puts " that nil-tolerance didn't handle the bad input, it LAUNDERED it;" +puts " some downstream ledger now owes a customer an explanation. the" +puts " confident version has one conditional posture: a barricade at" +puts " each door (inputs validated once, outputs too - honesty is also" +puts " a promise about what you return). inside, every line is a" +puts " declarative sentence about data that is KNOWN to be shaped." +puts " confidence isn't optimism - it's pushing all the doubt to the" +puts " boundary, where it can say no out loud." diff --git a/examples/contract_cop.rb b/examples/contract_cop.rb new file mode 100644 index 0000000..d97c578 --- /dev/null +++ b/examples/contract_cop.rb @@ -0,0 +1,101 @@ +# frozen_string_literal: true + +# The Contract Cop: RuboCop for capability specs. Contracts are the +# most-read documents in this framework - six tools consume them - +# so they deserve a style guide with teeth: named cops, an offense +# report, and autocorrection for everything mechanical. Style is not +# vanity; it's the cost of reading, paid down in advance. +# +# bundle exec ruby examples/contract_cop.rb +# +# Runs offline; a messy contract walks in, autocorrect walks it out. + +require_relative "../lib/agentic" + +# The defendant: a contract written at 6pm on a Friday +MESSY = { + name: "QuoteShipping", description: "", version: "1.0.0", + inputs: { + Mode: {type: "string", required: true, enum: %w[sea air road]}, + weightKg: {type: "number", required: true, min: 0, max: 5_000}, + customs_code: {type: "string"}, + ref: {}, + a: {type: "string", required: true}, + b: {type: "string", required: true}, + c: {type: "string", required: true}, + d: {type: "string", required: true}, + e: {type: "string", required: true} + }, + rules: { + check1: {fields: [:weightKg], check: ->(i) { i[:weightKg] < 5_000 }} + } +}.freeze + +# Each cop: name, check (spec-hash in, offenses out), correctable? +COPS = { + "Naming/SnakeCaseName" => { + check: ->(s) { (s[:name] =~ /\A[a-z][a-z0-9_]*\z/) ? [] : ["capability name '#{s[:name]}' is not snake_case"] }, + correct: ->(s) { s[:name] = s[:name].gsub(/([a-z])([A-Z])/, '\1_\2').downcase } + }, + "Naming/SnakeCaseFields" => { + check: ->(s) { s[:inputs].keys.reject { |k| k =~ /\A[a-z][a-z0-9_]*\z/ }.map { |k| "input :#{k} is not snake_case" } }, + correct: ->(s) { s[:inputs].transform_keys! { |k| k.to_s.gsub(/([a-z])([A-Z])/, '\1_\2').downcase.to_sym } } + }, + "Documentation/Description" => { + check: ->(s) { s[:description].to_s.empty? ? ["capability has no description"] : [] }, + correct: nil # a human must actually say what it does + }, + "Style/EnumOrder" => { + check: ->(s) { s[:inputs].select { |_, d| d[:enum] && d[:enum] != d[:enum].sort }.map { |k, _| "input :#{k} enum is not sorted" } }, + correct: ->(s) { s[:inputs].each_value { |d| d[:enum] = d[:enum].sort if d[:enum] } } + }, + "Lint/UntypedField" => { + check: ->(s) { s[:inputs].select { |_, d| d[:type].nil? }.map { |k, _| "input :#{k} has no type (and won't project into schemas)" } }, + correct: nil # guessing a type is how bugs get typed + }, + "Lint/OpaqueRuleWithoutMessage" => { + check: ->(s) { + s[:rules].select { |_, d| d.respond_to?(:call) || (d[:check] && !d[:message]) } + .map { |k, _| "rule :#{k} is opaque AND messageless - violations will say nothing" } + }, + correct: nil # the message is the author's testimony; can't forge it + }, + "Metrics/RequiredInputCount" => { + check: ->(s) { + required = s[:inputs].count { |_, d| d[:required] } + (required > 5) ? ["#{required} required inputs (max 5) - is this one capability or three?"] : [] + }, + correct: nil + } +}.freeze + +def inspect_spec(spec_hash) + COPS.flat_map { |cop, definition| definition[:check].call(spec_hash).map { |offense| [cop, offense] } } +end + +puts "CONTRACT COP (#{COPS.size} cops on the beat)" +puts +offenses = inspect_spec(MESSY) +puts " inspecting quote_shipping... #{offenses.size} offenses:" +offenses.each { |cop, offense| puts format(" %-33s %s", cop, offense) } +puts + +# --- autocorrect what's mechanical ---------------------------------------------- +corrected = {name: MESSY[:name].dup, description: MESSY[:description].dup, version: MESSY[:version], + inputs: MESSY[:inputs].transform_values(&:dup).dup, rules: MESSY[:rules].dup} +corrected[:inputs].each_value { |d| d[:enum] = d[:enum].dup if d[:enum] } +COPS.each_value { |definition| definition[:correct]&.call(corrected) } + +remaining = inspect_spec(corrected) +puts " after autocorrect (#{offenses.size - remaining.size} offenses fixed mechanically):" +remaining.each { |cop, offense| puts format(" %-33s %s", cop, offense) } +puts +puts " what autocorrect fixed, it fixed safely: names snake_cased," +puts " enums sorted - transformations with exactly one right answer." +puts " what remains is the honest residue: a missing description" +puts " (only the author knows what it does), an untyped field" +puts " (guessing types is how bugs get typed), an opaque messageless" +puts " rule, and seven required inputs' worth of scope creep. a linter's" +puts " job splits exactly there - automate the mechanical, and make" +puts " the judgment calls impossible to not-see. style is applied" +puts " empathy for the next reader, and contracts have six readers." diff --git a/examples/event_prof.rb b/examples/event_prof.rb new file mode 100644 index 0000000..367687f --- /dev/null +++ b/examples/event_prof.rb @@ -0,0 +1,72 @@ +# frozen_string_literal: true + +# EventProf for Plans: TestProf taught test suites to answer "where +# does the time GO?" by group, not by file. Same question for plans: +# tag every task by its kind (llm:, db:, render:), collect durations +# from the lifecycle hooks, and report task-seconds by tag - plus the +# number nobody computes: how much of that time ran in parallel, and +# how much of the wall clock one tag owns hostage. +# +# bundle exec ruby examples/event_prof.rb +# +# Runs offline; durations are scripted, accounting is real. + +require_relative "../lib/agentic" + +Agentic.logger.level = :fatal + +WORK = { + "db:fetch_users" => 0.03, "db:fetch_orders" => 0.04, "db:fetch_stock" => 0.03, + "llm:summarize" => 0.22, "llm:classify" => 0.18, "llm:draft" => 0.25, + "render:header" => 0.01, "render:body" => 0.02, "render:pdf" => 0.05 +}.freeze + +samples = [] +hooks = { + after_task_success: ->(task_id:, task:, result:, duration:) { + samples << [task.description, duration] + } +} + +orchestrator = Agentic::PlanOrchestrator.new(concurrency_limit: 3, lifecycle_hooks: hooks) +tasks = WORK.to_h { |name, cost| [name, Agentic::Task.new(description: name, agent_spec: {"name" => name, "instructions" => "w"})] } + +# db feeds llm feeds render - three stages, three lanes +db, llm, render = %w[db llm render].map { |prefix| tasks.select { |n, _| n.start_with?(prefix) }.values } +db.each { |t| orchestrator.add_task(t, agent: ->(task) { sleep(WORK[task.description]) }) } +llm.each { |t| orchestrator.add_task(t, db, agent: ->(task) { sleep(WORK[task.description]) }) } +render.each { |t| orchestrator.add_task(t, llm, agent: ->(task) { sleep(WORK[task.description]) }) } + +wall_start = Process.clock_gettime(Process::CLOCK_MONOTONIC) +orchestrator.execute_plan +wall = Process.clock_gettime(Process::CLOCK_MONOTONIC) - wall_start + +# --- the profile ---------------------------------------------------------------- +by_tag = samples.group_by { |name, _| name[/\A\w+/] } + .transform_values { |rows| {seconds: rows.sum { |_, d| d }, count: rows.size, worst: rows.max_by { |_, d| d }} } +task_seconds = samples.sum { |_, d| d } + +puts "EVENT PROF (task-seconds by tag; wall clock #{(wall * 1000).round}ms, 3 lanes)" +puts +puts format(" %-8s %-10s %-8s %-8s %s", "tag", "seconds", "share", "tasks", "worst offender") +by_tag.sort_by { |_, v| -v[:seconds] }.each do |tag, stats| + share = stats[:seconds] / task_seconds * 100 + puts format(" %-8s %6.0fms %5.1f%% %-8d %s (%.0fms) %s", + tag, stats[:seconds] * 1000, share, stats[:count], + stats[:worst][0], stats[:worst][1] * 1000, "#" * (share / 3).round) +end + +parallelism = task_seconds / wall +puts +puts format(" task-seconds: %.0fms across %.0fms of wall = %.1fx effective parallelism", task_seconds * 1000, wall * 1000, parallelism) +puts +llm_share = by_tag["llm"][:seconds] / task_seconds * 100 +puts " the TestProf move is reading the SHARE column before touching any" +puts format(" code: llm owns %.0f%% of all task-seconds, so a 20%% win there is", llm_share) +puts " worth more than deleting the entire render stage - optimizing" +puts " db: or render: is polishing doorknobs on a burning building." +puts format(" and the parallelism line is the second lesson: %.1fx on 3 lanes", parallelism) +puts " means the stage barriers are eating part of the overlap -" +puts " llm tasks can't start until ALL db tasks finish. profile by" +puts " group, fix the biggest group, re-profile. boring, effective," +puts " and the hooks made it fifteen lines." diff --git a/examples/gentle_deprecations.rb b/examples/gentle_deprecations.rb new file mode 100644 index 0000000..1e32daf --- /dev/null +++ b/examples/gentle_deprecations.rb @@ -0,0 +1,108 @@ +# frozen_string_literal: true + +# Gentle Deprecations: the hard part of maintaining a framework isn't +# adding the better name - it's the two years of not breaking anyone +# who used the old one. This shims a renamed contract field through +# three release phases: translate-and-warn (once per call site, with +# the caller named), count everything for the migration report, and +# a strict mode that turns warnings into CI failures ON YOUR schedule, +# not the users'. +# +# bundle exec ruby examples/gentle_deprecations.rb +# +# Runs offline; three "apps" call the API from three code sites. + +require_relative "../lib/agentic" + +Agentic.logger.level = :fatal + +# v2 renamed weight: -> weight_kg:. The contract only knows the new world. +CONTRACT = Agentic::CapabilitySpecification.new( + name: "quote_shipping", description: "Quote a shipment", version: "2.0.0", + inputs: { + mode: {type: "string", required: true, enum: %w[air sea]}, + weight_kg: {type: "number", required: true, min: 1} + }, + outputs: {price_cents: {type: "number", required: true}} +) + +# The shim: old names translated at the door, warned once per call +# site, tallied for the report. Deprecation is DATA about your users. +class DeprecationShim + RENAMES = {weight: :weight_kg}.freeze + + attr_reader :hits + + def initialize(strict: false) + @strict = strict + @warned = {} + @hits = Hash.new(0) + end + + def translate(inputs) + RENAMES.each do |old_name, new_name| + next unless inputs.key?(old_name) + + # The interesting frame is the USER's: skip the shim and the API + # boundary, blame the first frame that belongs to neither + site_location = caller_locations.find { |l| !l.label.include?("translate") && !%w[each quote].include?(l.label) } + site = "#{site_location.label} (#{site_location.to_s[/[^\/]+:\d+/]})" + @hits["#{old_name} at #{site}"] += 1 + if @strict + raise ArgumentError, "DEPRECATED input :#{old_name} (use :#{new_name}) - strict mode refuses it" + end + unless @warned["#{old_name}-#{site}"] + @warned["#{old_name}-#{site}"] = true + warn " DEPRECATION: :#{old_name} is now :#{new_name} (called from #{site}; this warning shows once per site)" + end + inputs = inputs.dup + inputs[new_name] = inputs.delete(old_name) + end + inputs + end +end + +SHIM = DeprecationShim.new +VALIDATOR = Agentic::CapabilityValidator.new(CONTRACT) + +def quote(inputs) + inputs = SHIM.translate(inputs) + VALIDATOR.validate_inputs!(inputs) + {price_cents: (inputs[:weight_kg] * ((inputs[:mode] == "air") ? 9 : 2) * 100).round} +end + +# Three call sites: one migrated, two still on the old name +def legacy_billing_job = quote(mode: "air", weight: 12) + +def legacy_admin_panel = quote(mode: "sea", weight: 400) + +def migrated_checkout = quote(mode: "air", weight_kg: 3) + +puts "GENTLE DEPRECATIONS (rename shipped; nobody broken; everybody counted)" +puts +3.times { legacy_billing_job } +2.times { legacy_admin_panel } +4.times { migrated_checkout } + +puts +puts " the migration report (deprecation is data about your users):" +SHIM.hits.each { |site, count| puts format(" %-46s %d call(s)", site, count) } +puts " migrated call sites warn nothing and appear nowhere." +puts + +# Release N+2: strict mode - the same shim becomes the enforcement +strict = DeprecationShim.new(strict: true) +begin + strict.translate(mode: "air", weight: 12) +rescue ArgumentError => e + puts " strict mode (release N+2, or CI today): #{e.message}" +end +puts +puts " the choreography, straight from the Rails playbook: release N" +puts " adds the new name and the shim - old code runs, warns once per" +puts " call site (per-site, or your logs become the outage), and the" +puts " tally tells you exactly who still needs a PR. release N+1 you" +puts " chase the tally to zero. release N+2 flips strict and deletes" +puts " the shim on YOUR schedule - because the deadline was enforced" +puts " by CI failures in the laggards' builds, not by breaking their" +puts " production. renames are cheap; broken trust compounds." diff --git a/examples/honest_doubles.rb b/examples/honest_doubles.rb new file mode 100644 index 0000000..7011b5e --- /dev/null +++ b/examples/honest_doubles.rb @@ -0,0 +1,101 @@ +# frozen_string_literal: true + +# Honest Doubles: every fake LLM in every agent test suite is lying a +# little - the question is whether anyone checks. The discipline: +# (1) don't mock what you don't own - wrap the vendor in an adapter +# whose interface YOU define; (2) verify every double against that +# interface (methods AND arity), so a rename breaks the test suite +# loudly instead of letting a thousand fakes drift into fiction. +# +# bundle exec ruby examples/honest_doubles.rb +# +# Runs offline; one double is honest, one drifted. Guess which passes. + +require_relative "../lib/agentic" + +Agentic.logger.level = :fatal + +# --- the owned boundary ---------------------------------------------------------- +# We do NOT stub Agentic::LlmClient (we don't own it; its interface +# can change under us at gem-update speed). We define OUR port: +class CompletionPort + # The whole vendor surface we permit ourselves to use, in one place + def complete(prompt, max_tokens:) + raise NotImplementedError + end +end + +# The real adapter would wrap Agentic::LlmClient. For tests, doubles: +class HonestDouble < CompletionPort + def initialize(scripted) + @scripted = scripted + end + + def complete(prompt, max_tokens:) + @scripted.fetch(prompt[/\w+/]) + end +end + +# This one was written against LAST QUARTER's port and nobody noticed +# the port grew a keyword since - classic double drift +class DriftedDouble + def complete(prompt) + "sure, whatever you say" + end +end + +# --- the verifier: doubles must match the port they claim to be ----------------- +def verify_double!(double, port) + port_methods = port.public_instance_methods(false) + port_methods.each do |name| + unless double.respond_to?(name) + raise ArgumentError, "double #{double.class} is missing ##{name}" + end + + expected = port.instance_method(name).parameters + actual = double.method(name).parameters + # Compare shapes: required/optional/keyword names must line up + if expected.map { |kind, n| [kind, n] } != actual.map { |kind, n| [kind, n] } + raise ArgumentError, "double #{double.class}##{name} has drifted: " \ + "port takes #{expected.inspect}, double takes #{actual.inspect}" + end + end +end + +# --- a consumer under test ------------------------------------------------------- +def triage(port, ticket) + label = port.complete("classify: #{ticket}", max_tokens: 5) + {ticket: ticket, label: label} +end + +puts "HONEST DOUBLES (verify the fake against the port, every time)" +puts + +honest = HonestDouble.new("classify" => "billing") +verify_double!(honest, CompletionPort) +puts " honest double: verified against CompletionPort - method AND arity match" +result = triage(honest, "I was charged twice") +puts " triage under test: #{result.inspect}" +puts + +drifted = DriftedDouble.new +begin + verify_double!(drifted, CompletionPort) + puts " drifted double: verified?! the verifier has no teeth" + exit(1) +rescue ArgumentError => e + puts " drifted double: REJECTED before any test ran -" + puts " #{e.message}" +end +puts +puts " without the verifier, the drifted double PASSES every test you" +puts " write with it - `complete` responds, strings come back, green" +puts " everywhere - while the real adapter takes max_tokens: and would" +puts " raise ArgumentError on the very first production call. that's" +puts " the treachery of unverified fakes: they don't fail, they VOUCH." +puts " the two rules, cheap to follow: own the boundary (one port class" +puts " names everything you use from the vendor - the census says the" +puts " smaller that surface, the better), and verify every double" +puts " against it in the double's own definition, so interface drift" +puts " breaks the suite at load time, not the demo. your tests are" +puts " only as honest as their most casual fake." diff --git a/examples/hostile_inputs.rb b/examples/hostile_inputs.rb new file mode 100644 index 0000000..238b8a6 --- /dev/null +++ b/examples/hostile_inputs.rb @@ -0,0 +1,69 @@ +# frozen_string_literal: true + +# Hostile Inputs: a parser's real spec is what it does with input +# nobody intended. The journal's replay parses a file that - by the +# journal's own reason for existing - may end mid-write. This probe +# feeds replay the whole rogues' gallery: torn tails, binary garbage, +# giant lines, wrong-shaped JSON. The verdict on the torn tail is the +# one that matters, and today it draws blood. Exit 1 by design. +# +# bundle exec ruby examples/hostile_inputs.rb +# +# Runs offline; exits 1 until torn-tail recovery ships. + +require_relative "../lib/agentic" +require "tmpdir" +require "json" + +GOOD = %({"event":"task_succeeded","task_id":"t1","description":"t1","duration":0.1,"output":"ok"}) + +def replay_verdict(lines) + path = File.join(Dir.tmpdir, "agentic_hostile.jsonl") + File.write(path, lines.join("\n")) + state = Agentic::ExecutionJournal.replay(path: path) + [:recovered, state.completed_task_ids.size] +rescue => e + [:crashed, e.class.to_s] +end + +PROBES = { + "clean file (control)" => [GOOD, GOOD.sub("t1", "t2")], + "torn tail (crash mid-write)" => [GOOD, %({"event":"task_succ)], + "binary garbage line" => [GOOD, "\x00\x01\xFFnot json at all"], + "empty + whitespace lines" => [GOOD, "", " ", GOOD.sub("t1", "t2")], + "8MB single line" => [GOOD, %({"event":"task_succeeded","task_id":"big","description":"big","duration":0.1,"output":"#{"x" * 8_000_000}"})], + "valid JSON, wrong shape" => [GOOD, %({"event":"task_succeeded","task_id":42,"duration":"fast"})], + "unknown event type" => [GOOD, %({"event":"solar_flare","task_id":"t9"})], + "duplicate success lines" => [GOOD, GOOD] +}.freeze + +puts "HOSTILE INPUTS (#{PROBES.size} probes against ExecutionJournal.replay)" +puts +blood = [] +PROBES.each do |name, lines| + verdict, detail = replay_verdict(lines) + ok = verdict == :recovered + blood << name unless ok + puts format(" %-30s %s", name, + ok ? "recovered (#{detail} task(s) salvaged)" : "CRASHED: #{detail} - ALL recovery denied") +end + +puts +if blood.empty? + puts " every probe salvaged what was salvageable. the tail is tolerated." +else + puts " #{blood.size} probe(s) drew blood: #{blood.join("; ")}." + puts + puts " the torn tail is the indefensible one. a journal exists FOR the" + puts " crash - fsync guarantees completed lines survive, but the line" + puts " being written AT the crash may land torn, and that is the exact" + puts " file every real recovery will read. today one torn byte at the" + puts " tail throws JSON::ParserError past every rescue that says" + puts " ValidationError, and 100% of the events that WERE durable" + puts " become unreachable. nokogiri's whole life is this lesson:" + puts " parsers meet real input, and real input is damaged. filed as" + puts " the round-13 ask: replay must salvage every whole line and" + puts " report (not raise on) a torn tail - recovery tools don't get" + puts " to be the second thing that fails. exit 1 until." +end +exit(blood.empty? ? 0 : 1) diff --git a/examples/onboarding_trail.rb b/examples/onboarding_trail.rb new file mode 100644 index 0000000..9fe189f --- /dev/null +++ b/examples/onboarding_trail.rb @@ -0,0 +1,71 @@ +# frozen_string_literal: true + +# The Onboarding Trail: a codebase is a place people live, and new +# teammates don't need a map of every pipe - they need a TOUR: which +# room to enter first, and why each room makes sense given the ones +# you've seen. This computes the tour from the code itself: scan who +# mentions whom, then order the rooms so no stop assumes a concept +# you haven't met yet. +# +# bundle exec ruby examples/onboarding_trail.rb +# +# Runs offline; the trail is derived, not curated (mostly). + +require_relative "../lib/agentic" + +LIB = File.expand_path("../lib/agentic", __dir__) + +# What each room is FOR - the one sentence a tour guide adds that a +# dependency graph can't +ROOM_NOTES = { + "task_failure" => "how this house talks about things going wrong (failure as data)", + "task_result" => "the envelope every outcome arrives in", + "task" => "the unit of work: lifecycle, payloads, needs", + "rate_limit" => "the shared front door: ceilings, windows, resize", + "execution_journal" => "the house's memory: fsynced, replayable, per-shard", + "relation_rules" => "predicates as data - rules tools can read", + "capability_specification" => "contracts: declared inputs, outputs, rules", + "capability_validator" => "the barricade that enforces the contracts", + "plan_orchestrator" => "the living room where everything meets: scheduling, hooks, the graph" +}.freeze + +# Who mentions whom, from the source itself +files = ROOM_NOTES.keys.to_h do |name| + source = File.read(File.join(LIB, "#{name}.rb"), encoding: "UTF-8") + constants = source.scan(/\b(?:Agentic::)?([A-Z][A-Za-z]+)\b/).flatten.uniq + mentioned = ROOM_NOTES.keys.select { |other| + other != name && constants.include?(other.split("_").map(&:capitalize).join) + } + [name, {mentions: mentioned, lines: source.lines.size}] +end + +# The trail: repeatedly visit the room with the fewest unmet mentions +trail = [] +until trail.size == files.size + next_room = files.keys.reject { |f| trail.include?(f) } + .min_by { |f| [(files[f][:mentions] - trail).size, files[f][:lines]] } + trail << next_room +end + +puts "THE ONBOARDING TRAIL (computed from who-mentions-whom)" +puts +puts " day one, in order - no room assumes one you haven't seen:" +puts +trail.each_with_index do |room, index| + unmet = files[room][:mentions] - trail[0..index] + puts format(" %d. %-26s %4d lines %s", index + 1, room, files[room][:lines], ROOM_NOTES[room]) + puts format(" %s", "(mentions #{files[room][:mentions].join(", ")})") if files[room][:mentions].any? + puts " WARNING: tour visits this before #{unmet.join(", ")}" if unmet.any? +end + +puts +puts " why a trail instead of a map: a map answers \"where is\" and" +puts " nobody's first question is where - it's \"what should I read" +puts " FIRST so the rest makes sense?\" the ordering came from the" +puts " code (fewest unmet concepts next), and the one-line room notes" +puts " came from a human, which is the correct split: structure is" +puts " derivable, PURPOSE isn't. notice the trail starts with failure -" +puts " this house talks about failure before it talks about work, and" +puts " a new teammate who learns that on day one has learned the" +puts " house's values, not just its layout. codebases are places" +puts " people live; give the new roommate a tour, not a blueprint." diff --git a/examples/plan_dsl.rb b/examples/plan_dsl.rb new file mode 100644 index 0000000..600a128 --- /dev/null +++ b/examples/plan_dsl.rb @@ -0,0 +1,92 @@ +# frozen_string_literal: true + +# The Plan DSL: Sinatra's whole argument was that an API is a user +# interface, and a user interface should read like what it means. +# The orchestrator's API is honest but administrative - ids, task +# objects, add_task bookkeeping. Thirty lines of DSL later, a plan +# reads like a plan. No engine changes: sugar OVER the API, never +# reaching into it. +# +# bundle exec ruby examples/plan_dsl.rb +# +# Runs offline; the DSL builds a real orchestrator underneath. + +require_relative "../lib/agentic" + +Agentic.logger.level = :fatal + +# The whole DSL. Symbols in, wiring out; the block IS the agent. +module Plan + def self.define(&block) + builder = Builder.new + builder.instance_eval(&block) + builder + end + + class Builder + attr_reader :orchestrator + + def initialize + @orchestrator = Agentic::PlanOrchestrator.new + @tasks = {} + end + + def step(name, after: [], needs: nil, &work) + task = Agentic::Task.new(description: name.to_s, agent_spec: {"name" => name.to_s, "instructions" => name.to_s}) + @tasks[name] = task + deps = Array(after).map { |n| @tasks.fetch(n) } + named = needs&.transform_values { |n| @tasks.fetch(n) } + @orchestrator.add_task(task, deps, needs: named, agent: ->(t) { work&.call(t) }) + self + end + + def run + @orchestrator.execute_plan + end + + def output_of(name, result) + result.task_result(@tasks.fetch(name).id).output + end + end +end + +# --- a plan that reads like a plan ---------------------------------------------- +plan = Plan.define do + step :fetch_orders do + [{id: 1, total: 120}, {id: 2, total: 80}] + end + + step :fetch_refunds do + [{order_id: 2, amount: 80}] + end + + step :ledger, needs: {orders: :fetch_orders, refunds: :fetch_refunds} do |t| + t.needs[:orders].sum { |o| o[:total] } - t.needs[:refunds].sum { |r| r[:amount] } + end + + step :report, after: :ledger do |t| + "net revenue: $#{t.previous_output}" + end +end + +result = plan.run +puts "THE PLAN DSL (thirty lines of sugar over the real API)" +puts +puts " #{plan.output_of(:report, result)}" +puts + +graph = plan.orchestrator.graph +puts " and it's all real underneath: #{graph[:tasks].size} tasks, labeled edges" +puts " (#{graph[:edges].filter_map { |e| e[:label] }.join(", ")}), same graph every round-5-to-11 tool consumes." +puts +puts " what the sugar buys: names instead of ids (symbols resolve to" +puts " tasks at definition time, so a typo'd :fetch_order fails at" +puts " DEFINE, not at run); the block IS the agent (the work sits" +puts " inside the step that owns it); and after:/needs: read as" +puts " English. what the sugar refuses: reaching into the engine." +puts " every line delegates to public API - add_task, execute_plan," +puts " graph - so the DSL can never drift ahead of what the engine" +puts " supports, and anything the DSL can't express, you drop down" +puts " one layer without rewriting. Sinatra's rule: the frontend" +puts " should be a pleasure and the escape hatch should be a door," +puts " not a wall." diff --git a/examples/plan_flog.rb b/examples/plan_flog.rb new file mode 100644 index 0000000..f878c52 --- /dev/null +++ b/examples/plan_flog.rb @@ -0,0 +1,114 @@ +# frozen_string_literal: true + +# Plan Flog: flog gives every Ruby method a pain score; this gives +# every plan one. Fan-in hurts (joins hide coupling), depth hurts +# (chains hide latency), unlabeled edges hurt (anonymous data flow), +# and orphans hurt (dead code that runs). One number per task, one +# number per plan, and a threshold that means "refactor me". Yes, +# it's opinionated. So is flog. That's the point. +# +# bundle exec ruby examples/plan_flog.rb +# +# Runs offline; three plans walk in, one gets told the truth. + +require_relative "../lib/agentic" + +def task_named(name) + Agentic::Task.new(description: name, agent_spec: {"name" => name, "instructions" => "w"}) +end + +def tidy_pipeline + o = Agentic::PlanOrchestrator.new + fetch, clean, render = %w[fetch clean render].map { |n| task_named(n) } + o.add_task(fetch) + o.add_task(clean, [fetch]) + o.add_task(render, [clean]) + o +end + +def labeled_diamond + o = Agentic::PlanOrchestrator.new + orders, refunds, ledger, report = %w[orders refunds ledger report].map { |n| task_named(n) } + o.add_task(orders) + o.add_task(refunds) + o.add_task(ledger, needs: {sales: orders, credits: refunds}) + o.add_task(report, [ledger]) + o +end + +def the_monster + o = Agentic::PlanOrchestrator.new + sources = 6.times.map { |i| task_named("src#{i}") } + sources.each { |s| o.add_task(s) } + god = task_named("do_everything") + o.add_task(god, sources) # six unlabeled inputs + chain = god + 4.times do |i| + step = task_named("then#{i}") + o.add_task(step, [chain]) + chain = step + end + o.add_task(task_named("orphan")) # added in a refactor, feeds nothing... but runs + o +end + +# The scoring, flog-style: pain per structural sin +def flog(graph) + stats = graph[:stats] + labeled = graph[:edges].count { |e| e[:label] } + scores = graph[:tasks].keys.to_h do |id| + fan_in = graph[:dependencies][id].size + fan_out = graph[:edges].count { |e| e[:from] == id } + unlabeled_in = (fan_in >= 2) ? graph[:edges].count { |e| e[:to] == id && !e[:label] } : 0 + orphan = (stats[:roots].include?(id) && stats[:leaves].include?(id) && graph[:tasks].size > 1) ? 5.0 : 0 + score = [fan_in - 1, 0].max * 1.5 + # a pipe is free; every EXTRA join input is coupling + [fan_out - 2, 0].max * 1.0 + # fan-out past 2 spreads blame + (stats[:depth][id] - 3).clamp(0, 99) * 0.8 + # depth past 3 hides latency + unlabeled_in * 1.2 + # anonymous inputs, where they can be confused + orphan # runs, feeds nothing: pay attention + [id, score] + end + [scores, labeled] +end + +PLANS = { + "tidy pipeline" => tidy_pipeline, + "labeled diamond" => labeled_diamond, + "the monster" => the_monster +}.freeze + +puts "PLAN FLOG (pain points per plan; > 12 total means refactor me)" +puts +PLANS.each do |name, orchestrator| + graph = orchestrator.graph + scores, = flog(graph) + total = scores.values.sum + names = graph[:tasks].transform_values(&:description) + worst = scores.max_by(2) { |_, s| s }.select { |_, s| s > 0 } + + verdict = if total > 12 + "REFACTOR ME" + else + ((total > 6) ? "watch it" : "fine") + end + puts format(" %-18s %5.1f %-12s %s", name, total, verdict, + worst.map { |id, s| "#{names[id]}=#{s.round(1)}" }.join(" ")) +end + +puts +monster = the_monster.graph +scores, = flog(monster) +names = monster[:tasks].transform_values(&:description) +top = scores.max_by { |_, s| s } +total = scores.values.sum +puts " the monster's breakdown, because a score you can't argue with" +puts " is a score you can't learn from: #{names[top[0]]} costs #{top[1].round(1)} - five" +puts " EXTRA join inputs at 1.5 coupling each, plus six anonymous ones" +puts " at 1.2. the orphan costs 5.0 flat: it runs on every execution" +puts " and feeds nothing, which is either a bug or a billing strategy." +puts " and the tidy pipeline scores 0.0, because a pipe is free and" +puts " boring plans should be." +puts +puts " numbers don't refactor code and they don't refactor plans -" +puts " but they end the meeting about whether the monster is fine." +puts " it's a #{total.round}. it's not fine." diff --git a/examples/plan_kata.rb b/examples/plan_kata.rb new file mode 100644 index 0000000..36ee4a7 --- /dev/null +++ b/examples/plan_kata.rb @@ -0,0 +1,94 @@ +# frozen_string_literal: true + +# The Plan Kata: red, green, refactor - for a plan. The "tests" are +# assertions about the graph (one root, one leaf, labeled joins, +# nothing too deep), written BEFORE any tasks exist. Each step adds +# the smallest thing that moves a red line green, and the refactor +# step changes structure with the assertions standing guard. You've +# TDD'd methods; plans deserve the same discipline. +# +# bundle exec ruby examples/plan_kata.rb +# +# Runs offline; exits 1 if the kata ends with a red assertion. + +require_relative "../lib/agentic" + +def task_named(name) + Agentic::Task.new(description: name, agent_spec: {"name" => name, "instructions" => "w"}) +end + +# The test list, written first - what a GOOD ingest plan looks like, +# structurally, before we know what the tasks are +ASSERTIONS = { + "has exactly one entry point" => ->(g) { g[:stats][:roots].size == 1 }, + "has exactly one deliverable" => ->(g) { g[:stats][:leaves].size == 1 }, + "every join names its inputs" => ->(g) { + g[:dependencies].select { |_, d| d.size >= 2 }.keys.all? { |id| + g[:edges].select { |e| e[:to] == id }.all? { |e| e[:label] } + } + }, + "no deeper than four stages" => ->(g) { g[:stats][:max_depth] <= 4 }, + "no orphan tasks" => ->(g) { + g[:tasks].size < 2 || (g[:stats][:roots] & g[:stats][:leaves]).empty? + } +}.freeze + +def check(orchestrator) + graph = orchestrator.graph + ASSERTIONS.transform_values { |assertion| assertion.call(graph) } +end + +def report(step, results) + reds = results.count { |_, ok| !ok } + puts " step: #{step}" + results.each { |name, ok| puts format(" %-32s %s", name, ok ? "green" : "RED") } + puts format(" -> %d red", reds) + puts +end + +puts "THE PLAN KATA (assertions first, tasks second)" +puts + +# RED: no tasks at all - most assertions can't hold on emptiness +o = Agentic::PlanOrchestrator.new +report("empty plan (the honest starting point)", check(o)) + +# GREEN, smallest step: one task satisfies one-root-one-leaf trivially +ingest = task_named("ingest") +o.add_task(ingest) +report("add the entry point", check(o)) + +# Grow: parse feeds off ingest; deliverable moves - still green +parse = task_named("parse") +o.add_task(parse, [ingest]) +report("add parse", check(o)) + +# RED on purpose: a second source creates a second root, and an +# unlabeled join - two assertions object, and they name the problem +prices = task_named("prices") +merge = task_named("merge") +o.add_task(prices) +o.add_task(merge, [parse, prices]) +report("bolt on a price feed (two sins)", check(o)) + +# GREEN again: REFACTOR IN PLACE - the round-12 release gave plans +# rewire_task, so fixing the shape no longer means demolishing it. +# Route the price feed through the one door, and give the merge its +# labels; the assertions stand guard the whole time. +o.rewire_task(prices, [ingest]) +o.rewire_task(merge, needs: {parsed: parse, prices: prices}) +report_task = task_named("report") +o.add_task(report_task, [merge]) +final = check(o) +report("refactor in place: rewire, relabel", final) + +reds = final.count { |_, ok| !ok } +puts " the kata's shape is the point: the assertions existed before" +puts " the plan did, every addition was the smallest thing that moved" +puts " a line, and the two deliberate sins were CAUGHT and NAMED by" +puts " tests written when nobody was defensive about the design yet." +puts " and the refactor was a real refactor this time: rewire_task" +puts " (round 12, this kata's own ask) changed the plan's shape without" +puts " demolishing its identity - red, green, REFACTOR, with all three" +puts " words meaning what they say. #{(reds == 0) ? "kata complete, all green." : "KATA INCOMPLETE."}" +exit((reds == 0) ? 0 : 1) diff --git a/examples/ports_and_adapters.rb b/examples/ports_and_adapters.rb new file mode 100644 index 0000000..2ee1b9a --- /dev/null +++ b/examples/ports_and_adapters.rb @@ -0,0 +1,96 @@ +# frozen_string_literal: true + +# Ports and Adapters: the domain is the part of your app that would +# survive a framework migration - IF you kept it clean. Here the +# use-case (quote a shipment) is pure Ruby speaking only to PORTS; +# the adapters live at the edge; and Agentic is the delivery +# mechanism, replaced in the second act by a bare call to prove the +# domain never knew it was there. The proof is mechanical: the +# domain's source is scanned for framework constants. +# +# bundle exec ruby examples/ports_and_adapters.rb +# +# Runs offline; exits 1 if the domain mentions the framework. + +require_relative "../lib/agentic" + +Agentic.logger.level = :fatal + +# --- the domain (would survive the migration) ----------------------------------- +DOMAIN_SOURCE = <<~RUBY + class QuoteShipment + Result = Struct.new(:price_cents, :carrier, keyword_init: true) + + def initialize(rate_source:, quote_repository:) + @rate_source = rate_source # port: #rate_for(mode) + @quote_repository = quote_repository # port: #save(result) + end + + def call(mode:, weight:) + rate = @rate_source.rate_for(mode) + result = Result.new(price_cents: (weight * rate).round, carrier: rate > 5 ? "premium" : "standard") + @quote_repository.save(result) + result + end + end +RUBY +eval(DOMAIN_SOURCE) # standard:disable Security/Eval -- the string exists so the purity scan below is honest + +# --- the adapters (edge; disposable) -------------------------------------------- +class StaticRates + def rate_for(mode) = {"air" => 9, "sea" => 2}.fetch(mode) +end + +class MemoryQuotes + def all = @all ||= [] + + def save(result) = all << result +end + +# --- act one: Agentic as the delivery mechanism --------------------------------- +repo = MemoryQuotes.new +use_case = QuoteShipment.new(rate_source: StaticRates.new, quote_repository: repo) + +orchestrator = Agentic::PlanOrchestrator.new +quote_task = Agentic::Task.new( + description: "quote", agent_spec: {"name" => "quote", "instructions" => "quote"}, + payload: {mode: "air", weight: 120} +) +orchestrator.add_task(quote_task, agent: ->(t) { use_case.call(**t.payload) }) +orchestrator.execute_plan + +puts "PORTS AND ADAPTERS (the domain would survive the migration)" +puts +puts " act one - delivered by Agentic:" +puts " plan ran the use-case: #{repo.all.last.to_h}" +puts + +# --- act two: the framework leaves; the domain doesn't notice ------------------- +bare = use_case.call(mode: "sea", weight: 300) +puts " act two - delivered by a bare method call:" +puts " same use-case, no orchestrator: #{bare.to_h}" +puts " repository holds #{repo.all.size} quotes; the domain never knew who called." +puts + +# --- the proof: scan the domain for framework constants ------------------------- +leaks = DOMAIN_SOURCE.scan(/\b(?:Agentic|PlanOrchestrator|Task|CapabilityS\w+)\b/).uniq - ["Task"] +leaks += DOMAIN_SOURCE.scan(/\bAgentic::\w+/) +puts " the purity scan: grep the domain's source for framework constants" +if leaks.empty? + puts " 0 mentions of the framework in the domain. the dependency" + puts " arrow points ONE way: the edge knows the center; the center" + puts " has never heard of the edge." +else + puts " LEAKED: #{leaks.join(", ")} - the domain is coupled to its delivery." +end +puts +puts " what Agentic added in act one wasn't the business logic - it was" +puts " everything AROUND it: retry policy, lifecycle hooks, journaling," +puts " concurrency, the graph. that's the correct division of labor:" +puts " frameworks orchestrate; domains decide. the ports (#rate_for," +puts " #save) are the entire vocabulary the domain needs from the" +puts " world, and both adapters fit in six lines because the ports" +puts " asked for so little. clean architecture isn't ceremony - it's" +puts " the freedom to change your mind about everything but the truth." + +exit(leaks.empty? ? 0 : 1) diff --git a/examples/process_drill.rb b/examples/process_drill.rb new file mode 100644 index 0000000..d45a279 --- /dev/null +++ b/examples/process_drill.rb @@ -0,0 +1,71 @@ +# frozen_string_literal: true + +# The Process Drill: threads share a Mutex; PROCESSES share nothing +# but the file. The journal claims flock+fsync, which is a promise +# about processes - so this drill forks real ones, points them all at +# one journal, and lets the kernel referee. Then replay must find +# every event whole: no torn lines, no interleaved halves, no losses. +# +# bundle exec ruby examples/process_drill.rb +# +# Runs offline; exits 1 if any process's write was torn or lost. + +require_relative "../lib/agentic" +require "tmpdir" +require "json" + +Agentic.logger.level = :fatal + +PROCESSES = 4 +EVENTS = 250 +PATH = File.join(Dir.tmpdir, "agentic_process_drill.journal.jsonl") +File.delete(PATH) if File.exist?(PATH) + +pids = PROCESSES.times.map do |p| + fork do + journal = Agentic::ExecutionJournal.new(path: PATH) + EVENTS.times do |i| + journal.record(:task_succeeded, + task_id: "p#{p}-#{i}", description: "p#{p}-#{i}", + duration: 0.001, output: "payload-#{p}-" + ("x" * (50 + (i % 100)))) + end + exit!(0) + end +end +statuses = pids.map { |pid| Process.wait2(pid).last.exitstatus } + +# --- the referee ---------------------------------------------------------------- +lines = File.readlines(PATH) +torn = lines.reject do |line| + JSON.parse(line) + true +rescue JSON::ParserError + false +end +state = Agentic::ExecutionJournal.replay(path: PATH) +expected = PROCESSES * EVENTS +per_process = PROCESSES.times.map { |p| + state.completed_task_ids.count { |id| id.start_with?("p#{p}-") } +} + +puts "PROCESS DRILL (#{PROCESSES} forked writers x #{EVENTS} events, one journal)" +puts +puts format(" processes exited cleanly: %s", statuses.all?(&:zero?) ? "yes" : "NO") +puts format(" lines on disk: %d/%d", lines.size, expected) +puts format(" torn lines: %d", torn.size) +puts format(" replay recovered per proc: %s", per_process.join(", ")) +puts + +ok = statuses.all?(&:zero?) && lines.size == expected && torn.empty? && per_process.all?(EVENTS) +if ok + puts " the flock claim is now a certificate, not a comment: four" + puts " processes - separate GVLs, separate heaps, separate everything -" + puts " interleaved a thousand writes into one file and the kernel's" + puts " advisory lock kept every line whole. this is the half of the" + puts " journal's promise the threads drill couldn't reach: a Mutex" + puts " means nothing across fork(2); only the fd-level lock does." + puts " crash-recovery tooling stands on exactly this property." +else + puts " DRILL FAILED - the promise about processes is broken." +end +exit(ok ? 0 : 1) diff --git a/examples/projection_agreement.rb b/examples/projection_agreement.rb index 6d970cb..85d61a4 100644 --- a/examples/projection_agreement.rb +++ b/examples/projection_agreement.rb @@ -90,8 +90,13 @@ def validator_allows?(validator, payload) puts format(" typed field: validator %-7s (per-key: nil isn't a boolean)", validator_allows?(validator, frontier) ? "allows" : "rejects") puts format(" schema %-7s (the property EXISTS - dependencies fire)", schema_allows?(schema, frontier) ? "allows" : "rejects") -# The TRUE divergence needs an untyped field, where nil sails past -# per-key checks and the two presence semantics finally face off +# In round 10 an untyped field exposed the true divergence: nil +# sailed past per-key checks, the validator's relation read it as +# absent, and the schema's dependencies read null as present. The +# round-11 release closes it from the projection side: a relation +# over any UNTYPED field stays out of the draft-07 keywords entirely +# (it still travels in x-agentic-rules), so the schema never claims +# a law it can't render faithfully. untyped = Agentic::CapabilitySpecification.new( name: "connect", description: "x", version: "1.0.0", inputs: {express: {}, customs_code: {type: "string"}}, @@ -99,13 +104,16 @@ def validator_allows?(validator, payload) ) ruby = validator_allows?(Agentic::CapabilityValidator.new(untyped), frontier) json = schema_allows?(untyped.to_json_schema, frontier) +projected = untyped.to_json_schema.key?("dependencies") puts format(" untyped field: validator %-7s (nil is ABSENT - rule not triggered)", ruby ? "allows" : "rejects") -puts format(" schema %-7s (null is PRESENT - rule fires)", json ? "allows" : "rejects") +puts format(" schema %-7s (projection %s)", json ? "allows" : "rejects", + projected ? "STILL EMITTED - divergence is back!" : "declined - the keyword was never emitted") puts puts " on the nil-free plane the projection is faithful, point by point." -puts " explicit null is where Ruby's nil and JSON's null part ways - but" -puts " only for UNTYPED fields, because typing guards the frontier." -puts " senders: omit keys, don't null them. (filed as the round-11 ask:" -puts " align or officially document presence semantics at the boundary.)" +puts " and at the frontier the projection now knows its own limits: a" +puts " relation over untyped fields is not rendered into keywords it" +puts " cannot render truthfully - it rides x-agentic-rules instead." +puts " a projection that declines is honest; one that guesses is a trap." +exit(1) if projected exit(disagreements.zero? ? 0 : 1) diff --git a/examples/relation_prober.rb b/examples/relation_prober.rb index e4bce8f..bcd32d7 100644 --- a/examples/relation_prober.rb +++ b/examples/relation_prober.rb @@ -4,13 +4,14 @@ # predicates deserve hostility. Each relation is probed with edge # inputs - zeros, negatives, floats, missing keys, nils - and every # verdict is checked against an independent hand-written oracle. -# The prober also walks off the paved road on purpose: a rule that -# references an undeclared field meets a string, and the resulting -# TypeError escapes raw. Exit 1 by design - the sharp edge is real. +# The prober also walks off the paved road on purpose: in round 10 +# a rule referencing an undeclared field met a string and escaped as +# a raw TypeError; the round-11 release files that edge down, and +# this prober is the acceptance test that proves it stays down. # # bundle exec ruby examples/relation_prober.rb # -# Runs offline; exits 1 because the last probe draws blood. +# Runs offline; exits 1 if any probe draws blood again. require_relative "../lib/agentic" @@ -75,31 +76,40 @@ def verdict(spec, payload) puts # --- off the paved road --------------------------------------------------------- -# A rule may reference a field the contract never declared. Per-key -# validation can't type-check what isn't declared, so a string sails -# through to sum_lte's arithmetic - and the failure is a raw -# TypeError, not a ValidationError. Callers rescuing the documented -# error class will not catch this. -sharp = spec_for( - {r: {relation: :sum_lte, fields: [:a, :undeclared], limit: 10}}, - {a: {type: "number"}} -) -puts " off the road: sum_lte over an UNDECLARED field, fed a string" -begin - Agentic::CapabilityValidator.new(sharp).validate_inputs!(a: 5, undeclared: "5") - puts " ...allowed?! the prober expected blood and found none" - exit(divergences.zero? ? 0 : 1) -rescue Agentic::Errors::ValidationError - puts " rejected with ValidationError - the edge has been filed down" - exit(divergences.zero? ? 0 : 1) -rescue TypeError => e - puts " RAW #{e.class}: #{e.message.inspect}" - puts - puts " a validator's one job is to convert bad input into its OWN" - puts " error type. here, bad input crashes the validator instead -" - puts " rescue Agentic::Errors::ValidationError won't catch it, so" - puts " the 422 path becomes a 500 path. filed as the round-11 ask:" - puts " relation rules must either type-check their fields at" - puts " declaration time or wrap evaluation failures. exit 1 until." - exit(1) +# In round 10, a rule referencing an undeclared field let a string +# reach sum_lte's arithmetic: raw TypeError, a 422 path turned 500 +# path. The round-11 fix refuses at CONSTRUCTION - the typo fails at +# boot, where it names itself, before any request can find it. +edges = { + "sum_lte over an UNDECLARED field" => + [{r: {relation: :sum_lte, fields: [:a, :undeclared], limit: 10}}, {a: {type: "number"}}], + "sum_lte over a declared STRING" => + [{r: {relation: :sum_lte, fields: [:a, :b], limit: 10}}, {a: {type: "number"}, b: {type: "string"}}], + "requires with a typo'd field (fail-open)" => + [{r: {relation: :requires, fields: [:a, :customs_kode]}}, {a: {type: "number"}, customs_code: {type: "string"}}] +} + +blood = 0 +puts " off the paved road: rules that must refuse to construct" +edges.each do |name, (rules, inputs)| + Agentic::CapabilityValidator.new(spec_for(rules, inputs)) + blood += 1 + puts format(" %-42s CONSTRUCTED - the edge is back", name) +rescue ArgumentError => e + puts format(" %-42s refused at boot: %s", name, e.message[0, 40] + "...") +rescue => e + blood += 1 + puts format(" %-42s wrong uniform: %s", name, e.class) +end + +puts +if divergences.zero? && blood.zero? + puts " the paved road holds and the roadside refuses construction." + puts " note the third edge: a typo'd field in requires used to fail" + puts " OPEN - the rule just never fired, which no test of valid inputs" + puts " would ever notice. now the typo can't boot. a validator's" + puts " errors must wear its uniform, and its typos must not compile." +else + puts " BLOOD DRAWN: #{divergences} divergence(s), #{blood} escaped edge(s)." end +exit((divergences + blood).zero? ? 0 : 1) diff --git a/examples/require_cost.rb b/examples/require_cost.rb new file mode 100644 index 0000000..47af4bc --- /dev/null +++ b/examples/require_cost.rb @@ -0,0 +1,77 @@ +# frozen_string_literal: true + +# The Require Cost Report: `require` is a purchase - memory, objects, +# and boot time, paid again by every process you fork and every +# worker you scale. This measures what the gem and each major +# dependency cost AT REQUIRE TIME, each in a clean subprocess so +# nobody's cost gets billed to a neighbor's account. +# +# bundle exec ruby examples/require_cost.rb +# +# Runs offline; each row is an isolated child process. + +require "open3" +require "rbconfig" + +RUBY = RbConfig.ruby +LIB = File.expand_path("../lib", __dir__) + +# Measure inside a pristine child: RSS and allocated objects, before +# and after the require - so each row is that require's WHOLE bill, +# transitive dependencies included +PROBE = <<~'RUBY' + def rss_kb = File.read("/proc/self/status")[/VmRSS:\s+(\d+)/, 1].to_i + target, touch = ARGV + objects_before = GC.stat(:total_allocated_objects) + rss_before = rss_kb + t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC) + require target + eval(touch) if touch && !touch.empty? # standard:disable Security/Eval + ms = (Process.clock_gettime(Process::CLOCK_MONOTONIC) - t0) * 1000 + puts [rss_kb - rss_before, GC.stat(:total_allocated_objects) - objects_before, ms.round(1)].join(",") +RUBY + +def cost_of(target, touch = "") + out, status = Open3.capture2(RUBY, "-I", LIB, "-e", PROBE, target, touch) + raise "probe failed for #{target}" unless status.success? + + rss_kb, objects, ms = out.strip.split(",") + {rss_mb: rss_kb.to_f / 1024, objects: objects.to_i, ms: ms.to_f} +end + +TARGETS = { + "json (stdlib)" => ["json"], + "zeitwerk" => ["zeitwerk"], + "async" => ["async"], + "dry-schema" => ["dry/schema"], + "agentic (require only)" => ["agentic"], + "agentic + first real touch" => ["agentic", + "Agentic::PlanOrchestrator.new; Agentic::CapabilityValidator"] +}.freeze + +puts "REQUIRE COST REPORT (each row measured in a pristine child process)" +puts +puts format(" %-28s %10s %14s %10s", "require", "RSS", "objects", "time") +rows = TARGETS.transform_values { |target, touch| cost_of(target, touch || "") } +rows.each do |name, cost| + puts format(" %-28s %8.1fMB %14d %8.0fms %s", + name, cost[:rss_mb], cost[:objects], cost[:ms], "#" * (cost[:rss_mb] * 2).round) +end + +bare = rows["agentic (require only)"] +touched = rows["agentic + first real touch"] + +puts +puts " the bill, read like a Heroku support ticket - and it's a plot" +puts format(" twist: `require \"agentic\"` costs %.1fMB / %dms, nearly FREE,", bare[:rss_mb], bare[:ms]) +puts " because Zeitwerk (the round-1 cleanup) defers every constant." +puts format(" the first real touch is where the bill lands: %.1fMB and %dms,", touched[:rss_mb], touched[:ms]) +puts " as async and dry-schema come in through the autoloader. deferred" +puts " is not free - it's a bill that arrives during your first" +puts " request instead of your boot, which is either exactly what you" +puts " want (CLI tools, tiny scripts pay only for what they touch) or" +puts " exactly what you don't (a web worker's first request eats the" +puts " latency). the moves this report funds: eager_load in servers +" +puts " preload_app (pay once, share copy-on-write), stay lazy in CLIs," +puts " and run this script in CI so a new dependency's bill arrives in" +puts " the PR that adds it - not in the invoice at month's end." diff --git a/examples/retry_budget.rb b/examples/retry_budget.rb index 08d5be9..3b78366 100644 --- a/examples/retry_budget.rb +++ b/examples/retry_budget.rb @@ -19,22 +19,23 @@ MAX_RETRIES = 3 JOBS = 12 -# The wallet: a windowed retry allowance with NON-BLOCKING admission. -# (A RateLimit wants to make you wait; a budget wants to tell you no. -# Round-11 ask: RateLimit#try_acquire, so this class can retire.) +# The wallet is a windowed RateLimit asked politely: try_acquire +# (round 11) answers false RIGHT NOW instead of making the caller +# wait for capacity - and waiting for retry capacity during an +# outage would just be the storm with extra steps. The custom +# budget class this example shipped with has retired. class RetryBudget - def initialize(allowance) - @allowance = allowance + def initialize(allowance, per: 60) + @wallet = Agentic::RateLimit.new(allowance, per: per) @spent = 0 end attr_reader :spent def spend? - return false if @spent >= @allowance - - @spent += 1 - true + admitted = @wallet.try_acquire + @spent += 1 if admitted + admitted end end @@ -105,4 +106,6 @@ def drill(strategy, budget: nil) puts " \"should I try again?\"; the budget answers \"should ANYONE?\" -" puts " round 9's breaker asked that per-upstream, this asks it per-" puts " window, and both read the same journaled verdicts. retries are" -puts " a shared resource. give them a wallet, not a habit." +puts " a shared resource. give them a wallet, not a habit. (and the" +puts " wallet is now a real windowed RateLimit - try_acquire says no" +puts " without making anyone wait for it, which is the entire point.)" diff --git a/examples/self_correcting_output.rb b/examples/self_correcting_output.rb new file mode 100644 index 0000000..cfe3b00 --- /dev/null +++ b/examples/self_correcting_output.rb @@ -0,0 +1,88 @@ +# frozen_string_literal: true + +# Self-Correcting Output: the pattern that makes LLM components +# shippable. The model's output is validated against the capability's +# contract; violations don't raise to the user - they become the +# CORRECTION PROMPT for a bounded retry. The contract is the editor, +# the model is the writer, and the loop converges or fails honestly +# with the full paper trail. +# +# bundle exec ruby examples/self_correcting_output.rb +# +# Runs offline; the "model" is scripted to be sloppy, then coachable. + +require_relative "../lib/agentic" +require "json" + +Agentic.logger.level = :fatal + +CONTRACT = Agentic::CapabilitySpecification.new( + name: "extract_invoice", description: "Extract structured invoice data", version: "1.0.0", + inputs: {text: {type: "string", required: true}}, + outputs: { + vendor: {type: "string", required: true, non_empty: true}, + total_cents: {type: "number", required: true, min: 0}, + currency: {type: "string", required: true, enum: %w[USD EUR GBP]}, + due_date: {type: "string", required: true} + } +) +VALIDATOR = Agentic::CapabilityValidator.new(CONTRACT) + +# The "model": pass 1 is what models actually do to schemas; pass 2 +# reads the corrections like a chastened intern +MODEL = lambda do |prompt, attempt| + if attempt == 1 + # currency invented, total as a string, due_date forgotten + {vendor: "Initech Supply Co", total_cents: "4200", currency: "usd"} + else + # the correction prompt names each violation; the model complies + {vendor: "Initech Supply Co", total_cents: 4200, currency: "USD", due_date: "2026-08-01"} + end +end + +def correction_prompt(violations) + lines = violations.map { |field, messages| "- #{field}: #{messages.join("; ")}" } + "Your previous answer violated the output contract:\n#{lines.join("\n")}\n" \ + "Return the SAME data corrected to satisfy every constraint. Do not apologize; return JSON." +end + +MAX_ATTEMPTS = 3 +INVOICE = "Invoice from Initech Supply Co, total $42.00, due Aug 1 2026" + +puts "SELF-CORRECTING OUTPUT (the contract is the editor)" +puts +attempt = 0 +output = nil +prompt = "Extract the invoice fields from: #{INVOICE}" +loop do + attempt += 1 + output = MODEL.call(prompt, attempt) + puts " attempt #{attempt}: #{JSON.generate(output)}" + begin + VALIDATOR.validate_outputs!(output) + puts " -> contract satisfied. shipped after #{attempt} attempt(s)." + break + rescue Agentic::Errors::ValidationError => e + if attempt >= MAX_ATTEMPTS + puts " -> #{MAX_ATTEMPTS} attempts exhausted; failing HONESTLY with the paper trail." + raise + end + puts " -> rejected by the contract; violations become the next prompt:" + correction = correction_prompt(e.violations) + correction.lines.each { |l| puts " #{l.chomp}" } + prompt = correction + end +end + +puts +puts " the shape of the trick: nothing here trusts the model, and" +puts " nothing here burdens the user. the contract that documents the" +puts " capability (rounds 5-11 built six tools on it) turns out to be" +puts " the exact artifact a correction loop needs - violations arrive" +puts " pre-written as actionable feedback (\"currency: must be one of:" +puts " USD, EUR, GBP\"), which beats \"please try again\" by exactly the" +puts " margin your production error rate will show. the loop is" +puts " bounded (#{MAX_ATTEMPTS} attempts - unbounded self-correction is a billing" +puts " strategy), each retry costs one more model call and is worth" +puts " it, and when it fails it fails with every draft on record." +puts " ship the editor with the writer; never ship the writer alone." diff --git a/examples/telemetry_bus.rb b/examples/telemetry_bus.rb new file mode 100644 index 0000000..596110f --- /dev/null +++ b/examples/telemetry_bus.rb @@ -0,0 +1,108 @@ +# frozen_string_literal: true + +# The Telemetry Bus: lifecycle hooks are callbacks - one producer, +# one consumer, coupled at configuration time. A telemetry bus +# inverts that: the orchestrator emits NAMED EVENTS into a bus, and +# any number of handlers attach, detach, and crash independently. +# The producer never learns who is listening. This is the :telemetry +# pattern Elixir converged on, because every library inventing its +# own instrumentation callbacks was the worse world. +# +# bundle exec ruby examples/telemetry_bus.rb +# +# Runs offline; three handlers listen, one detaches mid-flight. + +require_relative "../lib/agentic" + +Agentic.logger.level = :fatal + +# The bus: names, payloads, and isolation. A crashing handler is +# detached and reported - it never takes the plan down with it. +class TelemetryBus + def initialize + @handlers = Hash.new { |h, k| h[k] = {} } + end + + def attach(id, event, &handler) + @handlers[event][id] = handler + end + + def detach(id) + @handlers.each_value { |hs| hs.delete(id) } + end + + def execute(event, measurements, metadata = {}) + @handlers[event].each do |id, handler| + handler.call(measurements, metadata) + rescue => e + detach(id) + puts " [bus] handler #{id} crashed (#{e.class}) - detached, plan unharmed" + end + end +end + +BUS = TelemetryBus.new + +# The bridge: hooks in, events out. This is the ONLY place the +# orchestrator and the bus know about each other. +def telemetry_hooks(bus) + { + after_task_success: ->(task_id:, task:, result:, duration:) { + bus.execute([:agentic, :task, :success], {duration: duration}, {task: task.description}) + }, + after_task_failure: ->(task_id:, task:, failure:, duration:) { + bus.execute([:agentic, :task, :failure], {duration: duration}, {task: task.description, type: failure.type}) + }, + plan_completed: ->(plan_id:, status:, execution_time:, tasks:, results:) { + bus.execute([:agentic, :plan, :completed], {execution_time: execution_time}, {status: status}) + } + } +end + +# Handler 1: a metrics counter - knows nothing about logging or tracing +metrics = Hash.new(0) +BUS.attach(:metrics, [:agentic, :task, :success]) { |m, _| metrics[:tasks] += 1 } +BUS.attach(:metrics2, [:agentic, :task, :failure]) { |m, _| metrics[:failures] += 1 } + +# Handler 2: a slow-task tracer - only speaks when something is worth saying +BUS.attach(:tracer, [:agentic, :task, :success]) do |measurements, metadata| + puts " [trace] SLOW: #{metadata[:task]} took #{(measurements[:duration] * 1000).round}ms" if measurements[:duration] > 0.05 +end + +# Handler 3: a fragile exporter someone deployed on a Friday +BUS.attach(:exporter, [:agentic, :task, :success]) do |_m, metadata| + raise IOError, "export endpoint down" if metadata[:task] == "enrich" +end + +def run_plan(bus) + orchestrator = Agentic::PlanOrchestrator.new(lifecycle_hooks: telemetry_hooks(bus)) + fetch = Agentic::Task.new(description: "fetch", agent_spec: {"name" => "w", "instructions" => "w"}) + enrich = Agentic::Task.new(description: "enrich", agent_spec: {"name" => "w", "instructions" => "w"}) + publish = Agentic::Task.new(description: "publish", agent_spec: {"name" => "w", "instructions" => "w"}) + orchestrator.add_task(fetch, agent: ->(_t) { sleep(0.01) }) + orchestrator.add_task(enrich, [fetch], agent: ->(_t) { sleep(0.08) }) + orchestrator.add_task(publish, [enrich], agent: ->(_t) { sleep(0.01) }) + orchestrator.execute_plan +end + +puts "TELEMETRY BUS (three handlers, one bridge, zero coupling)" +puts +puts " run 1 - all handlers attached:" +run_plan(BUS) +puts " [metrics] #{metrics.inspect}" +puts + +puts " run 2 - tracer detached at runtime (ops got tired of it):" +BUS.detach(:tracer) +run_plan(BUS) +puts " [metrics] #{metrics.inspect}" +puts +puts " the orchestrator emitted the same events both runs - it cannot" +puts " tell that the tracer left or that the exporter crashed, and" +puts " that ignorance is the feature. hooks couple one producer to" +puts " one consumer at configuration time; a bus decouples N handlers" +puts " at RUNTIME, with isolation (the Friday exporter died alone)." +puts " event names are namespaced tuples, measurements are separated" +puts " from metadata - steal the whole :telemetry design; it was" +puts " right. the framework's hooks made the bridge ten lines, which" +puts " is exactly what hooks are for: being the floor a bus stands on." diff --git a/examples/tenant_shards.rb b/examples/tenant_shards.rb new file mode 100644 index 0000000..0900180 --- /dev/null +++ b/examples/tenant_shards.rb @@ -0,0 +1,89 @@ +# frozen_string_literal: true + +# Tenant Shards: at scale, "the plan" becomes "the plan, per shard" - +# same pipeline, isolated blast radius. Each shard gets its own +# journal (its own recovery story) and its own rate limit (its own +# noisy neighbor containment), under one control plane. Shard 2 +# crashes mid-run; the rerun resumes ONLY what shard 2 didn't finish, +# because recovery, like everything at scale, must be per-shard. +# +# bundle exec ruby examples/tenant_shards.rb +# +# Runs offline; the crash is scripted, the resume is real. + +require_relative "../lib/agentic" +require "tmpdir" + +Agentic.logger.level = :fatal + +SHARDS = { + "shard_1" => %w[acme globex], + "shard_2" => %w[initech umbrella hooli], + "shard_3" => %w[wonka] +}.freeze +PIPELINE = %w[extract transform load].freeze + +def journal_path(shard) = File.join(Dir.tmpdir, "agentic_#{shard}.journal.jsonl") + +# One shard's run: its own journal, its own limiter, resume-aware +def run_shard(shard, tenants, crash_at: nil) + journal = Agentic::ExecutionJournal.new(path: journal_path(shard)) + done = Agentic::ExecutionJournal.replay(path: journal_path(shard)) + limiter = Agentic::RateLimit.new(2) # per-shard: a hot shard can't starve the others + + orchestrator = Agentic::PlanOrchestrator.new( + concurrency_limit: 2, lifecycle_hooks: journal.lifecycle_hooks, + retry_policy: {max_retries: 0, retryable_errors: []} + ) + ran = 0 + skipped = 0 + tenants.each do |tenant| + previous = nil + PIPELINE.each do |step| + name = "#{tenant}:#{step}" + if done.completed?(name) + skipped += 1 + next + end + + task = Agentic::Task.new(description: name, agent_spec: {"name" => name, "instructions" => "run"}) + orchestrator.add_task(task, previous ? [previous] : [], agent: ->(t) { + raise "power cut" if crash_at == t.description + + limiter.acquire { sleep(0.002) } + ran += 1 + :ok + }) + previous = task + end + end + status = orchestrator.execute_plan.status + [status, ran, skipped] +end + +puts "TENANT SHARDS (#{SHARDS.size} shards, #{SHARDS.values.sum(&:size)} tenants, pipeline: #{PIPELINE.join(" -> ")})" +puts +SHARDS.each_key { |shard| File.delete(journal_path(shard)) if File.exist?(journal_path(shard)) } + +puts " run 1 - shard_2 loses power mid-tenant:" +SHARDS.each do |shard, tenants| + crash = (shard == "shard_2") ? "umbrella:transform" : nil + status, ran, = run_shard(shard, tenants, crash_at: crash) + puts format(" %-9s %-16s %2d steps ran%s", shard, status, ran, + crash ? " <- crashed at #{crash}" : "") +end +puts + +puts " run 2 - control plane reruns everything; journals decide what that means:" +SHARDS.each do |shard, tenants| + status, ran, skipped = run_shard(shard, tenants) + puts format(" %-9s %-16s %2d steps ran, %2d skipped (already journaled)", shard, status, ran, skipped) +end +puts +puts " the rerun was issued fleet-wide - the control plane doesn't" +puts " track which shard crashed, and shouldn't have to. shard 1 and 3" +puts " skipped everything (their journals proved completion); shard 2" +puts " re-ran only from the crash point. that's the sharding contract:" +puts " one plan definition, N isolated executions, N recovery stories," +puts " N rate limits - and a blast radius that ends at the shard" +puts " boundary. scale isn't a bigger machine; it's smaller failures." diff --git a/examples/threads_drill.rb b/examples/threads_drill.rb new file mode 100644 index 0000000..7004def --- /dev/null +++ b/examples/threads_drill.rb @@ -0,0 +1,93 @@ +# frozen_string_literal: true + +# The Threads Drill: fibers are polite; threads are not. Everything +# in this gem that claims to be shared-safe gets hammered by real +# Ruby threads - the kind that run truly parallel on JRuby, where +# there is no GVL to be your accidental bodyguard. The journal and +# registry hold. The windowed limiter's bookkeeping is the one to +# watch, and the drill says so out loud. +# +# bundle exec ruby examples/threads_drill.rb +# +# Runs offline; exits 1 if a guaranteed-safe structure corrupts. + +require_relative "../lib/agentic" +require "tmpdir" +require "json" + +Agentic.logger.level = :fatal + +THREADS = 8 +EVENTS = 150 +failures = 0 + +# --- drill 1: the journal under parallel writers -------------------------------- +path = File.join(Dir.tmpdir, "agentic_threads.journal.jsonl") +File.delete(path) if File.exist?(path) +journal = Agentic::ExecutionJournal.new(path: path) + +THREADS.times.map { |t| + Thread.new do + EVENTS.times { |i| journal.record(:task_succeeded, task_id: "t#{t}-#{i}", description: "t#{t}-#{i}", duration: 0.001, output: "x" * 64) } + end +}.each(&:join) + +lines = File.readlines(path) +parseable = lines.count do |line| + JSON.parse(line) + true +rescue JSON::ParserError + false +end +torn = lines.size - parseable +expected = THREADS * EVENTS +failures += 1 if lines.size != expected || torn.positive? +puts " drill 1 - journal, #{THREADS} threads x #{EVENTS} events:" +puts format(" %d/%d lines written, %d torn - %s", parseable, expected, torn, + (torn.zero? && lines.size == expected) ? "mutex + flock + fsync held" : "CORRUPTED") +puts + +# --- drill 2: the registry under concurrent registration ------------------------ +registry = Agentic::AgentCapabilityRegistry.instance +THREADS.times.map { |t| + Thread.new do + 50.times do |i| + spec = Agentic::CapabilitySpecification.new( + name: "cap-#{t}-#{i}", description: "x", version: "1.0.0", + inputs: {a: {type: "number", required: true}} + ) + Agentic.register_capability(spec, Agentic::CapabilityProvider.new(capability: spec, implementation: ->(inputs) { inputs })) + registry.get_provider("cap-#{t}-#{i}")&.execute(a: 1) + end + end +}.each(&:join) + +missing = THREADS.times.sum { |t| 50.times.count { |i| registry.get_provider("cap-#{t}-#{i}").nil? } } +failures += 1 if missing.positive? +puts " drill 2 - registry, #{THREADS} threads x 50 register+execute:" +puts format(" %d registrations lost - %s", missing, missing.zero? ? "registry held" : "RACE") +puts + +# --- drill 3: the windowed limiter's check-then-act ------------------------------ +# In round 11 this bookkeeping had no mutex and the drill called it +# "luck wearing a lab coat". The round-12 release put a real Mutex +# around the stamp dance, so the drill now ASSERTS what it could +# previously only observe. +limit = Agentic::RateLimit.new(50, per: 60) +admitted = THREADS.times.map { + Thread.new { 200.times.count { limit.try_acquire } } +}.map(&:value).sum + +failures += 1 if admitted != 50 +puts " drill 3 - windowed try_acquire, #{THREADS} threads x 200 attempts (ceiling 50):" +puts format(" admitted %d/50 - %s", admitted, + (admitted == 50) ? "the stamp bookkeeping holds a real Mutex now" : "OVER-ADMISSION - the lock is gone") +puts +puts " all three structures now hold under real threads for the right" +puts " reason: real locks (two Mutexes, flock, fsync), not scheduling" +puts " luck. this drill went from characterization to CERTIFICATION" +puts " when the round-12 release paid the limiter's lock debt - the" +puts " answer is now the same on every Ruby, which is the only kind" +puts " of thread-safety worth writing in a README." + +exit(failures.zero? ? 0 : 1) diff --git a/examples/write_path_profile.rb b/examples/write_path_profile.rb new file mode 100644 index 0000000..309b2a1 --- /dev/null +++ b/examples/write_path_profile.rb @@ -0,0 +1,80 @@ +# frozen_string_literal: true + +# The Write Path Profile: everyone's first instinct about a slow +# journal is "switch JSON libraries". Before holding that opinion, +# weigh each layer of the write separately - serialize, write, flush, +# fsync - because optimization budgets get spent where the profiler +# points or they get wasted. Spoiler: the disk's honesty is the +# product, and it is also the bill. +# +# bundle exec ruby examples/write_path_profile.rb +# +# Runs offline; timings are real syscalls on this machine. + +require_relative "../lib/agentic" +require "tmpdir" +require "json" + +EVENTS = 300 +PAYLOAD = {event: "task_succeeded", task_id: "t-123", description: "sync:orders", + duration: 0.412, output: "x" * 200}.freeze + +def bench(events = EVENTS) + t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC) + events.times { |i| yield(i) } + (Process.clock_gettime(Process::CLOCK_MONOTONIC) - t0) / events * 1_000_000 # us/event +end + +dir = Dir.mktmpdir("agentic_write_path") + +# Layer 1: serialization only +serialize = bench { JSON.generate(PAYLOAD) } + +# Layer 2: + buffered write (kernel may keep it in page cache forever) +buffered_file = File.open(File.join(dir, "buffered.jsonl"), "a") +buffered = bench { buffered_file.puts(JSON.generate(PAYLOAD)) } + +# Layer 3: + flush (userland buffer to kernel, still not durable) +flushed_file = File.open(File.join(dir, "flushed.jsonl"), "a") +flushed = bench { |i| + flushed_file.puts(JSON.generate(PAYLOAD)) + flushed_file.flush +} + +# Layer 4: the real thing - open, flock, puts, flush, FSYNC per event +journal = Agentic::ExecutionJournal.new(path: File.join(dir, "real.jsonl")) +real = bench { |i| journal.record(:task_succeeded, PAYLOAD.merge(task_id: "t-#{i}")) } + +# The alternative shape: group commit - buffer N, fsync once +group_file = File.open(File.join(dir, "group.jsonl"), "a") +group = bench { |i| + group_file.puts(JSON.generate(PAYLOAD)) + group_file.fsync if (i % 20) == 19 +} + +puts "WRITE PATH PROFILE (#{EVENTS} events per layer, microseconds each)" +puts +rows = { + "JSON.generate only" => serialize, + "+ buffered write" => buffered, + "+ flush to kernel" => flushed, + "journal.record (flock+fsync)" => real, + "group commit (fsync per 20)" => group +} +rows.each do |name, us| + puts format(" %-30s %9.1fus %s", name, us, "#" * [(Math.log10([us, 1].max) * 12).round, 1].max) +end + +puts +json_share = serialize / real * 100 +puts format(" the ledger: serialization is %.1f%% of the real write. swapping", json_share) +puts " JSON libraries would optimize a rounding error - the other" +puts format(" %.1f%% is the price of the fsync, which is to say the price of", 100 - json_share) +puts " the journal's ONLY promise (a crash cannot unwrite what record" +puts " returned from). the honest knob is group commit: batch 20" +puts format(" events per fsync and the write drops to %.0fus - but now a crash", group) +puts " can eat up to 19 acknowledged events, so it's not an" +puts " optimization, it's a DIFFERENT PROMISE, and only the caller" +puts " knows which promise their recovery story needs. profile first," +puts " name the tradeoff second, and never let anyone optimize the" +puts " layer the profiler acquitted." diff --git a/lib/agentic/capability_specification.rb b/lib/agentic/capability_specification.rb index aeecae7..5fd43b1 100644 --- a/lib/agentic/capability_specification.rb +++ b/lib/agentic/capability_specification.rb @@ -150,10 +150,19 @@ def to_json_schema(side = :inputs) # one exists: requires -> dependencies, mutually_exclusive -> a # not-required clause per pair. sum_lte has no JSON Schema keyword # and lives only in x-agentic-rules. + # + # Projection requires every referenced field to carry a declared + # type. Ruby presence is "given and non-nil" while JSON Schema's + # keywords count an explicit null as present; typed fields guard + # that frontier (per-key checks reject nil first), untyped fields + # would let the two renderings diverge - so for those the rule + # stays in x-agentic-rules and out of the keywords. # @param schema [Hash] The schema being built (mutated) # @param definition [Hash] The relation rule definition # @return [void] def project_relation!(schema, definition) + return unless definition.fetch(:fields).all? { |f| inputs.dig(f, :type) } + fields = definition.fetch(:fields).map(&:to_s) case definition[:relation] when :requires diff --git a/lib/agentic/capability_validator.rb b/lib/agentic/capability_validator.rb index 9b6f3f2..b1a12bd 100644 --- a/lib/agentic/capability_validator.rb +++ b/lib/agentic/capability_validator.rb @@ -16,9 +16,14 @@ module Agentic # must be present. class CapabilityValidator # @param specification [CapabilitySpecification] The capability specification + # @raise [ArgumentError] When a relation-typed rule is ill-declared + # (unknown relation, undeclared fields, sum_lte over non-numbers) - + # bad rules fail at construction, not mid-validation in the wrong + # error class def initialize(specification) @specification = specification @schemas = {} + validate_rule_declarations! end # Validates inputs against the capability's declared inputs @@ -107,6 +112,19 @@ def validate_rules!(inputs) ) end + # Fail-fast check of every relation-typed rule against the declared + # inputs, at construction time + def validate_rule_declarations! + rules = @specification.respond_to?(:rules) ? @specification.rules : nil + return if rules.nil? || rules.empty? + + rules.each do |key, definition| + next if definition.respond_to?(:call) || !definition[:relation] + + RelationRules.validate_declaration!(key, definition, @specification.inputs) + end + end + def schema_for(kind, declared) @schemas[kind] ||= Dry::Schema.define do declared.each do |name, definition| diff --git a/lib/agentic/execution_journal.rb b/lib/agentic/execution_journal.rb index 3874427..fc709b9 100644 --- a/lib/agentic/execution_journal.rb +++ b/lib/agentic/execution_journal.rb @@ -2,6 +2,7 @@ require "json" require "fileutils" +require "time" # Time#iso8601 - do not depend on another file loading it first module Agentic # Durable, append-only journal of plan execution. diff --git a/lib/agentic/plan_orchestrator.rb b/lib/agentic/plan_orchestrator.rb index 6d3277d..52ebffb 100644 --- a/lib/agentic/plan_orchestrator.rb +++ b/lib/agentic/plan_orchestrator.rb @@ -94,6 +94,66 @@ def add_task(task, dependencies = [], agent: nil, needs: nil) @execution_state[:pending].add(task_id) end + # Removes a pending task from the plan. Refactoring needs demolition + # to be surgical: only tasks that haven't run can leave, and a task + # that others depend on refuses to go (name the dependents, make the + # caller rewire them first - silent cascade deletes are how plans + # lose limbs). + # @param task [Task, String] The task (or its id) to remove + # @return [Task] The removed task + # @raise [ArgumentError] If unknown, already started, or depended upon + def remove_task(task) + task_id = task.respond_to?(:id) ? task.id : task + raise ArgumentError, "unknown task #{task_id}" unless @tasks.key?(task_id) + unless @execution_state[:pending].include?(task_id) + raise ArgumentError, "task #{task_id} has already started; only pending tasks can be removed" + end + + dependents = @dependencies.select { |_, deps| deps.include?(task_id) }.keys + if dependents.any? + names = dependents.map { |id| @tasks[id].description } + raise ArgumentError, "cannot remove #{@tasks[task_id].description}: " \ + "#{names.join(", ")} depend(s) on it - rewire them first" + end + + @execution_state[:pending].delete(task_id) + @dependencies.delete(task_id) + @task_needs.delete(task_id) + @task_agents.delete(task_id) + @tasks.delete(task_id) + end + + # Replaces a pending task's dependencies in place - the refactoring + # seam. Same shapes as add_task: positional dependencies, or needs: + # for named (labeled) ones. + # @param task [Task, String] The task (or its id) to rewire + # @param dependencies [Array] The new dependencies + # @param needs [Hash{Symbol=>Task,String}, nil] Named dependencies + # @return [void] + # @raise [ArgumentError] If unknown, already started, or wired to a + # task that isn't in the plan + def rewire_task(task, dependencies = [], needs: nil) + task_id = task.respond_to?(:id) ? task.id : task + raise ArgumentError, "unknown task #{task_id}" unless @tasks.key?(task_id) + unless @execution_state[:pending].include?(task_id) + raise ArgumentError, "task #{task_id} has already started; only pending tasks can be rewired" + end + + deps = Array(dependencies).map { |dep| dep.respond_to?(:id) ? dep.id : dep } + named = needs&.transform_values { |dep| dep.respond_to?(:id) ? dep.id : dep } + deps |= named.values if named + + unknown = deps.reject { |dep| @tasks.key?(dep) } + raise ArgumentError, "cannot wire to unknown task(s) #{unknown.join(", ")}" if unknown.any? + + @dependencies[task_id] = deps + if named + @task_needs[task_id] = named + else + @task_needs.delete(task_id) + end + end + # A read-only snapshot of the plan's topology, for tools that render, # review, or analyze the graph without executing it # @return [Hash] :tasks (id => Task), :dependencies (id => [ids]), @@ -219,20 +279,34 @@ def cancel_task(task_id) false end - # Cancels execution of the entire plan + # Cancels execution of the entire plan - promptly. Queued work is + # marked canceled before the scheduler can start it, and every + # scheduled fiber (waiting for a slot or mid-flight) is stopped. + # Stopping the fibers rather than the reactor keeps the promise + # when execute_plan has joined a host reactor: in-flight agents + # stop where they are instead of running to completion and billing + # for results nobody will read. # @return [void] def cancel_plan - # Stop the reactor to cancel all async tasks - @reactor&.stop - - # Move all pending and in_progress tasks to canceled state - @execution_state[:pending].each do |task_id| + # ALL bookkeeping first: stopping a fiber releases its concurrency + # slot and synchronously admits the next waiter - which must + # already read as canceled by then, or it will start (and bill) + # in the instant before its own stop arrives + @execution_state[:pending].dup.each do |task_id| transition_task_state(task_id, from: :pending, to: :canceled) end - - @execution_state[:in_progress].each do |task_id| + stopping = @execution_state[:in_progress].dup + stopping.each do |task_id| transition_task_state(task_id, from: :in_progress, to: :canceled) end + + # Then stop every scheduled fiber. Never stop the calling fiber + # itself - a lifecycle hook may cancel the plan from inside a task + current = Async::Task.current? + stopping.each do |task_id| + fiber = @async_tasks[task_id] + fiber&.stop unless fiber.nil? || fiber == current + end end # Determines if a task failure is retryable based on retry policy @@ -406,6 +480,10 @@ def schedule_task(task_id, agent_provider, semaphore, barrier) scheduled_at = Time.now async_task = barrier.async do semaphore.acquire do + # A task canceled while waiting for its slot must not run - + # its fiber may win the slot race against its own stop + next unless @execution_state[:in_progress].include?(task_id) + @lifecycle_hooks[:task_slot_acquired].call( task_id: task_id, task: task, diff --git a/lib/agentic/rate_limit.rb b/lib/agentic/rate_limit.rb index 6bc912e..a1ea412 100644 --- a/lib/agentic/rate_limit.rb +++ b/lib/agentic/rate_limit.rb @@ -36,6 +36,10 @@ def initialize(ceiling, per: nil) @per = per @semaphore = Async::Semaphore.new(ceiling) unless per @stamps = [] + # Windowed bookkeeping is check-then-act over shared state; a + # real Mutex makes the answer the same on every Ruby VM instead + # of relying on the GVL's scheduling habits + @window_mutex = Mutex.new @in_flight = 0 @high_water = 0 end @@ -54,6 +58,35 @@ def acquire # @return [Integer] Acquisitions currently inside the ceiling attr_reader :in_flight + # Non-blocking admission: runs the block (if given) when a slot or + # window stamp is available RIGHT NOW, and answers false otherwise + # instead of waiting. acquire is for work that must happen; this is + # for work that should only happen if capacity is to spare - retry + # budgets, best-effort refreshes, opportunistic prefetch. A budget + # wants to say no, not to make you wait for a yes. + # @yield The admitted work, if any + # @return [Boolean] True when admitted (block, if given, was run) + def try_acquire(&block) + if @per + admitted = @window_mutex.synchronize do + now = clock + @stamps.reject! { |stamp| stamp <= now - @per } + @stamps << now if @stamps.size < @ceiling + end + return false unless admitted + + track { block&.call } + return true + end + + return false if @semaphore.blocking? + + @semaphore.acquire do + track { block&.call } + end + true + end + # Changes the ceiling while the limiter is live - the seam adaptive # throttles steer. Raising a concurrency ceiling resumes waiters # immediately; lowering it lets in-flight work finish and admits @@ -115,26 +148,41 @@ def and(other) private + # Counter updates share the window mutex so in_flight/high_water + # stay truthful under real threads too; the yielded work itself + # runs OUTSIDE the lock (holding it there would serialize all + # windowed work, which is the opposite of a rate limit) def track - @in_flight += 1 - @high_water = [@high_water, @in_flight].max + @window_mutex.synchronize do + @in_flight += 1 + @high_water = [@high_water, @in_flight].max + end yield ensure - @in_flight -= 1 + @window_mutex.synchronize { @in_flight -= 1 } end # Rolling-window admission: wait until fewer than ceiling - # acquisitions have started within the last `per` seconds + # acquisitions have started within the last `per` seconds. The + # check-and-stamp is atomic under the mutex; only the sleep + # happens outside it. def windowed_acquire loop do - now = clock - @stamps.reject! { |stamp| stamp <= now - @per } - break if @stamps.size < @ceiling - - sleep(@stamps.first + @per - now) + wait = @window_mutex.synchronize do + now = clock + @stamps.reject! { |stamp| stamp <= now - @per } + if @stamps.size < @ceiling + @stamps << now + nil + else + @stamps.first + @per - now + end + end + break if wait.nil? + + sleep(wait) end - @stamps << clock track { yield } end diff --git a/lib/agentic/relation_rules.rb b/lib/agentic/relation_rules.rb index 97da800..fbdc4a4 100644 --- a/lib/agentic/relation_rules.rb +++ b/lib/agentic/relation_rules.rb @@ -19,8 +19,52 @@ module Agentic module RelationRules SUPPORTED = %i[sum_lte requires mutually_exclusive].freeze + # Field types sum_lte may lawfully add together + NUMERIC_TYPES = %w[number integer].freeze + module_function + # Fail-fast validation of a relation declaration against the + # contract's declared inputs. A rule referencing an undeclared + # field is a typo that would otherwise fail open (presence + # relations) or crash mid-validation in the wrong error class + # (sum_lte over a string). Refusing to construct moves the failure + # to boot, where it names itself. + # @param id [Symbol, String] The rule's id (for the error message) + # @param definition [Hash] The rule definition + # @param inputs [Hash] The capability's declared inputs + # @raise [ArgumentError] For unknown relations, undeclared or + # ill-typed fields, or a missing limit + # @return [void] + def validate_declaration!(id, definition, inputs) + relation = definition[:relation] + unless SUPPORTED.include?(relation) + raise ArgumentError, "rule :#{id} has unknown relation #{relation.inspect} " \ + "(supported: #{SUPPORTED.map(&:inspect).join(", ")})" + end + + fields = definition[:fields] + if !fields.is_a?(Array) || fields.empty? + raise ArgumentError, "rule :#{id} (#{relation}) must declare fields: as a non-empty Array" + end + + undeclared = fields.reject { |field| inputs.key?(field) } + if undeclared.any? + raise ArgumentError, "rule :#{id} (#{relation}) references undeclared " \ + "input#{(undeclared.size == 1) ? "" : "s"} #{undeclared.map(&:inspect).join(", ")}" + end + + if relation == :sum_lte + raise ArgumentError, "rule :#{id} (sum_lte) must declare limit:" unless definition.key?(:limit) + + non_numeric = fields.reject { |field| NUMERIC_TYPES.include?(inputs[field][:type]) } + if non_numeric.any? + raise ArgumentError, "rule :#{id} (sum_lte) can only sum declared numbers; " \ + "#{non_numeric.map(&:inspect).join(", ")} #{(non_numeric.size == 1) ? "is" : "are"} not" + end + end + end + # Builds the predicate a relation declaration describes # @param definition [Hash] The rule definition ({relation:, fields:, ...}) # @return [Proc] inputs -> Boolean diff --git a/spec/agentic/round10_features_spec.rb b/spec/agentic/round10_features_spec.rb index a406dda..6d9861a 100644 --- a/spec/agentic/round10_features_spec.rb +++ b/spec/agentic/round10_features_spec.rb @@ -62,9 +62,10 @@ rules: {odd: {relation: :sum_gte, fields: [:a], limit: 1}} ) + # Since round 11 this fails even earlier - at validator construction expect { - Agentic::CapabilityValidator.new(bad).validate_inputs!(a: 1) - }.to raise_error(ArgumentError, /unknown rule relation :sum_gte/) + Agentic::CapabilityValidator.new(bad) + }.to raise_error(ArgumentError, /unknown relation :sum_gte/) end it "projects expressible relations into draft-07 keywords and x-agentic-rules" do diff --git a/spec/agentic/round11_features_spec.rb b/spec/agentic/round11_features_spec.rb new file mode 100644 index 0000000..4b1f77b --- /dev/null +++ b/spec/agentic/round11_features_spec.rb @@ -0,0 +1,136 @@ +# frozen_string_literal: true + +require "spec_helper" + +RSpec.describe "round 11 framework features" do + def task_named(description) + Agentic::Task.new(description: description, agent_spec: {"name" => "w", "instructions" => "work"}) + end + + describe "prompt cancel_plan" do + it "stops in-flight fibers and never starts queued tasks" do + agent_runs = [] + orchestrator = Agentic::PlanOrchestrator.new(concurrency_limit: 2) + 6.times do |i| + orchestrator.add_task(task_named("job#{i}"), agent: ->(t) { + agent_runs << t.description + sleep(0.1) + :ok + }) + end + + wall = nil + result = nil + Sync do + started = Process.clock_gettime(Process::CLOCK_MONOTONIC) + runner = Async { result = orchestrator.execute_plan } + Async do + sleep(0.02) + orchestrator.cancel_plan + end + runner.wait + wall = Process.clock_gettime(Process::CLOCK_MONOTONIC) - started + end + + expect(result.status).to eq(:canceled) + expect(wall).to be < 0.09 # not the 300ms a full run would take + expect(agent_runs.size).to eq(2) # only the two that were mid-flight + end + + it "can be called from inside a lifecycle hook without stopping itself" do + orchestrator = Agentic::PlanOrchestrator.new( + concurrency_limit: 1, + lifecycle_hooks: { + after_task_success: ->(task_id:, task:, result:, duration:) { orchestrator.cancel_plan } + } + ) + first = task_named("first") + second = task_named("second") + orchestrator.add_task(first, agent: ->(_t) { :ok }) + orchestrator.add_task(second, [first], agent: ->(_t) { :ok }) + + result = orchestrator.execute_plan + + expect(result.status).to eq(:canceled) + expect(orchestrator.execution_state[:completed]).to include(first.id) + end + end + + describe "fail-fast relation declarations" do + it "refuses to construct a validator for a sum_lte over an undeclared field" do + spec = Agentic::CapabilitySpecification.new( + name: "x", description: "x", version: "1.0.0", + inputs: {a: {type: "number"}}, + rules: {fits: {relation: :sum_lte, fields: [:a, :ghost], limit: 10}} + ) + + expect { + Agentic::CapabilityValidator.new(spec) + }.to raise_error(ArgumentError, /references undeclared input :ghost/) + end + + it "refuses sum_lte over a declared non-number" do + spec = Agentic::CapabilitySpecification.new( + name: "x", description: "x", version: "1.0.0", + inputs: {a: {type: "number"}, b: {type: "string"}}, + rules: {fits: {relation: :sum_lte, fields: [:a, :b], limit: 10}} + ) + + expect { + Agentic::CapabilityValidator.new(spec) + }.to raise_error(ArgumentError, /can only sum declared numbers; :b is not/) + end + + it "refuses presence relations over undeclared fields (fail-open typos)" do + spec = Agentic::CapabilitySpecification.new( + name: "x", description: "x", version: "1.0.0", + inputs: {express: {type: "boolean"}}, + rules: {customs: {relation: :requires, fields: [:express, :customs_kode]}} + ) + + expect { + Agentic::CapabilityValidator.new(spec) + }.to raise_error(ArgumentError, /undeclared input :customs_kode/) + end + end + + describe "RateLimit#try_acquire" do + it "admits without blocking in windowed mode, then answers false" do + budget = Agentic::RateLimit.new(2, per: 60) + + expect(budget.try_acquire).to be(true) + expect(budget.try_acquire).to be(true) + expect(budget.try_acquire).to be(false) # would have BLOCKED under acquire + end + + it "runs the block only when admitted, in concurrency mode" do + limit = Agentic::RateLimit.new(1) + ran = [] + + Sync do + holder = Async { limit.acquire { sleep(0.03) } } + sleep(0.01) + expect(limit.try_acquire { ran << :rejected_path }).to be(false) + holder.wait + end + expect(limit.try_acquire { ran << :admitted_path }).to be(true) + + expect(ran).to eq([:admitted_path]) + end + end + + describe "typed-fields-only projection" do + it "keeps relations over untyped fields out of draft-07 keywords but in x-agentic-rules" do + spec = Agentic::CapabilitySpecification.new( + name: "x", description: "x", version: "1.0.0", + inputs: {express: {}, customs_code: {type: "string"}}, + rules: {customs: {relation: :requires, fields: [:express, :customs_code]}} + ) + + schema = spec.to_json_schema + + expect(schema).not_to have_key("dependencies") + expect(schema["x-agentic-rules"].first["relation"]).to eq("requires") + end + end +end diff --git a/spec/agentic/round12_features_spec.rb b/spec/agentic/round12_features_spec.rb new file mode 100644 index 0000000..8fb9215 --- /dev/null +++ b/spec/agentic/round12_features_spec.rb @@ -0,0 +1,109 @@ +# frozen_string_literal: true + +require "spec_helper" + +RSpec.describe "round 12 framework features" do + def task_named(description) + Agentic::Task.new(description: description, agent_spec: {"name" => "w", "instructions" => "work"}) + end + + describe "remove_task" do + it "removes a pending leaf and its bookkeeping" do + orchestrator = Agentic::PlanOrchestrator.new + root = task_named("root") + leaf = task_named("leaf") + orchestrator.add_task(root, agent: ->(_t) { :ok }) + orchestrator.add_task(leaf, [root], agent: ->(_t) { :ok }) + + orchestrator.remove_task(leaf) + graph = orchestrator.graph + + expect(graph[:tasks].keys).to eq([root.id]) + expect(graph[:stats][:leaves]).to eq([root.id]) + expect(orchestrator.execution_state[:pending]).not_to include(leaf.id) + end + + it "refuses to orphan dependents, naming them" do + orchestrator = Agentic::PlanOrchestrator.new + root = task_named("root") + leaf = task_named("leaf") + orchestrator.add_task(root) + orchestrator.add_task(leaf, [root]) + + expect { + orchestrator.remove_task(root) + }.to raise_error(ArgumentError, /cannot remove root: leaf depend\(s\) on it/) + end + end + + describe "rewire_task" do + it "replaces dependencies and labels in place" do + orchestrator = Agentic::PlanOrchestrator.new + ingest = task_named("ingest") + prices = task_named("prices") + merge = task_named("merge") + orchestrator.add_task(ingest) + orchestrator.add_task(prices) # accidental second root + orchestrator.add_task(merge, [ingest, prices]) + + orchestrator.rewire_task(prices, [ingest]) + orchestrator.rewire_task(merge, needs: {base: ingest, prices: prices}) + + graph = orchestrator.graph + expect(graph[:stats][:roots]).to eq([ingest.id]) + labels = graph[:edges].select { |e| e[:to] == merge.id }.map { |e| e[:label] } + expect(labels).to contain_exactly(:base, :prices) + end + + it "refuses wiring to tasks outside the plan" do + orchestrator = Agentic::PlanOrchestrator.new + lone = task_named("lone") + orchestrator.add_task(lone) + + expect { + orchestrator.rewire_task(lone, ["ghost-id"]) + }.to raise_error(ArgumentError, /unknown task\(s\) ghost-id/) + end + + it "executes with the rewired shape" do + orchestrator = Agentic::PlanOrchestrator.new + first = task_named("first") + second = task_named("second") + orchestrator.add_task(first, agent: ->(_t) { "from-first" }) + orchestrator.add_task(second, agent: ->(t) { t.needs[:input] || "unwired" }) + + orchestrator.rewire_task(second, needs: {input: first}) + result = orchestrator.execute_plan + + expect(result.task_result(second.id).output).to eq("from-first") + end + end + + describe "thread-safe windowed bookkeeping" do + it "never over-admits under contending threads" do + limit = Agentic::RateLimit.new(50, per: 60) + + admitted = 8.times.map { + Thread.new { 500.times.count { limit.try_acquire } } + }.sum(&:value) + + expect(admitted).to eq(50) + end + + it "keeps blocking acquire correct under the mutex" do + limit = Agentic::RateLimit.new(3, per: 0.1) + stamps = [] + + Sync do + 6.times.map { + Async do + limit.acquire { stamps << Process.clock_gettime(Process::CLOCK_MONOTONIC) } + end + }.each(&:wait) + end + + expect(stamps.size).to eq(6) + expect(stamps.sort[3] - stamps.min).to be >= 0.09 + end + end +end