feat: unified testcase view#4394
Open
ashrafchowdury wants to merge 46 commits into
Open
Conversation
…rect testset imports Replace barrel import from `../testset` with direct imports from specific testset files (`revisionEntity`, `revisionSchema`, `store`) to break circular dependency that caused "Cannot access before initialization" errors in standalone consumers. Update pnpm workspace and package.json to include `apps/*` for design-mockups app support.
…mockup app references Update JSON-string UX design docs to reflect production component audit (2026-05-04). Add references to new Next.js mockup app at `web/apps/design-mockups/` which supersedes static HTML mockups. Clarify unique contributions of gaps 02, 03, 05, and 06 after discovering production already ships capabilities originally claimed as missing (CellContentPopover, collapsedFields machinery, groupColumns, ChatMessageList).
…-to-string editing Introduce gap-09 (variable provenance + usage state in playground execution item) to overview, variants README, and TypeChip. Add `unused`, `draft`, and `chain` chip variants for four-state variable classification (used/chain/draft/unused). Update PlaygroundExecutionItemCompact to unify string and null editing into one controlled-input branch so typing into a null field initializes as string without losing focus
…reamline mockup shell Add root-level serialized view (JSON/YAML) to ProposedDrillIn via Segmented toggle in row header. Users can paste a full testcase payload in one shot instead of typing field-by-field; edits auto-apply on keystroke with inline parse-error feedback. Remove `competitiveNotes` prop and styling from MockupPageShell to simplify page headers. Update already-shipped, alt-tree-pane, gap-01, and gap-02 page copy to focus
…pages Update gap-01, gap-03, gap-04, gap-05, and gap-06 solution link descriptions to explicitly reference the "kitchen-sink Vanuatu row" fixture instead of generic "pick a fixture" instructions. Specify which columns/fields demonstrate each gap's unique contribution (e.g., nested `geo.coordinates` for auto-expand, literal `"geo.region"` collision, 5-turn messages array with tool calls). Clarify that Tuvalu and Kiribati rows in the testset demonstrate missing-
…s and add kitchen-sink schema Add `FIXTURE_KITCHEN_SINK_INFERRED_SCHEMA` covering every column in the testset's known-union (country, population, nested geo/inputs/outputs, literal-dotted keys, messages array, stringified metadata). Update solutions-drill-in to always render the schema-aware form section regardless of active fixture—switch between Tuvalu schema and kitchen-sink schema based on selection. Update gap-07 page copy
…able validation from index Remove gap-07 (schema-aware edit form) and gap-08 (playground variable validation) entries from design mockups index page. These gaps were surfaced by competitive analysis but are being deprioritized or consolidated into other work.
…nt axes Separate type classification into two orthogonal axes per 2026-05-05 feedback: type primitive (what the value IS—string/number/boolean/null/object/array) and optional render hint (how it renders—markdown/stringified/messages/tool-calls). Remove `long-str` variant; long-form strings now show `[string][markdown]` chips. Update all classify functions across PlaygroundExecutionItem, PlaygroundExecutionItemCompact, Play
…rings in table cells Introduce `markdown` render hint alongside existing `stringified`, `messages`, and `tool-calls` hints. Cells containing strings >100 chars or with newlines now display `[string][markdown]` chip pair plus a one-line preview (first non-empty line, stripped of markdown syntax, truncated to 60 chars) with char/line count metadata. Column type detection promotes string columns to `markdown` hint when all observed strings match the heuristic; `stringified` takes precedence when
…own render-hint chip Relocate the short/long editor-mode toggle from the `[string]` type chip to the `[markdown]` render-hint chip per 2026-05-05 feedback. When a markdown render hint is visible, its popover owns the mode-switching action (plain inline input ↔ Lexical editor with markdown preview); the type chip's popover retains only type conversions. When no render hint exists (plain string), the type chip keeps the mode toggle
… use concrete collision values
Add `flexWrap: wrap` to ProposedDrillIn field header left section so type/render-hint/name chips (dotted-key, collision, shadowed) wrap to second line when narrow instead of being clipped by parent overflow:hidden. Replace placeholder AI-meta labels in dot-key collision fixtures with concrete values: literal `"geo.region"` = "Melanesia" (what `{{geo.region}}` resolves to under literal-key-first), nested `geo.region` = "Pacific Islands" (what `{{$.geo.region}}` JSONPath resolves to). Makes
…on items and prompt config view Add collapsible footer to PlaygroundExecutionItem and PlaygroundExecutionItemCompact showing testcase columns the prompt chain doesn't reference. Defaults closed; click to reveal column names as inline chips. Filter kitchen-sink execution-item inputs to only variables actually used/chain/draft (via IN_USE_VARIABLE_NAMES set) instead of dumping all 12 testcase columns—unused columns stay reachable via
…tionPlaygroundShell and add prompt config pairing Move PromptVariableValidation component logic into ProductionPlaygroundShell.tsx and delete the standalone file. Update solutions-playground to render prompt config alongside execution items in three-way compare grid, making the variable contract visible (prompt defines variables → testcase fills them). Add production playground shell section showing today's live surface before
… solutions-drill-in
Solutions · Drill-in is now three stacked side-by-side comparisons (status quo
vs proposal) at production drawer width:
- Original proposal — ProposedDrillIn (chips + auto-expand + inline messages
+ dotted-key handling).
- New proposal — ProposalV2DrillIn: per-field "View as ▾" dropdown
(Text / Markdown / Chat / Form / JSON / YAML) replacing the old row of
icons; recursive form view with always-on vertical rail; production
ChatMessageList for chat fields; SharedEditor for strings/code with a
scoped white-gutter override.
- Schema-aware form alternative — kept as the third row for comparison.
Other supporting changes:
- ProdTestcaseDrawerSurface: production-faithful drawer chrome (close arrow,
nav, "Testcase N" title, Add to queue, Fields/JSON Segmented, footer).
- SideBySide: optional third column for ad-hoc 3-way comparisons.
- StubDrillIn: forwards showAddControls / showDeleteControls /
getDefaultValueForType so mockups can match the production testcase drawer.
…event multiple-instance errors Add webpack resolve aliases pinning react (19.2.5/19.2.6), react-dom, jsx-runtime, and jotai (three workspace versions) to design-mockups' node_modules copies. Prevents "null dispatcher.useContext" runtime error and "Multiple Jotai instances" warning when transpiled workspace packages (@agenta/ui, @agenta/oss) resolve different copies than the host app.
…of abbreviations Replace abbreviated type labels (str/num/bool/obj/arr) with full names (string/number/boolean/object/array) in TypeChip and ChipConversionPopover. Makes chips more readable for SMEs who shouldn't need to mentally translate shorthand. Variant keys stay full-name (`json-object`, `json-array`) for TS precision; visible labels now drop the `json-` prefix for legibility.
…ropdown and consolidate warnings into single indicator Replace inline render-hint chips (markdown/stringified/messages/tool-calls) with per-field "View as ▾" dropdown matching the vocabulary used in per-object Form/JSON/YAML selects. String fields get Text/Markdown toggle; stringified gets Text/JSON/YAML; messages gets Chat/JSON/YAML; objects/arrays keep Form/JSON/YAML. Chip area now only carries type primitives + warnings. Consolid
…ware headers, and info indicator to ProposedDrillIn Add per-level expand/collapse-all buttons mirroring top-level toolbar so every nesting level acts the same—clicking the button in a field's header propagates forceCollapsed/collapseSignal to its children. Introduce depth-aware header backgrounds (four tints from #FAFAFA to #E2E6ED) so nesting depth reads at a glance; add hover/focus affordances via [data-drill-in-header] CSS
…h variable insertion and controlled testcase state
Add always-editable contenteditable body to ProductionPromptTemplate MessageCards with atomic `{{var}}` token spans (contenteditable="false") so users can type around tokens without breaking them. Add "+ Variable" popover per message that inserts tokens at caret position. Introduce `onMessagesChange` callback for controlled mode—parent pages can lift message state to derive referenced
feat(design-mockups): consolidate table cell warnings into single indicator and add column-type awareness Move per-cell warning chips ([dotted-key], [⚠ collision], [mixed]) into consolidated CellWarningsIndicator matching FieldWarningsIndicator pattern from drill-in—single Warning icon + tooltip listing all applicable warnings instead of repeating chips on every row. Add columnType prop so cells suppress their own type chip when value matches column's declared type (header already carries
Four-phase implementation plan covering TypeChip primitive extraction, testset table column headers (Mahmoud's proposal), testcase drawer ProposalV2 direction, and playground compact variable list. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…h view mode dropdowns and type chips for improved testcase editing.
… and test case visualization in a focused drawer view
…awerAdapter and integrate TestcaseDrawer for improved UI consistency
…seDrawerNavigation hook and update playground and evaluation components
…w mode support and improved type handling
…mponents - Migrate utility functions related to data type detection and message parsing from fieldUtils to drill-in utilities. - Update imports in DrillInContent, EntityDualViewEditor, and other components to use the new utility functions. - Remove redundant utility functions and streamline code in EvalRunDetails components. - Refactor view mode handling in DrillInFieldHeader and related components to use a consistent ViewMode type. - Enhance type safety by replacing string types with specific ViewMode types in relevant components. - Clean up unused imports and variables across multiple files for better readability and maintainability.
…oss various files
… and loadable state management
…alue retrieval in TestcasesTableShell
…r improved chip visibility management
…proved error handling
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
dosubot
Bot
added
the
size:XXL
ashrafchowdury
changed the title
testcase view
Contributor
Railway Preview Environment
|
…component from TurnMessageHeaderOptions
…ies for improved message handling
…de and collapse functionality
…ling and type inference - Simplified JSON parsing logic in DrillInContent to handle native values directly. - Updated isExpandable and getItemCount functions to differentiate between native and stringified values. - Enhanced detectDataType to support both native and string modes, ensuring accurate type detection. - Removed redundant JSON parsing in various utility functions, focusing on native types. - Adjusted TextField and other renderers to accommodate new value modes, ensuring consistent behavior across components. - Cleaned up unused imports and functions related to stringified JSON handling. - Removed inferRenderHint utility as it was deemed unnecessary with the new type inference approach.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.