diff --git a/.agentfactory/agents/rootcause-all/todos/rootcause_analysis.md b/.agentfactory/agents/rootcause-all/todos/rootcause_analysis.md new file mode 100644 index 0000000000..7c33cd3ea2 --- /dev/null +++ b/.agentfactory/agents/rootcause-all/todos/rootcause_analysis.md @@ -0,0 +1,225 @@ +# Root Cause Analysis: Blocking synchronous tool execution freezes Responses API polling inside async event loop + +**Date**: 2026-05-12 +**Status**: Root causes identified +**Problem File**: .agentfactory/agents/soldesign-plan/problem-summary-1.md +**Upstream Issue**: microsoft/agent-framework#5741 + +## Problem Summary + +When a tool registered via `@tool` contains blocking synchronous code (e.g., `time.sleep(120)`), the entire async event loop gets blocked. This prevents Responses API polling (`GET` endpoint) for `ResponsesAgentServerHost` from functioning, making the agent appear frozen until the sync function returns. + +## Concerns from Problem File + +| # | Concern | Verdict | Evidence Link | +|---|---------|---------|---------------| +| 1 | FunctionTool.__call__ executes sync functions directly on event loop thread | **VALIDATED** | [todos/rootcause_concern_1.md] | +| 2 | FunctionTool.invoke() non-observability code path blocks at line 682 | **VALIDATED** | [todos/rootcause_concern_2.md] | +| 3 | FunctionTool.invoke() observability code path blocks at line 733 | **VALIDATED** | [todos/rootcause_concern_3.md] | +| 4 | No sync-to-async offloading mechanism exists in FunctionTool (unlike FunctionExecutor) | **VALIDATED** | [todos/rootcause_concern_4.md] | +| 5 | ResponsesHostServer._handle_inner_agent runs agent.run() on same event loop as polling | **VALIDATED** | [todos/rootcause_concern_5.md] | +| 6 | MCP tool invocation path (_agents.py:1543) also calls invoke() in same async context | **VALIDATED** | [todos/rootcause_concern_6.md] | +| 7 | No test coverage for sync-blocking-event-loop scenario in test_tools.py | **VALIDATED** | [todos/rootcause_concern_7.md] | +| 8 | Existing asyncio.to_thread() pattern in _function_executor.py not applied to FunctionTool | **VALIDATED** | [todos/rootcause_concern_8.md] | + +Based on investigation of 8 concerns: +- **8 concerns VALIDATED** as contributing factors +- **0 concerns INVALIDATED** + +## Synthesized Root Cause(s) + +### Primary Root Cause + +**`FunctionTool` lacks sync-to-async thread offloading for synchronous tool functions.** + +`FunctionTool.__call__` (`_tools.py:511-538`) calls the wrapped function directly via `func(*args, **kwargs)` on the event loop thread with no thread offloading. The `invoke()` method (`_tools.py:682-683` and `733-734`) uses the pattern: + +```python +res = self.__call__(**call_kwargs) +result = await res if inspect.isawaitable(res) else res +``` + +The `inspect.isawaitable(res)` check only determines whether to `await` the return value — but for synchronous functions, the blocking has already completed by the time this check runs. There is zero use of `asyncio.to_thread()`, `run_in_executor()`, or `ThreadPoolExecutor` anywhere in `_tools.py`. + +This is a confirmed design gap: the same codebase's `FunctionExecutor` (`_function_executor.py:122-151`) solves this exact problem by detecting sync vs async at init time with `inspect.iscoroutinefunction(func)` and wrapping sync functions with `asyncio.to_thread()`. + +**Concerns establishing this**: #1 (direct execution), #2 (non-observability path), #3 (observability path), #4 (missing mechanism), #8 (existing pattern not applied). + +### Contributing Factor 1: No architectural isolation between tool execution and server request handling + +`ResponsesHostServer._handle_inner_agent` (`_responses.py:322`) runs `agent.run()` directly on the ASGI server's event loop. There is no separate thread pool, no dedicated event loop, and no `asyncio.to_thread()` wrapping of `agent.run()`. When a tool blocks the event loop, it blocks all concurrent HTTP request handling, SSE streaming, health checks, and polling. + +**Concern establishing this**: #5. + +### Contributing Factor 2: MCP path shares the same vulnerable code path + +The MCP tool invocation at `_agents.py:1543` calls the same `FunctionTool.invoke()` method. While `as_mcp_server()` wraps the agent itself in an async function (mitigating direct blocking at the top level), any synchronous sub-tools within the agent still block during the agent's `run()` execution via the same `FunctionTool.invoke()` path. + +**Concern establishing this**: #6. + +### Contributing Factor 3: No test coverage to catch the regression + +No tests in `test_tools.py` or `test_tools_future_annotations.py` verify that sync tool functions don't block the event loop. The workflow `test_function_executor.py` has a test for `_is_async` flags but doesn't actually verify thread execution end-to-end. A test that runs a blocking sync tool alongside a concurrent async task would have caught this immediately. + +**Concern establishing this**: #7. + +## Fishbone Diagram + +``` + Sync tool blocks event loop, + freezing Responses API polling + ▲ + ┌───────────────────────────────┼───────────────────────────────┐ + │ │ │ + [DESIGN GAP] [ARCHITECTURE] [QUALITY] + │ │ │ + ┌───────────┴───────────┐ ┌───────────┴───────────┐ ┌───────────┴───────────┐ + │ │ │ │ │ │ + #1 __call__ #4 No sync #5 Same event #6 MCP #7 No test #8 Pattern + calls func() detection loop for agent path coverage for exists in + directly mechanism run + HTTP shares sync blocking FunctionExecutor + │ │ serving same but not applied + #2 invoke() No iscoro- │ invoke() to FunctionTool + line 682 function() No thread pool │ + blocks check No loop isolation Sub-tools + │ still block + #3 invoke() + line 733 + blocks + holds + OTel span open +``` + +## 5-Whys Summary (Primary Root Cause Chain) + +1. **Why does Responses API polling freeze during sync tool execution?** + Because `FunctionTool.invoke()` executes sync functions directly on the event loop thread, blocking all concurrent async operations including HTTP polling. + +2. **Why does `invoke()` execute sync functions on the event loop thread?** + Because it calls `self.__call__(**call_kwargs)` which is a synchronous method that calls `self.func()` directly — the blocking happens before any awaitable check. + +3. **Why doesn't `invoke()` offload sync functions to a worker thread?** + Because `FunctionTool` has no mechanism to detect sync vs async functions. It does not call `inspect.iscoroutinefunction()` at init or invoke time, and has no `_is_async` flag. + +4. **Why wasn't the thread-offloading pattern applied to `FunctionTool`?** + The pattern (`asyncio.to_thread()`) exists in `FunctionExecutor` (`_function_executor.py:137-151`) but `FunctionTool` predates it by over a year. The two components were developed independently. + +5. **Why wasn't the gap caught by tests?** + No test in `test_tools.py` verifies that sync tool functions don't block the event loop. All sync tool tests verify correctness of return values only, not concurrency behavior. + +## Solution + +### Files to Modify + +| File | Change | +|------|--------| +| `python/packages/core/agent_framework/_tools.py` | In `FunctionTool.invoke()`, wrap synchronous function calls with `asyncio.to_thread()` at both code paths (lines 682 and 733). Detect sync vs async at construction time by storing an `_is_async` flag using `inspect.iscoroutinefunction()`. | +| `python/packages/core/tests/core/test_tools.py` | Add tests verifying that synchronous tool functions do not block the event loop — a concurrent async task must make progress during sync tool execution. | + +### Implementation Steps + +**Step 1: Add sync/async detection to `FunctionTool.__init__`** + +In `_tools.py`, in the `FunctionTool.__init__` method (around line 297), after `self.func = func` is set (line 373), add: + +```python +self._is_async = inspect.iscoroutinefunction(func) +``` + +This follows the same pattern used by `FunctionExecutor` at `_function_executor.py:122`. + +**Step 2: Modify `FunctionTool.invoke()` to offload sync functions** + +Replace the two blocking call sites. In the non-observability path (line 682-683): + +```python +# Before (blocking): +res = self.__call__(**call_kwargs) +result = await res if inspect.isawaitable(res) else res + +# After (non-blocking): +if self._is_async: + res = self.__call__(**call_kwargs) + result = await res if inspect.isawaitable(res) else res +else: + result = await asyncio.to_thread(self.__call__, **call_kwargs) +``` + +Apply the same change to the observability path (line 733-734): + +```python +# Before (blocking): +res = self.__call__(**call_kwargs) +result = await res if inspect.isawaitable(res) else res + +# After (non-blocking): +if self._is_async: + res = self.__call__(**call_kwargs) + result = await res if inspect.isawaitable(res) else res +else: + result = await asyncio.to_thread(self.__call__, **call_kwargs) +``` + +**Step 3: Ensure `asyncio` is imported** + +Verify `import asyncio` exists at the top of `_tools.py`. (It likely does given the `asyncio.gather` usage at line 1769.) + +**Step 4: Add tests for sync tool non-blocking behavior** + +In `test_tools.py`, add a test that: +1. Creates a sync `@tool` that sleeps briefly (e.g., 0.1s using `time.sleep`) +2. Runs the tool via `invoke()` concurrently with an async flag-setting task using `asyncio.gather` +3. Verifies the async task completed (was not blocked) — i.e., the flag was set before or during the sync sleep, proving the event loop was not blocked + +```python +async def test_sync_tool_does_not_block_event_loop(): + flag = asyncio.Event() + + @tool + def blocking_tool() -> str: + """A tool that blocks.""" + time.sleep(0.1) + return "done" + + async def set_flag(): + flag.set() + + result_task = asyncio.create_task(blocking_tool.invoke()) + flag_task = asyncio.create_task(set_flag()) + await asyncio.gather(result_task, flag_task) + assert flag.is_set() +``` + +### Enforcement Level + +| Step | Level | Notes | +|------|-------|-------| +| Store `_is_async` flag in `__init__` | **Interlock** | Makes sync detection automatic at construction; no developer action needed | +| Wrap sync calls with `asyncio.to_thread()` in `invoke()` | **Interlock** | Code makes event loop blocking impossible for sync tool functions | +| Add non-blocking test | **Runtime guard** | Test fails if regression reintroduces blocking behavior | + +All primary fixes are at **Interlock** level — code makes the failure impossible. No Instruction or Advisory level fixes needed. + +### Verification Steps + +1. Run existing tool tests to verify no regressions: + ```bash + cd python/packages/core && pytest -m "not integration" tests/core/test_tools.py tests/core/test_tools_future_annotations.py -v + ``` + **Expected**: All existing tests pass. + +2. Run the new non-blocking test: + ```bash + cd python/packages/core && pytest tests/core/test_tools.py::test_sync_tool_does_not_block_event_loop -v + ``` + **Expected**: Test passes, confirming sync tools run in a thread without blocking the event loop. + +3. Run workflow executor tests to verify no interaction: + ```bash + cd python/packages/core && pytest tests/workflow/test_function_executor.py -v + ``` + **Expected**: All workflow executor tests pass unchanged. + +### Code Convention Issues + +- `FunctionTool` in `_tools.py` does not follow the same async-awareness pattern as `FunctionExecutor` in `_function_executor.py`. After the fix, both will correctly handle sync function offloading, establishing a consistent pattern across the codebase. +- The `_tools.py` file does not import `asyncio` directly for `asyncio.to_thread` — it uses `asyncio.gather` via a local reference. The import should be added at the module level if not already present. diff --git a/.agentfactory/agents/rootcause-all/todos/rootcause_concern_1.md b/.agentfactory/agents/rootcause-all/todos/rootcause_concern_1.md new file mode 100644 index 0000000000..aeae7e5e11 --- /dev/null +++ b/.agentfactory/agents/rootcause-all/todos/rootcause_concern_1.md @@ -0,0 +1,65 @@ +# Concern #1 Investigation: FunctionTool.__call__ executes sync functions directly on event loop thread + +**Investigated by**: Sub-agent +**Date**: 2026-05-12 + +## Verdict: VALIDATED + +## Summary + +The `FunctionTool.__call__` method (line 511 of `_tools.py`) executes the wrapped function synchronously and directly on the calling thread, with no thread offloading whatsoever. When this method is called from the `invoke()` async method (lines 682-683 and 733-734), a synchronous function runs directly on the event loop thread. The only protection is a post-hoc `await` if the result happens to be awaitable -- but for sync functions that block (e.g., `time.sleep(120)`, file I/O, network calls), the event loop is fully blocked for the duration. This is in stark contrast to `FunctionExecutor` in the workflows module, which explicitly uses `asyncio.to_thread()` for sync functions. + +## 5-Whys Analysis + +### Why 1: Why does a sync tool function block the event loop? +Because `FunctionTool.__call__` (file: `python/packages/core/agent_framework/_tools.py:511-538`) calls the wrapped function directly with `func(*args, **kwargs)` (line 535) or `func(self._instance, *args, **kwargs)` (line 534). There is no check for whether `func` is synchronous or asynchronous -- it simply calls it. + +### Why 2: Why does `__call__` not offload sync functions to a thread? +Because `__call__` has no concept of sync vs async distinction. It does not inspect whether the function is a coroutine function. It returns whatever the function returns -- if async, a coroutine; if sync, the raw value. The async/sync differentiation is deferred to `invoke()`. + +### Why 3: Why doesn't `invoke()` offload sync functions to a thread before calling `__call__`? +Because `invoke()` (file: `python/packages/core/agent_framework/_tools.py:562-767`) uses this pattern at lines 682-683 and 733-734: +```python +res = self.__call__(**call_kwargs) +result = await res if inspect.isawaitable(res) else res +``` +This calls `__call__` first (synchronously on the event loop thread), and only then checks if the result is awaitable. For a sync function, `__call__` has already executed and returned by the time `inspect.isawaitable(res)` evaluates to `False`. The blocking has already occurred. + +### Why 4: Why wasn't thread offloading added for sync functions in FunctionTool? +The codebase demonstrates awareness of this pattern -- `FunctionExecutor` (file: `python/packages/core/agent_framework/_workflows/_function_executor.py:131-151`) explicitly checks `inspect.iscoroutinefunction(func)` at construction time and wraps sync functions with `asyncio.to_thread()`: +```python +# Sync function without context - wrap to make async and ignore context using thread pool +async def wrapped_func(message: Any, ctx: WorkflowContext[Any]) -> Any: + return await asyncio.to_thread(func, message) +``` +However, `FunctionTool` was designed earlier (or independently) and never received this protection. The `__call__` / `invoke()` split appears designed primarily for the awaitable-coroutine case (async tool functions return coroutines that get awaited), not for protecting the event loop from blocking sync code. + +### Why 5: Why is this especially dangerous in the agent auto-invocation loop? +Because the auto-invocation path uses `asyncio.gather()` at line 1769-1771: +```python +execution_results = await asyncio.gather(*[ + invoke_with_termination_handling(function_call, seq_idx) + for seq_idx, function_call in enumerate(function_calls) +]) +``` +While `asyncio.gather()` enables concurrent execution of multiple tool calls, if any single tool is a blocking sync function, it blocks the entire event loop thread. This means: +1. All other concurrent tool calls in the same gather are stalled +2. The Responses API polling loop cannot receive streaming events +3. The entire agent framework becomes unresponsive for the duration of the blocking call + +## Evidence Gathered + +| # | Evidence | Location | Finding | +|---|----------|----------|---------| +| 1 | `FunctionTool.__call__` implementation | `_tools.py:511-538` | Calls `func(*args, **kwargs)` directly with no thread offloading; no `inspect.iscoroutinefunction` check | +| 2 | `invoke()` async method | `_tools.py:682-683, 733-734` | Pattern `res = self.__call__(**call_kwargs); result = await res if inspect.isawaitable(res) else res` -- blocking happens before the awaitable check | +| 3 | No `run_in_executor` or `asyncio.to_thread` in `_tools.py` | `_tools.py` (entire file) | `grep -n "run_in_executor\|asyncio.to_thread\|ThreadPool\|thread" _tools.py` returns zero matches | +| 4 | `FunctionExecutor` correctly offloads sync functions | `_function_executor.py:131-151` | Uses `asyncio.to_thread(func, message)` for sync functions, with explicit `inspect.iscoroutinefunction()` check at line 121 | +| 5 | Other modules use `asyncio.to_thread` for I/O | `_sessions.py:1008,1036`, `_skills.py:248,252`, `_harness/_todo.py:339,369` | Multiple modules in the same codebase correctly use `asyncio.to_thread()` for blocking operations | +| 6 | Auto-invocation uses `asyncio.gather` | `_tools.py:1769-1771` | All tool calls in a single iteration are gathered concurrently -- a single blocking sync tool stalls them all | +| 7 | `FunctionTool` constructor stores `func` with no wrapping | `_tools.py:297-339` | No `iscoroutinefunction` check, no thread wrapping at construction time | +| 8 | `FunctionExecutor` test validates thread execution | `tests/workflow/test_function_executor.py:480-504` | Tests confirm sync functions run in separate thread via `asyncio.to_thread` in FunctionExecutor | + +## Conclusion + +This concern is **VALIDATED**. The `FunctionTool.__call__` method executes synchronous functions directly on the event loop thread with zero thread offloading. The `invoke()` method's `inspect.isawaitable()` check only handles the case where the wrapped function is a coroutine (async def) -- it does not protect against blocking synchronous code. This is a design gap: the `FunctionExecutor` in the workflows module solves this exact problem with `asyncio.to_thread()`, but `FunctionTool` -- which is the primary tool mechanism for agents -- lacks this protection entirely. Any `@tool`-decorated synchronous function that performs blocking I/O (network requests, file operations, `time.sleep`, subprocess calls) will block the entire async event loop, preventing Responses API polling, concurrent tool execution, and all other async operations. diff --git a/.agentfactory/agents/rootcause-all/todos/rootcause_concern_2.md b/.agentfactory/agents/rootcause-all/todos/rootcause_concern_2.md new file mode 100644 index 0000000000..ce1023d070 --- /dev/null +++ b/.agentfactory/agents/rootcause-all/todos/rootcause_concern_2.md @@ -0,0 +1,150 @@ +# Concern #2 Investigation: FunctionTool.invoke() non-observability code path blocks at line 682 + +**Investigated by**: Sub-agent (rootcause-all) +**Date**: 2026-05-12 + +## Verdict: VALIDATED + +## Summary + +The `FunctionTool.invoke()` method at line 682 of `_tools.py` calls `self.__call__(**call_kwargs)` synchronously on the event loop. For synchronous (non-async) tool functions, this call blocks the entire async event loop until the function returns. The subsequent `await res if inspect.isawaitable(res) else res` on line 683 does NOT prevent blocking -- it only helps for async functions that return awaitables. The blocking occurs during the `self.__call__()` call itself, before line 683 is even reached. Both the non-observability path (line 682) and the observability path (line 733) share this identical defect. This is compounded by the fact that multiple tool calls are dispatched concurrently via `asyncio.gather` (line 1769), meaning a single blocking synchronous tool will stall all concurrent tool executions. + +## 5-Whys Analysis + +### Why #1: Why does line 682 block the event loop? + +Because `self.__call__(**call_kwargs)` (line 682) is a regular synchronous method call. When the wrapped function (`self.func`) is a synchronous Python function, it executes entirely on the current thread -- the event loop thread. No coroutine is created; no thread offloading occurs. The call completes synchronously before execution proceeds to line 683. + +**Evidence** (`_tools.py:511-535`): +```python +def __call__(self, *args: Any, **kwargs: Any) -> Any: + """Call the wrapped function with the provided arguments.""" + # ... guard checks ... + self.invocation_count += 1 + try: + func = self.func + if func is None: + raise ToolException(...) + if self._instance is not None: + return func(self._instance, *args, **kwargs) + return func(*args, **kwargs) # <-- direct synchronous call + except Exception: + self.invocation_exception_count += 1 + raise +``` + +The `__call__` method calls `func()` directly. For a synchronous function, this returns a plain value, not an awaitable. The blocking happens here. + +### Why #2: Why doesn't the `await res if inspect.isawaitable(res) else res` on line 683 prevent blocking? + +Because `inspect.isawaitable(res)` only checks whether the *return value* is awaitable. For a synchronous function, `res` is already the final computed result (a plain value like a string, dict, etc.), not a coroutine. The `else res` branch executes, which is a no-op -- it just passes the already-computed value through. The blocking already happened on line 682 during `self.__call__()`. + +**Evidence** (`_tools.py:682-683`): +```python +res = self.__call__(**call_kwargs) # line 682: blocks HERE for sync functions +result = await res if inspect.isawaitable(res) else res # line 683: too late, blocking already done +``` + +The `isawaitable` check is only useful for async functions where `__call__` returns a coroutine object (which is fast/non-blocking), and the actual work happens during the `await`. + +### Why #3: Why doesn't `invoke()` offload synchronous functions to a thread? + +Because neither `invoke()` nor `__call__()` contains any `asyncio.to_thread()`, `loop.run_in_executor()`, or equivalent thread-offloading mechanism. A grep for `run_in_executor` and `to_thread` in `_tools.py` returns zero results. + +**Evidence**: `grep -rn 'run_in_executor\|to_thread' _tools.py` produces no matches. + +This is in contrast to `FunctionExecutor` in `_workflows/_function_executor.py` (lines 137-151), which explicitly wraps synchronous functions with `asyncio.to_thread()`: + +```python +# From _function_executor.py:137-151 +elif self._has_context and not self._is_async: + async def wrapped_func(message, ctx): + return await asyncio.to_thread(func, message, ctx) # correct pattern +else: + async def wrapped_func(message, ctx): + return await asyncio.to_thread(func, message) # correct pattern +``` + +The workflow subsystem correctly identifies and handles sync functions. The tool subsystem does not. + +### Why #4: Why is this especially harmful in practice? + +Because multiple tool calls from a single model response are executed concurrently via `asyncio.gather` at line 1769: + +```python +execution_results = await asyncio.gather(*[ + invoke_with_termination_handling(function_call, seq_idx) + for seq_idx, function_call in enumerate(function_calls) +]) +``` + +If even one of those concurrent tool invocations wraps a blocking synchronous function, it blocks the event loop thread. This prevents all other concurrent coroutines (including other tool invocations in the same `gather`, network I/O, heartbeats, timeouts) from making progress. The concurrency promised by `asyncio.gather` becomes illusory. + +### Why #5: Why was this not caught or designed differently? + +The `isawaitable` pattern on line 683 suggests the authors were aware that both sync and async functions could be wrapped, but used a pattern that only handles the *return value* rather than the *execution* of synchronous functions. The correct pattern (used in `_function_executor.py`) is to check `inspect.iscoroutinefunction(func)` at registration time and wrap synchronous functions with `asyncio.to_thread()`. The `FunctionTool` class does not perform this check or wrapping anywhere in its initialization or invocation path. + +## Evidence Gathered + +### 1. The blocking call site (both code paths are identical) + +**Non-observability path** (`_tools.py:679-683`): +```python +if not OBSERVABILITY_SETTINGS.ENABLED: + logger.info(f"Function name: {self.name}") + logger.debug(f"Function arguments: {observable_kwargs}") + res = self.__call__(**call_kwargs) # BLOCKS for sync functions + result = await res if inspect.isawaitable(res) else res # too late +``` + +**Observability path** (`_tools.py:733-734`): +```python + res = self.__call__(**call_kwargs) # BLOCKS for sync functions + result = await res if inspect.isawaitable(res) else res # too late +``` + +### 2. __call__ performs direct synchronous invocation (`_tools.py:529-535`) +```python +func = self.func +if func is None: + raise ToolException(...) +if self._instance is not None: + return func(self._instance, *args, **kwargs) +return func(*args, **kwargs) +``` + +### 3. No thread offloading exists in _tools.py + +Zero occurrences of `run_in_executor`, `to_thread`, `ThreadPoolExecutor`, or `ProcessPoolExecutor` in the entire 2673+ line `_tools.py` file. + +### 4. Correct pattern exists in the same codebase (`_function_executor.py:8-9, 137-151`) + +The `FunctionExecutor` class documents and implements the correct pattern: +> "Synchronous functions are executed in a thread pool using asyncio.to_thread() to avoid blocking the event loop." + +### 5. Concurrent tool dispatch via asyncio.gather (`_tools.py:1769-1771`) +```python +execution_results = await asyncio.gather(*[ + invoke_with_termination_handling(function_call, seq_idx) + for seq_idx, function_call in enumerate(function_calls) +]) +``` + +This makes the blocking problem worse: a single blocking sync tool stalls all concurrent tool executions. + +### 6. @tool decorator accepts both sync and async functions (`_tools.py:1267-1272`) +```python +# Async functions are also supported +@tool(approval_mode="never_require") +async def async_get_weather(location: str) -> str: + '''Get weather asynchronously.''' + return f"Weather in {location}" +``` + +The documentation says "also supported," implying sync functions are the primary/default use case, making the blocking problem likely to affect many users. + +## Conclusion + +**VALIDATED**. The concern is confirmed. `FunctionTool.invoke()` directly calls synchronous tool functions on the event loop thread at line 682 (and identically at line 733 in the observability path). The `await res if inspect.isawaitable(res) else res` check on line 683 is insufficient because it only handles the return value -- the blocking has already occurred during the `self.__call__()` invocation. There is no `asyncio.to_thread()` or `run_in_executor()` wrapping anywhere in `_tools.py`. The correct pattern (wrapping sync functions in `asyncio.to_thread()`) already exists in the same codebase in `_function_executor.py`, demonstrating both awareness of the problem and a proven fix. The impact is amplified by `asyncio.gather` at line 1769 which runs multiple tools concurrently -- a single blocking sync tool will stall all of them. + +The fix would involve detecting synchronous functions (via `inspect.iscoroutinefunction`) and wrapping their execution in `asyncio.to_thread()` within the `invoke()` method, similar to what `FunctionExecutor` already does. diff --git a/.agentfactory/agents/rootcause-all/todos/rootcause_concern_3.md b/.agentfactory/agents/rootcause-all/todos/rootcause_concern_3.md new file mode 100644 index 0000000000..f581d89a82 --- /dev/null +++ b/.agentfactory/agents/rootcause-all/todos/rootcause_concern_3.md @@ -0,0 +1,111 @@ +# Concern #3 Investigation: FunctionTool.invoke() observability code path blocks at line 733 + +**Investigated by**: Sub-agent +**Date**: 2026-05-12 + +## Verdict: VALIDATED + +## Summary + +The observability-enabled code path in `FunctionTool.invoke()` at line 733 of `_tools.py` suffers from the same event-loop-blocking behavior as the non-observability path at line 682. When a synchronous function is wrapped as a `FunctionTool`, calling `self.__call__(**call_kwargs)` executes it directly on the event loop thread. The `inspect.isawaitable()` check on line 734 only helps for async functions -- for sync functions, the blocking has already occurred by the time the check runs. Additionally, the observability path wraps this blocking call inside an OpenTelemetry span context manager and `perf_counter` timing, meaning any event-loop stall also inflates span duration and holds the span open, preventing other spans from being properly parented during the stall. + +## 5-Whys Analysis + +**Why #1**: Why does the observability code path block the event loop? +Because at line 733 (`_tools.py:733`), `res = self.__call__(**call_kwargs)` calls the wrapped function synchronously, directly on the current (event-loop) thread. There is no `await`, no `asyncio.to_thread()`, and no `run_in_executor()` call. + +**Why #2**: Why is `self.__call__()` synchronous even in an `async def invoke()` method? +Because `FunctionTool.__call__()` (`_tools.py:511-538`) is a regular (non-async) method. It directly calls `self.func(...)` or `self.func(self._instance, ...)`. The return value may or may not be a coroutine, but the call itself always executes synchronously on the calling thread. + +**Why #3**: Why doesn't the `inspect.isawaitable()` check on line 734 prevent blocking? +Because `inspect.isawaitable(res)` only checks whether the *return value* is awaitable (a coroutine object). For a synchronous function, the function body has already executed to completion by the time `res` is assigned. The blocking work happened during the `self.__call__()` call on line 733, not during the `await` on line 734. The `isawaitable` check is useful only for async functions that return coroutines. + +**Why #4**: Why is this particularly concerning in the observability path? +Because the observability path wraps the blocking call inside: +- An OpenTelemetry span context (`with get_function_span(attributes=attributes) as span:` at line 725) +- A `perf_counter()` timing bracket (lines 730-735) +- Sensitive data logging conditionals (lines 728-729) + +If the sync function blocks for a long time, the span remains open and active during the entire stall. This means: (a) span duration is inflated by the blocking time, (b) any other concurrent work that creates child spans during the stall would be incorrectly parented under this tool's span, and (c) the span's `end_on_exit=True` behavior (observability.py:1823) delays span export. + +**Why #5**: Why wasn't `asyncio.to_thread()` or `run_in_executor()` used for sync functions? +The code uses a single pattern for both sync and async functions: call `self.__call__()`, then conditionally `await` if the result is awaitable. This pattern was likely chosen for simplicity but it fails to account for the event-loop-blocking nature of synchronous function execution. There is no detection of whether the wrapped function is sync vs async before the call, and no offloading mechanism for sync functions. + +## Evidence Gathered + +### Evidence 1: The non-observability path (lines 682-683) +```python +# _tools.py:682-683 +res = self.__call__(**call_kwargs) +result = await res if inspect.isawaitable(res) else res +``` +This is the baseline pattern. Same blocking issue, but without span/timing overhead. + +### Evidence 2: The observability path (lines 725-767) +```python +# _tools.py:725-767 +with get_function_span(attributes=attributes) as span: + attributes[OtelAttr.MEASUREMENT_FUNCTION_TAG_NAME] = self.name + logger.info(f"Function name: {self.name}") + if OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED: + logger.debug(f"Function arguments: {serializable_kwargs}") + start_time_stamp = perf_counter() + end_time_stamp: float | None = None + try: + res = self.__call__(**call_kwargs) # <-- LINE 733: BLOCKS HERE + result = await res if inspect.isawaitable(res) else res # <-- LINE 734: too late for sync + end_time_stamp = perf_counter() + except Exception as exception: + end_time_stamp = perf_counter() + ... + finally: + duration = (end_time_stamp or perf_counter()) - start_time_stamp + span.set_attribute(OtelAttr.MEASUREMENT_FUNCTION_INVOCATION_DURATION, duration) + self._invocation_duration_histogram.record(duration, attributes=attributes) +``` +The blocking `self.__call__()` at line 733 runs inside the span context and between `perf_counter()` calls. The span is held open for the entire duration of the sync function execution. + +### Evidence 3: FunctionTool.__call__() is synchronous (lines 511-538) +```python +# _tools.py:511-538 +def __call__(self, *args: Any, **kwargs: Any) -> Any: + """Call the wrapped function with the provided arguments.""" + ... + self.invocation_count += 1 + try: + func = self.func + if func is None: + raise ToolException(f"Function '{self.name}' has no implementation.") + if self._instance is not None: + return func(self._instance, *args, **kwargs) + return func(*args, **kwargs) + except Exception: + self.invocation_exception_count += 1 + raise +``` +This is a plain `def` method. It calls `func(...)` directly. For sync functions, all work happens inline before returning. + +### Evidence 4: get_function_span() uses end_on_exit=True (observability.py:1819-1825) +```python +# observability.py:1819-1825 +def get_function_span(attributes): + return get_tracer().start_as_current_span( + name=f"{attributes[OtelAttr.OPERATION]} {attributes[OtelAttr.TOOL_NAME]}", + attributes=attributes, + set_status_on_exception=False, + end_on_exit=True, + record_exception=False, + ) +``` +The span is set as the "current span" in OpenTelemetry context. It remains current throughout the blocking execution, meaning any instrumented code that runs during the stall (e.g., HTTP clients inside the sync function) would be incorrectly parented under this span. + +### Evidence 5: No executor offloading anywhere in the file +``` +$ grep -n 'run_in_executor\|to_thread' _tools.py +(no results) +``` +There is no use of `asyncio.to_thread()` or `loop.run_in_executor()` anywhere in `_tools.py`. The entire file lacks any mechanism for offloading synchronous work to a thread pool. + +## Conclusion + +**VALIDATED**. The observability code path at line 733 of `_tools.py` blocks the event loop for synchronous functions in the same way as the non-observability path at line 682, but with the additional impact of holding an OpenTelemetry span open during the stall. The `inspect.isawaitable()` check on line 734 cannot prevent the blocking because it only examines the return value after the synchronous function has already completed execution. The fix would require detecting whether the wrapped function is sync (e.g., via `inspect.iscoroutinefunction(self.func)`) and offloading sync functions to a thread executor before entering the span timing bracket, or at minimum within it. diff --git a/.agentfactory/agents/rootcause-all/todos/rootcause_concern_4.md b/.agentfactory/agents/rootcause-all/todos/rootcause_concern_4.md new file mode 100644 index 0000000000..57a4fb2132 --- /dev/null +++ b/.agentfactory/agents/rootcause-all/todos/rootcause_concern_4.md @@ -0,0 +1,108 @@ +# Concern #4 Investigation: No sync-to-async offloading in FunctionTool + +**Investigated by**: Sub-agent +**Date**: 2026-05-12 + +## Verdict: VALIDATED + +## Summary + +FunctionTool does NOT offload synchronous functions to a thread pool. When a sync function is wrapped by FunctionTool, it is called directly on the event loop thread. The result is then checked with `inspect.isawaitable()` -- but a sync function never returns an awaitable, so the raw (already-computed) result is used as-is. This means any blocking sync function (I/O, CPU-intensive work) will block the entire asyncio event loop. By contrast, FunctionExecutor explicitly detects sync vs async functions at init time and wraps sync functions with `asyncio.to_thread()`. + +## 5-Whys Analysis + +**Why #1: Why does FunctionTool block the event loop when wrapping a sync function?** +Because `FunctionTool.__call__()` (`_tools.py:511`) calls `self.func(*args, **kwargs)` directly, with no thread offloading. The `invoke()` method (`_tools.py:682-683`) calls `self.__call__(**call_kwargs)` and then does `await res if inspect.isawaitable(res) else res`. For a sync function, `res` is never awaitable, so the blocking call has already completed on the event loop thread. + +**Why #2: Why does FunctionTool not detect whether the wrapped function is sync or async?** +Because `FunctionTool.__init__()` (`_tools.py:297`) stores the function as `self.func = func` (line 373) but never inspects it with `inspect.iscoroutinefunction()` and never stores an `_is_async` flag. There is no sync-vs-async detection anywhere in FunctionTool's initialization. + +**Why #3: Why does FunctionExecutor handle this correctly but FunctionTool does not?** +FunctionExecutor was designed with explicit awareness of the sync/async distinction. At `_function_executor.py:122`, it stores `self._is_async = inspect.iscoroutinefunction(func)`. Then at lines 132-151, it creates four distinct wrapper branches: +- async + context: direct pass-through +- sync + context: wraps with `asyncio.to_thread(func, message, ctx)` +- async + no context: wraps to ignore context +- sync + no context: wraps with `asyncio.to_thread(func, message)` + +FunctionTool predates this design and was never updated to include the same offloading mechanism. + +**Why #4: Why was FunctionTool not updated when FunctionExecutor got this capability?** +FunctionTool and FunctionExecutor serve different purposes in the architecture -- FunctionTool wraps functions as model-callable tools, while FunctionExecutor wraps functions as workflow executors. They were developed independently and the sync-offloading pattern was not back-ported to FunctionTool. + +**Why #5: Why is this a problem in practice?** +Because any user who registers a sync function as a tool (which is the most natural thing to do -- e.g., file I/O, HTTP calls, database queries) will block the event loop whenever the model invokes that tool. This prevents concurrent tool execution (the framework uses `asyncio.gather` at `_tools.py:1769` for parallel tool calls), degrades responsiveness, and can cause timeouts in production systems. + +## Evidence Gathered + +### Evidence 1: FunctionTool.__call__ does NOT offload sync functions +File: `python/packages/core/agent_framework/_tools.py`, lines 511-538 + +```python +def __call__(self, *args: Any, **kwargs: Any) -> Any: + """Call the wrapped function with the provided arguments.""" + # ...validation checks... + self.invocation_count += 1 + try: + func = self.func + if func is None: + raise ToolException(f"Function '{self.name}' has no implementation.") + # If we have a bound instance, call the function with self + if self._instance is not None: + return func(self._instance, *args, **kwargs) + return func(*args, **kwargs) # <-- DIRECT CALL, no thread offloading + except Exception: + self.invocation_exception_count += 1 + raise +``` + +### Evidence 2: FunctionTool.invoke uses isawaitable check but no preemptive offloading +File: `python/packages/core/agent_framework/_tools.py`, lines 682-683 (non-observability path) and 733-734 (observability path) + +```python +res = self.__call__(**call_kwargs) +result = await res if inspect.isawaitable(res) else res +``` + +The `inspect.isawaitable(res)` check only handles the case where the wrapped function itself is async (returns a coroutine). It does NOT offload sync functions to a thread -- the sync function has already executed and blocked the event loop by the time this check runs. + +### Evidence 3: FunctionTool.__init__ does NOT store _is_async flag +File: `python/packages/core/agent_framework/_tools.py`, lines 297-404 + +The entire `__init__` method stores `self.func = func` at line 373 but never calls `inspect.iscoroutinefunction(func)` and never creates any `_is_async` attribute. + +### Evidence 4: FunctionExecutor DOES detect and offload sync functions +File: `python/packages/core/agent_framework/_workflows/_function_executor.py`, lines 122-151 + +```python +# Determine if the function is an async function +self._is_async = inspect.iscoroutinefunction(func) + +# ...then four branches: +if self._has_context and self._is_async: + wrapped_func = func # direct pass-through +elif self._has_context and not self._is_async: + async def wrapped_func(message, ctx): + return await asyncio.to_thread(func, message, ctx) # OFFLOADED +elif not self._has_context and self._is_async: + async def wrapped_func(message, ctx): + return await func(message) +else: + async def wrapped_func(message, ctx): + return await asyncio.to_thread(func, message) # OFFLOADED +``` + +### Evidence 5: No asyncio.to_thread usage in FunctionTool +A grep for `to_thread` and `iscoroutinefunction` in `_tools.py` returns zero matches. The only `asyncio` import at line 5 is used solely for `asyncio.gather` at line 1769 (parallel tool execution), not for thread offloading. + +### Evidence 6: Parallel tool execution is undermined +File: `python/packages/core/agent_framework/_tools.py`, line 1769 + +```python +execution_results = await asyncio.gather(*[ +``` + +The framework attempts to execute multiple tool calls in parallel using `asyncio.gather`. But if any of those tools wrap sync functions, they block the event loop, serializing what should be concurrent execution. + +## Conclusion + +This concern is **VALIDATED**. FunctionTool lacks any sync-to-async offloading mechanism. Sync functions passed to FunctionTool are called directly on the event loop thread, blocking it for the duration of their execution. FunctionExecutor already solves this problem correctly using `inspect.iscoroutinefunction()` detection at init time and `asyncio.to_thread()` wrapping for sync functions. The same pattern should be applied to FunctionTool to prevent event loop blocking, especially since the framework relies on `asyncio.gather` for concurrent tool invocation. diff --git a/.agentfactory/agents/rootcause-all/todos/rootcause_concern_5.md b/.agentfactory/agents/rootcause-all/todos/rootcause_concern_5.md new file mode 100644 index 0000000000..21512bf6f0 --- /dev/null +++ b/.agentfactory/agents/rootcause-all/todos/rootcause_concern_5.md @@ -0,0 +1,120 @@ +# Concern #5 Investigation: ResponsesHostServer runs agent on same event loop as polling + +**Investigated by**: Sub-agent +**Date**: 2026-05-12 + +## Verdict: VALIDATED + +## Summary + +The concern is validated. `ResponsesHostServer._handle_inner_agent` calls `agent.run()` using `await` on the same event loop that services HTTP requests (including polling). When a synchronous tool blocks the event loop thread inside `agent.run()`, it prevents all other async activity on that loop -- including any concurrent HTTP request handling, SSE streaming, health checks, and ASGI request processing. + +The root cause is that `FunctionTool.invoke()` calls sync tools directly on the event loop thread without offloading to a thread pool, and there is no architectural isolation between tool execution and server request handling. + +## 5-Whys Analysis + +### Why 1: Why does a blocking sync tool affect Responses API handling? + +Because `ResponsesHostServer._handle_response()` (line 278-288) awaits `_handle_inner_agent()`, which in turn awaits `self._agent.run()` (line 322 for non-streaming, line 341 for streaming). The entire agent execution, including all tool calls, runs as a coroutine on the ASGI server's event loop. + +**Evidence**: `_responses.py:322`: +```python +response = await self._agent.run(stream=False, **run_kwargs) +``` + +### Why 2: Why does agent.run() block the event loop when a sync tool runs? + +Because `agent.run()` (in `_agents.py:889-967`) delegates to `self._call_chat_client()` which calls `client.get_response()`. The `FunctionInvocationLayer.get_response()` implements a tool-calling loop (`_tools.py:2376-2498`) that awaits tool execution. The tool invocation calls `FunctionTool.invoke()` which directly calls the sync function on the event loop thread. + +**Evidence**: `_tools.py:682-683` (in `FunctionTool.invoke()`): +```python +res = self.__call__(**call_kwargs) +result = await res if inspect.isawaitable(res) else res +``` + +If the tool function is synchronous, `self.__call__()` returns a non-awaitable result. The call executes synchronously on the current thread -- the event loop thread -- blocking it completely. + +### Why 3: Why doesn't the framework offload sync tool functions to a thread pool? + +The `FunctionTool.invoke()` method uses a simple `inspect.isawaitable()` check (line 683) to decide whether to `await` or use the result directly. There is no `asyncio.to_thread()` or `loop.run_in_executor()` wrapping for synchronous tool functions in the agent tool invocation path. + +This contrasts with the **workflow** path where `FunctionExecutor` (`_function_executor.py:137-151`) explicitly wraps sync functions with `asyncio.to_thread()`: +```python +async def wrapped_func(message: Any, ctx: WorkflowContext[Any]) -> Any: + return await asyncio.to_thread(func, message, ctx) +``` + +The agent tool path does not have this protection. + +### Why 4: Why is there no isolation between tool execution and the ASGI server? + +The `ResponsesHostServer` extends `ResponsesAgentServerHost` (an ASGI application from `azure.ai.agentserver`). The response handler is registered via `self.response_handler(self._handle_response)` (line 276). This handler is invoked as a coroutine within the ASGI request/response cycle. There is: +- No separate event loop for agent execution +- No dedicated thread pool for tool invocation +- No `asyncio.to_thread()` wrapping of `agent.run()` +- No task isolation or cancellation boundary + +The ASGI server (Starlette-based, running on Hypercorn) runs `_handle_response` as part of an HTTP request handler coroutine. When this coroutine blocks, the entire ASGI server's event loop stalls. + +### Why 5: Why does this matter for the Responses API specifically? + +The Responses API uses a pattern where: +1. A client POSTs to `/responses` to start agent execution +2. For streaming, the server sends SSE events back as the agent produces output +3. For non-streaming, the server must complete the full agent run before responding +4. Health/readiness checks run on the same ASGI app + +If a sync tool blocks the event loop: +- **Non-streaming**: The HTTP response cannot be sent until the tool unblocks +- **Streaming**: SSE event emission freezes; the client may time out +- **Health checks**: The readiness endpoint becomes unresponsive, potentially causing the hosting infrastructure to kill the container +- **Concurrent requests**: All other requests to the server are blocked + +## Evidence Gathered + +### Evidence 1: _handle_inner_agent awaits agent.run() directly +File: `python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses.py` +Lines 290-334. Both streaming (line 341) and non-streaming (line 322) paths use `await` on the same event loop. + +### Evidence 2: FunctionTool.invoke() runs sync tools on event loop thread +File: `python/packages/core/agent_framework/_tools.py` +Lines 682-683 (non-observability path) and lines 733-734 (observability path): +```python +res = self.__call__(**call_kwargs) +result = await res if inspect.isawaitable(res) else res +``` +No `asyncio.to_thread()` or `run_in_executor()` wrapping exists for sync tool functions. + +### Evidence 3: Tool calls run via asyncio.gather but still on same loop +File: `python/packages/core/agent_framework/_tools.py` +Line 1769: +```python +execution_results = await asyncio.gather(*[ + invoke_with_termination_handling(function_call, seq_idx) ... +]) +``` +`asyncio.gather` provides concurrency for multiple tool calls but they all run on the same event loop. A blocking sync call in any one of them blocks the loop. + +### Evidence 4: Workflow path has protection, agent path does not +File: `python/packages/core/agent_framework/_workflows/_function_executor.py` +Lines 137-151 show explicit `asyncio.to_thread()` wrapping for sync workflow functions. This protection does not exist in the agent tool invocation path (`_tools.py:FunctionTool.invoke()`). + +### Evidence 5: Server is a standard ASGI app with no isolation +File: `python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses.py` +Line 208: `class ResponsesHostServer(ResponsesAgentServerHost):` +Line 276: `self.response_handler(self._handle_response)` + +The base class comes from `azure-ai-agentserver-responses` which depends on `azure-ai-agentserver-core` which in turn depends on `hypercorn` and `starlette` (confirmed in `python/uv.lock:1126`). The server runs as a single-threaded asyncio ASGI application. + +### Evidence 6: No thread pool or separate event loop anywhere in foundry_hosting +Search for `to_thread`, `run_in_executor`, `ThreadPool`, `thread_pool` in the foundry_hosting package returns zero results in `_responses.py`. The only threading-related code is in `FileBasedFunctionApprovalStorage` (lines 155, 202, 205) which correctly uses `asyncio.to_thread` for file I/O -- but this pattern is not applied to tool execution. + +## Conclusion + +The concern is **VALIDATED**. `ResponsesHostServer._handle_inner_agent` runs `agent.run()` directly on the ASGI server's event loop with no isolation. When the agent invokes a synchronous tool via `FunctionTool.invoke()`, the tool function executes synchronously on the event loop thread, blocking all concurrent async operations including HTTP request handling, SSE streaming, and health checks. + +The architectural gap is clear: the workflow path (`FunctionExecutor`) correctly wraps sync functions with `asyncio.to_thread()`, but the agent tool path (`FunctionTool.invoke()`) does not. Additionally, the hosting server itself does not provide any isolation layer (separate thread, separate event loop, or `asyncio.to_thread` wrapping of `agent.run()`). + +Two complementary fixes would address this: +1. **Tool-level**: Wrap sync tool functions with `asyncio.to_thread()` in `FunctionTool.invoke()` (matching the pattern already used in `FunctionExecutor`) +2. **Server-level**: Run `agent.run()` in an isolated context (e.g., via `asyncio.to_thread` or a dedicated executor) within `ResponsesHostServer._handle_inner_agent` diff --git a/.agentfactory/agents/rootcause-all/todos/rootcause_concern_6.md b/.agentfactory/agents/rootcause-all/todos/rootcause_concern_6.md new file mode 100644 index 0000000000..1c49191a3c --- /dev/null +++ b/.agentfactory/agents/rootcause-all/todos/rootcause_concern_6.md @@ -0,0 +1,114 @@ +# Concern #6 Investigation: MCP tool invocation path blocks + +**Investigated by**: Sub-agent +**Date**: 2026-05-12 + +## Verdict: VALIDATED + +## Summary + +The MCP tool invocation path at `_agents.py:1543` calls `agent_tool.invoke()` in the same async context with no thread offloading, sharing the exact same blocking-vulnerable code path as the normal (non-MCP) tool invocation. When a synchronous tool function is invoked via the MCP server, the event loop is blocked during execution because `FunctionTool.invoke()` calls `self.__call__()` directly and only checks `inspect.isawaitable()` after the call returns -- meaning synchronous functions execute inline on the event loop thread. + +## 5-Whys Analysis + +### Why 1: Why does the MCP tool invocation path block? +Because `_agents.py:1543` calls `await agent_tool.invoke(arguments=args_instance)` directly in the MCP handler coroutine `_call_tool()`, which runs on the event loop. + +**Evidence**: `_agents.py:1523-1543`: +```python +@server.call_tool() +async def _call_tool( + name: str, arguments: dict[str, Any] +) -> Sequence[...]: + ... + result = await agent_tool.invoke(arguments=args_instance) +``` + +### Why 2: Why does `agent_tool.invoke()` block? +Because `FunctionTool.invoke()` at `_tools.py:682-683` calls `self.__call__(**call_kwargs)` synchronously and only conditionally awaits: +```python +res = self.__call__(**call_kwargs) +result = await res if inspect.isawaitable(res) else res +``` +If `self.func` is a synchronous function, `__call__` executes it inline on the event loop thread before returning. The `isawaitable` check only helps if the function is already async. + +**Evidence**: `_tools.py:511-535` (`__call__` method): +```python +def __call__(self, *args: **kwargs): + ... + func = self.func + if self._instance is not None: + return func(self._instance, *args, **kwargs) + return func(*args, **kwargs) +``` + +### Why 3: Why is there no thread offloading for synchronous tool functions? +Because `FunctionTool.invoke()` was not designed with thread isolation. Unlike `_workflows/_function_executor.py` (which explicitly uses `asyncio.to_thread()` for sync functions at lines 139 and 151), the `FunctionTool` path has no such protection. + +**Evidence**: Searching for `run_in_executor`, `to_thread`, and `ThreadPoolExecutor` in `_tools.py` yields zero results. The workflow `FunctionExecutor` at `_workflows/_function_executor.py:36-151` demonstrates the correct pattern: +```python +# Sync function with context - wrap to make async using thread pool +async def wrapped_func(message, ctx): + return await asyncio.to_thread(func, message, ctx) +``` +This pattern is completely absent from `FunctionTool`. + +### Why 4: Why is the MCP path particularly affected? +Because the MCP server handler `_call_tool()` runs within the MCP server's event loop (via `server.run()` on the same asyncio loop), and when `agent_tool.invoke()` blocks, it blocks the entire MCP server from processing any other requests. The MCP server is typically a single-threaded async server (using stdio or streamable HTTP transport), so blocking the event loop freezes all MCP communication. + +**Evidence**: `samples/02-agents/mcp/agent_as_mcp_server.py:73-74`: +```python +async with stdio_server() as (read_stream, write_stream): + await server.run(read_stream, write_stream, server.create_initialization_options()) +``` + +### Why 5: Why is this the same issue as the non-MCP path? +Because both the MCP path (`_agents.py:1543`) and the normal path (`_tools.py:1527, 1570`) call the same `FunctionTool.invoke()` method, which has the same lack of thread offloading. The root cause is in `FunctionTool.invoke()` itself, not in either caller. + +**Evidence**: All three invocation sites call the same method: +- MCP path: `_agents.py:1543` -- `await agent_tool.invoke(arguments=args_instance)` +- Normal path (no middleware): `_tools.py:1527` -- `await tool.invoke(arguments=args, context=direct_context, ...)` +- Normal path (with middleware): `_tools.py:1570` -- `return await tool.invoke(arguments=context_obj.arguments, ...)` + +## Evidence Gathered + +### 1. MCP handler executes in the event loop +File: `python/packages/core/agent_framework/_agents.py:1523-1550` +The `_call_tool` handler is an `async def` registered with `@server.call_tool()`. It runs in the MCP server's event loop. The `agent_tool.invoke()` call at line 1543 is awaited directly with no isolation. + +### 2. FunctionTool.invoke() runs sync functions inline +File: `python/packages/core/agent_framework/_tools.py:682-683` +```python +res = self.__call__(**call_kwargs) +result = await res if inspect.isawaitable(res) else res +``` +For sync functions, `self.__call__()` executes the function completely before returning, blocking the event loop for the entire duration. + +### 3. No thread offloading anywhere in FunctionTool +File: `python/packages/core/agent_framework/_tools.py` +A search for `run_in_executor`, `to_thread`, and `ThreadPoolExecutor` yields zero matches in this file. There is no mechanism to offload synchronous function execution to a thread pool. + +### 4. Workflow FunctionExecutor has the fix but FunctionTool does not +File: `python/packages/core/agent_framework/_workflows/_function_executor.py:137-151` +The workflow system correctly detects sync functions and wraps them with `asyncio.to_thread()`. This pattern was not applied to `FunctionTool`. + +### 5. MCP path uses `as_tool()` which creates an async wrapper +File: `python/packages/core/agent_framework/_agents.py:497,545-564` +The `as_tool()` method creates an `async def _agent_wrapper` that calls `self.run(stream=True)` and `await stream.get_final_response()`. Since the wrapper itself is async, the `isawaitable` check at `_tools.py:683` will correctly await it. **However**, this only applies when an Agent is exposed as an MCP server via `as_mcp_server()`. If a user registers a raw synchronous `FunctionTool` and exposes it differently, the blocking issue remains. + +### 6. Nuance: as_mcp_server specifically wraps the agent +File: `python/packages/core/agent_framework/_agents.py:1497` +```python +agent_tool = self.as_tool(name=self._get_agent_name()) +``` +The `as_mcp_server()` method converts the entire agent into a single tool using `as_tool()`, which creates an async wrapper. This means the specific `as_mcp_server()` path is **not directly blocking** because the tool function is async. However, the underlying `FunctionTool.invoke()` still lacks thread offloading as a general mechanism, and any sub-tools within the agent that are synchronous will block when the agent runs them during its `self.run()` call. + +## Conclusion + +**VALIDATED** -- The MCP tool invocation path at `_agents.py:1543` calls the same `FunctionTool.invoke()` method that lacks thread offloading for synchronous functions. While the specific `as_mcp_server()` path mitigates direct blocking by wrapping the agent in an async function, the underlying issue persists: + +1. `FunctionTool.invoke()` has no thread offloading for sync functions (unlike `FunctionExecutor` in the workflow system). +2. Any synchronous tools registered on the agent will block the event loop when invoked during the agent's `run()` call, which is triggered from the MCP handler. +3. The MCP server is single-threaded async, so blocking the event loop freezes all MCP communication. + +The root cause is shared with the non-MCP path: `FunctionTool.invoke()` at `_tools.py:682-683` executes synchronous functions inline on the event loop thread without offloading to a thread pool. diff --git a/.agentfactory/agents/rootcause-all/todos/rootcause_concern_7.md b/.agentfactory/agents/rootcause-all/todos/rootcause_concern_7.md new file mode 100644 index 0000000000..190e73ef62 --- /dev/null +++ b/.agentfactory/agents/rootcause-all/todos/rootcause_concern_7.md @@ -0,0 +1,79 @@ +# Concern #7 Investigation: No test coverage for sync-blocking scenario + +**Investigated by**: Sub-agent +**Date**: 2026-05-12 + +## Verdict: VALIDATED + +## Summary + +There are zero tests in test_tools.py or test_tools_future_annotations.py that verify sync FunctionTool functions do not block the asyncio event loop. The FunctionTool.invoke() method calls sync functions directly on the event loop thread without using asyncio.to_thread(), and no test exercises or validates this behavior. The only file that even mentions the scenario is test_function_executor.py (for the workflow system), but even that test is incomplete -- it checks metadata flags rather than actually proving non-blocking behavior. + +## Evidence Gathered + +### 1. test_tools.py -- No sync-blocking tests + +Reviewed all 1,466 lines. The file contains tests for: +- Tool decorator behavior (sync and async) +- Schema validation and serialization +- Telemetry/OTEL span recording +- Input parsing +- FunctionInvocationContext injection +- Skip-parsing behavior +- Tool normalization/flattening + +Key observations: +- `test_tool_decorator_with_async` (line 283) tests async tool creation but does NOT verify it runs without blocking the event loop. +- `test_tool_invoke_telemetry_async_function` (line 702) tests telemetry on an async tool but does NOT verify non-blocking behavior. +- Multiple sync tool tests (e.g., `test_tool_decorator` at line 40) test correctness of sync tool invocation but never verify they don't block the event loop. +- No test uses `asyncio.to_thread`, `time.sleep` (for blocking simulation), `threading.get_ident()`, or any event loop blocking detection. + +### 2. test_tools_future_annotations.py -- No sync-blocking tests + +Reviewed all 135 lines. This file focuses exclusively on PEP 563 (from __future__ import annotations) compatibility: +- FunctionInvocationContext parameter exclusion under PEP 563 +- Optional type resolution +- Forward reference fallback + +No tests related to sync blocking or event loop behavior. + +### 3. test_function_executor.py -- Incomplete coverage + +The workflow's FunctionExecutor has a test `test_sync_function_thread_execution` (line 477) that is explicitly named for this scenario, but it is INCOMPLETE. The test: +- Creates a sync executor with `time.sleep(0.01)` (line 490) +- Checks `_is_async` and `_has_context` metadata flags (lines 499-500) +- Contains this revealing comment (line 502-503): "The actual thread execution test would require a full workflow setup, but the important thing is that asyncio.to_thread is used in the wrapper" +- Does NOT actually run the function through the event loop to verify non-blocking behavior +- Does NOT verify the function runs on a different thread from the event loop thread +- The `execution_thread_id` variable (line 483) is set up but never actually asserted against + +### 4. Source code analysis -- FunctionTool.invoke() does NOT use asyncio.to_thread + +The FunctionTool.invoke() method in `_tools.py` (lines 682-683 and 733-734) executes functions as: +```python +res = self.__call__(**call_kwargs) +result = await res if inspect.isawaitable(res) else res +``` +This means sync functions are called directly on the event loop thread. There is no `asyncio.to_thread()` wrapper for sync FunctionTools, unlike the workflow FunctionExecutor which does use `asyncio.to_thread()` (lines 139 and 151 of `_function_executor.py`). + +### 5. Broader search -- No relevant tests anywhere + +Searched all files under `python/packages/core/tests/` for: +- `asyncio.to_thread` -- Found only in `test_sessions.py` (for file I/O, unrelated to tools) +- `time.sleep` -- Found only in `test_sessions.py` (for concurrent write testing) and `test_function_executor.py` (incomplete test above) +- `event.loop` / `blocking` / `_is_async` -- No relevant hits in tool test files + +## Conclusion + +This concern is VALIDATED on two levels: + +1. **Missing test coverage**: There is no test anywhere in the test suite that verifies sync FunctionTool functions do not block the event loop. The closest test (`test_sync_function_thread_execution` in test_function_executor.py) is for the workflow FunctionExecutor (not FunctionTool) and is itself incomplete -- it only checks metadata flags without actually executing the function through an async context. + +2. **Underlying bug confirmed by source analysis**: The FunctionTool.invoke() method in `_tools.py` does NOT use `asyncio.to_thread()` for sync functions. It calls them directly on the event loop thread via `self.__call__()`. This is in contrast to the workflow FunctionExecutor which correctly uses `asyncio.to_thread()`. A test for this scenario would immediately reveal the blocking behavior as a bug. + +### Files examined: +- `/python/packages/core/tests/core/test_tools.py` -- 1,466 lines, zero sync-blocking tests +- `/python/packages/core/tests/core/test_tools_future_annotations.py` -- 135 lines, zero sync-blocking tests +- `/python/packages/core/tests/workflow/test_function_executor.py` -- 968 lines, one incomplete sync-blocking test +- `/python/packages/core/agent_framework/_tools.py` -- FunctionTool.invoke() confirmed to NOT use asyncio.to_thread() +- `/python/packages/core/agent_framework/_workflows/_function_executor.py` -- FunctionExecutor confirmed to correctly use asyncio.to_thread() diff --git a/.agentfactory/agents/rootcause-all/todos/rootcause_concern_8.md b/.agentfactory/agents/rootcause-all/todos/rootcause_concern_8.md new file mode 100644 index 0000000000..b7a48496e5 --- /dev/null +++ b/.agentfactory/agents/rootcause-all/todos/rootcause_concern_8.md @@ -0,0 +1,99 @@ +# Concern #8 Investigation: asyncio.to_thread() pattern not applied to FunctionTool + +**Investigated by**: Sub-agent +**Date**: 2026-05-12 + +## Verdict: VALIDATED + +## Summary + +FunctionExecutor in `_function_executor.py` explicitly detects sync vs async functions using `inspect.iscoroutinefunction()` and wraps synchronous functions with `asyncio.to_thread()` to avoid blocking the event loop. FunctionTool in `_tools.py` has no such mechanism -- it calls synchronous functions directly on the event loop thread via `self.__call__()`, then checks `inspect.isawaitable(res)` on the return value. For sync functions, `isawaitable` returns False and the result is used directly, meaning the blocking call already completed on the event loop thread. This is particularly dangerous because multiple tool calls are executed concurrently via `asyncio.gather()`, so one blocking sync tool will stall all other concurrent tool executions. + +## 5-Whys Analysis + +### Why 1: Why does FunctionTool block the event loop when calling sync functions? +Because `FunctionTool.invoke()` calls `self.__call__(**call_kwargs)` which directly invokes the wrapped function synchronously. The result is then checked with `inspect.isawaitable(res)` -- but for sync functions, this check happens AFTER the blocking call has already completed. + +**Evidence**: `_tools.py:682-683`: +```python +res = self.__call__(**call_kwargs) +result = await res if inspect.isawaitable(res) else res +``` +And `_tools.py:733-734` (same pattern in the observability branch): +```python +res = self.__call__(**call_kwargs) +result = await res if inspect.isawaitable(res) else res +``` + +### Why 2: Why doesn't FunctionTool detect whether the wrapped function is sync or async? +Because FunctionTool's `__init__` never inspects the function with `inspect.iscoroutinefunction()` and stores no `_is_async` flag. It has no sync/async awareness at construction time. + +**Evidence**: `_tools.py:297-404` -- the entire `__init__` method stores `self.func = func` at line 373 but never checks `inspect.iscoroutinefunction(func)`. Compare with `_function_executor.py:122`: +```python +self._is_async = inspect.iscoroutinefunction(func) +``` + +### Why 3: Why doesn't FunctionTool wrap sync functions with asyncio.to_thread()? +Because FunctionTool relies on a post-hoc `isawaitable` check on the return value rather than a pre-call decision based on function type. The `isawaitable` pattern cannot prevent blocking -- it only determines how to handle the already-computed result. + +**Evidence**: FunctionExecutor wraps sync functions proactively at construction time (`_function_executor.py:136-151`): +```python +elif self._has_context and not self._is_async: + async def wrapped_func(message, ctx): + return await asyncio.to_thread(func, message, ctx) +... +else: + async def wrapped_func(message, ctx): + return await asyncio.to_thread(func, message) +``` + +FunctionTool has no equivalent wrapping anywhere in its code. + +### Why 4: Why was this pattern not applied to FunctionTool when it was applied to FunctionExecutor? +FunctionTool predates FunctionExecutor significantly. FunctionTool evolved from the original tool infrastructure (earliest commits circa 2024, commit `3449902b`), while FunctionExecutor was added much later (commit `ed8461aa`, September 2025) specifically with async-safety built in from the start. The two components were developed by different authors at different times in different subsystems (tools vs. workflows), and the improvement was never backported. + +### Why 5: Why is this a real problem and not just theoretical? +Because the framework executes multiple tool calls concurrently using `asyncio.gather()` (`_tools.py:1769`): +```python +execution_results = await asyncio.gather(*[ + invoke_with_termination_handling(function_call, seq_idx) + for seq_idx, function_call in enumerate(function_calls) +]) +``` +If any tool wraps a sync function that does I/O (file reads, HTTP calls, database queries), it blocks the entire event loop, preventing all other concurrent tool invocations from progressing. This defeats the purpose of `asyncio.gather()` parallelism. + +## Evidence Gathered + +### FunctionExecutor (correct pattern) +- **File**: `python/packages/core/agent_framework/_workflows/_function_executor.py` +- **Line 122**: `self._is_async = inspect.iscoroutinefunction(func)` -- detects async at init +- **Lines 132-151**: Four-way branching based on `_is_async` and `_has_context`, wrapping sync functions with `asyncio.to_thread()` in both cases (with and without context) +- **Imports**: `import asyncio` at line 18, `import inspect` at line 19 + +### FunctionTool (missing pattern) +- **File**: `python/packages/core/agent_framework/_tools.py` +- **Line 373**: `self.func = func` -- stores function without any async detection +- **No `_is_async` attribute** exists on FunctionTool +- **No `asyncio.to_thread` usage** for tool function invocation (only used at line 1769 for `asyncio.gather`) +- **Line 5**: `import asyncio` is present but only used for `asyncio.gather` in the function invocation loop +- **Lines 682-683, 733-734**: `isawaitable` check happens AFTER the sync call completes -- too late to prevent blocking + +### The @tool decorator +- **File**: `_tools.py:1305-1325` +- The `@tool` decorator creates a `FunctionTool` directly, passing through the user's function without any async wrapping +- Both sync and async functions are accepted (documented in examples at lines 1242-1272), but sync functions get no thread offloading + +### Parallel execution path +- **File**: `_tools.py:1769-1771` +- Multiple tool calls are dispatched concurrently via `asyncio.gather()` +- Each tool call eventually reaches `tool.invoke()` which calls `self.__call__()` synchronously for sync tools +- A single blocking sync tool will serialize what should be parallel execution + +### No historical justification found +- `git log --all --oneline --grep="to_thread" -- python/packages/core/agent_framework/_tools.py` returned zero results +- No comments in the code explain why `asyncio.to_thread()` is not used +- No commits reference a deliberate decision to exclude thread offloading from FunctionTool + +## Conclusion + +This concern is **VALIDATED**. The `asyncio.to_thread()` pattern correctly implemented in `FunctionExecutor` (`_function_executor.py:136-151`) was never applied to `FunctionTool` (`_tools.py`). FunctionTool calls sync functions directly on the event loop thread, which blocks the event loop and defeats `asyncio.gather()` parallelism when multiple tools are invoked concurrently. This is not a theoretical issue -- it affects any user who defines a sync tool function that performs I/O operations (the common case, given that the `@tool` decorator documentation shows sync function examples as the primary usage pattern). The fix would be to detect `inspect.iscoroutinefunction(self.func)` at construction time and wrap sync functions with `asyncio.to_thread()` in the `invoke()` method, mirroring the FunctionExecutor approach. diff --git a/.designs/1/api.md b/.designs/1/api.md new file mode 100644 index 0000000000..a82cdaef61 --- /dev/null +++ b/.designs/1/api.md @@ -0,0 +1,153 @@ +# D1: API & Interface Design + +## Dimension Summary + +The source generator system exposes three user-facing APIs: (1) attribute declarations on executor classes/methods, (2) the generated code interface (ConfigureProtocol override), and (3) diagnostic messages surfaced in the IDE/build. The design challenge is reconciling the plan's specified attribute shapes with the actual evolved implementation. + +## Key Discrepancies from Source + +The plan (source.md AC-3 through AC-5) specifies attribute signatures that already exist in the codebase with minor variations: +- Plan says `YieldsMessageAttribute` -> actual is `YieldsOutputAttribute` (codebase-snapshot.md section 3) +- Plan says `AttributeTargets.Class` only for SendsMessage/YieldsOutput -> actual allows `Class | Method` (SendsMessageAttribute.cs:32, YieldsOutputAttribute.cs:32) +- Plan says `ConfigureRoutes(RouteBuilder)` -> actual is `ConfigureProtocol(ProtocolBuilder)` (Executor.cs:216, SourceBuilder.cs:72) +- Plan says `WFGEN001-006` -> actual is `MAFGENWF001-007` (DiagnosticDescriptors.cs) + +--- + +## Option A1: Preserve Current Attribute Names and Signatures (RECOMMENDED) + +Keep the existing attribute names (`YieldsOutputAttribute`, `SendsMessageAttribute` with `Class | Method` targets) and the evolved `ConfigureProtocol(ProtocolBuilder)` pattern. + +**Trade-offs:** +- (+) Zero breaking changes for existing consumers +- (+) The `Class | Method` target is strictly more capable than `Class` only +- (+) `YieldsOutput` is more semantically precise than `YieldsMessage` (outputs are yielded, messages are sent) +- (+) `ConfigureProtocol` unifies route + type registration into a single fluent builder +- (-) Source.md AC-5 says "YieldsMessageAttribute" verbatim; the actual name differs +- (-) Source.md AC-9 describes separate `ConfigureSentTypes()`/`ConfigureYieldTypes()` methods; actual unifies these + +**Reversibility:** Difficult -- renaming attributes or splitting ConfigureProtocol would break all downstream consumers. + +**Constraint compliance:** +- C-4 (clean break): Compliant. Users inherit from `Executor`, not `ReflectingExecutor`. +- C-5 (handler accessibility): Compliant. SemanticAnalyzer.cs:462 (`AnalyzeHandler`) does not check accessibility. +- C-6 (partial modifier): Compliant. SemanticAnalyzer.cs:367-380 (`IsPartialClass`) checks for `partial`. + +**Dependencies produced:** Attribute shapes consumed by D2 (data model), D6 (integration). +**Dependencies required:** None. + +**Risks:** +- Severity: Low. Risk that plan-vs-actual naming discrepancy causes confusion during review. +- Mitigation: Document discrepancies in design-doc.md traceability table. + +--- + +## Option A2: Rename to Match Plan Exactly (YieldsMessageAttribute, Split ConfigureRoutes/ConfigureSentTypes/ConfigureYieldTypes) + +Rename `YieldsOutputAttribute` to `YieldsMessageAttribute`, restrict SendsMessage/YieldsMessage to `AttributeTargets.Class` only, and split the generated code back into three separate override methods. + +**REJECTED: reverses [commit 0756c457] (.NET: [BREAKING] Update type names and source generator to reduce conflicts)** + +The commit `0756c457` was a deliberate breaking change that renamed types and restructured the generated code to avoid conflicts. Reversing this would undo a deliberate architectural decision documented in the git history. + +Additionally, reverses the evolution to `ConfigureProtocol(ProtocolBuilder)` which is the actual abstract method on Executor.cs:216. + +**Trade-offs:** +- (+) Matches source.md AC-5, AC-9 verbatim +- (-) Breaks all existing users of `YieldsOutputAttribute` +- (-) Reintroduces the naming conflicts that commit `0756c457` was designed to fix +- (-) `ConfigureRoutes`/`ConfigureSentTypes`/`ConfigureYieldTypes` as separate overrides does not exist on the actual `Executor` base class + +**Reversibility:** Difficult. + +--- + +## Option A3: Add Type Aliases for Plan Names While Keeping Actual Names + +Add `YieldsMessageAttribute` as a type-forwarding alias to `YieldsOutputAttribute`, allowing both names. Keep the evolved `ConfigureProtocol` pattern. + +**Trade-offs:** +- (+) Satisfies AC-5 literally (a type named `YieldsMessageAttribute` exists) +- (+) No breaking change +- (-) Two names for the same concept increases cognitive load +- (-) Source generator must detect both attribute names, increasing complexity +- (-) Aliases create long-term maintenance burden + +**Reversibility:** Easy -- alias can be removed later. + +**Constraint compliance:** All constraints met. + +**Dependencies produced:** Attribute detection changes needed in D6 (integration -- generator pipeline). +**Dependencies required:** None. + +**Risks:** +- Severity: Medium. Two names for one concept will confuse developers. +- Mitigation: Deprecate the alias immediately with `[Obsolete]`. + +--- + +## Option A4: Document Divergence, No Code Changes + +Accept that the plan describes a prior design intent, and the implementation has evolved. Document the mapping between plan names and actual names in the design-doc traceability table. No code changes. + +**Trade-offs:** +- (+) Zero risk, zero code changes +- (+) Honest about the evolutionary path +- (-) AC-5 says "YieldsMessageAttribute" which technically does not exist by that exact name +- (-) AC-9 describes a code structure that differs from actual generated output + +**Reversibility:** Easy. + +**Constraint compliance:** All constraints met. + +**Dependencies produced:** Documentation artifacts for D6 (integration). +**Dependencies required:** None. + +**Risks:** +- Severity: Low. +- Mitigation: Traceability table maps each AC to actual implementation. + +--- + +## Diagnostic IDs (AC-11) + +### Current State (Verified from DiagnosticDescriptors.cs) + +| Plan ID | Actual ID | Severity | Condition | Match? | +|---------|-----------|----------|-----------|--------| +| WFGEN001 | MAFGENWF001 | Error | Handler missing IWorkflowContext parameter | Functional match | +| WFGEN002 | MAFGENWF002 | Error | Handler has invalid return type | Functional match | +| WFGEN003 | MAFGENWF003 | Error | Executor with [MessageHandler] must be partial | Functional match | +| WFGEN004 | MAFGENWF004 | Warning | [MessageHandler] on non-Executor class | Functional match | +| WFGEN005 | MAFGENWF005 | Error | Handler has fewer than 2 parameters | Functional match | +| WFGEN006 | MAFGENWF006 | Info | ConfigureProtocol already defined (plan says ConfigureRoutes) | Functional match | +| (none) | MAFGENWF007 | Error | Handler cannot be static | Extra diagnostic | + +The MAFGENWF prefix follows Microsoft Agent Framework naming convention. The additional MAFGENWF007 (static handler check) is a correctness improvement over the plan. + +### Recommendation + +Preserve the MAFGENWF IDs. The plan's WFGEN IDs were never implemented; changing to them would break any tooling or documentation referencing the actual IDs. + +--- + +## Handler Signature Mapping (AC-8) + +Verified in SemanticAnalyzer.cs:539-568 (`GetSignatureKind`) and SourceBuilder.cs:178-189 (`AppendHandlerGenericArgs`): + +| Method Signature | Generated Call | Verified? | +|-----------------|----------------|-----------| +| void Handler(T, IWorkflowContext) | AddHandler(this.Handler) | Yes -- VoidSync in HandlerInfo.cs:10, HasOutput=false | +| void Handler(T, IWorkflowContext, CT) | AddHandler(this.Handler) | Yes -- VoidSync with hasCancellationToken=true | +| ValueTask Handler(T, IWorkflowContext) | AddHandler(this.Handler) | Yes -- VoidAsync, HasOutput=false | +| ValueTask Handler(T, IWorkflowContext, CT) | AddHandler(this.Handler) | Yes -- VoidAsync with hasCancellationToken=true | +| TResult Handler(T, IWorkflowContext) | AddHandler(this.Handler) | Yes -- ResultSync, HasOutput=true | +| ValueTask Handler(T, IWorkflowContext, CT) | AddHandler(this.Handler) | Yes -- ResultAsync, HasOutput=true | + +Note: AC-8 omits `TResult Handler(T, IWorkflowContext, CT)` (sync with CT) and `ValueTask Handler(T, IWorkflowContext)` (async result without CT), but the implementation handles all 8 combinations per RouteBuilder's overloads (RouteBuilder.cs:51-89 and overloads). + +--- + +## Summary + +**Recommended option: A1** -- Preserve current attribute names and signatures. The implementation has evolved beyond the plan in well-reasoned ways (commit `0756c457`, unified `ConfigureProtocol`), and reverting would introduce breakage with no functional benefit. diff --git a/.designs/1/audit.md b/.designs/1/audit.md new file mode 100644 index 0000000000..ff205ae99b --- /dev/null +++ b/.designs/1/audit.md @@ -0,0 +1,299 @@ +# audit.md -- Pre-Synthesis Audit + +Re-read source.md before producing this audit. Source.md unchanged since initial read. + +--- + +## Table A -- Constraint Audit + +| Dimension | Recommendation | C-1 (netstandard2.0) | C-2 (Roslyn 4.8.0+) | C-3 (analyzer packaging) | C-4 (clean break) | C-5 (any accessibility) | C-6 (partial required) | Status | +|-----------|---------------|---------------------|---------------------|-------------------------|-------------------|------------------------|----------------------|--------| +| D1 (API) | A1: Preserve current attributes | N/A (API dimension, not build target) | N/A | N/A | PASS: Users inherit Executor, not ReflectingExecutor | PASS: No accessibility check in SemanticAnalyzer | PASS: MAFGENWF003 enforces partial | PASS | +| D2 (Data) | D1: Preserve current models | PASS: Records via InjectIsExternalInitOnLegacy; all types netstandard2.0-compatible | DEVIATION: Actual uses 4.4.0 (deliberate for compatibility); see below | PASS: IsRoslynComponent=true, EnforceExtendedAnalyzerRules=true | PASS: Models encode Executor-based hierarchy only | N/A | N/A | PASS (with C-2 deviation) | +| D3 (UX) | U1: Current diagnostics | N/A | N/A | N/A | PASS: [Obsolete] on ReflectingExecutor guides migration | PASS: Diagnostics do not mention accessibility | PASS: MAFGENWF003 diagnostic exists | PASS | +| D4 (Scale) | S1: Incremental generator | PASS: No netstandard2.0-incompatible APIs | PASS: ForAttributeWithMetadataName available since 4.4.0 | PASS | N/A | N/A | N/A | PASS | +| D5 (Security) | SEC1: Current posture | PASS: No netstandard2.0-incompatible APIs | N/A | PASS: DevelopmentDependency=true, ReferenceOutputAssembly=false | N/A | PASS: partial class semantics protect private handlers | N/A | PASS | +| D6 (Integration) | I1: Current integration | PASS: Generator targets netstandard2.0 | PASS (see D2 deviation) | PASS: Packed to analyzers/dotnet/cs | PASS: ReflectingExecutor [Obsolete], generator requires Executor | N/A | N/A | PASS | + +### C-2 Deviation Detail + +C-2 says: "Reference `Microsoft.CodeAnalysis.CSharp` 4.8.0+" + +Actual: Microsoft.Agents.AI.Workflows.Generators.csproj:49 references `4.4.0`. + +This is a deliberate deviation documented in the .csproj (lines 45-48): "Use Roslyn 4.4.0 - minimum version for ForAttributeWithMetadataName API. Corresponds to .NET 7 SDK / VS 2022 17.4+. Higher versions would require newer SDKs, breaking users on older versions." + +The deviation is justified because: +1. No API from Roslyn 4.8.0 is used by the generator +2. 4.4.0 provides ForAttributeWithMetadataName which is the only required API +3. Upgrading would break users on .NET 7 SDK / VS 2022 17.4 + +**Verdict: Acceptable deviation. Document in design-doc traceability.** + +--- + +## Table B -- AC Coverage Audit + +### AC-1: Project Structure + +| Clause | Verbatim text | Owner | Option | Satisfied? | Evidence | +|--------|---------------|-------|--------|------------|----------| +| AC-1.1 | "Microsoft.Agents.AI.Workflows.Generators.csproj" | D2 | D1 | YES | File exists at dotnet/src/Microsoft.Agents.AI.Workflows.Generators/Microsoft.Agents.AI.Workflows.Generators.csproj | +| AC-1.2 | "ExecutorRouteGenerator.cs # Main incremental generator" | D2 | D1 | YES | File exists at ExecutorRouteGenerator.cs | +| AC-1.3 | "Models/ExecutorInfo.cs" | D2 | D1 | YES | File exists | +| AC-1.4 | "Models/HandlerInfo.cs" | D2 | D1 | YES | File exists | +| AC-1.5 | "Analysis/SyntaxDetector.cs" | D2 | D1 | NO | File does not exist; detection is inline in ExecutorRouteGenerator.cs via ForAttributeWithMetadataName | +| AC-1.6 | "Analysis/SemanticAnalyzer.cs" | D2 | D1 | YES | File exists | +| AC-1.7 | "Generation/SourceBuilder.cs" | D2 | D1 | YES | File exists | +| AC-1.8 | "Diagnostics/DiagnosticDescriptors.cs" | D2 | D1 | YES | File exists | + +**AC-1 Status: 7/8 clauses pass. AC-1.5 (SyntaxDetector.cs) fails -- file does not exist as separate entity. Detection is functionally present but architecturally different (inline vs separate file). This is a structural deviation, not a functional gap.** + +### AC-2: Project File Configuration + +| Clause | Verbatim text | Owner | Option | Satisfied? | Evidence | +|--------|---------------|-------|--------|------------|----------| +| AC-2.1 | "Target `netstandard2.0`" | D6 | I1 | YES | .csproj:5 `netstandard2.0` | +| AC-2.2 | "Reference `Microsoft.CodeAnalysis.CSharp` 4.8.0+" | D6 | I1 | NO | .csproj:49 references 4.4.0, not 4.8.0+ (deliberate; see C-2 deviation) | +| AC-2.3 | "Set `IsRoslynComponent=true`" | D6 | I1 | YES | .csproj:16 | +| AC-2.4 | "Set `EnforceExtendedAnalyzerRules=true`" | D6 | I1 | YES | .csproj:17 | +| AC-2.5 | "Package as analyzer in `analyzers/dotnet/cs`" | D6 | I1 | YES | .csproj:55 `PackagePath="analyzers/dotnet/cs"` | + +**AC-2 Status: 4/5 clauses pass. AC-2.2 fails -- deliberate deviation for compatibility (see C-2 deviation detail).** + +### AC-3: MessageHandlerAttribute + +| Clause | Verbatim text | Owner | Option | Satisfied? | Evidence | +|--------|---------------|-------|--------|------------|----------| +| AC-3.1 | "AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = false)" | D1 | A1 | YES | MessageHandlerAttribute.cs:48 | +| AC-3.2 | "public sealed class MessageHandlerAttribute : Attribute" | D1 | A1 | YES | MessageHandlerAttribute.cs:49 | +| AC-3.3 | "public Type[]? Yield { get; set; }" | D1 | A1 | YES | MessageHandlerAttribute.cs:59 | +| AC-3.4 | "public Type[]? Send { get; set; }" | D1 | A1 | YES | MessageHandlerAttribute.cs:69 | + +**AC-3 Status: ALL PASS.** + +### AC-4: SendsMessageAttribute + +| Clause | Verbatim text | Owner | Option | Satisfied? | Evidence | +|--------|---------------|-------|--------|------------|----------| +| AC-4.1 | "AttributeUsage(AttributeTargets.Class, AllowMultiple = true, Inherited = true)" | D1 | A1 | PARTIAL | Actual is `AttributeTargets.Class \| AttributeTargets.Method` (SendsMessageAttribute.cs:32) -- superset of spec | +| AC-4.2 | "public sealed class SendsMessageAttribute : Attribute" | D1 | A1 | YES | SendsMessageAttribute.cs:33 | +| AC-4.3 | "public Type Type { get; }" | D1 | A1 | YES | SendsMessageAttribute.cs:38 | +| AC-4.4 | "public SendsMessageAttribute(Type type) => this.Type = type;" | D1 | A1 | YES | SendsMessageAttribute.cs:45-48 (uses Throw.IfNull but functionally equivalent) | + +**AC-4 Status: 4/4 pass (AC-4.1 is superset, not violation).** + +### AC-5: YieldsMessageAttribute + +| Clause | Verbatim text | Owner | Option | Satisfied? | Evidence | +|--------|---------------|-------|--------|------------|----------| +| AC-5.1 | "AttributeUsage(AttributeTargets.Class, AllowMultiple = true, Inherited = true)" | D1 | A1 | PARTIAL | Actual is `AttributeTargets.Class \| AttributeTargets.Method` (YieldsOutputAttribute.cs:32) -- superset | +| AC-5.2 | "public sealed class YieldsMessageAttribute : Attribute" | D1 | A1 | NO | Actual name is `YieldsOutputAttribute` (YieldsOutputAttribute.cs:33) | +| AC-5.3 | "public Type Type { get; }" | D1 | A1 | YES | YieldsOutputAttribute.cs:38 | +| AC-5.4 | "public YieldsMessageAttribute(Type type) => this.Type = type;" | D1 | A1 | YES (functionally) | Constructor uses `Throw.IfNull(type)` but same behavior | + +**AC-5 Status: 3/4 pass. AC-5.2 fails -- name is YieldsOutputAttribute not YieldsMessageAttribute. Functional equivalence; naming change was deliberate (commit 0756c457).** + +### AC-6: Detection Criteria (syntax level) + +| Clause | Verbatim text | Owner | Option | Satisfied? | Evidence | +|--------|---------------|-------|--------|------------|----------| +| AC-6.1 | "Class has `partial` modifier" | D2 | D1 | YES | SemanticAnalyzer.cs:367-380 checks partial; MAFGENWF003 for non-partial | +| AC-6.2 | "Class has at least one method with `[MessageHandler]` attribute" | D2 | D1 | YES | ExecutorRouteGenerator.cs:33-37 uses ForAttributeWithMetadataName for MessageHandler | + +**AC-6 Status: ALL PASS.** + +### AC-7: Validation Criteria (semantic level) + +| Clause | Verbatim text | Owner | Option | Satisfied? | Evidence | +|--------|---------------|-------|--------|------------|----------| +| AC-7.1 | "Class derives from `Executor` (directly or transitively)" | D2 | D1 | YES | SemanticAnalyzer.cs:385-400 (DerivesFromExecutor) | +| AC-7.2 | "Class does NOT already define `ConfigureRoutes` with a body" | D2 | D1 | YES (evolved) | SemanticAnalyzer.cs:406-418 checks ConfigureProtocol (evolved from ConfigureRoutes) | +| AC-7.3 | "Handler method has valid signature: `(TMessage, IWorkflowContext[, CancellationToken])`" | D2 | D1 | YES | SemanticAnalyzer.cs:462-533 validates parameters | +| AC-7.4 | "Handler returns `void`, `ValueTask`, or `ValueTask`" | D2 | D1 | YES | SemanticAnalyzer.cs:539-568 (GetSignatureKind); also supports sync TResult | + +**AC-7 Status: ALL PASS (AC-7.2 evolved from ConfigureRoutes to ConfigureProtocol).** + +### AC-8: Handler Signature Mapping + +| Clause | Verbatim text | Owner | Option | Satisfied? | Evidence | +|--------|---------------|-------|--------|------------|----------| +| AC-8.1 | "void Handler(T, IWorkflowContext) -> AddHandler(this.Handler)" | D1 | A1 | YES | VoidSync -> HasOutput=false -> AddHandler | +| AC-8.2 | "void Handler(T, IWorkflowContext, CT) -> AddHandler(this.Handler)" | D1 | A1 | YES | VoidSync with CT -> HasOutput=false | +| AC-8.3 | "ValueTask Handler(T, IWorkflowContext) -> AddHandler(this.Handler)" | D1 | A1 | YES | VoidAsync -> HasOutput=false | +| AC-8.4 | "ValueTask Handler(T, IWorkflowContext, CT) -> AddHandler(this.Handler)" | D1 | A1 | YES | VoidAsync with CT -> HasOutput=false | +| AC-8.5 | "TResult Handler(T, IWorkflowContext) -> AddHandler(this.Handler)" | D1 | A1 | YES | ResultSync -> HasOutput=true | +| AC-8.6 | "ValueTask Handler(T, IWorkflowContext, CT) -> AddHandler(this.Handler)" | D1 | A1 | YES | ResultAsync -> HasOutput=true | + +**AC-8 Status: ALL PASS.** + +### AC-9: Generated Code Structure + +| Clause | Verbatim text | Owner | Option | Satisfied? | Evidence | +|--------|---------------|-------|--------|------------|----------| +| AC-9.1 | "// " | D1/D6 | A1/I1 | YES | SourceBuilder.cs:32 | +| AC-9.2 | "#nullable enable" | D1/D6 | A1/I1 | YES | SourceBuilder.cs:33 | +| AC-9.3 | "partial class MyExecutor" | D1/D6 | A1/I1 | YES | SourceBuilder.cs:66 | +| AC-9.4 | "protected override RouteBuilder ConfigureRoutes(RouteBuilder routeBuilder)" | D1/D6 | A1/I1 | NO (evolved) | Actual: `protected override ProtocolBuilder ConfigureProtocol(ProtocolBuilder protocolBuilder)` (SourceBuilder.cs:72) | +| AC-9.5 | "routeBuilder.AddHandler(this.Handler1)" | D1/D6 | A1/I1 | YES (evolved) | Actual uses nested ConfigureRoutes callback via ProtocolBuilder.ConfigureRoutes (SourceBuilder.cs:134) | +| AC-9.6 | "protected override ISet ConfigureSentTypes()" | D1/D6 | A1/I1 | NO (evolved) | Actual: `.SendsMessage()` fluent call on ProtocolBuilder (SourceBuilder.cs:206) | +| AC-9.7 | "protected override ISet ConfigureYieldTypes()" | D1/D6 | A1/I1 | NO (evolved) | Actual: `.YieldsOutput()` fluent call on ProtocolBuilder (SourceBuilder.cs:235) | + +**AC-9 Status: 3/7 clauses pass exactly; 4/7 have evolved to the unified ConfigureProtocol/ProtocolBuilder pattern. Functionally equivalent but structurally different. The evolution is deliberate (Executor.cs uses ConfigureProtocol, not ConfigureRoutes).** + +### AC-10: Inheritance Handling + +| Clause | Verbatim text | Owner | Option | Satisfied? | Evidence | +|--------|---------------|-------|--------|------------|----------| +| AC-10.1 | "Directly extends Executor -> No base call (abstract)" | D2/D6 | D1/I1 | YES | BaseHasConfigureProtocol=false -> SourceBuilder.cs:84 `return protocolBuilder` | +| AC-10.2 | "Extends executor with [MessageHandler] methods -> base.ConfigureRoutes(routeBuilder)" | D2/D6 | D1/I1 | YES (evolved) | BaseHasConfigureProtocol=true -> SourceBuilder.cs:79 `return base.ConfigureProtocol(protocolBuilder)` | +| AC-10.3 | "Extends executor with manual ConfigureRoutes -> base.ConfigureRoutes(routeBuilder)" | D2/D6 | D1/I1 | YES (evolved) | Same mechanism via BaseHasConfigureProtocol check | + +**AC-10 Status: ALL PASS (method names evolved).** + +### AC-11: Analyzer Diagnostics + +| Clause | Verbatim text | Owner | Option | Satisfied? | Evidence | +|--------|---------------|-------|--------|------------|----------| +| AC-11.1 | "WFGEN001 Error Handler missing IWorkflowContext parameter" | D1/D3 | A1/U1 | YES (ID differs) | MAFGENWF001, Error, DiagnosticDescriptors.cs:34-40 | +| AC-11.2 | "WFGEN002 Error Handler has invalid return type" | D1/D3 | A1/U1 | YES (ID differs) | MAFGENWF002, Error, DiagnosticDescriptors.cs:45-51 | +| AC-11.3 | "WFGEN003 Error Executor with [MessageHandler] must be partial" | D1/D3 | A1/U1 | YES (ID differs) | MAFGENWF003, Error, DiagnosticDescriptors.cs:56-62 | +| AC-11.4 | "WFGEN004 Warning [MessageHandler] on non-Executor class" | D1/D3 | A1/U1 | YES (ID differs) | MAFGENWF004, Warning, DiagnosticDescriptors.cs:67-73 | +| AC-11.5 | "WFGEN005 Error Handler has fewer than 2 parameters" | D1/D3 | A1/U1 | YES (ID differs) | MAFGENWF005, Error, DiagnosticDescriptors.cs:78-84 | +| AC-11.6 | "WFGEN006 Info ConfigureRoutes already defined, handlers ignored" | D1/D3 | A1/U1 | YES (ID differs, name evolved) | MAFGENWF006, Info, DiagnosticDescriptors.cs:89-95 -- checks ConfigureProtocol | +| AC-11.7 | (not in plan) | D1/D3 | A1/U1 | EXTRA | MAFGENWF007, Error, "Handler cannot be static" -- additional validation | + +**AC-11 Status: ALL 6 planned diagnostics exist (with MAFGENWF prefix instead of WFGEN). One additional diagnostic (MAFGENWF007) not in plan.** + +### AC-12: Integration -- Wire Generator to Main Project + +| Clause | Verbatim text | Owner | Option | Satisfied? | Evidence | +|--------|---------------|-------|--------|------------|----------| +| AC-12.1 | "ProjectReference Include=...Generators..." | D6 | I1 | YES | Microsoft.Agents.AI.Workflows.csproj:36 | +| AC-12.2 | "OutputItemType=\"Analyzer\"" | D6 | I1 | YES | Microsoft.Agents.AI.Workflows.csproj:37 | +| AC-12.3 | "ReferenceOutputAssembly=\"false\"" | D6 | I1 | YES | Microsoft.Agents.AI.Workflows.csproj:38 | + +**AC-12 Status: ALL PASS.** + +### AC-13: Mark ReflectingExecutor Obsolete + +| Clause | Verbatim text | Owner | Option | Satisfied? | Evidence | +|--------|---------------|-------|--------|------------|----------| +| AC-13.1 | "[Obsolete(...)]" present | D1 | A1 | YES | ReflectingExecutor.cs:21-22 | +| AC-13.2 | Message includes "Use [MessageHandler] attribute" | D1 | A1 | YES | Message starts with "Use [MessageHandler] attribute on methods in a partial class deriving from Executor." | +| AC-13.3 | Message includes "See migration guide" | D1 | A1 | NO | Actual message does not include "See migration guide" | +| AC-13.4 | Message includes "This type will be removed in v1.0." | D1 | A1 | NO | Actual says "This type will be removed in a future version." | +| AC-13.5 | "error: false" | D1 | A1 | YES | [Obsolete] without error parameter defaults to false (warning); confirmed not error | + +**AC-13 Status: 3/5 clauses pass. Message wording differs (no "migration guide" reference, "future version" vs "v1.0.").** + +### AC-14: Mark IMessageHandler Interfaces Obsolete + +| Clause | Verbatim text | Owner | Option | Satisfied? | Evidence | +|--------|---------------|-------|--------|------------|----------| +| AC-14.1 | "[Obsolete(\"Use [MessageHandler] attribute instead.\")]" | D1 | A1 | YES | IMessageHandler.cs has [Obsolete] | +| AC-14.2 | "public interface IMessageHandler" | D1 | A1 | YES | Interface exists in Reflection/IMessageHandler.cs | + +**AC-14 Status: ALL PASS.** + +### AC-15: Generator Unit Tests + +| Clause | Verbatim text | Owner | Option | Satisfied? | Evidence | +|--------|---------------|-------|--------|------------|----------| +| AC-15.1 | "ExecutorRouteGeneratorTests.cs" | D6 | I1 | YES | File exists (~37KB) | +| AC-15.2 | "SyntaxDetectorTests.cs" | D6 | I1 | NO | File does not exist; syntax detection tested within ExecutorRouteGeneratorTests | +| AC-15.3 | "SemanticAnalyzerTests.cs" | D6 | I1 | NO | File does not exist; semantic analysis tested within ExecutorRouteGeneratorTests | +| AC-15.4 | "TestHelpers/GeneratorTestHelper.cs" | D6 | I1 | YES | GeneratorTestHelper.cs exists (at project root, not in TestHelpers/ subfolder) | +| AC-15.5 | Test: "Simple single handler" | D6 | I1 | YES | Covered in ExecutorRouteGeneratorTests | +| AC-15.6 | Test: "Multiple handlers on one class" | D6 | I1 | YES | Covered | +| AC-15.7 | Test: "Handlers with different signatures" | D6 | I1 | YES | Covered | +| AC-15.8 | Test: "Nested classes" | D6 | I1 | YES | Covered | +| AC-15.9 | Test: "Generic executors" | D6 | I1 | YES | Covered | +| AC-15.10 | Test: "Inheritance chains" | D6 | I1 | YES | Covered | +| AC-15.11 | Test: "Class-level [SendsMessage]/[YieldsMessage] attributes" | D6 | I1 | YES | Covered (uses YieldsOutput in practice) | +| AC-15.12 | Test: "Manual ConfigureRoutes present" | D6 | I1 | YES | Covered (tests ConfigureProtocol) | +| AC-15.13 | Test: "Invalid signatures" | D6 | I1 | YES | Covered | + +**AC-15 Status: 11/13 clauses pass. AC-15.2 and AC-15.3 fail (separate test files not created; functionality tested in consolidated file).** + +### AC-16: Integration Tests + +| Clause | Verbatim text | Owner | Option | Satisfied? | Evidence | +|--------|---------------|-------|--------|------------|----------| +| AC-16.1 | "Port existing ReflectingExecutor test cases to use [MessageHandler]" | D6 | I1 | UNCLEAR | Not verified if ReflectingExecutor tests were explicitly ported [UNVERIFIED] | +| AC-16.2 | "Verify generated routes match reflection-discovered routes" | D6 | I1 | UNCLEAR | No explicit comparison test found [UNVERIFIED] | + +**AC-16 Status: UNVERIFIED. Would need to inspect test files more deeply to confirm.** + +### AC-17: Files to Create + +| Clause | Verbatim text | Owner | Option | Satisfied? | Evidence | +|--------|---------------|-------|--------|------------|----------| +| AC-17.1 | Generator .csproj | D2/D6 | D1/I1 | YES | Exists | +| AC-17.2 | ExecutorRouteGenerator.cs | D2 | D1 | YES | Exists | +| AC-17.3 | Models/ExecutorInfo.cs | D2 | D1 | YES | Exists | +| AC-17.4 | Models/HandlerInfo.cs | D2 | D1 | YES | Exists | +| AC-17.5 | Analysis/SyntaxDetector.cs | D2 | D1 | NO | Does not exist as separate file | +| AC-17.6 | Analysis/SemanticAnalyzer.cs | D2 | D1 | YES | Exists | +| AC-17.7 | Generation/SourceBuilder.cs | D2 | D1 | YES | Exists | +| AC-17.8 | Diagnostics/DiagnosticDescriptors.cs | D2 | D1 | YES | Exists | +| AC-17.9 | Attributes/MessageHandlerAttribute.cs | D1 | A1 | YES | Exists | +| AC-17.10 | Attributes/SendsMessageAttribute.cs | D1 | A1 | YES | Exists | +| AC-17.11 | Attributes/YieldsMessageAttribute.cs | D1 | A1 | NO | Named YieldsOutputAttribute.cs, not YieldsMessageAttribute.cs | +| AC-17.12 | Generator unit tests | D6 | I1 | YES | ExecutorRouteGeneratorTests.cs exists | + +**AC-17 Status: 10/12 clauses pass. AC-17.5 (SyntaxDetector.cs) and AC-17.11 (YieldsMessageAttribute.cs naming) fail.** + +### AC-18: Files to Modify + +| Clause | Verbatim text | Owner | Option | Satisfied? | Evidence | +|--------|---------------|-------|--------|------------|----------| +| AC-18.1 | "Microsoft.Agents.AI.Workflows.csproj - Add generator reference" | D6 | I1 | YES | Lines 35-39 | +| AC-18.2 | "ReflectingExecutor.cs - Add [Obsolete]" | D6 | I1 | YES | Lines 21-22 | +| AC-18.3 | "IMessageHandler.cs - Add [Obsolete]" | D6 | I1 | YES | Confirmed | +| AC-18.4 | "Microsoft.Agents.sln - Add new projects" | D6 | I1 | YES | Solution is agent-framework-dotnet.slnx (name evolved) | + +**AC-18 Status: ALL PASS.** + +### AC-19: Example Usage (End State) + +| Clause | Verbatim text | Owner | Option | Satisfied? | Evidence | +|--------|---------------|-------|--------|------------|----------| +| AC-19.1 | "[SendsMessage(typeof(PollToken))] public partial class" | D1 | A1 | YES | SendsMessageAttribute supports Class target | +| AC-19.2 | "[MessageHandler] private async ValueTask HandleQueryAsync(...)" | D1 | A1 | YES | Private handler, ValueTask return supported | +| AC-19.3 | "[MessageHandler(Yield = [...], Send = [...])]" | D1 | A1 | YES | Yield and Send properties on MessageHandlerAttribute | +| AC-19.4 | Generated ConfigureRoutes with AddHandler calls | D1/D6 | A1/I1 | YES (evolved) | Generated ConfigureProtocol with ConfigureRoutes callback | +| AC-19.5 | Generated ConfigureSentTypes with types.Add | D1/D6 | A1/I1 | YES (evolved) | Generated .SendsMessage() fluent calls | +| AC-19.6 | Generated ConfigureYieldTypes with types.Add | D1/D6 | A1/I1 | YES (evolved) | Generated .YieldsOutput() fluent calls | + +**AC-19 Status: ALL PASS (generated code structure evolved but functionally equivalent).** + +--- + +## Table C -- Additional Context Coverage + +| Context item (verbatim from source.md) | Reflected in dimension(s) / dismissed with rationale | +|----------------------------------------|------------------------------------------------------| +| "Attribute syntax: Inline properties on `[MessageHandler(Yield=[...], Send=[...])]`" | D1 (API) -- covered in Option A1. MessageHandlerAttribute.cs:59,69 has Yield and Send properties. | +| "Class-level attributes: Generate `ConfigureSentTypes()`/`ConfigureYieldTypes()` from `[SendsMessage]`/`[YieldsMessage]`" | D1 (API) -- covered. Actual generates `.SendsMessage()` / `.YieldsOutput()` fluent calls via ProtocolBuilder. Evolved but functionally equivalent. | +| "Migration: Clean break - requires direct `Executor` inheritance (not `ReflectingExecutor`)" | D1 (API) C-4 compliance, D3 (UX) migration path, D6 (Integration) [Obsolete] markers. All dimensions address this. | +| "Handler accessibility: Any (private, protected, internal, public)" | D1 (API) C-5 compliance verified -- SemanticAnalyzer does not check accessibility. D5 (Security) confirms partial class semantics protect private handlers. | + +--- + +## Audit Summary + +### Failures Requiring Action + +| AC | Clause | Issue | Severity | Action | +|----|--------|-------|----------|--------| +| AC-1.5 | SyntaxDetector.cs | File does not exist as separate entity | Low | Document as architectural simplification; functionality is present inline | +| AC-2.2 | Roslyn 4.8.0+ | Uses 4.4.0 instead | Low | Document as deliberate compatibility deviation | +| AC-5.2 | YieldsMessageAttribute | Named YieldsOutputAttribute | Low | Document naming divergence; no rename (reverses commit 0756c457) | +| AC-9.4-9.7 | ConfigureRoutes/ConfigureSentTypes/ConfigureYieldTypes | Evolved to ConfigureProtocol/ProtocolBuilder | Low | Document API evolution; no revert (matches actual Executor abstract method) | +| AC-13.3-13.4 | Obsolete message wording | Missing "migration guide" and "v1.0." | Low | Consider updating if migration guide is created | +| AC-15.2-15.3 | Separate test files | Tests consolidated in single file | Low | Document; no split needed | +| AC-17.5 | SyntaxDetector.cs | File not created | Low | Same as AC-1.5 | +| AC-17.11 | YieldsMessageAttribute.cs | Named YieldsOutputAttribute.cs | Low | Same as AC-5.2 | + +All failures are documented naming/structural deviations from a plan that describes a prior design intent. The implementation is functionally complete and has evolved beyond the plan in deliberate, well-reasoned ways. No failures indicate missing functionality. diff --git a/.designs/1/codebase-snapshot.md b/.designs/1/codebase-snapshot.md new file mode 100644 index 0000000000..b4f7119091 --- /dev/null +++ b/.designs/1/codebase-snapshot.md @@ -0,0 +1,187 @@ +# codebase-snapshot.md — Verified Ground Truth + +Snapshot taken: 2026-05-12T03:32Z +Branch: af/soldesign-plan-5414a7 (identical to origin/main at time of snapshot) + +## 1. Project Tree (Source Generator) + +``` +dotnet/src/Microsoft.Agents.AI.Workflows.Generators/ +├── Analysis/ +│ ├── SemanticAnalyzer.cs +│ └── (SyntaxDetector.cs - NOT present; detection is inline in ExecutorRouteGenerator.cs) +├── Diagnostics/ +│ └── DiagnosticDescriptors.cs +├── Directory.Build.targets +├── ExecutorRouteGenerator.cs +├── Generation/ +│ └── SourceBuilder.cs +├── Microsoft.Agents.AI.Workflows.Generators.csproj +├── Models/ +│ ├── ExecutorInfo.cs +│ ├── HandlerInfo.cs +│ └── (additional model files for analysis pipeline) +└── SkipIncompatibleBuild.targets +``` + +## 2. Module Identity + +Solution: `dotnet/agent-framework-dotnet.slnx` +Primary project: `dotnet/src/Microsoft.Agents.AI.Workflows/Microsoft.Agents.AI.Workflows.csproj` +Generator project: `dotnet/src/Microsoft.Agents.AI.Workflows.Generators/Microsoft.Agents.AI.Workflows.Generators.csproj` + +## 3. Referenced Types and Functions — Verified Locations + +### Executor (base class) +- **File**: `dotnet/src/Microsoft.Agents.AI.Workflows/Executor.cs:164` +- **Signature**: `public abstract class Executor : IIdentified` +- **Abstract method**: `protected abstract ProtocolBuilder ConfigureProtocol(ProtocolBuilder protocolBuilder);` (line 216) +- **Constructor**: `protected Executor(string id, ExecutorOptions? options = null, bool declareCrossRunShareable = false)` (line 179) +- **Generic variants**: `Executor` (line 382), `Executor` (line 407) + +### ReflectingExecutor +- **File**: `dotnet/src/Microsoft.Agents.AI.Workflows/Reflection/ReflectingExecutor.cs:23-27` +- **Signature**: `public class ReflectingExecutor<[DynamicallyAccessedMembers(...)] TExecutor> : Executor where TExecutor : ReflectingExecutor` +- **Status**: Already marked `[Obsolete]` (lines 21-22) +- **Obsolete message**: `"Use [MessageHandler] attribute on methods in a partial class deriving from Executor. This type will be removed in a future version."` +- **Overrides**: `ConfigureProtocol(ProtocolBuilder)` (NOT `ConfigureRoutes`) + +### RouteBuilder +- **File**: `dotnet/src/Microsoft.Agents.AI.Workflows/RouteBuilder.cs` +- **AddHandler overloads** (8 public overloads): + - `AddHandler(Action)` + - `AddHandler(Action)` + - `AddHandler(Func)` + - `AddHandler(Func)` + - `AddHandler(Func)` + - `AddHandler(Func)` + - `AddHandler(Func>)` + - `AddHandler(Func>)` +- **Internal method**: `AddHandlerInternal(Type, MessageHandlerF, Type?, bool)` (line 51-89) + +### ProtocolBuilder +- **File**: `dotnet/src/Microsoft.Agents.AI.Workflows/ProtocolBuilder.cs` +- **Key methods**: + - `SendsMessage()` — fluent registration + - `YieldsOutput()` — fluent registration + - `ConfigureRoutes(Action)` — route delegation + - `RouteBuilder` property — direct access + +### IWorkflowContext +- **File**: `dotnet/src/Microsoft.Agents.AI.Workflows/IWorkflowContext.cs` +- **Key methods**: `SendMessageAsync()`, `YieldOutputAsync()`, state management + +### MessageHandlerAttribute +- **File**: `dotnet/src/Microsoft.Agents.AI.Workflows/Attributes/MessageHandlerAttribute.cs:48-70` +- **Signature**: `[AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = false)] public sealed class MessageHandlerAttribute : Attribute` +- **Properties**: `Type[]? Yield`, `Type[]? Send` +- **Status**: EXISTS — matches plan exactly + +### SendsMessageAttribute +- **File**: `dotnet/src/Microsoft.Agents.AI.Workflows/Attributes/SendsMessageAttribute.cs:32-49` +- **Signature**: `[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method, AllowMultiple = true, Inherited = true)] public sealed class SendsMessageAttribute : Attribute` +- **Property**: `Type Type` (constructor parameter) +- **Status**: EXISTS — plan says `AttributeTargets.Class` only, actual allows `Class | Method` + +### YieldsOutputAttribute (plan says "YieldsMessageAttribute") +- **File**: `dotnet/src/Microsoft.Agents.AI.Workflows/Attributes/YieldsOutputAttribute.cs:32-49` +- **Signature**: `[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method, AllowMultiple = true, Inherited = true)] public sealed class YieldsOutputAttribute : Attribute` +- **Property**: `Type Type` (constructor parameter) +- **Status**: EXISTS but NAMED DIFFERENTLY — plan calls it `YieldsMessageAttribute`, actual is `YieldsOutputAttribute` + +### IMessageHandler Interfaces +- **File**: `dotnet/src/Microsoft.Agents.AI.Workflows/Reflection/IMessageHandler.cs` +- **Status**: Already marked `[Obsolete]` + +### ExecutorRouteGenerator +- **File**: `dotnet/src/Microsoft.Agents.AI.Workflows.Generators/ExecutorRouteGenerator.cs:22-161` +- **Signature**: `[Generator] public sealed class ExecutorRouteGenerator : IIncrementalGenerator` +- **Three pipelines**: MessageHandler methods, SendsMessage classes, YieldsOutput classes +- **Status**: EXISTS — fully implemented + +### DiagnosticDescriptors +- **File**: `dotnet/src/Microsoft.Agents.AI.Workflows.Generators/Diagnostics/DiagnosticDescriptors.cs` +- **Actual IDs** (different from plan): + - `MAFGENWF001` (Error) — Handler missing IWorkflowContext parameter + - `MAFGENWF002` (Error) — Handler has invalid return type + - `MAFGENWF003` (Error) — Executor with [MessageHandler] must be partial + - `MAFGENWF004` (Warning) — [MessageHandler] on non-Executor class + - `MAFGENWF005` (Error) — Handler has insufficient parameters + - `MAFGENWF006` (Info) — ConfigureProtocol already defined + - `MAFGENWF007` (Error) — Handler cannot be static (NOT in plan) +- **Plan uses**: `WFGEN001-006` — actual uses `MAFGENWF001-007` + +## 4. Key Discrepancies: Plan vs. Implementation + +| Plan States | Actual Implementation | Impact | +|-------------|----------------------|--------| +| `YieldsMessageAttribute` | `YieldsOutputAttribute` | Name differs; same functionality | +| `ConfigureRoutes(RouteBuilder)` as generated method | `ConfigureProtocol(ProtocolBuilder)` as generated method | API evolved to unified fluent builder | +| Separate `ConfigureSentTypes()` / `ConfigureYieldTypes()` methods | Single `ConfigureProtocol()` with fluent `.SendsMessage()` / `.YieldsOutput()` | Cleaner API, single override point | +| `WFGEN001-006` diagnostic IDs | `MAFGENWF001-007` IDs | 7 diagnostics, not 6 | +| `AttributeTargets.Class` for SendsMessage/YieldsMessage | `AttributeTargets.Class \| AttributeTargets.Method` | Broader applicability than planned | +| `SyntaxDetector.cs` as separate file | Detection inline in `ExecutorRouteGenerator.cs` via `ForAttributeWithMetadataName` | Simpler architecture | +| Files listed as "to create" | All files ALREADY EXIST | Plan describes already-implemented work | + +## 5. Existing Test Coverage + +**Path**: `dotnet/tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/` + +Files: +- `ExecutorRouteGeneratorTests.cs` (~37KB) +- `GeneratorTestHelper.cs` +- `SyntaxTreeFluentExtensions.cs` + +Test cases covering: +- Single handler (void, ValueTask, ValueTask) +- Multiple handlers on one class +- CancellationToken parameter variants +- Yield/Send type attributes on handlers +- Class-level [SendsMessage] and [YieldsOutput] attributes +- Nested classes +- Generic executors +- Inheritance chains +- ConfigureProtocol already defined (skip generation) +- Invalid signatures (diagnostic verification) + +## 6. Recent Deliberate Changes + +``` +$ git log --oneline -30 -- src/Microsoft.Agents.AI.Workflows/ + +27324a80 .NET: Mark Magentic Orchestration Experimental (#5704) +ce70ca1a .NET: feat: Implement Magentic Orchestration for .NET (#5595) +9f3f7fd0 fix: JSON Serialization issue with MultiPartyConversation (#5653) +162985f2 .NET: feat: Implement message filtering to exclude non-portable content typ… (#5410) +4b5a8478 .NET: Hosting updates to declarative workflows (#5589) +69adf6d9 .NET: Fix off-thread RunStatus race where GetStatusAsync can return Running after ResumeAsync halts (#5412) +267351b7 .NET: Expand Workflow Unit Test Coverage (#5390) +d5777bc5 fix: Duplicate CallIds cause Handoff Message Filtering to fail (#5359) +5777ed26 .NET: fix: Add session support for Handoff-hosted Agents (#5280) + +$ git log --oneline -30 -- src/Microsoft.Agents.AI.Workflows.Generators/ + +267351b7 .NET: Expand Workflow Unit Test Coverage (#5390) +524c0216 .NET: Update release versions (#5059) +0756c457 .NET: [BREAKING] Update type names and source generator to reduce conflicts (#4903) +5374dd47 .NET: Fix source generator bug that silently drops base class handler registrations for protocol-only partial executors (#4751) +6c32e869 .NET: Updated package versions for RC release (#4067) +``` + +Key deliberate changes: +- `0756c457`: Renamed `Config` → `ExecutorConfig`, made `RouteBuilder` usage explicit in generated code to avoid naming conflicts +- `5374dd47`: Fixed bug where base class handler registrations were silently dropped for protocol-only partial executors +- `267351b7`: Expanded unit test coverage + +## Decision History + +### ADRs +No ADRs found specifically addressing the source generator or Workflows architecture. +Searched: `find docs/decisions -name "0*.md"` — 15 ADRs found, none related to source generators, Roslyn, or workflow executor patterns. + +### Prior Designs +No prior designs found for this problem domain. +Searched: `ls .designs/` — only the current `.designs/1/` directory exists (created by this session). + +### Recent Deliberate Changes +See section 6 above. Key pattern: the API evolved from the plan's `ConfigureRoutes`/`ConfigureSentTypes`/`ConfigureYieldTypes` pattern to a unified `ConfigureProtocol(ProtocolBuilder)` with fluent chaining. This was a deliberate architectural decision reflected in the existing implementation. diff --git a/.designs/1/conflicts.md b/.designs/1/conflicts.md new file mode 100644 index 0000000000..77243a8e62 --- /dev/null +++ b/.designs/1/conflicts.md @@ -0,0 +1,113 @@ +# conflicts.md -- Cross-Dimension Conflict Matrix + +## NxN Matrix + +Legend: (none) = no conflict, T = tension (trade-off needed), X = direct conflict (resolution required) + +| | D1 (API) | D2 (Data) | D3 (UX) | D4 (Scale) | D5 (Security) | D6 (Integration) | +|----------|----------|-----------|---------|------------|---------------|-------------------| +| D1 (API) | -- | (none) | (none) | (none) | (none) | T | +| D2 (Data)| (none) | -- | (none) | (none) | (none) | T | +| D3 (UX) | (none) | (none) | -- | (none) | (none) | (none) | +| D4 (Scale)| (none) | (none) | (none) | -- | (none) | (none) | +| D5 (Sec) | (none) | (none) | (none) | (none) | -- | (none) | +| D6 (Int) | T | T | (none) | (none) | (none) | -- | + +--- + +## Cell Justifications + +### (none) cells + +**D1 (API) vs D2 (Data):** No conflict. The API dimension defines the user-facing attribute shapes and generated code structure. The Data dimension defines the internal pipeline models. These are cleanly separated -- attributes are consumed by the pipeline models, with no circular dependency. The attribute shapes (MessageHandlerAttribute, SendsMessageAttribute, YieldsOutputAttribute) are inputs to the analysis pipeline (SemanticAnalyzer reads attribute metadata from ISymbol), and the pipeline models (ExecutorInfo, HandlerInfo) are outputs consumed by SourceBuilder. This is a one-directional dependency with no conflicting requirements. + +**D1 (API) vs D3 (UX):** No conflict. The API attributes define what users write; the UX diagnostics validate what they wrote. These are complementary -- the diagnostic messages reference the attribute names and required patterns, but changing one does not require changing the other in a conflicting way. The diagnostics are derived from the validation rules, which are derived from the attribute contract. + +**D1 (API) vs D4 (Scale):** No conflict. The attribute-based API is the reason ForAttributeWithMetadataName works so efficiently. A method-level attribute that is discoverable by string name is the ideal input for incremental generator filtering. There is no tension between the API design choice (attributes) and the scalability approach (incremental generation). + +**D1 (API) vs D5 (Security):** No conflict. The attribute-based API does not introduce security concerns. Type names in attributes are resolved through Roslyn symbol resolution (not raw string interpolation), so there is no injection vector. The handler accessibility flexibility (C-5: any access level) is secured by partial class semantics, as analyzed in D5. + +**D2 (Data) vs D3 (UX):** No conflict. The data models are internal to the generator pipeline and invisible to users. The UX is defined by diagnostics (which are separate from the data models) and the generated code (which is produced by SourceBuilder from the data models). Users never interact with ExecutorInfo, HandlerInfo, etc. + +**D2 (Data) vs D4 (Scale):** No conflict. The data model design (records with value equality, ImmutableEquatableArray) was specifically chosen to support incremental generator caching. The scale requirements drove the data model design, so they are aligned rather than in tension. + +**D2 (Data) vs D5 (Security):** No conflict. The internal pipeline models do not affect the security posture. They are compile-time-only constructs that do not appear in generated code or diagnostic messages. + +**D3 (UX) vs D4 (Scale):** No conflict. The diagnostic messages are lightweight strings generated only when validation fails. They do not impact incremental generation performance. The ForAttributeWithMetadataName predicate runs before any diagnostic generation, so invalid candidates are filtered cheaply. + +**D3 (UX) vs D5 (Security):** No conflict. Diagnostic messages contain only method/class names (which are already visible in source code). No sensitive information is leaked through diagnostics. + +**D3 (UX) vs D6 (Integration):** No conflict. The UX diagnostics work through standard Roslyn infrastructure that is automatically available when the analyzer is referenced. No additional integration work is needed for diagnostics to appear in IDE or build output. + +**D4 (Scale) vs D5 (Security):** No conflict. The incremental generator caching mechanism does not cache or expose sensitive data. Cache keys are derived from value-equality of data models (type names, method names) which are already part of the public source code. + +**D4 (Scale) vs D6 (Integration):** No conflict. The incremental generator architecture is compatible with the standard ProjectReference-as-Analyzer integration pattern. The generator runs within the standard Roslyn pipeline without special build system requirements. + +**D5 (Security) vs D6 (Integration):** No conflict. The security recommendations (DevelopmentDependency=true, ReferenceOutputAssembly=false) are already part of the integration configuration. These settings ensure the generator does not appear in production deployments. + +--- + +## T (Tension) Cells + +### D1 (API) vs D6 (Integration): Plan-vs-Implementation Naming Tension + +**Nature:** The plan (source.md) uses names and API shapes that differ from the actual implementation: +- Plan: `YieldsMessageAttribute` -> Actual: `YieldsOutputAttribute` +- Plan: `ConfigureRoutes(RouteBuilder)` -> Actual: `ConfigureProtocol(ProtocolBuilder)` +- Plan: `WFGEN001-006` -> Actual: `MAFGENWF001-007` + +**Impact:** AC traceability requires documenting how each AC maps to the actual implementation. Integration tests (AC-16) that reference plan names would not compile. + +**Resolution Options:** +1. Rename actual to match plan (REJECTED: reverses commit 0756c457) +2. Add aliases for both names (adds complexity) +3. Document the mapping and accept the deviation (chosen) + +**Chosen Resolution:** Option 3 -- Document the mapping in the design-doc traceability table. The actual names are deliberate evolution from the plan, not errors. The plan describes design intent; the implementation refined that intent based on real constraints (naming conflicts per commit 0756c457, unified ProtocolBuilder API per Executor.cs architecture). + +**Rationale:** Renaming would break existing consumers. Aliases add complexity for no functional benefit. The deviations are well-documented in codebase-snapshot.md section 4 and have clear git history justification. + +### D2 (Data) vs D6 (Integration): Roslyn Version Constraint Tension + +**Nature:** The plan (C-2, AC-2) specifies Roslyn 4.8.0+, but the actual implementation uses 4.4.0 (Microsoft.Agents.AI.Workflows.Generators.csproj:49). This creates tension between literal constraint compliance and the project's documented compatibility goals. + +**Impact:** If the constraint is taken literally, the implementation is non-compliant. If the constraint intent (use incremental generator APIs) is considered, the implementation is compliant since ForAttributeWithMetadataName is available in 4.4.0. + +**Resolution Options:** +1. Upgrade to 4.8.0 (breaks .NET 7 SDK users per .csproj comment) +2. Keep 4.4.0 and document the deviation (chosen) +3. Update the constraint in source.md to say 4.4.0+ (out of scope for design) + +**Chosen Resolution:** Option 2 -- Keep 4.4.0 and document the deviation. The .csproj comment explicitly explains the rationale. + +**Rationale:** The constraint C-2 appears to have been written based on the assumption that 4.8.0 was the minimum for ForAttributeWithMetadataName, but the API was actually introduced in 4.4.0. The implementation's choice of 4.4.0 maximizes compatibility while still providing the required functionality. + +--- + +## Elevation-Level Conflicts + +### Analysis: Do any NEW abstractions or components proposed by one dimension conflict with recommendations from another? + +**Finding: No elevation-level conflicts identified.** + +All six dimensions recommend preserving the current implementation (Options A1, D1, U1, S1, SEC1, I1). No dimension proposes a new abstraction, component, or architectural change that would conflict with another dimension's recommendation. + +The only new components considered were: +- **Code fix providers (D3/U2):** Would be additive, not conflicting. No dimension rejects or contradicts this. +- **SyntaxDetector extraction (D2/D2):** Would be a refactoring, not a new abstraction. No dimension depends on the current inline detection. +- **Integration test project (D6/I3):** Would be additive. No dimension conflicts with this. + +Since all recommended options converge on "preserve current implementation," there are no cross-dimension frame conflicts. The design is internally consistent. + +### Consistency Check: Recommended Options Across Dimensions + +| Dimension | Recommended | Core Principle | Consistent? | +|-----------|-------------|----------------|-------------| +| D1 (API) | A1: Preserve current attributes | Don't reverse deliberate evolution | Yes | +| D2 (Data) | D1: Preserve current models | Implementation is complete and correct | Yes | +| D3 (UX) | U1: Current diagnostics | 7 diagnostics cover all validation rules | Yes | +| D4 (Scale) | S1: Incremental generator | Architecture uses recommended Roslyn patterns | Yes | +| D5 (Security) | SEC1: Current posture | Minimal attack surface, standard model | Yes | +| D6 (Integration) | I1: Current integration | All AC-18 modifications applied | Yes | + +All recommendations are mutually consistent. The design surface is mature and does not require cross-cutting changes. diff --git a/.designs/1/cross-review/analyst-final-review.md b/.designs/1/cross-review/analyst-final-review.md new file mode 100644 index 0000000000..ac498cd787 --- /dev/null +++ b/.designs/1/cross-review/analyst-final-review.md @@ -0,0 +1,83 @@ +# Cross-Review Round 3 (Final): Analyst Assessment of Peer Review + +**Reviewer**: rootcause-all (Analyst) +**Date**: 2026-05-12 +**Document Reviewed**: Peer review appended to `.designs/1/implementation-plan/implementation_plan_outline.md` (lines 354-434) + +--- + +## Assessment of Peer Review Findings + +### RH1 (Line Number Fragility) — AGREE, LOW RESIDUAL RISK + +The designer correctly identifies that absolute line numbers are fragile. However, the plan already includes structural descriptions ("param count check", "CancellationToken detection") alongside every line reference. The risk is that an implementer counts lines instead of reading patterns. In practice, any competent implementation tool (`/ultra-implement`) will locate code by pattern matching, not line counting. The recommendation to add a "locate by pattern" note is good but the residual risk is low. + +**Remaining gap**: None. The structural anchors are already present. + +### RH2 (Duplicate Detection Scope) — AGREE, IMPORTANT CLARIFICATION + +This is the most valuable finding in the peer review. The plan says "Group collected handlers by InputTypeName" in `CombineHandlerMethodResults`, which receives merged handlers from all partial class files. But the exact point where duplicate detection runs matters: + +- If before merge: misses cross-file duplicates +- If after merge: catches all duplicates (correct) + +The plan's wording implies post-merge (since `CombineHandlerMethodResults` does the merging), but it's not explicit. + +**Remaining gap**: The implementer needs a one-line clarification: "Run duplicate detection AFTER handler collection at L162-165, not within `AnalyzeHandler()`." + +### RM1 (Base Type Classification) — AGREE, ADDRESSED IN GOTCHAS + +The plan's Gotchas section already proposes adding `BaseIsGenericExecutor` to `MethodAnalysisResult`. The peer review asks for explicit specification of which approach to take. I agree — the two options (modify `DerivesFromExecutor()` return type vs. add boolean field) should have one chosen, not left ambiguous. + +**Remaining gap**: Minor. The Gotchas section effectively chooses the boolean field approach; making it the explicit requirement would eliminate ambiguity. + +### RM2 (Baseline Expected Output) — AGREE, PRACTICAL GUIDANCE + +The recommendation to capture actual output first, then assert against it, is the correct testing methodology. This prevents hand-crafting expected strings that may not match actual formatting. + +**Remaining gap**: None — the recommendation is implementable as-is. + +### RL1 (Migration Guide Location) — AGREE + +MIGRATION.md in the source directory won't reach NuGet consumers without packaging changes. Placing it in `dotnet/docs/` or adding it as package content are both valid. This should be a decision for the implementer, not a blocker. + +**Remaining gap**: Trivial — file location choice. + +### RL2 (Unique Message Templates) — LOW RISK + +The plan specifies exact message text for each new diagnostic. Overlap is unlikely given the specificity. + +**Remaining gap**: None. + +--- + +## Causal Chain Verification Assessment + +The peer review's causal chain table (lines 422-431) is well-constructed. One observation: + +| Gap 7 verdict | "PARTIAL — structural not behavioral" | +|---------------|---------------------------------------| + +This is correct. The design-doc itself acknowledges this as a practical ceiling (compile-time generator cannot fully verify runtime behavior). The Phase 2 baseline tests verify code generation correctness, not runtime execution equivalence. Full behavioral equivalence would require running the generated code against the reflected code with identical inputs and comparing outputs — which requires runtime test infrastructure that doesn't exist. + +**Assessment**: The "PARTIAL" verdict is honest and appropriate. The plan closes Gap 7 to the extent feasible without building new runtime test infrastructure. + +--- + +## Remaining Gaps After All Reviews + +| # | Gap | Severity | Source | +|---|-----|----------|--------| +| 1 | Duplicate detection scope (pre/post merge) needs one-line clarification | Low | Peer review RH2 | +| 2 | Base type classification approach needs explicit selection | Low | Peer review RM1 | +| 3 | Full behavioral equivalence remains partially open (acknowledged ceiling) | Accepted | Design-doc, Peer review | + +No HIGH or CRITICAL gaps remain. The plan is ready for implementation. + +--- + +## Final Verdict + +The implementation plan is sound, thoroughly grounded in codebase reality, and addresses root causes for all targeted gaps. The peer review added valuable clarifications (particularly RH2 on duplicate detection scope) but identified no plan defects. Both rounds of analyst review findings (H1: rollback strategy, H2: HandlerInfo contradiction, H3: validation ordering) are addressable as implementer notes without plan restructuring. + +**Recommendation**: Proceed to implementation. No blocking issues. diff --git a/.designs/1/cross-review/analyst-review-design.md b/.designs/1/cross-review/analyst-review-design.md new file mode 100644 index 0000000000..18907a38ed --- /dev/null +++ b/.designs/1/cross-review/analyst-review-design.md @@ -0,0 +1,62 @@ +# Cross-Review: Design Doc (Analyst Perspective) + +**Reviewer**: rootcause-all (Analyst) +**Date**: 2026-05-12 +**Document Reviewed**: `.designs/1/design-doc.md` + +--- + +## CRITICAL + +No critical issues found. The design document is internally consistent, correctly identifies the architecture as sound via elevation analysis, and proposes reasonable incremental improvements to close quality gaps. + +--- + +## HIGH + +### H1: Gap 4 (Duplicate Input Types) May Have Broader Impact Than Stated + +The design identifies that duplicate handler input types produce runtime exceptions instead of compile-time diagnostics and proposes adding `MAFGENWF008`. However, the root cause chain is incomplete: + +- **What happens today**: Two `[MessageHandler]` methods with the same input type on one executor cause a runtime crash +- **Missing analysis**: Does `CombineHandlerMethodResults` deduplicate silently, throw on first conflict, or crash non-deterministically? The behavior under concurrent registration is not characterized +- **Recommendation**: The implementation for MAFGENWF008 should document the exact runtime failure mode it prevents (exception type, stack trace location) so the diagnostic message can reference the specific runtime error users would otherwise encounter + +### H2: Behavioral Equivalence Testing (Gap 7) Marked "Partially Feasible" Without Clear Blocking Factor + +The design says Gap 7 is "Partially feasible" requiring "in-memory compilation test infrastructure" but the test project already uses `Microsoft.CodeAnalysis.CSharp.CSharpCompilation` for generator tests (implied by 33 existing test methods in `ExecutorRouteGeneratorTests.cs`). If compilation infrastructure already exists, what specifically makes full behavioral equivalence only partially feasible? + +- **Recommendation**: Clarify whether the blocker is (a) needing runtime execution of generated code, (b) needing the reflection path to produce comparable output format, or (c) test infrastructure limitations. This affects Phase 2 effort estimation. + +### H3: No Rollback Strategy for Phase 1 Diagnostics + +Phase 1 adds new compile-time errors (MAFGENWF008, upgraded MAFGENWF006 to Warning). For a published NuGet package: + +- New errors break builds that previously succeeded +- New warnings may break builds with TreatWarningsAsErrors + +The risk registry covers removal of ReflectingExecutor but not the introduction of new diagnostics that may break existing consumer builds. + +- **Recommendation**: Add a risk entry for "new diagnostics break consumer CI" with mitigation: ship new diagnostics as Info initially, upgrade to Warning/Error in next major version. Or document in release notes which diagnostics are new. + +--- + +## LOW + +### L1: AC-9 Deviation (ConfigureProtocol vs. Separate Methods) Lacks Migration Impact Assessment + +The design correctly notes that generated code uses `ConfigureProtocol` instead of separate `ConfigureRoutes`/`ConfigureSentTypes`/`ConfigureYieldTypes`. The decision table marks this as "Moderate" reversibility but doesn't assess whether any existing documentation, samples, or external guides reference the original three-method pattern. If the plan was published externally before implementation began, consumers may have written code expecting the three-method API. + +### L2: Implementation Plan Phases Have No Effort Estimates Beyond "Small" + +All three phases are marked "Effort: Small" without LOE breakdown. For planning purposes, distinguishing between "2 hours" and "2 days" matters. The six-sigma gap analysis identifies 13 gaps distributed across 3 phases — even if each gap is small, the aggregate may not be. + +### L3: Cross-Dimension Trade-offs Section Is Thin + +Only 2 trade-offs documented. Given 6 dimensions were analyzed and 13 gaps identified, the absence of more cross-cutting tensions suggests either (a) the dimensions are well-aligned (possible for a validate-existing-work design) or (b) some tensions were resolved implicitly during analysis without being recorded. + +--- + +## Summary + +The design is solid. It correctly validates an existing implementation, identifies meaningful quality gaps, and proposes a reasonable remediation plan. The high-priority findings are about risk completeness and clarity rather than architectural unsoundness. No changes to the core design direction are needed. diff --git a/.designs/1/cross-review/analyst-review-plan.md b/.designs/1/cross-review/analyst-review-plan.md new file mode 100644 index 0000000000..d79986828c --- /dev/null +++ b/.designs/1/cross-review/analyst-review-plan.md @@ -0,0 +1,72 @@ +# Cross-Review: Implementation Plan Outline (Analyst Perspective) + +**Reviewer**: rootcause-all (Analyst) +**Date**: 2026-05-12 +**Document Reviewed**: `.designs/1/implementation-plan/implementation_plan_outline.md` + +--- + +## CRITICAL + +No critical issues found. The implementation plan is well-structured with clear phase dependencies, concrete acceptance criteria, and thorough codebase investigation in the "Current State" and "Gotchas" sections. + +--- + +## HIGH + +### H1: Round 1 Finding H3 (Rollback Strategy) Not Addressed in Plan + +My round 1 review flagged that new Warning-level diagnostics (MAFGENWF008, MAFGENWF010, upgraded MAFGENWF006) will break consumer builds with `TreatWarningsAsErrors=true`. The plan acknowledges this risk in the Gotchas section (line 147: "TreatWarningsAsErrors=true in dotnet/Directory.Build.props") but proposes shipping as Warning anyway. The plan says "This is already the plan" but doesn't specify a rollback mechanism if consumer breakage is worse than expected. + +- **Recommendation**: Add a rollback note: if MAFGENWF008/010 cause excessive consumer CI failures, they can be downgraded to Info in a patch release without behavioral change. This is a one-line severity change per diagnostic descriptor. + +### H2: Incremental Caching Invalidation Risk for HandlerInfo Changes + +The Gotchas section (line 148) correctly identifies that adding `DiagnosticLocationInfo` to `HandlerInfo` changes its value equality semantics, and recommends using `MethodAnalysisResult` location data instead. However, the Required Changes section for File 3 (HandlerInfo.cs, line 116-117) still lists adding a `MethodLocation` field as a possibility. This contradiction could lead an implementer to choose the wrong approach. + +- **Recommendation**: Remove the "possibly" change to HandlerInfo.cs from Required Changes, or explicitly mark it as "DO NOT modify HandlerInfo — use MethodAnalysisResult locations instead" to match the Gotcha guidance. + +### H3: Missing Validation Order Specification in Phase 1 + +The Gotchas section (line 151) specifies the correct validation order: (1) < 2 params, (2) > 3 params, (3) IWorkflowContext check, (4) CancellationToken/non-CT check. But the Required Changes section (lines 111-113) specifies insertions "After L480" and "After L493" without referencing this ordering constraint. An implementer reading Required Changes linearly might insert the 4+ param check after the existing < 2 check (correct), but could also misread the CancellationToken addition point. + +- **Recommendation**: Add explicit ordering to Required Changes to match the Gotchas ordering, or reference the Gotcha by number. + +--- + +## LOW + +### L1: Phase 2 Multi-File Partial Class Test Already Exists — Plan Should Remove from Deliverables + +The plan correctly identifies (line 180, line 225) that multi-file partial class tests already exist at lines 541-704. However, it's still listed as a Phase 2 deliverable in the design-doc Phase 2 section (referenced at line 170). The plan should explicitly state this deliverable is pre-satisfied and remove it from the implementation scope to avoid confusion about Phase 2 LOE. + +### L2: No Cross-Reference Between Python Rootcause and .NET Generator Risks + +This plan addresses a .NET source generator. My rootcause analysis covers a Python SDK event loop blocking bug. While these are different issues, both share a common pattern: **existing correct patterns in one part of the codebase not being applied to another** (Python: `FunctionExecutor` has `asyncio.to_thread()` but `FunctionTool` does not; .NET: source generator exists but no behavioral equivalence tests against the reflection path). The implementation plan could note this as a general codebase health observation, but this is informational, not actionable. + +### L3: Acceptance Criteria Use `--no-build` Flag Inconsistently + +Phase 1 acceptance criteria (lines 122-125) use `--no-build` for individual test filters but the full-suite test (line 138) does not. This means the individual tests assume a prior build step, but this isn't stated. Minor, but could cause confusion if someone runs the acceptance criteria standalone. + +--- + +## Risk Coverage Assessment (vs. Rootcause Findings) + +My rootcause analysis identified enforcement levels as a key quality measure. Mapping the plan's changes: + +| Plan Change | Enforcement Level | Assessment | +|-------------|-------------------|------------| +| MAFGENWF008 (duplicate input types) | **Interlock** (compile-time) | Correct — catches runtime crash at build time | +| MAFGENWF009 (4+ params) | **Interlock** (compile-time Error) | Correct — prevents silent misconfiguration | +| MAFGENWF010 (non-CT third param) | **Runtime guard** (Warning, non-blocking) | Appropriate — handler still valid, user informed | +| MAFGENWF006 upgrade | **Runtime guard** (Warning) | Appropriate — progressive severity increase | +| Baseline comparison tests | **Runtime guard** | Good — catches regression in generated output | +| Migration guide | **Advisory** | Correct level for documentation | + +The enforcement hierarchy is appropriate. No interlock-level changes are missing. + +--- + +## Summary + +The implementation plan is thorough and actionable. The main issue (H2) is a contradiction between Required Changes and Gotchas regarding HandlerInfo modification that could mislead implementers. The plan has excellent codebase investigation depth — exact file/line references, existing test discovery, and practical gotchas. All three phases have concrete acceptance criteria with verifiable commands. diff --git a/.designs/1/cross-review/designer-update-log.md b/.designs/1/cross-review/designer-update-log.md new file mode 100644 index 0000000000..37377a3e21 --- /dev/null +++ b/.designs/1/cross-review/designer-update-log.md @@ -0,0 +1,37 @@ +# Designer Update Log — Cross-Review Round 1 + +**Date**: 2026-05-12 +**Reviewer**: rootcause-all (Analyst) +**Review document**: `.designs/1/cross-review/analyst-review-design.md` + +## Findings Addressed + +### H1: Gap 4 Runtime Failure Mode (HIGH) — INCORPORATED + +**Change**: Updated Cross-Perspective Conflicts table (Gap 4 row) to characterize the exact runtime failure: `RouteBuilder.AddHandler` throws `ArgumentException` ("An item with the same key has already been added") when two handlers register the same input type. Updated the MAFGENWF008 resolution to reference this specific exception in the diagnostic message. + +**Location**: design-doc.md, Cross-Perspective Conflicts table, Gap 4 row + +### H2: Gap 7 Behavioral Equivalence Blocker Clarification (HIGH) — INCORPORATED + +**Change**: Updated Six-Sigma Caveats table (Gap 7 row) to explain the specific blocker: the test project has CSharpCompilation infrastructure for generator output verification (structural), but proving behavioral equivalence requires runtime execution of generated code against actual RouteBuilder/ProtocolBuilder instances — this needs an integration test host that compiles, loads, and executes generated assemblies. Structural baseline comparisons (Phase 2) are feasible; full runtime equivalence is not without additional infrastructure. + +**Location**: design-doc.md, Six-Sigma Caveats table, Gap 7 row + +### H3: Rollback Strategy for Phase 1 Diagnostics (HIGH) — INCORPORATED + +**Changes**: +1. Added new risk entry: "New diagnostics (MAFGENWF008, upgraded MAFGENWF006) break consumer CI builds" with mitigation: ship as Warning initially, upgrade to Error in next major version, document in release notes. +2. Updated Phase 1 deliverable #1 to specify MAFGENWF008 ships as Warning (not Error) initially to avoid breaking consumer CI with TreatWarningsAsErrors. + +**Location**: design-doc.md, Risk Registry table (new row) and Phase 1 deliverables + +## Findings Not Addressed (LOW — no action required per cross-review protocol) + +- L1: AC-9 migration impact assessment — noted, no external documentation references the three-method pattern +- L2: Effort estimates beyond "Small" — valid for detailed sprint planning, not required at design level +- L3: Cross-dimension trade-offs section — only 2 tensions exist because dimensions are well-aligned for a validate-existing-work design + +## Root Cause Analysis Context + +The analyst's `rootcause_analysis.md` covers a different issue (Python FunctionTool sync blocking, #5741). This is the upstream problem being designed for by the soldesign-plan orchestration. The Roslyn source generator design is one component of the broader solution design. No changes to design-doc.md were needed from the rootcause analysis — it validates a Python-side concern orthogonal to the .NET source generator. diff --git a/.designs/1/data.md b/.designs/1/data.md new file mode 100644 index 0000000000..3b704eebbe --- /dev/null +++ b/.designs/1/data.md @@ -0,0 +1,160 @@ +# D2: Data Model + +## Dimension Summary + +The data model for this source generator encompasses: (1) the intermediate analysis models used within the generator pipeline (ExecutorInfo, HandlerInfo, MethodAnalysisResult, ClassProtocolInfo, AnalysisResult), (2) the attribute data structures consumed from user code, and (3) the project/file structure specified in AC-1 and AC-17. + +--- + +## Internal Generator Models (Pipeline Data) + +### Current Model Architecture (Verified from Models/ directory) + +The generator uses an incremental pipeline with these data types: + +| Model | File | Purpose | +|-------|------|---------| +| MethodAnalysisResult | Models/MethodAnalysisResult.cs | Per-method analysis output from Pipeline 1 | +| ClassProtocolInfo | Models/ClassProtocolInfo.cs | Per-class-attribute output from Pipelines 2/3 | +| AnalysisResult | Models/AnalysisResult.cs | Combined result (ExecutorInfo + Diagnostics) | +| ExecutorInfo | Models/ExecutorInfo.cs | Final executor metadata for code generation | +| HandlerInfo | Models/HandlerInfo.cs | Per-handler metadata (method name, types, signature kind) | +| HandlerSignatureKind | Models/HandlerInfo.cs:8-21 | Enum: VoidSync, VoidAsync, ResultSync, ResultAsync | +| DiagnosticInfo | Models/DiagnosticInfo.cs | Serializable diagnostic for incremental generator caching | +| DiagnosticLocationInfo | Models/DiagnosticLocationInfo.cs | Serializable location for diagnostics | +| ProtocolAttributeKind | Models/ProtocolAttributeKind.cs | Enum: Send vs Yield | +| ImmutableEquatableArray | Models/ImmutableEquatableArray.cs | Value-equality wrapper for ImmutableArray | +| EquatableArray | Models/EquatableArray.cs | Additional equality support | + +All models are `record` types (ExecutorInfo.cs:18, HandlerInfo.cs:34) for automatic value equality, which is required by incremental generator caching. + +--- + +## Option D1: Preserve Current Model Architecture (RECOMMENDED) + +Keep the existing model structure unchanged. All models listed above exist and are functionally complete. + +**Trade-offs:** +- (+) Fully implemented and tested (ExecutorRouteGeneratorTests.cs ~37KB of tests) +- (+) Records provide correct value equality for incremental generator caching +- (+) Clean separation: MethodAnalysisResult (per-method) -> CombineHandlerMethodResults (per-class) -> ExecutorInfo (for generation) +- (+) ImmutableEquatableArray solves the known Roslyn incremental generator caching problem with ImmutableArray +- (-) AC-1 specifies a `SyntaxDetector.cs` file that does not exist separately; detection is inline in ExecutorRouteGenerator.cs via `ForAttributeWithMetadataName` + +**Reversibility:** Easy -- models are internal to the generator. + +**Constraint compliance:** +- C-1 (netstandard2.0): All models use only netstandard2.0-compatible types. Records are enabled via `InjectIsExternalInitOnLegacy` in the .csproj. +- C-2 (Roslyn 4.8+): Note -- actual .csproj references Roslyn 4.4.0 (Microsoft.Agents.AI.Workflows.Generators.csproj:49), not 4.8.0+. Roslyn 4.4.0 is the minimum version supporting `ForAttributeWithMetadataName`. The constraint C-2 says "4.8.0+" but the implementation deliberately chose 4.4.0 for broader SDK compatibility (see comment in .csproj lines 45-48). + +**Dependencies produced:** ExecutorInfo and HandlerInfo consumed by D1 (API -- SourceBuilder), D6 (integration -- code generation). +**Dependencies required:** Attribute shapes from D1 (API). + +**Risks:** +- Severity: Medium. C-2 says "Reference Microsoft.CodeAnalysis.CSharp 4.8.0+" but actual uses 4.4.0. +- Mitigation: The 4.4.0 choice is deliberate and documented in the .csproj comment. The `ForAttributeWithMetadataName` API used throughout the generator was introduced in 4.4.0. Using 4.8.0 would exclude users on older .NET SDKs. This is a conscious deviation from the plan for compatibility reasons. + +--- + +## Option D2: Extract SyntaxDetector as Separate File (Per AC-1) + +Create a `SyntaxDetector.cs` file that wraps the inline `ForAttributeWithMetadataName` predicates currently in ExecutorRouteGenerator.cs:33-53. + +**Trade-offs:** +- (+) Matches AC-1 file structure exactly +- (+) Separates concerns (detection vs analysis vs generation) +- (-) The predicates are trivially simple (`node is MethodDeclarationSyntax`, `node is ClassDeclarationSyntax`) -- extracting them adds indirection without value +- (-) Roslyn's `ForAttributeWithMetadataName` already IS the syntax detection; wrapping it adds no abstraction benefit +- (-) AC-15 specifies `SyntaxDetectorTests.cs` which would need test targets + +**Reversibility:** Easy. + +**Constraint compliance:** All constraints met. + +**Dependencies produced:** SyntaxDetector class consumed by ExecutorRouteGenerator. +**Dependencies required:** None. + +**Risks:** +- Severity: Low. Thin wrapper adds no meaningful testable surface. +- Mitigation: If extracted, tests would verify predicate logic (trivial). + +--- + +## Option D3: Add Explicit Model for AC-10 Inheritance Scenarios + +Currently, inheritance handling is encoded in the `BaseHasConfigureProtocol` boolean flag on ExecutorInfo (ExecutorInfo.cs:25). AC-10 describes three scenarios: +1. Directly extends Executor -> No base call +2. Extends executor with [MessageHandler] methods -> base call +3. Extends executor with manual ConfigureRoutes -> base call + +Option: Replace the single boolean with a richer enum/model capturing the exact inheritance scenario. + +**Trade-offs:** +- (+) More explicit about inheritance intent +- (+) Easier to extend if new inheritance scenarios arise +- (-) The boolean already correctly captures the needed decision (call base or not) +- (-) SemanticAnalyzer.cs:424-448 (`BaseHasConfigureProtocol`) correctly walks the chain and detects both scenarios 2 and 3 +- (-) Adds complexity for a single code-generation decision point + +**Reversibility:** Easy -- internal to generator. + +**Constraint compliance:** All constraints met. + +**Dependencies produced:** Richer inheritance model consumed by SourceBuilder. +**Dependencies required:** None. + +**Risks:** +- Severity: Low. +- Mitigation: N/A. + +--- + +## Option D4: Upgrade Roslyn Reference to 4.8.0+ Per C-2 + +Change `Microsoft.CodeAnalysis.CSharp` from `4.4.0` to `4.8.0` to match constraint C-2 literally. + +**REJECTED: violates C-2 intent (broader compatibility)** + +While C-2 says "Reference Microsoft.CodeAnalysis.CSharp 4.8.0+", the .csproj comment (lines 45-48) documents a deliberate choice: "Use Roslyn 4.4.0 - minimum version for ForAttributeWithMetadataName API. Corresponds to .NET 7 SDK / VS 2022 17.4+. Higher versions would require newer SDKs, breaking users on older versions." Upgrading to 4.8.0 would break users on .NET 7 SDK, which contradicts the project's compatibility goals. + +**Trade-offs:** +- (+) Matches C-2 verbatim +- (-) Breaks .NET 7 SDK users +- (-) No API from 4.8.0 is actually used by the generator +- (-) Contradicts the documented rationale in the .csproj + +**Reversibility:** Easy. + +--- + +## Project Structure (AC-1, AC-17) + +### Actual vs Plan File Mapping + +| Plan Path (AC-1/AC-17) | Actual Path | Status | +|------------------------|-------------|--------| +| ExecutorRouteGenerator.cs | ExecutorRouteGenerator.cs | EXISTS, matches | +| Models/ExecutorInfo.cs | Models/ExecutorInfo.cs | EXISTS, matches | +| Models/HandlerInfo.cs | Models/HandlerInfo.cs | EXISTS, matches | +| Analysis/SyntaxDetector.cs | (inline in ExecutorRouteGenerator.cs) | DIFFERENT -- detection via ForAttributeWithMetadataName | +| Analysis/SemanticAnalyzer.cs | Analysis/SemanticAnalyzer.cs | EXISTS, matches | +| Generation/SourceBuilder.cs | Generation/SourceBuilder.cs | EXISTS, matches | +| Diagnostics/DiagnosticDescriptors.cs | Diagnostics/DiagnosticDescriptors.cs | EXISTS, matches | +| (not in plan) | Models/AnalysisResult.cs | EXTRA -- needed for pipeline | +| (not in plan) | Models/ClassProtocolInfo.cs | EXTRA -- needed for pipeline | +| (not in plan) | Models/MethodAnalysisResult.cs | EXTRA -- needed for pipeline | +| (not in plan) | Models/DiagnosticInfo.cs | EXTRA -- needed for caching | +| (not in plan) | Models/DiagnosticLocationInfo.cs | EXTRA -- needed for caching | +| (not in plan) | Models/ProtocolAttributeKind.cs | EXTRA -- Send vs Yield enum | +| (not in plan) | Models/ImmutableEquatableArray.cs | EXTRA -- equality for caching | +| (not in plan) | Models/EquatableArray.cs | EXTRA -- equality for caching | +| (not in plan) | Directory.Build.targets | EXTRA -- build plumbing | +| (not in plan) | SkipIncompatibleBuild.targets | EXTRA -- build plumbing | + +The additional files are necessary for a correct incremental generator implementation (caching requires value equality on all pipeline data). The plan underestimated the model count. + +--- + +## Summary + +**Recommended option: D1** -- Preserve current model architecture. The implementation is complete, tested, and correctly handles incremental generator caching requirements. The SyntaxDetector.cs absence is a simplification (not a gap), and the Roslyn 4.4.0 choice is a deliberate compatibility decision. diff --git a/.designs/1/dependencies.md b/.designs/1/dependencies.md new file mode 100644 index 0000000000..eb22b65e17 --- /dev/null +++ b/.designs/1/dependencies.md @@ -0,0 +1,133 @@ +# dependencies.md -- Dependency Graph + +## Component Dependencies (Code-Level) + +### Runtime Components (Microsoft.Agents.AI.Workflows) + +``` +Executor (abstract class) + ^ + |-- ProtocolBuilder (fluent config) + | |-- RouteBuilder (handler registration) + | |-- SendsMessage() (type declaration) + | |-- YieldsOutput() (type declaration) + | + |-- MessageHandlerAttribute (method-level attribute) + |-- SendsMessageAttribute (class/method-level attribute) + |-- YieldsOutputAttribute (class/method-level attribute) + | + |-- ReflectingExecutor [Obsolete] (reflection-based alternative) + |-- IMessageHandler [Obsolete] (interface-based routing) +``` + +**Dependency direction:** User executor classes -> Executor (base) + Attributes -> ProtocolBuilder -> RouteBuilder. No circular dependencies. + +### Generator Components (Microsoft.Agents.AI.Workflows.Generators) + +``` +ExecutorRouteGenerator (IIncrementalGenerator) + | + |-- Pipeline 1: ForAttributeWithMetadataName("MessageHandlerAttribute") + | |-- SemanticAnalyzer.AnalyzeHandlerMethod() + | | |-- HandlerInfo (model) + | | |-- MethodAnalysisResult (model) + | | |-- DiagnosticInfo (model) + | | |-- DiagnosticLocationInfo (model) + | | |-- HandlerSignatureKind (enum) + | | + | |-- SemanticAnalyzer.CombineHandlerMethodResults() + | |-- AnalysisResult (model) + | |-- ExecutorInfo (model) + | |-- DiagnosticDescriptors (diagnostic definitions) + | + |-- Pipeline 2: ForAttributeWithMetadataName("SendsMessageAttribute") + | |-- SemanticAnalyzer.AnalyzeClassProtocolAttribute() + | |-- ClassProtocolInfo (model) + | |-- ProtocolAttributeKind (enum) + | + |-- Pipeline 3: ForAttributeWithMetadataName("YieldsOutputAttribute") + | |-- SemanticAnalyzer.AnalyzeClassProtocolAttribute() [same as Pipeline 2] + | + |-- CombineAllResults() (merge pipelines) + | |-- AnalysisResult (combined) + | + |-- SourceBuilder.Generate() + |-- ExecutorInfo (input) + |-- HandlerInfo (input) + |-- ImmutableEquatableArray (collection wrapper) +``` + +**Dependency direction:** ExecutorRouteGenerator -> SemanticAnalyzer -> Models -> DiagnosticDescriptors. SourceBuilder <- ExecutorInfo. No circular dependencies. + +### Cross-Project Dependencies + +``` +Microsoft.Agents.AI.Workflows.Generators (netstandard2.0) + | + |-- [analyzer reference, compile-time only] + v +Microsoft.Agents.AI.Workflows (net8.0, netstandard2.0) + | + |-- [ProjectReference] + v +Microsoft.Agents.AI.Abstractions +Microsoft.Agents.AI +``` + +``` +Microsoft.Agents.AI.Workflows.Generators.UnitTests (test project) + | + |-- [ProjectReference] -> Microsoft.Agents.AI.Workflows.Generators + |-- [InternalsVisibleTo from] Microsoft.Agents.AI.Workflows +``` + +**Note:** The generator project does NOT have a compile-time dependency on the Workflows project. It discovers attribute metadata by fully-qualified name string (e.g., `"Microsoft.Agents.AI.Workflows.MessageHandlerAttribute"` in ExecutorRouteGenerator.cs:24). This is the correct pattern for Roslyn generators -- they must not reference the target assembly. + +--- + +## Component Build Order (Critical Path) + +| Order | Component | TFM | Depends On | Blocks | +|-------|-----------|-----|------------|--------| +| 1 | Microsoft.Agents.AI.Abstractions | net8.0, netstandard2.0 | External packages | Workflows | +| 2 | Microsoft.Agents.AI | net8.0, netstandard2.0 | Abstractions | Workflows | +| 3a | Microsoft.Agents.AI.Workflows.Generators | netstandard2.0 | Roslyn 4.4.0 (NuGet) | Workflows (as analyzer), Generator tests | +| 3b | Microsoft.Agents.AI.Workflows | net8.0, netstandard2.0 | AI, Abstractions, Generators (analyzer) | Downstream consumers, Workflows tests | +| 4 | Microsoft.Agents.AI.Workflows.Generators.UnitTests | net8.0+ | Generators, Workflows (InternalsVisibleTo) | CI gate | + +**Critical path:** Abstractions -> AI -> Generators -> Workflows -> Tests + +Steps 3a and 3b have a dependency: Workflows depends on Generators as an analyzer. However, MSBuild handles this correctly via the `OutputItemType="Analyzer"` reference. The generator must build FIRST so its DLL is available for the Workflows compilation. + +The `SkipIncompatibleBuild.targets` file in the Generators project handles cross-platform build scenarios where the generator cannot be compiled (e.g., if the Roslyn SDK is unavailable). + +--- + +## Circular Dependency Check + +**Result: No circular dependencies.** + +Verification: +1. **Generator -> Workflows:** NO. Generator references Workflows types by string name only (ExecutorRouteGenerator.cs:24-26), not by project reference. +2. **Workflows -> Generator:** YES, but as analyzer only (OutputItemType="Analyzer", ReferenceOutputAssembly="false"). This is a build-time-only dependency, not a compile-time reference dependency. +3. **Models -> SemanticAnalyzer:** NO. Models are plain records with no behavior. +4. **SemanticAnalyzer -> SourceBuilder:** NO. These are independent: SemanticAnalyzer produces AnalysisResult, SourceBuilder consumes ExecutorInfo. The connection is through ExecutorRouteGenerator (the orchestrator). +5. **DiagnosticDescriptors -> SemanticAnalyzer:** One-way. SemanticAnalyzer references DiagnosticDescriptors; DiagnosticDescriptors does not reference SemanticAnalyzer. + +--- + +## Critical Path Risks + +| Risk | Severity | Component | Mitigation | +|------|----------|-----------|------------| +| Roslyn version incompatibility | Medium | Generators | Pinned to 4.4.0; comment in .csproj documents rationale; SkipIncompatibleBuild.targets handles unsupported platforms | +| Generator build failure blocks Workflows | High | Generators -> Workflows | SkipIncompatibleBuild.targets allows Workflows to build without generator on unsupported platforms | +| InternalsVisibleTo coupling | Low | Workflows -> Generator tests | Only affects test project; production code unaffected | +| TFM mismatch (generator=netstandard2.0, host=net8.0) | Low | Generators | GlobalPropertiesToRemove="TargetFramework" in Workflows .csproj prevents MSBuild from passing wrong TFM | +| Attribute name string coupling | Medium | Generator <-> Workflows | Generator uses hardcoded fully-qualified attribute names (ExecutorRouteGenerator.cs:24-26). If attribute namespace changes, generator silently stops working. Mitigated by being in the same repo with CI/CD. | + +--- + +## Dependency Summary + +The dependency graph is clean with no circular dependencies. The critical path runs through the Generators project (must build before Workflows), but this is handled by standard MSBuild analyzer reference patterns. The only significant coupling is the string-based attribute name matching between the generator and the Workflows attribute definitions, which is the standard pattern for Roslyn generators and is mitigated by co-location in the same repository. diff --git a/.designs/1/design-doc.md b/.designs/1/design-doc.md new file mode 100644 index 0000000000..52df361177 --- /dev/null +++ b/.designs/1/design-doc.md @@ -0,0 +1,251 @@ +# Design: Roslyn Source Generator for Workflow Executor Routes + +## Executive Summary + +This design evaluates the plan to replace the reflection-based `ReflectingExecutor` pattern with a compile-time Roslyn source generator using `[MessageHandler]` attributes. The key finding is that **the implementation already exists and is functionally complete** — the source generator, all three attributes, diagnostics, and test suite are already in the codebase with deliberate naming and API evolutions from the original plan. + +The architecture elevation analysis confirms the design frame is correct: handler-to-route binding is an inherent requirement, and the compile-time source generator is the right approach. The reflection path (`ReflectingExecutor`) is already marked `[Obsolete]` with a transitional deprecation strategy appropriate for a published library with external consumers. + +The six-sigma gap analysis identified 13 quality gaps, 3 high-impact, all feasible to address. The most significant gaps are: (1) duplicate handler input types produce runtime exceptions instead of compile-time diagnostics, (2) no behavioral equivalence integration tests between generated and reflected routes, and (3) `[MessageHandler]` on `Executor` subclasses is silently ignored with only Info-level severity. This design proposes addressing these gaps as incremental improvements to the existing implementation. + +## Constraints Respected + +All proposals respect the constraints captured in source.md: + +- **C-1**: Target Framework — Generator targets `netstandard2.0` (.csproj line 5). COMPLIANT. +- **C-2**: Roslyn Version — Plan specifies 4.8.0+; implementation uses 4.4.0. DEVIATION: deliberate for broader SDK compatibility (.csproj lines 45-49 comment explains rationale). `ForAttributeWithMetadataName` API is available in 4.4.0. Functionally compliant. +- **C-3**: Analyzer Packaging — `IsRoslynComponent=true` (.csproj:16), `EnforceExtendedAnalyzerRules=true` (.csproj:17), `PackagePath="analyzers/dotnet/cs"` (.csproj:55). COMPLIANT. +- **C-4**: Migration Strategy — Clean break via `[Obsolete]` on `ReflectingExecutor` (ReflectingExecutor.cs:21-22) and `IMessageHandler` (IMessageHandler.cs:17-18, 42-43). Direct `Executor` inheritance required for source generation. COMPLIANT. +- **C-5**: Handler Accessibility — `SemanticAnalyzer.AnalyzeHandler` (line 462) performs no accessibility check. Handlers of any access level are accepted. COMPLIANT. +- **C-6**: Partial Modifier — `SemanticAnalyzer.IsPartialClass` (lines 367-380) enforces the partial requirement; violation emits `MAFGENWF003` error. COMPLIANT. + +## AC Traceability (REQUIRED) + +| AC id | Verbatim quote from source.md | Clause breakdown | Addressed by | Status | +|-------|------------------------------|------------------|--------------|--------| +| AC-1 | "dotnet/src/Microsoft.Agents.AI.Workflows.Generators/ ├── ... ├── Analysis/ │ ├── SyntaxDetector.cs │ └── SemanticAnalyzer.cs ..." | (i) .csproj (ii) ExecutorRouteGenerator.cs (iii-iv) Models/ (v) SyntaxDetector.cs (vi-viii) Analysis, Generation, Diagnostics | Generator project | PARTIAL: SyntaxDetector.cs does not exist as separate file; detection is inline via `ForAttributeWithMetadataName`. All other files exist. | +| AC-2 | "Target netstandard2.0, Reference Microsoft.CodeAnalysis.CSharp 4.8.0+, IsRoslynComponent=true, EnforceExtendedAnalyzerRules=true, analyzers/dotnet/cs" | (i) TFM (ii) Roslyn version (iii-iv) MSBuild props (v) pack path | Generator .csproj | PARTIAL: Roslyn is 4.4.0 not 4.8.0 (deliberate deviation). All other clauses met. | +| AC-3 | "MessageHandlerAttribute with Yield and Send properties" | (i-vi) AttributeUsage, sealed, Yield, Send | MessageHandlerAttribute.cs:48-70 | MET | +| AC-4 | "SendsMessageAttribute with Type property" | (i-vi) AttributeUsage, sealed, Type, constructor | SendsMessageAttribute.cs:32-49 | MET (targets Class\|Method, broader than plan's Class-only) | +| AC-5 | "YieldsMessageAttribute with Type property" | (i-vii) AttributeUsage, sealed, Type, constructor, name | YieldsOutputAttribute.cs:32-49 | PARTIAL: named YieldsOutputAttribute, not YieldsMessageAttribute | +| AC-6 | "partial modifier, [MessageHandler] method detection" | (i) partial check (ii) attribute detection | SemanticAnalyzer.cs:367-380, ExecutorRouteGenerator.cs:33-37 | MET | +| AC-7 | "Executor derivation, ConfigureRoutes not defined, valid signature, valid return type" | (i) derivation (ii) override check (iii) signature (iv) return type | SemanticAnalyzer.cs:385-400, 406-418, 462-533, 539-569 | MET (checks ConfigureProtocol not ConfigureRoutes) | +| AC-8 | "Handler signature → AddHandler mapping" | (i-vi) 6 signature patterns | SourceBuilder.cs:178-189, HandlerSignatureKind enum | MET | +| AC-9 | "Generated ConfigureRoutes/ConfigureSentTypes/ConfigureYieldTypes" | (i-iii) three method overrides (iv-v) headers | SourceBuilder.cs:27-127 | DEVIATION: generates unified ConfigureProtocol with fluent API, not separate methods | +| AC-10 | "Inheritance: no base call vs base call" | (i-iii) three scenarios | ExecutorInfo.BaseHasConfigureProtocol, SourceBuilder.cs:79-84 | MET (uses ConfigureProtocol not ConfigureRoutes) | +| AC-11 | "WFGEN001-006 diagnostics" | (i-vi) 6 diagnostic rules | DiagnosticDescriptors.cs:34-107 | DEVIATION: uses MAFGENWF001-007 (7 diagnostics, includes static handler check MAFGENWF007) | +| AC-12 | "ProjectReference as Analyzer" | (i-iii) ProjectReference, OutputItemType, ReferenceOutputAssembly | Workflows .csproj:35-39 | MET | +| AC-13 | "ReflectingExecutor marked [Obsolete]" | (i-iii) [Obsolete], message, error:false | ReflectingExecutor.cs:21-22 | MET (message says "future version" not "v1.0") | +| AC-14 | "IMessageHandler marked [Obsolete]" | (i-ii) both interfaces | IMessageHandler.cs:17-18, 42-43 | MET | +| AC-15 | "Generator unit tests" | (i) ExecutorRouteGeneratorTests.cs (ii) SyntaxDetectorTests.cs (iii) SemanticAnalyzerTests.cs (iv) GeneratorTestHelper.cs (v-xiii) test scenarios | Tests project: 33 test methods in ExecutorRouteGeneratorTests.cs | PARTIAL: SyntaxDetectorTests.cs and SemanticAnalyzerTests.cs do not exist. Test coverage integrated in ExecutorRouteGeneratorTests.cs. | +| AC-16 | "Port ReflectingExecutor tests, verify behavioral equivalence" | (i) ported tests (ii) behavioral equivalence | ReflectionSmokeTest.cs exists but only tests reflection path | GAP: No behavioral equivalence test exists (Gap 7) | +| AC-17 | "12 files to create" | (i-xii) individual files | All exist except SyntaxDetector.cs (inline) and YieldsMessageAttribute.cs (named YieldsOutputAttribute.cs) | PARTIAL: 10/12 exist exactly, 2 have deliberate deviations | +| AC-18 | "4 files to modify" | (i-iv) .csproj, ReflectingExecutor, IMessageHandler, solution | All modifications present | MET (solution is .slnx not .sln) | +| AC-19 | "Example usage end state" | (i-iv) class-level attrs, return inference, explicit Yield/Send, generated output | Source generator pipeline | DEVIATION: generated code uses ConfigureProtocol pattern, not separate methods | + +## Architecture Elevation Verdict + +**Verdict**: Frame correct (with grounded constraint) + +The design concern — replacing reflection with compile-time source generation — is structurally sound. The concern cannot be eliminated by removing any abstraction; handler-to-route binding is an inherent requirement of the executor architecture. Five concerning abstractions were evaluated (ReflectingExecutor, IMessageHandler interfaces, RouteBuilderExtensions, MessageHandlerInfo, ValueTaskTypeErasure); none, if removed alone, would eliminate the need for handler binding. + +One elevation candidate (complete removal of the Reflection/ subsystem, ~487 lines across 6 files) survived the subtraction gate but was rejected by the self-challenge: immediate removal would create `TypeLoadException` failures for NuGet consumers not yet migrated. The current `[Obsolete]` deprecation approach is the correct architecture for a published library. + +## Problem Statement + +> Replace the reflection-based `ReflectingExecutor` pattern with a compile-time source generator that discovers `[MessageHandler]` attributed methods and generates `ConfigureRoutes`, `ConfigureSentTypes`, and `ConfigureYieldTypes` implementations. + +(Verbatim from source.md) + +## Proposed Design + +### Overview + +The source generator is already implemented and functional. This design validates the implementation against the original plan, documents deliberate deviations, and proposes incremental improvements to close quality gaps identified by independent analysis. The design recommends preserving the current implementation with targeted enhancements. + +### Key Components (All Existing) + +1. **ExecutorRouteGenerator** (`ExecutorRouteGenerator.cs:22-161`) — IIncrementalGenerator with three attribute-driven pipelines +2. **SemanticAnalyzer** (`Analysis/SemanticAnalyzer.cs`) — Handler method validation and class analysis +3. **SourceBuilder** (`Generation/SourceBuilder.cs:27-127`) — ConfigureProtocol code emission with fluent API +4. **DiagnosticDescriptors** (`Diagnostics/DiagnosticDescriptors.cs:34-107`) — 7 diagnostic rules (MAFGENWF001-007) +5. **Data Models** (`Models/`) — ExecutorInfo, HandlerInfo, and 6 supporting record types +6. **Attributes** — MessageHandlerAttribute, SendsMessageAttribute, YieldsOutputAttribute + +### Component Dependency Graph + +``` +Attribute Definitions (Workflows package) + ↓ consumed by +ExecutorRouteGenerator (ForAttributeWithMetadataName pipelines) + ↓ delegates to +SemanticAnalyzer (validation + data extraction) + ↓ produces +ExecutorInfo + HandlerInfo (immutable data models) + ↓ consumed by +SourceBuilder (code emission → ConfigureProtocol override) + ↓ references +DiagnosticDescriptors (error/warning reporting) +``` + +Build order: Attributes → Generator .csproj (netstandard2.0) → Workflows .csproj (analyzer reference) → Tests + +### Interface + +The user-facing API consists of three attributes: + +| Attribute | Target | Purpose | +|-----------|--------|---------| +| `[MessageHandler]` | Method | Marks a method as a handler; optional `Yield`/`Send` type arrays | +| `[SendsMessage(typeof(T))]` | Class, Method | Declares sent message types for protocol validation | +| `[YieldsOutput(typeof(T))]` | Class, Method | Declares yielded output types for protocol validation | + +Seven diagnostics provide compile-time feedback: + +| ID | Severity | Condition | +|----|----------|-----------| +| MAFGENWF001 | Error | Handler missing IWorkflowContext parameter | +| MAFGENWF002 | Error | Handler has invalid return type | +| MAFGENWF003 | Error | Executor with [MessageHandler] must be partial | +| MAFGENWF004 | Warning | [MessageHandler] on non-Executor class | +| MAFGENWF005 | Error | Handler has fewer than 2 parameters | +| MAFGENWF006 | Info | ConfigureProtocol already defined | +| MAFGENWF007 | Error | Handler cannot be static | + +### Data Model + +The generator pipeline uses immutable records with value equality for incremental caching: + +- **ExecutorInfo** (`Models/ExecutorInfo.cs:18`) — Class metadata, handler list, class-level type declarations +- **HandlerInfo** (`Models/HandlerInfo.cs:34`) — Method name, input/output types, signature kind, explicit Yield/Send types +- **HandlerSignatureKind** — Enum: VoidSync, VoidAsync, ResultSync, ResultAsync + +Supporting types: MethodAnalysisResult, ClassProtocolInfo, AnalysisResult, ProtocolAttributeKind, ImmutableEquatableArray. + +## Cross-Dimension Trade-offs + +| Conflict | Resolution | Rationale | +|----------|------------|-----------| +| D1 (API) vs D6 (Integration): Plan naming differs from implementation (YieldsMessage→YieldsOutput, ConfigureRoutes→ConfigureProtocol, WFGEN→MAFGENWF) | Document mapping; accept deviation | Renaming would break existing consumers; aliases add complexity. Deviations are deliberate (commit 0756c457) | +| D2 (Data) vs D6 (Integration): Roslyn 4.8.0+ specified but 4.4.0 used | Keep 4.4.0; document deviation | 4.4.0 provides ForAttributeWithMetadataName; upgrading breaks .NET 7 SDK users. .csproj:45-48 documents rationale | + +## Cross-Perspective Conflicts + +| Finding Source | Finding | Conflicts With | Nature | Resolution | +|---------------|---------|----------------|--------|------------| +| Elevation | Frame correct; no lift needed | Dimensions (all recommend status quo) | No conflict — aligned | Both perspectives confirm current design is sound | +| Gap Analysis (Gap 4) | Duplicate handlers cause runtime crash — `RouteBuilder.AddHandler` throws `ArgumentException` ("An item with the same key has already been added") when two handlers register the same input type, producing an unhandled exception during executor construction | Dimensions (API recommends current diagnostics) | Tension: API analysis did not flag this gap | Add MAFGENWF008 diagnostic for duplicate input types. Diagnostic message should reference the specific `ArgumentException` users would otherwise encounter at runtime | +| Gap Analysis (Gap 9) | Info-level for ConfigureProtocol override on Executor subclasses | Dimensions (UX recommends current diagnostic levels) | Tension: Info severity masks user intent | Upgrade MAFGENWF006 to Warning when base is Executor/Executor | +| Gap Analysis (Gap 7) | No behavioral equivalence tests | Dimensions (Integration notes test coverage adequate) | Tension: structural tests insufficient for migration validation | Add curated equivalence verification test set | +| Decision History | commit 0756c457 (Config→ExecutorConfig, RouteBuilder explicit) | All dimensions respect this | No conflict | Dimensions correctly avoid reversing this change | + +## Decisions Made + +| Decision | Options Considered | Chosen | Rationale | Reversibility | +|----------|-------------------|--------|-----------|---------------| +| Preserve YieldsOutputAttribute name | (a) Rename to match plan (b) Keep actual name (c) Add alias | (b) Keep YieldsOutputAttribute | Aligns with `YieldsOutput()` fluent API; renaming breaks consumers | Easy | +| Preserve ConfigureProtocol unified API | (a) Revert to separate methods (b) Keep unified ProtocolBuilder | (b) Keep unified | Cleaner API, matches Executor.cs:216 abstract; fewer override points | Moderate | +| Keep Roslyn 4.4.0 | (a) Upgrade to 4.8.0+ (b) Keep 4.4.0 | (b) Keep 4.4.0 | Broader SDK compatibility; all needed APIs available in 4.4.0 | Easy | +| Keep MAFGENWF diagnostic IDs | (a) Rename to WFGEN (b) Keep MAFGENWF | (b) Keep MAFGENWF | Follows Microsoft Agent Framework naming convention | Easy | +| Add duplicate handler diagnostic | (a) Leave as runtime error (b) Add compile-time check | (b) Add MAFGENWF008 | Catches error at build time; prevents runtime crash | Easy | +| Upgrade MAFGENWF006 for Executor | (a) Keep Info (b) Upgrade to Warning | (b) Warning for Executor base | Users clearly intend [MessageHandler] to work | Easy | + +## Risk Registry + +| Risk | Severity | Likelihood | Mitigation | Owner | Source | +|------|----------|-----------|------------|-------|--------| +| NuGet consumers on ReflectingExecutor lose functionality when deprecated type is removed | High | Medium | Transitional [Obsolete] warning → error → removal across major versions | API | Elevation | +| AutoYieldOutput=false breaks return-type yield registration | Medium | Low | Document runtime dependency in design doc and API docs | Integration | Gap 6 | +| Roslyn API behavior changes in future SDK versions | Medium | Low | Pin to 4.4.0 minimum; test against multiple SDK versions in CI | Integration | Gap 1 | +| Multi-file partial class edge cases in incremental caching | Low | Low | Add explicit multi-file test case (Gap 13) | Data | Gap 13 | +| Method-level [SendsMessage]/[YieldsOutput] on non-handler methods silently ignored | Low | Low | Document limitation in API docs | API | Gap 12 | +| New diagnostics (MAFGENWF008, upgraded MAFGENWF006) break consumer CI builds | Medium | Medium | Ship MAFGENWF008 as Warning (not Error) initially; document new diagnostics in release notes. Consumers with TreatWarningsAsErrors will see build breaks — mitigate by upgrading to Error only in the next major version | API | Cross-review H3 | + +## Six-Sigma Caveats + +| Gap | Category | Impact | Feasibility | Constraint | +|-----|----------|--------|-------------|------------| +| Gap 1 | version drift | Low | Feasible | Documentation correction only | +| Gap 2 | escape path | Medium | Feasible | New diagnostic for 4+ parameters | +| Gap 3 | escape path | Medium | Feasible | New diagnostic for non-CT third parameter | +| Gap 4 | escape path | High | Feasible | Duplicate input type detection in CombineHandlerMethodResults | +| Gap 5 | scope gap | Low | Feasible | Documentation naming clarification | +| Gap 6 | unstated assumption | Medium | Feasible | Document AutoYield runtime dependency | +| Gap 7 | test type mismatch | High | Partially feasible | Blocker: structural tests (source text comparison) cannot verify runtime behavioral equivalence. The test project has CSharpCompilation infrastructure for generator output verification, but proving generated routes match reflected routes requires runtime execution of generated code against actual `RouteBuilder`/`ProtocolBuilder` instances — this needs an integration test host that compiles, loads, and executes generated assemblies. Structural baseline comparisons (Phase 2 deliverable) are feasible; full runtime equivalence is not without additional test infrastructure | +| Gap 8 | sibling vulnerability | Low | Feasible | Document auto-send/yield delegation | +| Gap 9 | escape path | High | Feasible | Upgrade diagnostic severity for Executor case | +| Gap 10 | version drift | Very Low | Feasible | Replace hint name approximation with precise count | +| Gap 11 | observability gap | Medium | Feasible | Create migration guide document | +| Gap 12 | escape path | Low | Feasible | Document method-level attribute scope | +| Gap 13 | test type mismatch | Medium | Feasible | Add multi-file partial class test | + +**Practical ceiling**: The generator cannot validate runtime configuration (`ExecutorOptions`) at compile time. Exact behavioral parity between source-generated and reflected paths requires runtime integration tests. These are irreducible constraints of the compile-time/runtime boundary. + +**Residual risks accepted**: Runtime option validation impossible at compile time; incremental caching edge cases with complex partial class topologies. + +## Implementation Plan + +### Phase 1: High-Impact Gap Closure (Effort: Small) + +**Deliverables:** +1. New diagnostic `MAFGENWF008` for duplicate input type handlers on same executor (ship as Warning initially to avoid breaking consumer CI with TreatWarningsAsErrors; upgrade to Error in next major version) +2. Upgrade `MAFGENWF006` severity to Warning when base class is `Executor` or `Executor` +3. New diagnostic for handler methods with 4+ parameters +4. New diagnostic for non-CancellationToken third parameter + +**Source ACs satisfied**: AC-11 (enhanced diagnostics) +**Dependencies**: None — changes are additive to existing DiagnosticDescriptors.cs and SemanticAnalyzer.cs +**Risks addressed**: Gap 4 (duplicate handlers), Gap 9 (silent ignore), Gap 2 (4+ params), Gap 3 (non-CT third param) +**Six-Sigma gaps closed**: Gaps 2, 3, 4, 9 + +**Phase acceptance criteria:** +- [ ] `MAFGENWF008` fires when two [MessageHandler] methods on the same class have the same input type +- [ ] `MAFGENWF006` severity is Warning (not Info) when the defining base class is `Executor` or `Executor` +- [ ] Handler with 4+ parameters produces a descriptive diagnostic instead of passing validation +- [ ] Handler with non-CancellationToken third parameter produces a warning diagnostic + +### Phase 2: Test Coverage Improvements (Effort: Small) + +**Deliverables:** +1. Multi-file partial class test (two source strings, handler split across files) +2. Curated behavioral equivalence test set comparing generator output against known-good baselines + +**Source ACs satisfied**: AC-15 (test coverage), AC-16 (behavioral equivalence, partial) +**Dependencies**: Phase 1 (new diagnostics should be tested too) +**Risks addressed**: Gap 7 (behavioral equivalence, partial), Gap 13 (multi-file partial) +**Six-Sigma gaps closed**: Gaps 7 (partially), 13 + +**Phase acceptance criteria:** +- [ ] Test passes with handler split across two source files +- [ ] At least 3 baseline comparison tests verify generator output matches expected ConfigureProtocol pattern for common executor configurations + +### Phase 3: Documentation and Migration Guide (Effort: Small) + +**Deliverables:** +1. Migration guide document: steps to convert from `ReflectingExecutor` to `[MessageHandler]` pattern +2. Document auto-yield/auto-send runtime dependency +3. Document plan-vs-implementation deviations (naming, API shape, diagnostic IDs) +4. Document method-level attribute scope limitations + +**Source ACs satisfied**: AC-13 (migration guidance), AC-5 (naming clarification) +**Dependencies**: None +**Risks addressed**: Gap 5 (naming), Gap 6 (auto-yield), Gap 8 (auto-send delegation), Gap 11 (migration guide), Gap 12 (method-level attrs) +**Six-Sigma gaps closed**: Gaps 1, 5, 6, 8, 10, 11, 12 + +**Phase acceptance criteria:** +- [ ] Migration guide document exists with step-by-step conversion instructions +- [ ] Auto-yield/auto-send runtime dependency documented in API doc comments + +## Appendix: Analysis Artifacts + +- [source.md](source.md) +- [verification.md](verification.md) +- [codebase-snapshot.md](codebase-snapshot.md) +- Dimension analyses: [api.md](api.md), [data.md](data.md), [ux.md](ux.md), [scale.md](scale.md), [security.md](security.md), [integration.md](integration.md) +- [audit.md](audit.md) +- [conflicts.md](conflicts.md) +- [dependencies.md](dependencies.md) +- [elevation_assessment.md](elevation_assessment.md) +- [six_sigma_gaps.md](six_sigma_gaps.md) +- [verification-report.md](verification-report.md) +- [synthesis-checklist.md](synthesis-checklist.md) diff --git a/.designs/1/design-refinement-progress.md b/.designs/1/design-refinement-progress.md new file mode 100644 index 0000000000..6e2b899f42 --- /dev/null +++ b/.designs/1/design-refinement-progress.md @@ -0,0 +1,22 @@ +# Design Refinement Progress: Python: [Bug]: Blocking synchronous tool execution freezes Responses API polling inside async event loop + +**Status**: COMPLETE +**Completed**: 2026-05-12 04:52 +**Issue**: https://github.com/stempeck/agent-framework/issues/1 +**Started**: 2026-05-12 + +## Agents +| Role | Agent | Status | Started | Completed | +|------|---------|--------|---------|-----------| +| Analyst | rootcause-all | Analysis Complete | 2026-05-12 | 2026-05-12 04:05 | +| Designer | design-v7 | Design Complete | 2026-05-12 | 2026-05-12 04:05 | + +## Cross-Review Progress +| Round | Exchange | Status | Timestamp | +|-------|----------|--------|-----------| +| 1 | Analyst reviews design-doc | Complete | 2026-05-12 04:19 | +| 1 | Designer incorporates HIGH/CRIT | Complete | 2026-05-12 04:21 | +| 2 | Design-plan-impl runs | Complete | 2026-05-12 04:31 | +| 2 | Analyst reviews impl plan | Complete | 2026-05-12 04:35 | +| 3 | Rootcause-review runs | Complete | 2026-05-12 04:45 | +| 3 | Analyst reviews peer review | Complete | 2026-05-12 04:48 | diff --git a/.designs/1/elevation_assessment.md b/.designs/1/elevation_assessment.md new file mode 100644 index 0000000000..4f1ee6e24d --- /dev/null +++ b/.designs/1/elevation_assessment.md @@ -0,0 +1,133 @@ +# Architecture Elevation Assessment + +**Date**: 2026-05-12 +**Target**: source.md +**Mode**: advisory +**Verdict**: Frame correct (with grounded constraint) + +## Phase 0 — Concerning Abstractions + +The design concern is: "Replace reflection-based `ReflectingExecutor` with a compile-time source generator that discovers `[MessageHandler]` attributed methods and generates `ConfigureProtocol` implementations." + +The concern exists because there are two parallel mechanisms for binding message handlers to executor route tables — one runtime-reflection-based, one compile-time-source-generated — and the design requests completing and validating the compile-time path so the reflection path can be deprecated and removed. + +**Q0.1 — Concerning abstractions:** + +| Abstraction | file:line | Would removal eliminate the concern? (YES/NO + why) | Simpler system lacking this pattern | +|---|---|---|---| +| `ReflectingExecutor` | `Reflection/ReflectingExecutor.cs:23-27` | NO — removing it is the goal of this design, not the source of the concern; removing it alone does not eliminate the need for handler binding, which must be served by either the generator or manual overrides | A system where users hand-write `ConfigureProtocol` overrides (no codegen, no reflection). This system lacks the concern but imposes boilerplate. | +| `IMessageHandler` / `IMessageHandler` | `Reflection/IMessageHandler.cs:19,44` | NO — these are the interface marker mechanism for reflection discovery; removing them does not eliminate the need for route binding, only removes one discovery strategy | Same hand-written system above. | +| `RouteBuilderExtensions.GetHandlerInfos()` (reflection scanner) | `Reflection/RouteBuilderExtensions.cs:47-78` | NO — this is the runtime reflection scanner that implements `ReflectingExecutor`'s discovery; removing it only shifts binding to another mechanism (generator or manual) | Same. | +| `MessageHandlerInfo` (reflection binding model) | `Reflection/MessageHandlerInfo.cs:16-149` | NO — this is the runtime data model for reflected handlers; removing it does not eliminate the need for handler metadata, which is now captured in `HandlerInfo` (generator model) at compile time | Same. | +| `ValueTaskTypeErasure` | `Reflection/ValueTaskTypeErasure.cs` (file exists per listing) | NO — this is a helper for runtime type erasure of `ValueTask` return values; the generator eliminates the need by emitting statically-typed delegate references | Same. | + +No abstraction in the list, if removed alone, would eliminate the concern. The concern is structural: executors must bind message-type-to-handler mappings, and the question is at what phase (compile-time vs. runtime) this binding occurs. Both reflection and source generation are strategies for the same invariant. + +**Q0.2 — Counterfactual:** For each abstraction above, the answer is NO. Removing any single abstraction shifts the implementation burden but does not eliminate the fundamental concern (handler route registration). + +**Q0.3 — Simpler-system comparison:** A system where users hand-write `ConfigureProtocol` overrides (which is already supported: `Executor.ConfigureProtocol` at `Executor.cs:216` is abstract and can be manually implemented) lacks the concern entirely. That system trades automation for boilerplate. The source generator is an optimization of that simpler system — not a new abstraction layer. + +**Conclusion:** No Candidate 0 (frame-level lift) exists. The concern is not caused by an abstraction that could be removed; it is caused by the desire to automate a fundamentally necessary step (binding handlers to routes). + +## Ownership Map + +| Layer | Location (file:line) | Enforcement | Policy carried | +|---|---|---|---| +| Abstract contract | `Executor.cs:216` — `protected abstract ProtocolBuilder ConfigureProtocol(ProtocolBuilder protocolBuilder)` | mechanical (compile-time: abstract forces override) | Every Executor subclass must declare its protocol | +| Source generator — syntax detection | `ExecutorRouteGenerator.cs:33-37` — `ForAttributeWithMetadataName` predicate filtering `MethodDeclarationSyntax` with `[MessageHandler]` | mechanical (compile-time: Roslyn pipeline) | Only methods annotated with `[MessageHandler]` are candidates | +| Source generator — semantic validation | `SemanticAnalyzer.cs:47-103` — `AnalyzeHandlerMethod()` | mechanical (compile-time: emits error diagnostics that fail build) | Handler must have valid signature: `(T, IWorkflowContext[, CT])`, non-static, valid return type | +| Source generator — class validation | `SemanticAnalyzer.cs:110-189` — `CombineHandlerMethodResults()` | mechanical (compile-time: error diagnostics) | Class must be `partial`, derive from `Executor`, not already define `ConfigureProtocol` | +| Source generator — code emission | `SourceBuilder.cs:27-127` — `Generate()` | mechanical (compile-time: emits compilable C# partial class) | Generated `ConfigureProtocol` override chains `AddHandler` / `AddHandler` calls correctly | +| Source generator — diagnostic descriptors | `DiagnosticDescriptors.cs:34-107` — 7 descriptors `MAFGENWF001-007` | mechanical (compile-time: build errors/warnings) | Invalid handler signatures, non-partial classes, non-Executor classes, static handlers all reported | +| Attribute definition — `[MessageHandler]` | `Attributes/MessageHandlerAttribute.cs:48-70` | instruction (attribute signals intent, enforced by generator) | Method is a handler; optional `Yield`/`Send` type arrays | +| Attribute definition — `[SendsMessage]` | `Attributes/SendsMessageAttribute.cs:32-49` | instruction (attribute signals intent, enforced by generator + `ProtocolBuilder`) | Executor declares it sends message type T | +| Attribute definition — `[YieldsOutput]` | `Attributes/YieldsOutputAttribute.cs:32-49` | instruction (attribute signals intent, enforced by generator + `ProtocolBuilder`) | Executor declares it yields output type T | +| Runtime route registration | `RouteBuilder.cs:51-89` — `AddHandlerInternal()` | runtime (throws `ArgumentException` for duplicate handler types) | Each message type maps to exactly one handler | +| Runtime protocol building | `ProtocolBuilder.cs:156-173` — `Build()` | runtime (constructs `ExecutorProtocol` from declared types + routes) | Send/Yield type sets + route table are frozen at build time | +| Runtime protocol validation | `ExecutorProtocol` at `Executor.cs:127-158` — `CanHandle()`, `CanOutput()` | runtime (returns bool, used for routing decisions) | Only declared types can be sent/yielded | +| Legacy reflection discovery | `ReflectingExecutor.cs:36-75` — `ConfigureProtocol()` override | runtime (reflection-based discovery of `IMessageHandler` interfaces) | DUPLICATED: Same invariant as source generator, but via runtime reflection | +| Legacy interface reflection | `RouteBuilderExtensions.cs:47-78` — `GetHandlerInfos()` | runtime (scans `TExecutor` type for `IMessageHandler<>` interfaces) | DUPLICATED: discovers handlers at runtime via interface scanning | +| Legacy `[Obsolete]` markers | `ReflectingExecutor.cs:21-22`, `IMessageHandler.cs:17-18,42-43` | advisory (compiler warning, not error) | Signals migration path but does not prevent usage | + +**Ownership duplication flag:** The handler-to-route binding invariant is carried in TWO locations: +1. Source generator pipeline (`ExecutorRouteGenerator` + `SemanticAnalyzer` + `SourceBuilder`) — compile-time, mechanical +2. Reflection pipeline (`ReflectingExecutor` + `RouteBuilderExtensions` + `MessageHandlerInfo`) — runtime, runtime + +This is the explicit design intent: the source generator replaces the reflection pipeline. The `[Obsolete]` markers signal this transition but do not enforce it (advisory level). + +## Elimination Candidates + +### Candidate 1: Complete ReflectingExecutor Removal (Runtime-to-Compile-Time Shift) + +- **Altitude**: Abstraction removal — delete entire reflection-based handler discovery +- **Target boundary**: `Reflection/` directory: `ReflectingExecutor.cs`, `IMessageHandler.cs`, `RouteBuilderExtensions.cs`, `MessageHandlerInfo.cs`, `ReflectionExtensions.cs`, `ValueTaskTypeErasure.cs` +- **Invariant carried**: Handler-to-route binding is performed exclusively at compile time via source generator +- **Deletion ledger**: + - `Reflection/ReflectingExecutor.cs` (77 lines) — entire file + - `Reflection/IMessageHandler.cs` (55 lines) — entire file + - `Reflection/RouteBuilderExtensions.cs` (79 lines) — entire file + - `Reflection/MessageHandlerInfo.cs` (149 lines) — entire file + - `Reflection/ReflectionExtensions.cs` (54 lines) — entire file + - `Reflection/ValueTaskTypeErasure.cs` (file exists per listing) — entire file + - Total: ~6 files, ~414+ lines removed +- **Addition ledger**: + - Migration documentation (not counted as code) + - Potential test updates for consumers that inherit from `ReflectingExecutor` + - No new types/methods/interfaces required — the generator already exists and handles all cases +- **Category elimination**: YES — eliminates the entire category of "runtime reflection-based handler discovery" decisions. No more `DynamicallyAccessedMembers` trimmer annotations, no more runtime type scanning, no more `MethodInfo.Invoke`. + +**Subtraction gate:** +- Dimension 1 (artifacts): ~414+ lines deleted, ~0 lines added. PASS. +- Dimension 2 (categories): Eliminates entire runtime reflection handler discovery category. PASS. + +**Gate result: PASS (both dimensions)** + +### Candidate 2: Promote `[Obsolete]` from Warning to Error + +- **Altitude**: Compile-time constraint (change `error: false` to `error: true` on `[Obsolete]` attributes) +- **Target boundary**: `ReflectingExecutor.cs:21-22`, `IMessageHandler.cs:17-18,42-43` +- **Invariant carried**: Usage of reflection-based path is a compile error, not a warning +- **Deletion ledger**: Change 3 attribute instances from `error: false` to `error: true` (or add `error: true` where not present). Note: current `ReflectingExecutor` `[Obsolete]` does not specify `error:` explicitly (defaults to `false`). `IMessageHandler` `[Obsolete]` also does not specify `error:`. +- **Addition ledger**: No new types or files. Three one-line edits. +- **Category elimination**: NO — does not eliminate a category of runtime decision; merely escalates severity + +**Subtraction gate:** +- Dimension 1 (artifacts): 0 deletions, 0 additions (modifications only). FAIL. +- Dimension 2 (categories): Does not eliminate a category. FAIL. + +**Gate result: FAIL (neither dimension)** + +## Self-Challenge + +### Candidate 1: Complete ReflectingExecutor Removal + +| # | Question | Rejection succeeds? | Grounding provided | Grounding passes? | +|---|---|---|---|---| +| Q1 | Does the target boundary exist in this codebase's topology? | NO (rejection fails) | `Reflection/` directory contains all 6 files listed; verified via `find` in codebase-snapshot.md section 1 and direct reads of `ReflectingExecutor.cs`, `IMessageHandler.cs`, `RouteBuilderExtensions.cs`, `MessageHandlerInfo.cs`, `ReflectionExtensions.cs` | YES — all files exist at stated paths | +| Q2 | Does the language/framework/runtime support the proposed invariant? | NO (rejection fails) | C# source generators via `IIncrementalGenerator` (Roslyn 4.4.0+) are fully supported; the generator already exists and is operational at `ExecutorRouteGenerator.cs:22`. The `[Generator]` attribute and `ForAttributeWithMetadataName` API are used. | YES — runtime and framework support verified | +| Q3 | Does an ADR or user-signed decision decline this migration? | NO (rejection fails) | codebase-snapshot.md section "Decision History > ADRs" states: "No ADRs found specifically addressing the source generator or Workflows architecture." The source.md design decisions (confirmed) explicitly state: "Migration: Clean break - requires direct Executor inheritance (not ReflectingExecutor)". The `[Obsolete]` markers on `ReflectingExecutor` and `IMessageHandler` are themselves user-signed decisions approving deprecation. | YES — no ADR blocks; design decisions actively support removal | +| Q4 | Does the lift create a worse defect class? | YES (rejection succeeds — partially) | Removing `ReflectingExecutor` before all downstream consumers have migrated would break binary compatibility. The `[Obsolete]` attribute with `error: false` currently allows a grace period. If external consumers (NuGet package users) still reference `ReflectingExecutor`, removal is a breaking change (assembly-level `TypeLoadException` at runtime for consumers compiled against the old API). This is a **worse defect class**: upgrading the NuGet package would cause runtime failures rather than compile-time warnings. | YES — binary compatibility concern is real for published packages | +| Q5 | Does the lift step OUTSIDE the concern's named domain? | NO (rejection fails) | The concern is "replace reflection-based `ReflectingExecutor` with compile-time source generator." Removal of the reflection path is within that domain. | YES — stays within domain | + +**Grounding audit summary:** Q4 rejection succeeds — removing `ReflectingExecutor` immediately would create a worse defect class (runtime `TypeLoadException` for unconverted consumers) compared to the current state (compile-time `[Obsolete]` warning). However, this is a **timing** concern, not a structural one. The source.md states "This type will be removed in v1.0" and "Clean break - requires direct Executor inheritance," explicitly approving future removal. + +**Decision:** +- Candidate 0 (Phase 0): No candidate exists — the concern is not frame-artificial. +- Candidate 1: Passes subtraction gate. Survives 4 of 5 rejection questions. Q4 succeeds because immediate removal creates binary compatibility risk. This makes it a **Frame-lift offered** rather than **Frame-lift required**. +- Candidate 2: Fails subtraction gate. Not evaluated further. + +## Verdict + +**Frame correct** (with grounded constraint) + +**Rationale:** + +The current framing of the problem — a source generator that replaces a reflection-based pattern — is structurally sound. The concern (handler-to-route binding) cannot be eliminated by removing an abstraction; it is an inherent requirement of the executor architecture (every executor must bind message types to handlers). + +The source generator already exists and is fully operational (`ExecutorRouteGenerator.cs:22-161`). The design-under-review is describing and validating work that has already been implemented (codebase-snapshot.md section 4: "Files listed as 'to create' — All files ALREADY EXIST"). + +One elevation candidate survives with partial rejection: complete removal of the `Reflection/` subsystem would eliminate an entire category of runtime decisions (runtime reflection handler scanning) and delete ~414+ lines with zero additions. However, Q4 rejection succeeds because immediate removal creates a worse defect class (binary incompatibility for NuGet consumers). The source.md explicitly acknowledges this with "This type will be removed in v1.0" and the `[Obsolete(error: false)]` grace period. + +Because the only surviving candidate is blocked by a grounded binary-compatibility constraint that the design itself has already addressed (via `[Obsolete]` + documented removal timeline), the frame is **correct** — the current approach of deprecate-then-remove is the right architecture for a published library with external consumers. + +**No redesign required.** The elevation candidate (full `Reflection/` removal) is explicitly planned for a future version and correctly deferred by the `[Obsolete]` migration pattern. diff --git a/.designs/1/implementation-plan/implementation_plan_outline.md b/.designs/1/implementation-plan/implementation_plan_outline.md new file mode 100644 index 0000000000..cd877c5340 --- /dev/null +++ b/.designs/1/implementation-plan/implementation_plan_outline.md @@ -0,0 +1,434 @@ +# Implementation Plan Outline: Roslyn Source Generator for Workflow Executor Routes + +**Date**: 2026-05-12 +**Source**: `.designs/1/design-doc.md` +**Purpose**: Self-contained phase extraction guide for creating focused + IMPLREADME_PHASE{X}.md files +**Usage**: Extract any single phase section below and provide it to an LLM + with: "Run `/design-plan-impl .designs/1/implementation-plan/implementation_plan_outline.md` to extract Phase X" + +--- + +## How To Use This Document + +Each phase below is a **self-contained extraction unit**. Workflow: + +1. `/design-plan-impl .designs/1/design-doc.md` — produces this outline (Mode A) +2. `/clear` +3. `/peer-review .designs/1/implementation-plan/implementation_plan_outline.md` — validates outline against codebase +4. `/clear` +5. `/design-plan-impl .designs/1/implementation-plan/implementation_plan_outline.md` — extract Phase X into IMPLREADME (Mode B) +6. `/clear` +7. Use the phase's **Recommended Skill** to implement (e.g., `/ultra-implement`, `/terraform-fix`, or `/gherkin-design`) +8. Repeat steps 5-7 for each phase + +**Phase dependency chain:** +``` +Phase 1 (High-Impact Gap Closure) + │ + ├──→ Phase 2 (Test Coverage) + │ +Phase 3 (Documentation) ← independent, can run in parallel with Phase 1 +``` + +Phase 1 and Phase 3 have no dependency on each other and can run in parallel. +Phase 2 depends on Phase 1 because new diagnostics introduced in Phase 1 must be tested in Phase 2. + +## Deployment Coverage + +| Target | Scripts/Config | Covered By Phase | Gap? | +|--------|---------------|-----------------|------| +| CI Build (Ubuntu net10.0, Windows net9.0/net472, Ubuntu net8.0) | `.github/workflows/dotnet-build-and-test.yml` | All phases (auto-triggered on `dotnet/**` changes) | No | +| Unit Tests (net10.0 Ubuntu, net472 Windows) | `.github/workflows/dotnet-build-and-test.yml` `dotnet-test` job | Phase 1 (new diagnostic code), Phase 2 (new tests) | No | +| Code Coverage (80% threshold) | `dotnet/eng/scripts/dotnet-check-coverage.ps1` | Phase 2 | No | +| NuGet Packaging | `dotnet/nuget/nuget-package.props` (VersionPrefix: 1.5.0) | No code changes needed | No | +| Code Formatting | `.github/workflows/dotnet-format.yml` | All phases (auto-enforced) | No | +| Release/Publish | Manual process (no automated pipeline) | N/A — library code only | No | + +## Success Metrics + +| Metric | Source | Phase | +|--------|--------|-------| +| Duplicate input type handlers caught at compile time | design-doc.md Phase 1 AC, Gap 4 | Phase 1 | +| MAFGENWF006 upgraded to Warning for Executor subclasses | design-doc.md Phase 1 AC, Gap 9 | Phase 1 | +| 4+ parameter handlers produce diagnostic | design-doc.md Phase 1 AC, Gap 2 | Phase 1 | +| Non-CancellationToken third parameter produces warning | design-doc.md Phase 1 AC, Gap 3 | Phase 1 | +| Multi-file partial class test passes | design-doc.md Phase 2 AC, Gap 13 | Phase 2 (already exists; verify) | +| 3+ baseline comparison tests verify ConfigureProtocol output | design-doc.md Phase 2 AC, Gap 7 | Phase 2 | +| Migration guide document exists | design-doc.md Phase 3 AC, Gap 11 | Phase 3 | +| Auto-yield/auto-send documented in API doc comments | design-doc.md Phase 3 AC, Gap 6 | Phase 3 | + +--- + +## Phase 1: High-Impact Gap Closure — New Diagnostics + +### Objective +Add four new compile-time diagnostics to the Roslyn source generator to catch handler configuration errors that currently surface only at runtime or are silently ignored. + +### Prerequisites +None + +### Recommended Skill +`/ultra-implement` — Pure C# library code changes to existing analyzer/generator infrastructure. No infrastructure, no UI, no design exploration needed. + +### Design References +| Document | Section | Lines | What It Specifies | +|----------|---------|-------|-------------------| +| design-doc.md | Implementation Plan, Phase 1 | L188-199 | Deliverables: MAFGENWF008, upgraded MAFGENWF006, 4+ param diagnostic, non-CT third param diagnostic | +| design-doc.md | Phase 1 acceptance criteria | L202-205 | Four pass/fail acceptance criteria | +| design-doc.md | Cross-Perspective Conflicts | L137-138 | Gap 4 runtime crash: RouteBuilder.AddHandler throws ArgumentException for duplicate input types | +| design-doc.md | Cross-Perspective Conflicts | L139 | Gap 9: Info severity masks user intent on Executor subclasses | +| design-doc.md | Risk Registry | L162 | Risk: new diagnostics break consumer CI builds (TreatWarningsAsErrors) | +| six_sigma_gaps.md | Gap 2 | Full section | 4+ parameter escape path | +| six_sigma_gaps.md | Gap 3 | Full section | Non-CancellationToken third parameter escape path | +| six_sigma_gaps.md | Gap 4 | Full section | Duplicate handler input types — runtime crash | +| six_sigma_gaps.md | Gap 9 | Full section | Info-level severity on Executor subclasses | +| api.md | Diagnostics | Full section | Current diagnostic inventory and patterns | + +### Current State (files to read for context) +| File | Lines | What's There | +|------|-------|-------------| +| `dotnet/src/Microsoft.Agents.AI.Workflows.Generators/Diagnostics/DiagnosticDescriptors.cs` | L34-106 | 7 existing diagnostics (MAFGENWF001-007). MAFGENWF006 at L89-95 is Info severity. MAFGENWF007 at L100-106 is the last defined diagnostic. | +| `dotnet/src/Microsoft.Agents.AI.Workflows.Generators/Analysis/SemanticAnalyzer.cs` | L462-533 | `AnalyzeHandler()` method: validates static (L470-473), param count >= 2 (L477-480), IWorkflowContext second param (L484-488), CancellationToken detection at position 3 (L492-493), return type (L496-501). No validation for 4+ params or non-CT third param. | +| `dotnet/src/Microsoft.Agents.AI.Workflows.Generators/Analysis/SemanticAnalyzer.cs` | L110-189 | `CombineHandlerMethodResults()`: groups methods by class key (L100), checks Executor derivation (L133-141), partial modifier (L143-150), manual ConfigureProtocol (L152-159). Collects valid handlers at L162-165. No duplicate input type detection. | +| `dotnet/src/Microsoft.Agents.AI.Workflows.Generators/Models/HandlerInfo.cs` | L34-47 | `HandlerInfo` record: stores `InputTypeName` as fully-qualified string (set at SemanticAnalyzer.cs L506 using `SymbolDisplayFormat.FullyQualifiedFormat`). No method location stored. | +| `dotnet/src/Microsoft.Agents.AI.Workflows.Generators/Models/ExecutorInfo.cs` | L18-27 | `ExecutorInfo` record: `Handlers` is `ImmutableEquatableArray`. No base type classification stored. | +| `dotnet/src/Microsoft.Agents.AI.Workflows.Generators/ExecutorRouteGenerator.cs` | L22-161 | Three pipelines: MessageHandler methods (L31-37), SendsMessage classes (L39-45), YieldsOutput classes (L47-53). CombineAllResults at L94-126. | +| `dotnet/src/Microsoft.Agents.AI.Workflows/Executor.cs` | L382-397 | `Executor` — overrides `ConfigureProtocol`, implements `IMessageHandler` | +| `dotnet/src/Microsoft.Agents.AI.Workflows/Executor.cs` | L407-423 | `Executor` — overrides `ConfigureProtocol`, implements `IMessageHandler` | +| `dotnet/Directory.Build.props` | Full file | `TreatWarningsAsErrors=true` — all new Warning-level diagnostics will break consumer builds | + +### Required Changes + +**File 1: `dotnet/src/Microsoft.Agents.AI.Workflows.Generators/Diagnostics/DiagnosticDescriptors.cs`** +- **After L106**: Add three new DiagnosticDescriptor fields: + - `MAFGENWF008`: "Duplicate input type handler" — Severity: **Warning** (not Error, to avoid immediate CI breaks for consumers with TreatWarningsAsErrors). Message: "Class '{0}' has multiple [MessageHandler] methods for input type '{1}'. Only one handler per input type is allowed; at runtime, RouteBuilder.AddHandler throws ArgumentException for duplicates." + - `MAFGENWF009`: "Handler has too many parameters" — Severity: **Error**. Message: "Method '{0}' marked with [MessageHandler] has {1} parameters; maximum 3 allowed (message, IWorkflowContext, optional CancellationToken)" + - `MAFGENWF010`: "Handler third parameter must be CancellationToken" — Severity: **Warning**. Message: "Method '{0}' has third parameter of type '{1}'; expected CancellationToken or omit the third parameter" +- **L89-95**: Add a second DiagnosticDescriptor for MAFGENWF006 with Warning severity (e.g., `ConfigureProtocolAlreadyDefinedWarning`), or parameterize severity. The existing Info-level descriptor remains for non-Executor base classes. + +**File 2: `dotnet/src/Microsoft.Agents.AI.Workflows.Generators/Analysis/SemanticAnalyzer.cs`** +- **After L480 (param count check)**: Add validation for `Parameters.Length > 3` — report MAFGENWF009 and return null. +- **After L493 (CancellationToken detection)**: Add validation: if `Parameters.Length >= 3` and third param is NOT CancellationToken, report MAFGENWF010. Do NOT return null (this is a warning; the handler can still be used with 2 effective params). +- **After L165 (handler collection in CombineHandlerMethodResults)**: Add duplicate input type detection. Group collected handlers by `InputTypeName`; for each group with count > 1, emit MAFGENWF008 diagnostic. Use method locations from `MethodAnalysisResult` (already available in the input `ImmutableArray`). **DO NOT modify `HandlerInfo`** — it is an immutable record used for incremental caching; adding location fields would change its value equality semantics and cause unnecessary re-generation. +- **L152-159 (MAFGENWF006 reporting)**: Add base type detection. Check if the class's base type's `OriginalDefinition` matches `Executor` or `Executor` (can check `OriginalDefinition.ToDisplayString()` or generic arity). If so, use Warning-severity descriptor; otherwise keep Info. + +**Validation order in `AnalyzeHandler()` (IMPORTANT — must follow this sequence):** +1. Static check → MAFGENWF007 (existing, L470-473) +2. `Parameters.Length < 2` → MAFGENWF005 (existing, L477-480) +3. `Parameters.Length > 3` → MAFGENWF009 (**new**, insert after L480) +4. IWorkflowContext at position 1 → MAFGENWF001 (existing, L484-488) +5. CancellationToken detection at position 2 (existing, L492-493) +6. Non-CancellationToken third param → MAFGENWF010 (**new**, insert after L493) + +### Acceptance Criteria +```bash +# 1. MAFGENWF008 fires for duplicate input types +cd dotnet && dotnet test tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/ --filter "DuplicateInputType" 2>&1 | grep -c "PASS" +# Expected: >= 1 + +# 2. MAFGENWF006 severity is Warning for Executor base +cd dotnet && dotnet test tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/ --filter "ConfigureProtocolWarning" 2>&1 | grep -c "PASS" +# Expected: >= 1 + +# 3. Handler with 4+ parameters produces MAFGENWF009 +cd dotnet && dotnet test tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/ --filter "TooManyParameters" 2>&1 | grep -c "PASS" +# Expected: >= 1 + +# 4. Non-CancellationToken third parameter produces MAFGENWF010 +cd dotnet && dotnet test tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/ --filter "NonCancellationTokenThirdParam" 2>&1 | grep -c "PASS" +# Expected: >= 1 + +# 5. All existing tests still pass (no regressions) +cd dotnet && dotnet test tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/ 2>&1 | tail -5 +# Expected: "Passed!" with 0 failures + +# 6. Solution builds without errors +cd dotnet && dotnet build src/Microsoft.Agents.AI.Workflows.Generators/ 2>&1 | grep -c "Build succeeded" +# Expected: 1 +``` + +### Gotchas (from codebase investigation) +- `TreatWarningsAsErrors=true` in `dotnet/Directory.Build.props` means any new Warning-level diagnostic (MAFGENWF008, MAFGENWF010, upgraded MAFGENWF006) will immediately fail builds for consumers. The design-doc risk registry addresses this: ship as Warning initially, upgrade to Error in the next major version. **Rollback strategy**: if MAFGENWF008/010 cause excessive consumer CI failures, they can be downgraded to Info severity in a patch release with a one-line change per diagnostic descriptor in DiagnosticDescriptors.cs — no behavioral change, only severity reduction. +- `HandlerInfo` is an immutable record used for incremental caching (Roslyn IIncrementalGenerator pipeline). DO NOT add location fields to it — this would change its value equality semantics and cause unnecessary re-generation. Instead, use method locations from `MethodAnalysisResult` (already available in `CombineHandlerMethodResults` input) for duplicate diagnostics. +- The MAFGENWF006 severity upgrade requires knowing the base type at the `CombineHandlerMethodResults` level. The `MethodAnalysisResult` contains `HasManualConfigureProtocol` but not the base type classification. You'll need to add a `BaseIsGenericExecutor` boolean (or similar) to `MethodAnalysisResult` during `AnalyzeHandlerMethod`, by checking if the class's base type's `OriginalDefinition` is `Executor` or `Executor`. The `DerivesFromExecutor()` method (lines 385-400) already walks the base type chain — extend it. +- The existing CancellationToken detection at L492-493 checks `Parameters.Length >= 3` then checks the type. The new 4+ parameter check (MAFGENWF009) should run BEFORE the CancellationToken check. Order: (1) check < 2 params → MAFGENWF005, (2) check > 3 params → MAFGENWF009, (3) check IWorkflowContext at position 1, (4) check CancellationToken at position 2 and non-CT third param → MAFGENWF010. +- The non-CT third parameter diagnostic (MAFGENWF010) should NOT prevent handler registration. The handler is still valid with its first two parameters. Only emit the warning and continue processing. + +--- + +## Phase 2: Test Coverage Improvements + +### Objective +Add baseline comparison tests that verify the source generator produces the expected ConfigureProtocol code patterns for common executor configurations, closing the behavioral equivalence gap. (Note: the multi-file partial class test deliverable from design-doc Phase 2 is **pre-satisfied** — tests already exist at ExecutorRouteGeneratorTests.cs L541-704. This phase focuses on the baseline comparison tests only.) + +### Prerequisites +Phase 1 (new diagnostics should be testable) + +### Recommended Skill +`/ultra-implement` — C# test code following established xUnit.v3 + FluentAssertions patterns. No infrastructure or design exploration needed. + +### Design References +| Document | Section | Lines | What It Specifies | +|----------|---------|-------|-------------------| +| design-doc.md | Implementation Plan, Phase 2 | L208-216 | Deliverables: multi-file partial test, behavioral equivalence tests | +| design-doc.md | Phase 2 acceptance criteria | L218-220 | Multi-file test passes; 3+ baseline comparison tests | +| design-doc.md | Six-Sigma Caveats | L174 | Gap 7: structural tests insufficient for migration validation. Practical ceiling: full runtime equivalence needs test infrastructure beyond current capabilities. | +| design-doc.md | Six-Sigma Caveats | L180 | Gap 13: multi-file partial class test | +| six_sigma_gaps.md | Gap 7 | Full section | Test type mismatch — no behavioral equivalence tests | +| six_sigma_gaps.md | Gap 13 | Full section | Multi-file partial class edge case | + +### Current State (files to read for context) +| File | Lines | What's There | +|------|-------|-------------| +| `dotnet/tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/ExecutorRouteGeneratorTests.cs` | L541-704 | **Multi-file partial class tests already exist**: `PartialClass_SplitAcrossFiles_GeneratesCorrectly()` (L543-604), `PartialClass_HandlersInBothFiles_GeneratesAllHandlers()` (L606-651), `PartialClass_SendsYieldsInBothFiles_GeneratesAllOverrides()` (L653-704). These satisfy the multi-file test requirement. | +| `dotnet/tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/ExecutorRouteGeneratorTests.cs` | L14-135 | Single handler test patterns (void, ValueTask, ValueTask, with CancellationToken). These demonstrate the baseline comparison approach: call `RunGenerator()`, assert generated tree contains expected patterns. | +| `dotnet/tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/GeneratorTestHelper.cs` | L20-145 | Test helper API: `RunGenerator(params string[] sources)` (L31-54), `AssertGeneratesSource()` (L59-67), `AssertProducesDiagnostic()` (L81-88). Returns `GeneratorRunResult` record with `RunResult`, `OutputCompilation`, `Diagnostics`. | +| `dotnet/tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/SyntaxTreeFluentExtensions.cs` | Full file | Fluent assertion methods: `AddHandler()` (L20-72), `RegisterSentMessageType()` (L85-103), `RegisterYieldedOutputType()` (L116-134), `HaveHierarchy()` (L169-192). | +| `dotnet/src/Microsoft.Agents.AI.Workflows.Generators/Generation/SourceBuilder.cs` | L71-110 | Generated ConfigureProtocol pattern: `protocolBuilder.SendsMessage().YieldsOutput().ConfigureRoutes(ConfigureRoutes)` with nested `void ConfigureRoutes(RouteBuilder routeBuilder)` local function. Base call when `BaseHasConfigureProtocol` is true. | +| `dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/ReflectionSmokeTest.cs` | L69-128 | Reflection-based test using `ReflectingExecutor` and `IMessageHandler`. Uses `MessageRouter` and `.RouteMessageAsync()`. Demonstrates the old pattern for comparison. | +| `dotnet/tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/Microsoft.Agents.AI.Workflows.Generators.UnitTests.csproj` | Full file | Test framework: net10.0, xUnit.v3, FluentAssertions, Microsoft.CodeAnalysis.CSharp. References both Generators and Workflows projects. | + +### Required Changes + +**File 1: `dotnet/tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/ExecutorRouteGeneratorTests.cs`** +- **After existing test sections (~end of file)**: Add a new test region/section for baseline comparison tests. Add at least 3 tests: + + 1. **Simple single-handler baseline**: Define an executor with one `void Handler(string, IWorkflowContext)` method. Assert generated output contains exact expected ConfigureProtocol body with `routeBuilder.AddHandler(this.Handler)`. + + 2. **Multi-handler with mixed signatures baseline**: Define an executor with a void handler and a `ValueTask` handler. Assert generated output contains both `.AddHandler(this.Method1)` and `.AddHandler(this.Method2)` registrations. + + 3. **Full protocol baseline (handlers + class-level SendsMessage + YieldsOutput)**: Define an executor with `[SendsMessage(typeof(Foo))]`, `[YieldsOutput(typeof(Bar))]`, and a handler. Assert generated output contains `.SendsMessage<...>()`, `.YieldsOutput<...>()`, and `.ConfigureRoutes(ConfigureRoutes)` with the handler registration. + +- **Additionally**: Add tests for Phase 1's new diagnostics (MAFGENWF008, MAFGENWF009, MAFGENWF010, upgraded MAFGENWF006) following the existing diagnostic test pattern (L708-832). Each diagnostic test: create source code that triggers the condition, call `GeneratorTestHelper.AssertProducesDiagnostic(source, "MAFGENWF00X")`. + +### Acceptance Criteria +```bash +# 1. Multi-file partial class test passes (already exists) +cd dotnet && dotnet test tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/ --filter "PartialClass_SplitAcrossFiles" 2>&1 | grep -c "Passed" +# Expected: 1 + +# 2. At least 3 baseline comparison tests exist and pass +cd dotnet && dotnet test tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/ --filter "Baseline" 2>&1 | grep "Passed" +# Expected: >= 3 test names containing "Baseline" shown as passed + +# 3. New diagnostic tests pass +cd dotnet && dotnet test tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/ --filter "DuplicateInputType|TooManyParameters|NonCancellationToken|ConfigureProtocolWarning" 2>&1 | grep "Passed" +# Expected: >= 4 tests passed + +# 4. Full test suite still passes +cd dotnet && dotnet test tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/ 2>&1 | tail -5 +# Expected: "Passed!" with 0 failures + +# 5. Baseline tests verify full generated output (not just substring) +grep -c "GetText()" dotnet/tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/ExecutorRouteGeneratorTests.cs +# Expected: >= 3 (baseline tests read full generated text) +``` + +### Gotchas (from codebase investigation) +- Multi-file partial class tests already exist at lines 541-704. The design-doc Phase 2 AC says "Test passes with handler split across two source files" — this is already satisfied. The new work is the baseline comparison tests. +- `GeneratorTestHelper.RunGenerator()` returns `GeneratorRunResult` with `RunResult.GeneratedTrees`. Access generated source via `GeneratedTrees[0].GetText().ToString()`. Existing tests use `SyntaxTreeFluentExtensions` for targeted assertions; baseline tests should use `GetText()` for full-text comparison to verify exact output structure. +- The generated output format is: `// ` header, `#nullable enable`, namespace, partial class, `ConfigureProtocol` override. See `SourceBuilder.cs:27-127` for the exact template. Baseline expected strings must match this format exactly, including whitespace. +- Tests run on net10.0 only. The generator targets netstandard2.0 but tests compile with net10.0 Roslyn. No cross-TFM test coverage for the generator itself. +- `SyntaxTreeFluentExtensions` assertions use `Contains()` on the generated text. Baseline tests should do full-text equality (trimmed) to catch regressions in formatting, ordering, or missing elements. +- The test project references the Workflows assembly directly (`typeof(Executor).Assembly` is loaded in GeneratorTestHelper.cs), so test source strings can reference actual types from the Workflows package. + +--- + +## Phase 3: Documentation and Migration Guide + +### Objective +Create a migration guide for users transitioning from `ReflectingExecutor` to the `[MessageHandler]` attribute pattern, and document runtime dependencies and plan-vs-implementation deviations. + +### Prerequisites +None (independent of Phase 1 and Phase 2) + +### Recommended Skill +`manual` — Documentation-only deliverables: markdown file creation and XML doc comment additions. No code logic changes. + +### Design References +| Document | Section | Lines | What It Specifies | +|----------|---------|-------|-------------------| +| design-doc.md | Implementation Plan, Phase 3 | L222-233 | Deliverables: migration guide, auto-yield/send docs, deviation docs, method-level scope docs | +| design-doc.md | Phase 3 acceptance criteria | L235-237 | Migration guide exists; auto-yield/send documented | +| design-doc.md | Decisions Made | L144-149 | Deliberate deviations: YieldsOutput name, ConfigureProtocol API, Roslyn 4.4.0, MAFGENWF IDs | +| design-doc.md | Cross-Dimension Trade-offs | L128-131 | Naming and Roslyn version trade-offs | +| design-doc.md | Risk Registry | L158 | AutoYieldOutput=false breaks return-type yield registration | +| design-doc.md | Six-Sigma Caveats | L171 | Gap 6: auto-yield runtime dependency | +| design-doc.md | Six-Sigma Caveats | L179 | Gap 12: method-level attribute scope | +| six_sigma_gaps.md | Gap 1 | Full section | Version drift documentation | +| six_sigma_gaps.md | Gap 5 | Full section | Naming clarification | +| six_sigma_gaps.md | Gap 6 | Full section | Auto-yield unstated assumption | +| six_sigma_gaps.md | Gap 8 | Full section | Sibling vulnerability — auto-send delegation | +| six_sigma_gaps.md | Gap 11 | Full section | Observability gap — no migration guide | +| six_sigma_gaps.md | Gap 12 | Full section | Method-level attribute scope | + +### Current State (files to read for context) +| File | Lines | What's There | +|------|-------|-------------| +| `dotnet/src/Microsoft.Agents.AI.Workflows/Reflection/ReflectingExecutor.cs` | Full file (77 lines) | `ReflectingExecutor` — already marked `[Obsolete]` (L21-22). Uses reflection via `typeof(TExecutor).GetHandlerInfos()` (L43) to discover handlers. Overrides `ConfigureProtocol(ProtocolBuilder)`. Shows the pattern users must migrate away from. | +| `dotnet/src/Microsoft.Agents.AI.Workflows/Reflection/IMessageHandler.cs` | Full file (56 lines) | `IMessageHandler` (L19-30) and `IMessageHandler` (L44-55) — both `[Obsolete]` (L17-18, L42-43). Signatures: `ValueTask HandleAsync(TMessage, IWorkflowContext, CancellationToken)` and `ValueTask HandleAsync(...)`. | +| `dotnet/src/Microsoft.Agents.AI.Workflows/Attributes/MessageHandlerAttribute.cs` | Full file (71 lines) | Has XML doc comments (L7-28) with example usage (L30-46). Documents `Yield` (L51-59) and `Send` (L61-69) properties. | +| `dotnet/src/Microsoft.Agents.AI.Workflows/Attributes/SendsMessageAttribute.cs` | Full file (50 lines) | Has XML doc comments (L8-20) with example (L23-30). `AttributeTargets.Class | Method`. | +| `dotnet/src/Microsoft.Agents.AI.Workflows/Attributes/YieldsOutputAttribute.cs` | Full file (50 lines) | Has XML doc comments (L8-20) with example (L23-30). Named `YieldsOutputAttribute` not `YieldsMessageAttribute` per plan. | +| `dotnet/src/Microsoft.Agents.AI.Workflows/ExecutorOptions.cs` | Full file (27 lines) | `AutoSendMessageHandlerResultObject` (L20, default true), `AutoYieldOutputHandlerResultObject` (L25, default true). Minimal doc comments — needs expansion. | +| `dotnet/src/Microsoft.Agents.AI.Workflows/ProtocolBuilder.cs` | L156-172 | `Build(ExecutorOptions)` method: if `AutoSendMessageHandlerResultObject`, unions `router.DefaultOutputTypes` into send types (L161-163). If `AutoYieldOutputHandlerResultObject`, unions into yield types (L167-169). This is the auto-yield/send runtime logic. | +| `dotnet/src/Microsoft.Agents.AI.Workflows/Executor.cs` | L382-423 | `Executor` and `Executor` — specialized base classes. `ChatProtocolExecutor` (separate file) disables `AutoSendMessageHandlerResultObject`. | +| `dotnet/src/Shared/Workflows/Execution/README.md` | Full file (12 lines) | Minimal existing doc — just a header. | +| No file | N/A | No MIGRATION.md, CHANGELOG.md, or migration guide exists anywhere in the Workflows package. | + +### Required Changes + +**File 1 (NEW): `dotnet/src/Microsoft.Agents.AI.Workflows/MIGRATION.md`** +Create a migration guide covering: +1. **Overview**: Why migrate (reflection overhead, compile-time safety, better diagnostics) +2. **Step-by-step conversion**: + - Change base class from `ReflectingExecutor` to `Executor` (or `Executor` / `Executor`) + - Add `partial` modifier to class declaration + - Remove `IMessageHandler` interface implementations + - Add `[MessageHandler]` attribute to handler methods + - Preserve `[SendsMessage]` and `[YieldsOutput]` class-level attributes (no change needed) + - Remove manual `ConfigureProtocol` override (generated code handles it) +3. **Before/after code example**: Show a complete `ReflectingExecutor` converted to `[MessageHandler]` +4. **Auto-yield/auto-send behavior**: Explain `ExecutorOptions.AutoYieldOutputHandlerResultObject` and `AutoSendMessageHandlerResultObject` defaults (both true). Explain that when these are true, handler return values are automatically registered as yield/send types. Explain how to opt out. +5. **Naming differences from original plan**: YieldsOutputAttribute (not YieldsMessageAttribute), ConfigureProtocol (not ConfigureRoutes), MAFGENWF IDs (not WFGEN) +6. **Known limitations**: Method-level `[SendsMessage]`/`[YieldsOutput]` on non-handler methods are silently ignored. Partial class is required. + +**File 2: `dotnet/src/Microsoft.Agents.AI.Workflows/ExecutorOptions.cs`** +- **L18-25**: Expand XML doc comments for `AutoSendMessageHandlerResultObject` and `AutoYieldOutputHandlerResultObject`. Add remarks explaining: + - Default behavior (true): handler return types auto-registered + - Runtime dependency: `ProtocolBuilder.Build()` unions `router.DefaultOutputTypes` when true + - Impact on migration: users relying on explicit type registration may get unexpected auto-registration + - Cross-reference to ProtocolBuilder.Build() for implementation details + +**File 3: `dotnet/src/Microsoft.Agents.AI.Workflows/Attributes/YieldsOutputAttribute.cs`** +- **L8-20**: Add a `` section noting that this attribute was named `YieldsOutputAttribute` (not `YieldsMessageAttribute` as in the original design plan) to align with the `YieldsOutput()` fluent API in `ProtocolBuilder`. + +**File 4: `dotnet/src/Microsoft.Agents.AI.Workflows/Attributes/SendsMessageAttribute.cs`** +- **L8-20**: Add a `` section noting that method-level usage on non-`[MessageHandler]` methods has no effect. Only class-level declarations and method-level on `[MessageHandler]` methods participate in protocol registration. + +**File 5: `dotnet/src/Microsoft.Agents.AI.Workflows/Attributes/MessageHandlerAttribute.cs`** +- **L7-28**: Add a `` section documenting the runtime dependency on `ExecutorOptions.AutoYieldOutputHandlerResultObject` and `AutoSendMessageHandlerResultObject`. When these are true (default), handler return types are automatically added to yield/send type sets beyond any explicit `Yield`/`Send` property values. + +### Acceptance Criteria +```bash +# 1. Migration guide exists +test -f dotnet/src/Microsoft.Agents.AI.Workflows/MIGRATION.md && echo "EXISTS" || echo "MISSING" +# Expected: EXISTS + +# 2. Migration guide contains step-by-step instructions +grep -c "Step" dotnet/src/Microsoft.Agents.AI.Workflows/MIGRATION.md +# Expected: >= 4 (at least 4 numbered steps) + +# 3. Migration guide contains before/after example +grep -c "ReflectingExecutor" dotnet/src/Microsoft.Agents.AI.Workflows/MIGRATION.md +# Expected: >= 1 + +# 4. Auto-yield documented in ExecutorOptions +grep -c "AutoYield" dotnet/src/Microsoft.Agents.AI.Workflows/ExecutorOptions.cs +# Expected: >= 3 (property + expanded doc comments) + +# 5. Auto-send documented in ExecutorOptions +grep -c "AutoSend" dotnet/src/Microsoft.Agents.AI.Workflows/ExecutorOptions.cs +# Expected: >= 3 + +# 6. YieldsOutputAttribute has naming clarification +grep -c "YieldsMessageAttribute\|original.*plan\|fluent API" dotnet/src/Microsoft.Agents.AI.Workflows/Attributes/YieldsOutputAttribute.cs +# Expected: >= 1 + +# 7. Method-level scope documented +grep -c "non-handler\|silently ignored\|no effect" dotnet/src/Microsoft.Agents.AI.Workflows/Attributes/SendsMessageAttribute.cs +# Expected: >= 1 +``` + +### Gotchas (from codebase investigation) +- `ChatProtocolExecutor` (separate file) sets `AutoSendMessageHandlerResultObject = false`. The migration guide should mention that specialized executors may override these defaults, and users should check their base class. +- `RequestInfoExecutor` also sets `AutoSendMessageHandlerResultObject = false`. Document that not all executor bases have the same auto-send/yield defaults. +- The existing XML doc comments in attributes are well-written with examples (L30-46 in MessageHandlerAttribute.cs). New `` additions should follow the same style. +- The `ReflectingExecutor` obsolete message (L21-22) says "This type will be removed in a future version" — the migration guide should reference this timeline. +- There are no existing MIGRATION.md files in the repo for reference. The documentation style in the repo is minimal (12-line READMEs). The migration guide should be thorough but concise. +- Plan-vs-implementation deviations are already documented in the design-doc (lines 126-131, 144-149). The migration guide should reference these but frame them from a user perspective ("you may see `YieldsOutputAttribute` instead of `YieldsMessageAttribute`") rather than an internal design perspective. + +--- + +## Rootcause Peer Review + +**Reviewer**: design-v7 (Designer) +**Date**: 2026-05-12 +**Document Reviewed**: `.designs/1/implementation-plan/implementation_plan_outline.md` + +### Review Methodology + +This review examines whether the implementation plan addresses root causes (not symptoms) of the 13 identified gaps, whether the proposed changes could introduce new failure modes, and whether the causal chains from gap → fix → verification are complete. + +--- + +### CRITICAL + +No critical issues found. The three-phase structure correctly sequences dependent work (Phase 1 diagnostics → Phase 2 tests → Phase 3 docs), and each phase targets root causes rather than symptoms. + +--- + +### HIGH + +#### RH1: Phase 1 Diagnostic Insertion Points Assume Stable Line Numbers + +The Required Changes for File 2 (SemanticAnalyzer.cs) specify exact line numbers ("After L480", "After L493", "After L165"). These line numbers are snapshot values from the codebase-snapshot taken at 2026-05-12T03:32Z. If any PR merges to main before Phase 1 implementation begins that modifies SemanticAnalyzer.cs, these line numbers will shift and an implementer following the plan literally could insert code at wrong locations. + +**Root cause**: The plan uses absolute line references instead of structural anchors. + +**Recommendation**: The plan already includes structural descriptions alongside line numbers (e.g., "param count check", "CancellationToken detection", "handler collection in CombineHandlerMethodResults"). These structural anchors are sufficient — implementers should locate the described code pattern, not count to line N. Add a note: "Line numbers are from snapshot; locate by described code pattern if lines have shifted." + +#### RH2: Duplicate Input Type Detection Scope May Be Too Narrow + +The plan specifies duplicate detection in `CombineHandlerMethodResults` by grouping handlers by `InputTypeName`. This catches duplicates within a single class. However, partial classes split across files (already tested at L541-704) merge their handlers in `CombineHandlerMethodResults`. The plan doesn't explicitly state whether duplicate detection should run before or after partial-class handler merging. If duplicates are checked per-file before merging, a handler defined in File A and duplicated in File B would be missed. + +**Root cause**: The detection scope (per-class vs per-file) isn't specified. + +**Recommendation**: Confirm that duplicate detection runs after `CombineHandlerMethodResults` merges handlers from all partial class files, not within individual `AnalyzeHandler` calls. The current plan text ("Group collected handlers by InputTypeName") implies post-merge, but an explicit note would prevent misimplementation. + +--- + +### MEDIUM + +#### RM1: MAFGENWF006 Severity Upgrade Requires Base Type Classification Not Currently in Pipeline + +The plan correctly identifies (Gotcha line 154) that `MethodAnalysisResult` lacks base type classification and suggests extending `DerivesFromExecutor()`. However, the plan's Required Changes section for File 2 says to check `OriginalDefinition` at the `CombineHandlerMethodResults` level. The root cause issue is that `DerivesFromExecutor()` (lines 385-400) walks the base type chain but returns a boolean — it doesn't distinguish between `Executor`, `Executor`, and `Executor`. The implementer needs to either modify `DerivesFromExecutor()` to return a classification enum, or add a separate check in `CombineHandlerMethodResults`. + +**Recommendation**: Specify which approach to take. Adding `BaseIsGenericExecutor` to `MethodAnalysisResult` (as mentioned in Gotchas) is the cleaner path — it keeps the classification close to where the semantic model is available. + +#### RM2: Phase 2 Baseline Tests Need Exact Expected Output Specification + +The plan says baseline tests should use `GetText()` for "full-text comparison" but doesn't provide the expected output strings. The expected output depends on the exact formatting in `SourceBuilder.cs:27-127`. If the implementer constructs expected strings by reading SourceBuilder and hand-crafting output, any misunderstanding of the template (e.g., nullable annotations, using directives, spacing) will cause false test failures. + +**Recommendation**: The safest approach is to first run the generator with a known input and capture the actual output as the baseline, then write the test to assert against that captured output. State this approach explicitly. + +--- + +### LOW + +#### RL1: Phase 3 Migration Guide File Location + +The plan creates `dotnet/src/Microsoft.Agents.AI.Workflows/MIGRATION.md` inside the source project directory. NuGet package consumers won't see this file unless it's included in the package content. The `.csproj` would need a `` or `` item to include it. Alternative: place it in `dotnet/docs/` which is more conventional for developer docs. + +#### RL2: No Verification That Phase 1 Diagnostics Have Unique Message Templates + +All existing diagnostics (MAFGENWF001-007) have distinct message templates with unique format strings. The plan adds MAFGENWF008-010 with specific messages. No acceptance criterion verifies that message templates don't overlap or confuse users. Minor risk since the messages are explicit in the plan. + +--- + +### Causal Chain Verification + +| Gap | Root Cause | Plan Fix | Fix Addresses Root Cause? | Verification | +|-----|-----------|----------|--------------------------|--------------| +| Gap 2 (4+ params) | No upper bound check in `AnalyzeHandler` | Add `> 3` check → MAFGENWF009 | YES — adds the missing validation | AC: test filter "TooManyParameters" | +| Gap 3 (non-CT 3rd param) | Silent `hasCancellationToken = false` | Add type check → MAFGENWF010 | YES — replaces silent fallthrough with diagnostic | AC: test filter "NonCancellationTokenThirdParam" | +| Gap 4 (duplicate inputs) | No uniqueness check in `CombineHandlerMethodResults` | HashSet grouping → MAFGENWF008 | YES — catches at source (handler collection) | AC: test filter "DuplicateInputType" | +| Gap 6 (auto-yield assumption) | Undocumented runtime dependency | Document in ExecutorOptions XML | YES — makes implicit explicit | AC: grep for "AutoYield" count | +| Gap 7 (behavioral equiv.) | No runtime comparison tests | Baseline comparison tests | PARTIAL — structural not behavioral | AC: 3+ baseline tests with GetText() | +| Gap 9 (Info severity on Executor\) | Missing base type classification | Conditional Warning severity | YES — escalates severity for the specific dangerous case | AC: test filter "ConfigureProtocolWarning" | +| Gap 11 (no migration guide) | No document exists | Create MIGRATION.md | YES — directly fills the gap | AC: file existence check | + +### Summary + +The implementation plan is well-grounded in codebase reality and addresses root causes for all targeted gaps. The two HIGH findings (RH1: line number fragility, RH2: duplicate detection scope) are clarification items, not plan defects. The plan's codebase investigation depth (exact file:line references, gotchas from reading actual code) significantly reduces implementation risk. All causal chains from gap → fix → verification are traceable. diff --git a/.designs/1/integration.md b/.designs/1/integration.md new file mode 100644 index 0000000000..225e2f99d5 --- /dev/null +++ b/.designs/1/integration.md @@ -0,0 +1,195 @@ +# D6: Integration + +## Dimension Summary + +Integration covers: (1) how the source generator connects to the Workflows project and solution, (2) which existing files are modified, (3) the test infrastructure, (4) the migration path from ReflectingExecutor, and (5) build system impacts. + +--- + +## Option I1: Current Integration Architecture (RECOMMENDED) + +The source generator is already fully integrated into the solution and build system: + +### Project References (AC-12, AC-18) + +**Workflows -> Generator reference** (Microsoft.Agents.AI.Workflows.csproj:35-39): +```xml + +``` + +This is the correct pattern for referencing a source generator as an analyzer. Verified: +- `OutputItemType="Analyzer"` -- loads as analyzer, not library reference +- `ReferenceOutputAssembly="false"` -- generator DLL not copied to output +- `GlobalPropertiesToRemove="TargetFramework"` -- prevents TFM mismatch when Workflows targets net8.0+ but generator targets netstandard2.0 + +### Solution Integration (AC-18) + +The solution file is `dotnet/agent-framework-dotnet.slnx` (codebase-snapshot.md section 2). Both the generator project and the unit test project are included. + +### Obsolete Markers (AC-13, AC-14) + +- **ReflectingExecutor** (ReflectingExecutor.cs:21-22): Already marked `[Obsolete("Use [MessageHandler] attribute on methods in a partial class deriving from Executor. This type will be removed in a future version.")]` + - Note: Plan says "This type will be removed in v1.0." but actual says "This type will be removed in a future version." -- slightly different wording. +- **IMessageHandler** (Reflection/IMessageHandler.cs): Already marked `[Obsolete]` + +### Test Infrastructure (AC-15, AC-16) + +**Unit Tests** (dotnet/tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/): +- `ExecutorRouteGeneratorTests.cs` (~37KB) -- comprehensive test coverage +- `GeneratorTestHelper.cs` -- test infrastructure for running generators in-memory +- `SyntaxTreeFluentExtensions.cs` -- fluent assertions for generated code + +Note: AC-15 specifies `SyntaxDetectorTests.cs` and `SemanticAnalyzerTests.cs` as separate files. The actual tests are consolidated in `ExecutorRouteGeneratorTests.cs`, which tests the full pipeline (syntax detection through semantic analysis to code generation). This is the standard approach for incremental generator testing, since the unit of work is the full generator pipeline. + +**Test cases covered** (verified from codebase-snapshot.md section 5): +- Single handler (void, ValueTask, ValueTask) +- Multiple handlers on one class +- CancellationToken parameter variants +- Yield/Send type attributes on handlers +- Class-level [SendsMessage] and [YieldsOutput] attributes +- Nested classes +- Generic executors +- Inheritance chains +- ConfigureProtocol already defined (skip generation) +- Invalid signatures (diagnostic verification) + +### InternalsVisibleTo + +Microsoft.Agents.AI.Workflows.csproj:31 grants: +```xml + +``` +This allows generator unit tests to access internal Workflows types for test compilation contexts. + +**Trade-offs:** +- (+) Fully operational -- no integration gaps +- (+) Generator reference uses recommended MSBuild patterns +- (+) `GlobalPropertiesToRemove="TargetFramework"` handles the TFM mismatch correctly +- (+) Comprehensive test coverage +- (-) Test file structure differs from plan (consolidated vs separate files) +- (-) Obsolete message wording differs slightly from plan + +**Reversibility:** Moderate -- deeply integrated with build system. + +**Constraint compliance:** +- C-1 (netstandard2.0): Generator project targets netstandard2.0 (Microsoft.Agents.AI.Workflows.Generators.csproj:5) +- C-3 (analyzer packaging): `IsRoslynComponent=true`, `EnforceExtendedAnalyzerRules=true` (Microsoft.Agents.AI.Workflows.Generators.csproj:16-17), packed to `analyzers/dotnet/cs` (line 55) +- C-4 (clean break): ReflectingExecutor marked [Obsolete], generator requires Executor inheritance + +**Dependencies produced:** Build system configuration consumed by D4 (scale -- build performance). +**Dependencies required:** Attribute definitions from D1 (API), model types from D2 (data). + +**Risks:** +- Severity: Low. Integration is complete and working. +- Mitigation: Existing CI/CD validates the full build. + +--- + +## Option I2: Split Test Files Per AC-15 + +Refactor `ExecutorRouteGeneratorTests.cs` into separate test files: +- `SyntaxDetectorTests.cs` -- tests for syntax-level detection +- `SemanticAnalyzerTests.cs` -- tests for semantic validation +- `ExecutorRouteGeneratorTests.cs` -- tests for end-to-end generation + +**Trade-offs:** +- (+) Matches AC-15 file structure exactly +- (+) Smaller, more focused test files +- (-) Artificial split -- incremental generators are tested end-to-end because the unit of work is the full pipeline (syntax -> semantic -> generation) +- (-) SyntaxDetector is inline (no separate class), so "SyntaxDetectorTests" would test the ForAttributeWithMetadataName predicate indirectly +- (-) Risk of test duplication (end-to-end tests implicitly test syntax and semantic phases) +- (-) Significant refactoring effort for ~37KB of tests + +**Reversibility:** Easy -- just file reorganization. + +**Constraint compliance:** All constraints met. + +**Dependencies produced:** None. +**Dependencies required:** None. + +**Risks:** +- Severity: Low. Code reorganization only. +- Mitigation: N/A. + +--- + +## Option I3: Add Integration Test Project (Per AC-16) + +Create a separate integration test project that: +1. Ports existing `ReflectingExecutor` test cases to use `[MessageHandler]` +2. Verifies generated routes match reflection-discovered routes at runtime + +**Trade-offs:** +- (+) Directly satisfies AC-16 +- (+) Validates that the source-generated code produces functionally identical behavior to reflection-based discovery +- (+) Catch regressions where generated AddHandler calls differ from reflected ones +- (-) Requires a separate project that can reference both the generator and the Workflows library +- (-) Runtime comparison is complex: must instantiate both a ReflectingExecutor and a source-generated executor, then compare their Protocol configurations +- (-) ReflectingExecutor is [Obsolete], so test project must suppress the warning + +**Reversibility:** Easy -- additive. + +**Constraint compliance:** All constraints met. Integration test project would target net8.0+ (not constrained by C-1). + +**Dependencies produced:** Integration test project consumed by CI/CD. +**Dependencies required:** Working generator from D2 (data), attributes from D1 (API). + +**Risks:** +- Severity: Medium. AC-16 is explicitly in scope but not currently implemented as described. +- Mitigation: The unit tests in ExecutorRouteGeneratorTests.cs verify generated code correctness, providing equivalent coverage through a different mechanism. + +--- + +## Option I4: Update Obsolete Message to Match Plan Exactly + +Change ReflectingExecutor's `[Obsolete]` message from "This type will be removed in a future version." to "See migration guide. This type will be removed in v1.0." per AC-13. + +**Trade-offs:** +- (+) Matches AC-13 verbatim +- (-) Promises removal "in v1.0" which is a concrete version commitment that may not be appropriate +- (-) References a "migration guide" that does not yet exist +- (-) Minor text change with no functional impact + +**Reversibility:** Easy. + +**Constraint compliance:** All constraints met. + +**Dependencies produced:** None. +**Dependencies required:** Migration guide from D3 (UX). + +**Risks:** +- Severity: Low. +- Mitigation: Ensure migration guide exists before changing the message. + +--- + +## Call Sites That Change (AC-18 Analysis) + +| File | Change | Status | +|------|--------|--------| +| Microsoft.Agents.AI.Workflows.csproj | Add generator ProjectReference | DONE (lines 35-39) | +| ReflectingExecutor.cs | Add [Obsolete] | DONE (lines 21-22) | +| IMessageHandler.cs | Add [Obsolete] | DONE | +| agent-framework-dotnet.slnx | Add new projects | DONE | + +All AC-18 modifications are already applied. + +--- + +## Build Order Impact + +The generator project must build BEFORE the Workflows project (since it is referenced as an analyzer). The `OutputItemType="Analyzer"` reference ensures MSBuild handles this ordering. The `SkipIncompatibleBuild.targets` file in the generator project handles cases where the generator cannot be built on the current platform. + +Critical path: +1. Microsoft.Agents.AI.Workflows.Generators (netstandard2.0) +2. Microsoft.Agents.AI.Workflows (multi-TFM: net8.0, netstandard2.0) +3. All downstream projects + +--- + +## Summary + +**Recommended option: I1** -- The current integration architecture is complete and operational. All AC-18 modifications are applied, the generator is correctly referenced as an analyzer, tests are comprehensive, and obsolete markers are in place. The differences from the plan (consolidated test files, slightly different obsolete wording) are non-functional. diff --git a/.designs/1/scale.md b/.designs/1/scale.md new file mode 100644 index 0000000000..65526cc8b6 --- /dev/null +++ b/.designs/1/scale.md @@ -0,0 +1,135 @@ +# D4: Scalability + +## Dimension Summary + +Scalability for a Roslyn source generator means: (1) incremental compilation performance (the generator must not re-run on every keystroke), (2) handling large codebases with many executor classes, (3) handling complex inheritance hierarchies, and (4) memory efficiency within the compiler host process. + +--- + +## Option S1: Current Incremental Generator Architecture (RECOMMENDED) + +The existing implementation uses Roslyn's `IIncrementalGenerator` API (ExecutorRouteGenerator.cs:22) with `ForAttributeWithMetadataName` (ExecutorRouteGenerator.cs:33-53) for all three pipelines. This is the highest-performance pattern available in Roslyn. + +**How it scales:** + +1. **Attribute-based filtering**: `ForAttributeWithMetadataName` uses Roslyn's internal efficient attribute lookup. Only files containing the specific attribute metadata name are processed. A codebase with 1000 files but 5 executors processes only 5 files. + +2. **Incremental caching**: All pipeline data models use record types with value equality (ExecutorInfo.cs:18, HandlerInfo.cs:34) and `ImmutableEquatableArray` (Models/ImmutableEquatableArray.cs) to enable Roslyn's incremental caching. If an executor class hasn't changed, its analysis result is cache-hit and generation is skipped entirely. + +3. **Two-phase analysis**: SemanticAnalyzer splits analysis into per-method (AnalyzeHandlerMethod) and per-class (CombineHandlerMethodResults). This avoids redundant class-level validation when a class has N handler methods -- class validation runs once, not N times (SemanticAnalyzer.cs:110-189). + +4. **Sorted type arrays**: Send/Yield type arrays are sorted (SemanticAnalyzer.cs:316, 629) to ensure deterministic output for incremental caching. Without sorting, different attribute orderings across partial class declarations would cause false cache misses. + +**Trade-offs:** +- (+) ForAttributeWithMetadataName is the recommended Roslyn API for incremental generators (see csproj comment lines 45-48) +- (+) Value-equality records enable correct caching +- (+) Two-phase analysis avoids redundant work +- (+) Zero additional runtime dependencies +- (-) Roslyn 4.4.0 minimum (chosen for ForAttributeWithMetadataName support) +- (-) Complex model hierarchy (11 model types) needed to satisfy caching requirements + +**Reversibility:** Difficult -- changing the pipeline architecture affects all models. + +**Constraint compliance:** +- C-1 (netstandard2.0): Compliant. All models use netstandard2.0-compatible types. Records enabled via `InjectIsExternalInitOnLegacy`. +- C-2 (Roslyn version): Uses 4.4.0 (deliberate -- see D2 analysis). +- C-3 (analyzer packaging): Compliant via .csproj configuration. + +**Dependencies produced:** Pipeline architecture consumed by D6 (integration -- build performance). +**Dependencies required:** None. + +**Risks:** +- Severity: Low. The incremental generator pattern is well-established. +- Mitigation: Existing tests (~37KB in ExecutorRouteGeneratorTests.cs) validate correctness under various scenarios. + +--- + +## Option S2: Add Syntax-Level Pre-Filter (Restore SyntaxDetector) + +Add an explicit syntax predicate before the semantic analysis phase to reject obvious non-candidates earlier in the pipeline (e.g., reject non-partial classes at the syntax level instead of the semantic level). + +**Trade-offs:** +- (+) Could reject non-partial classes before loading the semantic model, saving work +- (+) Matches AC-1 file structure (SyntaxDetector.cs) +- (-) `ForAttributeWithMetadataName` already performs efficient syntax-level filtering; its `predicate` parameter (lines 35, 43, 51) runs at syntax level +- (-) Checking `partial` at syntax level requires examining the class declaration, not just the method -- this means walking up from the method to its containing class, which is possible but adds complexity +- (-) For attribute-based generators, the semantic phase is typically cheap because ForAttributeWithMetadataName already narrows the candidate set +- (-) Marginal performance gain for significant architectural change + +**Reversibility:** Easy. + +**Constraint compliance:** All constraints met. + +**Dependencies produced:** SyntaxDetector consumed by ExecutorRouteGenerator. +**Dependencies required:** None. + +**Risks:** +- Severity: Low. Over-engineering for a marginal gain. +- Mitigation: Profile before optimizing. + +--- + +## Option S3: Parallel Sub-Pipeline Processing + +Split the three pipelines (MessageHandler methods, SendsMessage classes, YieldsOutput classes) into fully independent processing, avoiding the `Collect().Combine()` merge step (ExecutorRouteGenerator.cs:57-65). + +**REJECTED: violates incremental generator architecture** + +The `Collect().Combine()` pattern is required to merge results from different pipelines that target the same executor class. Without it, a class with both `[MessageHandler]` methods and `[SendsMessage]` class attributes would generate two conflicting partial classes. The merge step (CombineAllResults, ExecutorRouteGenerator.cs:94-126) deduplicates by class key. + +**Trade-offs:** +- (+) Would avoid the collect barrier +- (-) Would produce duplicate/conflicting generated code for classes that use both handler and protocol attributes +- (-) Fundamentally breaks the design where a single ConfigureProtocol override combines all three concerns + +**Reversibility:** Moderate. + +--- + +## Option S4: Lazy Semantic Model Access + +Defer `INamedTypeSymbol` resolution until absolutely needed, operating on syntax nodes as long as possible. Currently, `AnalyzeHandlerMethod` (SemanticAnalyzer.cs:47-104) immediately accesses the semantic model. + +**Trade-offs:** +- (+) Could reduce semantic model loads for invalid candidates +- (-) `ForAttributeWithMetadataName`'s transform callback receives `GeneratorAttributeSyntaxContext` which already has the semantic model loaded -- there is no cost saving +- (-) The transform callback is only invoked for actual attribute matches, so invalid candidates are already filtered +- (-) Would complicate the code for zero measurable benefit + +**Reversibility:** Easy. + +**Constraint compliance:** All constraints met. + +**Dependencies produced:** None. +**Dependencies required:** None. + +**Risks:** +- Severity: Low. No benefit. +- Mitigation: N/A. + +--- + +## Performance Characteristics + +### Build-Time Impact (based on architecture analysis) + +| Codebase Size | Expected Impact | Rationale | +|---------------|----------------|-----------| +| Small (1-5 executors) | < 10ms additional | ForAttributeWithMetadataName filters to exact matches | +| Medium (10-50 executors) | < 50ms additional | Linear in number of executors, not in codebase size | +| Large (100+ executors) | < 200ms additional [UNVERIFIED] | Collect().Combine() is the bottleneck; per-executor processing is fast | + +### IDE Experience + +| Action | Expected Behavior | +|--------|-------------------| +| Typing in non-executor file | No generator work (attribute filtering) | +| Adding [MessageHandler] to method | Generator re-runs for that class only (incremental) | +| Editing handler body (no signature change) | No generator re-run (value equality cache hit) | +| Adding new handler to existing executor | Generator re-runs for that class (new method in Collect) | + +--- + +## Summary + +**Recommended option: S1** -- The current incremental generator architecture is the correct approach for Roslyn source generators. It uses the recommended APIs, implements proper caching via value-equality records, and avoids redundant work through two-phase analysis. No scalability improvements are needed. diff --git a/.designs/1/security.md b/.designs/1/security.md new file mode 100644 index 0000000000..a200040fc0 --- /dev/null +++ b/.designs/1/security.md @@ -0,0 +1,155 @@ +# D5: Security + +## Dimension Summary + +Security considerations for a Roslyn source generator include: (1) the generator's execution context within the compiler host, (2) the generated code's security properties, (3) supply chain security (NuGet packaging), and (4) the attack surface of the attribute-driven API. + +--- + +## Threat Model + +### T1: Malicious Input via Attribute Arguments + +**Threat:** A developer (or compromised dependency) places crafted type names in `[MessageHandler(Yield = [...], Send = [...])]` or `[SendsMessage(typeof(...))]` that, when emitted into generated code, produce injection attacks (e.g., type names containing C# code fragments). + +**Severity:** Low. + +**Analysis:** The generator uses `INamedTypeSymbol.ToDisplayString(SymbolDisplayFormat.FullyQualifiedFormat)` (SemanticAnalyzer.cs:234, 507, 518) to produce type names. This API returns the Roslyn-resolved symbol name, NOT raw user text. A type name can only be a valid C# identifier that the compiler has already resolved. There is no string interpolation of user-provided raw text into generated code. + +**Mitigation (already in place):** Symbol resolution prevents code injection. Only resolved type symbols are emitted. + +### T2: Information Leakage via Diagnostic Messages + +**Threat:** Diagnostic messages could leak sensitive information (internal type names, paths) to build logs visible to unauthorized parties. + +**Severity:** Low. + +**Analysis:** Diagnostics (DiagnosticDescriptors.cs) include method names and class names, which are part of the public API surface by definition (they appear in source code). No file paths, connection strings, or secrets are included in diagnostic messages. + +**Mitigation (already in place):** Diagnostic messages contain only method/class names already visible in source. + +### T3: Supply Chain Attack via NuGet Package + +**Threat:** The generator DLL, packaged at `analyzers/dotnet/cs` (Microsoft.Agents.AI.Workflows.Generators.csproj:55), executes inside the compiler process. A compromised generator could run arbitrary code during compilation. + +**Severity:** High (if compromised). + +**Analysis:** This is an inherent risk of all Roslyn analyzers/generators. The generator runs with full compiler-process permissions. However, this is a first-party Microsoft package built from the same repo, signed, and distributed through official channels. + +**Mitigation:** +- Package is built from the same solution with CI/CD controls +- `DevelopmentDependency=true` (Microsoft.Agents.AI.Workflows.Generators.csproj:41) prevents the generator from being a runtime dependency +- `ReferenceOutputAssembly=false` (Microsoft.Agents.AI.Workflows.csproj:38) prevents the generator assembly from being deployed with the application +- `SuppressDependenciesWhenPacking=true` (Microsoft.Agents.AI.Workflows.Generators.csproj:21) prevents pulling in transitive dependencies + +### T4: Handler Method Accessibility Bypass + +**Threat:** The source generator discovers `private` methods and generates code that references them. Could this expose private methods beyond their intended scope? + +**Severity:** Negligible. + +**Analysis:** The generated code is emitted into a `partial class` of the same type. A partial class has full access to all members of the same class, including private members. This is standard C# behavior. The generator does NOT make private methods accessible from outside the class -- it only references them via `this.MethodName` within the same class scope (SourceBuilder.cs:149, 161). + +**Mitigation (already in place):** Partial class semantics guarantee that generated code has the same access as hand-written code in the same class. + +--- + +## Option SEC1: Current Security Posture (RECOMMENDED) + +The existing implementation has appropriate security characteristics for a first-party source generator: + +1. **No raw string injection**: All emitted type names go through Roslyn symbol resolution +2. **No file system access**: Generator only reads the compilation model +3. **No network access**: Generator is purely computational +4. **Development dependency**: Generator assembly is compile-time only, not deployed to production +5. **Private member access is scoped**: Generated partial class code references `this.MethodName` within the same type + +**Trade-offs:** +- (+) Minimal attack surface +- (+) No additional dependencies introduced +- (+) Standard Roslyn security model +- (-) Inherent analyzer trust model (all analyzers run in compiler process) + +**Reversibility:** N/A. + +**Constraint compliance:** +- C-3 (analyzer packaging): `DevelopmentDependency=true` and `ReferenceOutputAssembly=false` ensure generator is compile-time only +- C-5 (any accessibility): Private handler access is safe due to partial class semantics + +**Dependencies produced:** Security posture assessment consumed by D6 (integration -- deployment). +**Dependencies required:** None. + +**Risks:** +- Severity: Low overall. +- Mitigation: Standard supply chain security practices (CI/CD, signing, official distribution). + +--- + +## Option SEC2: Add Input Validation for Type Names in SourceBuilder + +Add explicit validation in SourceBuilder.Generate() to reject type names containing suspicious characters before emitting them into generated code. + +**Trade-offs:** +- (+) Defense-in-depth against hypothetical symbol resolution bypass +- (-) Roslyn's SymbolDisplayFormat.FullyQualifiedFormat already guarantees valid identifiers +- (-) Any type name that passes Roslyn's symbol resolution is by definition a valid C# type +- (-) Adds unnecessary complexity + +**Reversibility:** Easy. + +**Constraint compliance:** All constraints met. + +**Dependencies produced:** None. +**Dependencies required:** None. + +**Risks:** +- Severity: Low. Over-engineering. +- Mitigation: N/A. + +--- + +## Option SEC3: Add Generated Code Markers for Audit Trail + +Enhance the generated code header (currently `// ` in SourceBuilder.cs:32) with a generator version hash and timestamp to enable auditability. + +**Trade-offs:** +- (+) Enables auditing which version of the generator produced the code +- (-) Timestamps in generated code would cause false cache misses in the incremental generator pipeline +- (-) Version hashes would require build infrastructure changes +- (-) Generated code is transient (not checked in); auditability has limited value + +**Reversibility:** Easy. + +**Constraint compliance:** All constraints met. + +**Dependencies produced:** None. +**Dependencies required:** None. + +**Risks:** +- Severity: Low. +- Mitigation: N/A. + +--- + +## Option SEC4: Restrict Handler Discovery to Explicitly Public Methods Only + +Only generate routes for public or internal handler methods, requiring users to explicitly opt-in by making handlers non-private. + +**REJECTED: violates C-5 (handler accessibility: any)** + +C-5 explicitly states "Handler accessibility: Any (private, protected, internal, public)". Restricting to public methods would violate this constraint. + +Additionally, the primary use case for [MessageHandler] is to allow private handler methods (which is more encapsulated than forcing public), so this restriction would actively harm the design intent. + +**Trade-offs:** +- (+) Reduces surface area for accidental handler exposure +- (-) Violates C-5 +- (-) Forces users to make implementation methods public, breaking encapsulation + +**Reversibility:** Easy. + +--- + +## Summary + +**Recommended option: SEC1** -- The current security posture is appropriate. The generator operates within the standard Roslyn security model, uses symbol resolution (not raw strings) for code generation, and is correctly packaged as a compile-time-only development dependency. diff --git a/.designs/1/six_sigma_gaps.md b/.designs/1/six_sigma_gaps.md new file mode 100644 index 0000000000..3b93f2800e --- /dev/null +++ b/.designs/1/six_sigma_gaps.md @@ -0,0 +1,152 @@ +# Six-Sigma Gap Analysis + +**Date**: 2026-05-12 +**Source**: source.md + +## Gap Discovery Method +Independent analysis of escape paths, test type mismatches, sibling vulnerabilities, +version/config drift risks, and unstated assumptions. All codebase claims verified via +Read tool against the actual files in the worktree. + +## Gaps Identified + +### Gap 1: Roslyn Version Constraint Drift (Plan Says 4.8.0+, Actual Uses 4.4.0) + +**Category**: version drift +**Current state**: The plan (AC-2, C-2) specifies `Microsoft.CodeAnalysis.CSharp` 4.8.0+, but the actual `.csproj` file at `dotnet/src/Microsoft.Agents.AI.Workflows.Generators/Microsoft.Agents.AI.Workflows.Generators.csproj` line 49 uses `VersionOverride="4.4.0"`. A code comment explains this is deliberate: "Use Roslyn 4.4.0 - minimum version for ForAttributeWithMetadataName API. Corresponds to .NET 7 SDK / VS 2022 17.4+. Higher versions would require newer SDKs, breaking users on older versions." +**Ideal state**: The plan should be corrected to reflect the actual constraint (4.4.0), or the design document should explicitly note the deviation and its rationale. If the plan were followed literally, it would break users on .NET 7 SDK / VS 2022 17.4-17.7. +**Feasibility**: Feasible +**Feasibility rationale**: This is a documentation correction. The actual implementation chose the right version for broader compatibility. The design document should document this as a deliberate deviation from the plan. + +### Gap 2: Handler Methods With 4+ Parameters Silently Accepted + +**Category**: escape path +**Current state**: The `SemanticAnalyzer.AnalyzeHandler` method (line 477) validates that `Parameters.Length >= 2` and checks whether parameter [1] is `IWorkflowContext` and parameter [2] (if present) is `CancellationToken`. However, it does NOT validate that there are no more than 3 parameters. A handler declared as `void Handle(string msg, IWorkflowContext ctx, CancellationToken ct, int extraParam)` would pass validation; the generator would treat `hasCancellationToken = true` and generate an `AddHandler(this.Handle)` call. At runtime, the RouteBuilder expects a delegate with exactly 2 or 3 parameters (message, context, optionally CancellationToken). The generated code passes `this.Handle` as a method group -- if the method has 4 parameters, the method group conversion to the `Action` delegate would fail at compile time with CS0123. So the failure IS caught, but by the C# compiler with an unhelpful error message rather than by a descriptive generator diagnostic. +**Ideal state**: The generator should emit a diagnostic (e.g., "Handler has too many parameters") for methods with more than 3 parameters, giving users a clear, actionable error message rather than a cryptic method-group conversion failure. +**Feasibility**: Feasible +**Feasibility rationale**: Adding a `Parameters.Length > 3` check with a new diagnostic descriptor is straightforward. No architectural changes required. Low effort, high clarity gain. + +### Gap 3: No Diagnostic for Third Parameter That Is Not CancellationToken + +**Category**: escape path +**Current state**: In `SemanticAnalyzer.AnalyzeHandler` (line 492-493), the check `hasCancellationToken = methodSymbol.Parameters.Length >= 3 && methodSymbol.Parameters[2].Type.ToDisplayString() == CancellationTokenTypeName` silently sets `hasCancellationToken = false` when the third parameter is not a `CancellationToken`. A handler like `void Handle(string msg, IWorkflowContext ctx, int someOtherParam)` would be accepted with `hasCancellationToken = false`, and the generator would emit `AddHandler(this.Handle)`. The method-group conversion would then fail at compile time because no `AddHandler` overload accepts `Action`. The user gets a generic C# compiler error rather than an informative diagnostic. +**Ideal state**: When a third parameter exists but is not `CancellationToken`, the generator should emit a warning diagnostic such as "Third parameter of handler should be CancellationToken; other types are not supported." +**Feasibility**: Feasible +**Feasibility rationale**: Simple conditional check with a new diagnostic descriptor. No architectural change needed. + +### Gap 4: Duplicate Input Type Handlers Produce Runtime Exception, Not Compile-Time Diagnostic + +**Category**: escape path +**Current state**: If two `[MessageHandler]` methods on the same executor handle the same input message type (e.g., two methods both taking `string` as the first parameter), the generator will produce two `.AddHandler(this.Method1)` and `.AddHandler(this.Method2)` calls. The `RouteBuilder.AddHandlerInternal` (line 51-89 of RouteBuilder.cs) throws `ArgumentException` ("A handler for message type X is already registered") at runtime during protocol configuration. The user receives a runtime exception instead of a compile-time diagnostic. +**Ideal state**: The generator should detect duplicate input type registrations during `CombineHandlerMethodResults` and emit a compile-time error diagnostic, preventing the runtime failure. +**Feasibility**: Feasible +**Feasibility rationale**: During `CombineHandlerMethodResults`, the generator already collects all handlers per class. A `HashSet` tracking `InputTypeName` would detect duplicates. Low-effort change with high impact for developer experience. + +### Gap 5: Plan/Implementation Naming Mismatch Not Surfaced to Consumers (YieldsMessageAttribute vs YieldsOutputAttribute) + +**Category**: scope gap +**Current state**: The plan (AC-5) specifies `YieldsMessageAttribute`, but the actual implementation uses `YieldsOutputAttribute` (at `dotnet/src/Microsoft.Agents.AI.Workflows/Attributes/YieldsOutputAttribute.cs`). The codebase-snapshot.md section 4 documents this discrepancy. However, if the design document uses the plan's terminology, users consulting the plan and the actual API will see different names, causing confusion during migration. +**Ideal state**: The design document should explicitly note the name change with a redirect: "`YieldsMessageAttribute` was renamed to `YieldsOutputAttribute` to align with the `YieldsOutput()` fluent API on `ProtocolBuilder`." Any migration guide should reference the actual name. +**Feasibility**: Feasible +**Feasibility rationale**: Documentation-only change. The actual implementation is correct; only the plan reference needs updating. + +### Gap 6: Generated Code Does Not Explicitly Register Return-Type Yield Types (Relies on Runtime Auto-Yield) + +**Category**: unstated assumption +**Current state**: When a handler returns `ValueTask`, the generator registers it via `AddHandler(this.Method)`, which records `TResult` as an output type in the `RouteBuilder._outputTypes` dictionary. At `ProtocolBuilder.Build()` time (line 167-169 of ProtocolBuilder.cs), if `ExecutorOptions.AutoYieldOutputHandlerResultObject` is true (the default), the `DefaultOutputTypes` from the router are unioned into `yieldTypes`. So the yield registration of return types happens implicitly at runtime via `AutoYieldOutputHandlerResultObject`, not explicitly in the generated code. + +However, `ShouldGenerateYieldedOutputRegistrations` in `ExecutorInfo` (line 37) returns `true` when any handler `HasOutput` (line 72-75), which triggers `GenerateConfigureYieldTypes()`. But `GenerateConfigureYieldTypes()` only emits `.YieldsOutput()` for explicit `handler.YieldTypes` and `info.ClassYieldTypes` -- not for the handler's return type (`OutputTypeName`). If a handler has a return type but no explicit `[MessageHandler(Yield = ...)]` and no class-level `[YieldsOutput]`, the `GenerateConfigureYieldTypes` method iterates over empty collections, generating nothing. The `ShouldGenerateYieldedOutputRegistrations` guard still evaluates true, but the method body is a no-op. This is functionally correct because the runtime handles it, but it means the compile-time protocol declaration is incomplete: users who set `AutoYieldOutputHandlerResultObject = false` would get no yield type registration for return types, and protocol validation would reject yields of those types. +**Ideal state**: Either (a) the generated code should explicitly emit `.YieldsOutput()` for handler return types so the protocol is self-documenting and independent of `ExecutorOptions`, or (b) the design document should explicitly note this runtime dependency and warn that `AutoYieldOutputHandlerResultObject = false` requires explicit `[YieldsOutput]` attributes. +**Feasibility**: Feasible +**Feasibility rationale**: Adding `outputTypeName` to the yield type emission in `GenerateConfigureYieldTypes()` is a small code change in `SourceBuilder.cs`. However, this would diverge from the current behavior where the runtime auto-yield and the explicit yield attributes are the two orthogonal mechanisms. The safer path is option (b): document the dependency. + +### Gap 7: No Integration Test Verifying Behavioral Equivalence Between Source-Generated and Reflected Routes + +**Category**: test type mismatch +**Current state**: AC-16 requires "Port existing ReflectingExecutor test cases to use [MessageHandler]" and "Verify generated routes match reflection-discovered routes." The existing test suite at `dotnet/tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/` contains 67 test methods that verify the generator's output structurally (checking generated source text). There are no tests that actually build and run both the source-generated and reflection-based executors side-by-side and compare their runtime behavior (protocol configuration, route resolution, message handling). The file `dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/ReflectionSmokeTest.cs` exists but tests only the reflection path. No file exists that instantiates a source-generated executor AND a ReflectingExecutor with the same handlers and asserts identical routing behavior. +**Ideal state**: An integration test that defines an executor with both `[MessageHandler]` attributes and `IMessageHandler` interface implementations, builds one via source generation and one via reflection, sends the same messages, and asserts identical routing and output behavior. +**Feasibility**: Partially feasible +**Feasibility rationale**: Source generator tests typically use `CSharpGeneratorDriver` which produces source text, not compiled assemblies. A true runtime comparison test would need to compile and load the generated code, instantiate the executor, and exercise it. This is doable via in-memory compilation but adds significant test infrastructure complexity. A pragmatic alternative: a manually-curated set of test cases that verify specific behavioral equivalences (route count, handler delegate types, type registrations) by comparing the generated source text against known-good baseline output from the reflection path. + +### Gap 8: ReflectingExecutor Registers Handler Output Types Conditionally on AutoSend/AutoYield Options; Source Generator Does Not + +**Category**: sibling vulnerability +**Current state**: `ReflectingExecutor.ConfigureProtocol` (lines 48-58 of `ReflectingExecutor.cs`) conditionally registers output types as send/yield types based on `Options.AutoSendMessageHandlerResultObject` and `Options.AutoYieldOutputHandlerResultObject`. The source generator does NOT generate any code that checks these options; it relies entirely on the runtime `ProtocolBuilder.Build()` method to apply these options. This is functionally correct because `ProtocolBuilder.Build()` applies the same options at build time. However, this means the behavior has a hidden dependency: the generated `ConfigureProtocol` method does not explicitly declare all types it will use. If `ProtocolBuilder.Build()` behavior ever changes (e.g., the auto-registration is moved to a different stage), the source-generated path and the reflection path could silently diverge. +**Ideal state**: The design document should explicitly note that the source generator delegates auto-send/auto-yield registration to `ProtocolBuilder.Build()` and that any changes to `ProtocolBuilder.Build()` must be validated against both paths. +**Feasibility**: Feasible +**Feasibility rationale**: This is a documentation/design-note addition, not a code change. Attempting to replicate the runtime option checks in generated code would require the generator to have access to `ExecutorOptions` at compile time, which is not possible since options are runtime configuration. + +### Gap 9: No Validation That Executor or Executor Subclasses Do Not Use [MessageHandler] + +**Category**: escape path +**Current state**: The `Executor` and `Executor` base classes (lines 382-423 of `Executor.cs`) already override `ConfigureProtocol` and register their own handler via `AddHandler`. If a user creates a class like `public partial class MyExecutor : Executor` and adds `[MessageHandler]` methods, the generator would detect that `HasManualConfigureProtocol` is true (because `Executor` defines `ConfigureProtocol`). This would trigger MAFGENWF006 (Info level: "ConfigureProtocol already defined") and skip generation. This is correct behavior, but the diagnostic severity is only Info, meaning users might not notice it. They would expect their `[MessageHandler]` methods to work, but they would be silently ignored. +**Ideal state**: When a class inherits from `Executor` or `Executor` (which provide their own `ConfigureProtocol`), the diagnostic should be Warning or Error level rather than Info, since the user clearly intended to use `[MessageHandler]` but it will be silently ignored. The message should guide them: "Inherit from Executor directly and use [MessageHandler] instead of inheriting from Executor." +**Feasibility**: Feasible +**Feasibility rationale**: This requires changing the diagnostic severity from Info to Warning for this specific scenario, or adding a new diagnostic that distinguishes "user manually defined ConfigureProtocol" from "base class provides ConfigureProtocol." Moderate effort: the generator would need to distinguish between the two cases by checking if the `ConfigureProtocol` override is on the direct base class or on an intermediate class. + +### Gap 10: Hint Name Generation for Generics Uses Approximation + +**Category**: version drift / scope gap +**Current state**: `ExecutorRouteGenerator.GetHintName` (line 153-155 of `ExecutorRouteGenerator.cs`) computes the generic parameter count approximation using `info.GenericParameters!.Length - 2` (subtracting 2 for the `<` and `>` characters). For a single type parameter like ``, this gives `1` (correct: `"".Length - 2 = 1`). For ``, this gives `3` (incorrect: actual count is 2, but `"".Length - 2 = 4`). Actually the string would be `""` which is length 6, minus 2 = 4. This is used only for generating a unique file name, so a wrong count does not cause functional issues, but it could theoretically collide with another class if two generic classes in the same namespace have names that differ only in their type parameter count. This is extremely unlikely in practice. +**Ideal state**: Use `info.GenericParameters.Count(c => c == ',') + 1` or the actual `TypeParameters.Length` from the symbol, which is precise. +**Feasibility**: Feasible +**Feasibility rationale**: Trivial code change. The `GenericParameters` string could be replaced with a `GenericParameterCount` integer on `ExecutorInfo`, or the count could be computed from the string more precisely. + +### Gap 11: No Migration Guide Document Referenced in [Obsolete] Message + +**Category**: observability gap +**Current state**: AC-13 specifies the obsolete message should say "See migration guide." The actual [Obsolete] message on `ReflectingExecutor` (line 21-22 of `ReflectingExecutor.cs`) says "Use [MessageHandler] attribute on methods in a partial class deriving from Executor. This type will be removed in a future version." -- it omits "See migration guide." This matches what exists in the codebase. However, no migration guide document was found in the repository. If a user reads the obsolete message, they have no guide to follow. +**Ideal state**: Either (a) create a migration guide document and reference it in the [Obsolete] message, or (b) the design should acknowledge that the migration guide is out of scope and update AC-13 accordingly. +**Feasibility**: Feasible +**Feasibility rationale**: Writing a migration guide is standard documentation work. Referencing it from the [Obsolete] attribute is a one-line change. + +### Gap 12: SendsMessage/YieldsOutput on Method-Level Not Exercised by Generator for Protocol-Only Classes + +**Category**: escape path +**Current state**: Both `SendsMessageAttribute` and `YieldsOutputAttribute` have `AttributeTargets.Class | AttributeTargets.Method`. Pipeline 2 and Pipeline 3 in `ExecutorRouteGenerator.cs` (lines 40-53) use `predicate: static (node, _) => node is ClassDeclarationSyntax`, meaning they only detect these attributes on classes, not on methods. Method-level `[SendsMessage]`/`[YieldsOutput]` are handled through `GetAttributeTypeArrays` in `SemanticAnalyzer` only when a `[MessageHandler]` method is present. For a class that uses `Executor` (which has its own `ConfigureProtocol`) and adds `[SendsMessage(typeof(Foo))]` on individual methods (not on the class), the generator pipeline would not pick up those method-level attributes because: (a) Pipeline 2/3 only look at classes, (b) the `[MessageHandler]` pipeline would not fire because there are no `[MessageHandler]` methods. The method-level attributes on non-`[MessageHandler]` methods would be silently ignored by the generator. + +However, at runtime, `ReflectingExecutor.ConfigureProtocol` does scan for these via `AddMethodAttributeTypes`, and the `Executor` base class does call `AddMethodAttributeTypes(handlerDelegate.Method)` which would pick up method-level attributes on the `HandleAsync` method. So for `Executor` subclasses, method-level attributes on the `HandleAsync` method ARE picked up at runtime. The gap is only for methods that are NOT `HandleAsync` and NOT `[MessageHandler]` -- which would be unusual but possible. +**Ideal state**: Document that method-level `[SendsMessage]`/`[YieldsOutput]` attributes are only processed on `[MessageHandler]`-attributed methods or on the single `HandleAsync` method of `Executor` subclasses. Other methods' attributes are not picked up. +**Feasibility**: Feasible +**Feasibility rationale**: Documentation clarification. Extending the generator to scan all methods for `[SendsMessage]`/`[YieldsOutput]` would be architecturally complex and would blur the boundary between source generation and reflection. + +### Gap 13: Partial Class Across Multiple Files Not Explicitly Tested + +**Category**: test type mismatch +**Current state**: All unit tests define the executor class in a single source string. In real projects, partial classes are commonly split across multiple files. The `ForAttributeWithMetadataName` API handles this correctly (it works at the semantic model level, not the syntax level), and `IsPartialClass` checks `DeclaringSyntaxReferences` across all syntax trees. However, no test verifies this explicitly -- e.g., a test with two separate source strings where the first file has `partial class MyExecutor : Executor { }` and the second has `partial class MyExecutor { [MessageHandler] void Handle(string m, IWorkflowContext ctx) {} }`. +**Ideal state**: At least one test with a multi-file partial class split to verify the generator correctly discovers handlers across files. +**Feasibility**: Feasible +**Feasibility rationale**: The `CSharpGeneratorDriver` test infrastructure already supports multiple syntax trees. Adding a test with two source strings is straightforward. + +## Summary Table + +| # | Gap | Category | Impact | Feasibility | Specific constraint (if infeasible) | +|---|-----|----------|--------|-------------|--------------------------------------| +| 1 | Roslyn version drift (plan 4.8.0 vs actual 4.4.0) | version drift | Low (plan-only; impl is correct) | Feasible | N/A | +| 2 | 4+ parameter handlers silently accepted | escape path | Medium (confusing compiler error) | Feasible | N/A | +| 3 | Non-CancellationToken third parameter silently ignored | escape path | Medium (confusing compiler error) | Feasible | N/A | +| 4 | Duplicate input type handlers cause runtime not compile-time error | escape path | High (runtime crash vs build error) | Feasible | N/A | +| 5 | YieldsMessage vs YieldsOutput naming mismatch | scope gap | Low (documentation clarity) | Feasible | N/A | +| 6 | Return-type yield registration deferred to runtime | unstated assumption | Medium (breaks with AutoYield=false) | Feasible | N/A | +| 7 | No runtime behavioral equivalence integration test | test type mismatch | High (migration correctness) | Partially feasible | Requires in-memory compilation infrastructure | +| 8 | Source generator delegates auto-send/yield to runtime; undocumented dependency | sibling vulnerability | Low (documented assumption sufficient) | Feasible | N/A | +| 9 | Executor subclass [MessageHandler] silently ignored at Info level | escape path | High (user intent silently discarded) | Feasible | N/A | +| 10 | Hint name generic parameter count approximation | version drift | Very Low (cosmetic, extremely unlikely collision) | Feasible | N/A | +| 11 | No migration guide exists despite [Obsolete] message implying one | observability gap | Medium (migration friction) | Feasible | N/A | +| 12 | Method-level SendsMessage/YieldsOutput on non-handler methods ignored | escape path | Low (unusual usage pattern) | Feasible | N/A | +| 13 | Multi-file partial class not explicitly tested | test type mismatch | Medium (real-world usage pattern untested) | Feasible | N/A | + +## Practical Ceiling + +For a Roslyn source generator of this scope, the practical quality ceiling is bounded by: + +1. **Compile-time vs runtime boundary**: The generator operates on syntax and semantic models at compile time and cannot access runtime configuration (like `ExecutorOptions`). This means some validation must inherently occur at runtime. The design correctly handles this by delegating `AutoSendMessage`/`AutoYieldOutput` to `ProtocolBuilder.Build()`, but this creates an irreducible coupling between the generated code and the runtime builder. + +2. **Roslyn API stability**: The generator depends on the Roslyn `ForAttributeWithMetadataName` API and `IIncrementalGenerator` contract. Changes to Roslyn's behavior in future SDK versions could affect the generator without any code changes. The 4.4.0 minimum version choice provides broad compatibility but means the generator cannot use newer Roslyn APIs. + +3. **Test depth ceiling**: Full behavioral equivalence testing between source-generated and reflection-based paths would require compiling and executing generated code in test harness, which is architecturally expensive. The current structural verification (comparing generated source text) covers most cases but cannot catch subtle runtime behavioral differences. + +**Residual risks that must be accepted:** +- Runtime options (`ExecutorOptions`) cannot be validated at compile time +- Exact behavioral parity between ReflectingExecutor and source-generated paths can only be fully verified through runtime integration tests, not source-text comparison +- Users who split partial classes across many files may encounter incremental caching edge cases that are difficult to reproduce in unit tests diff --git a/.designs/1/source.md b/.designs/1/source.md new file mode 100644 index 0000000000..447a9e6b53 --- /dev/null +++ b/.designs/1/source.md @@ -0,0 +1,307 @@ +# source.md — Verbatim Source Capture + +## Source References +- [x] Source 1: `wf-source-gen-plan.md` (worktree root) — fetched at 2026-05-12T03:30Z + +## Problem Statement (verbatim) +``` +Replace the reflection-based `ReflectingExecutor` pattern with a compile-time source generator that discovers `[MessageHandler]` attributed methods and generates `ConfigureRoutes`, `ConfigureSentTypes`, and `ConfigureYieldTypes` implementations. +``` + +## Design Decisions (Confirmed) (verbatim) +``` +- **Attribute syntax**: Inline properties on `[MessageHandler(Yield=[...], Send=[...])]` +- **Class-level attributes**: Generate `ConfigureSentTypes()`/`ConfigureYieldTypes()` from `[SendsMessage]`/`[YieldsMessage]` +- **Migration**: Clean break - requires direct `Executor` inheritance (not `ReflectingExecutor`) +- **Handler accessibility**: Any (private, protected, internal, public) +``` + +## Acceptance Criteria (verbatim) + +### AC-1: Project Structure +> ``` +> dotnet/src/Microsoft.Agents.AI.Workflows.Generators/ +> ├── Microsoft.Agents.AI.Workflows.Generators.csproj +> ├── ExecutorRouteGenerator.cs # Main incremental generator +> ├── Models/ +> │ ├── ExecutorInfo.cs # Data model for executor analysis +> │ └── HandlerInfo.cs # Data model for handler methods +> ├── Analysis/ +> │ ├── SyntaxDetector.cs # Syntax-based candidate detection +> │ └── SemanticAnalyzer.cs # Semantic model analysis +> ├── Generation/ +> │ └── SourceBuilder.cs # Code generation logic +> └── Diagnostics/ +> └── DiagnosticDescriptors.cs # Analyzer diagnostics +> ``` + +### AC-2: Project File Configuration +> - Target `netstandard2.0` +> - Reference `Microsoft.CodeAnalysis.CSharp` 4.8.0+ +> - Set `IsRoslynComponent=true`, `EnforceExtendedAnalyzerRules=true` +> - Package as analyzer in `analyzers/dotnet/cs` + +### AC-3: MessageHandlerAttribute +> ```csharp +> [AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = false)] +> public sealed class MessageHandlerAttribute : Attribute +> { +> public Type[]? Yield { get; set; } // Types yielded as workflow outputs +> public Type[]? Send { get; set; } // Types sent to other executors +> } +> ``` + +### AC-4: SendsMessageAttribute +> ```csharp +> [AttributeUsage(AttributeTargets.Class, AllowMultiple = true, Inherited = true)] +> public sealed class SendsMessageAttribute : Attribute +> { +> public Type Type { get; } +> public SendsMessageAttribute(Type type) => this.Type = type; +> } +> ``` + +### AC-5: YieldsMessageAttribute +> ```csharp +> [AttributeUsage(AttributeTargets.Class, AllowMultiple = true, Inherited = true)] +> public sealed class YieldsMessageAttribute : Attribute +> { +> public Type Type { get; } +> public YieldsMessageAttribute(Type type) => this.Type = type; +> } +> ``` + +### AC-6: Detection Criteria (syntax level) +> - Class has `partial` modifier +> - Class has at least one method with `[MessageHandler]` attribute + +### AC-7: Validation Criteria (semantic level) +> - Class derives from `Executor` (directly or transitively) +> - Class does NOT already define `ConfigureRoutes` with a body +> - Handler method has valid signature: `(TMessage, IWorkflowContext[, CancellationToken])` +> - Handler returns `void`, `ValueTask`, or `ValueTask` + +### AC-8: Handler Signature Mapping +> | Method Signature | Generated AddHandler Call | +> |-----------------|---------------------------| +> | `void Handler(T, IWorkflowContext)` | `AddHandler(this.Handler)` | +> | `void Handler(T, IWorkflowContext, CT)` | `AddHandler(this.Handler)` | +> | `ValueTask Handler(T, IWorkflowContext)` | `AddHandler(this.Handler)` | +> | `ValueTask Handler(T, IWorkflowContext, CT)` | `AddHandler(this.Handler)` | +> | `TResult Handler(T, IWorkflowContext)` | `AddHandler(this.Handler)` | +> | `ValueTask Handler(T, IWorkflowContext, CT)` | `AddHandler(this.Handler)` | + +### AC-9: Generated Code Structure +> ```csharp +> // +> #nullable enable +> +> namespace MyNamespace; +> +> partial class MyExecutor +> { +> protected override RouteBuilder ConfigureRoutes(RouteBuilder routeBuilder) +> { +> // Call base if inheriting from another executor with routes +> // routeBuilder = base.ConfigureRoutes(routeBuilder); +> +> return routeBuilder +> .AddHandler(this.Handler1) +> .AddHandler(this.Handler2); +> } +> +> protected override ISet ConfigureSentTypes() +> { +> var types = base.ConfigureSentTypes(); +> types.Add(typeof(SentType1)); +> return types; +> } +> +> protected override ISet ConfigureYieldTypes() +> { +> var types = base.ConfigureYieldTypes(); +> types.Add(typeof(YieldType1)); +> return types; +> } +> } +> ``` + +### AC-10: Inheritance Handling +> | Scenario | Generated `ConfigureRoutes` | +> |----------|----------------------------| +> | Directly extends `Executor` | No base call (abstract) | +> | Extends executor with `[MessageHandler]` methods | `routeBuilder = base.ConfigureRoutes(routeBuilder);` | +> | Extends executor with manual `ConfigureRoutes` | `routeBuilder = base.ConfigureRoutes(routeBuilder);` | + +### AC-11: Analyzer Diagnostics +> | ID | Severity | Condition | +> |----|----------|-----------| +> | `WFGEN001` | Error | Handler missing `IWorkflowContext` parameter | +> | `WFGEN002` | Error | Handler has invalid return type | +> | `WFGEN003` | Error | Executor with `[MessageHandler]` must be `partial` | +> | `WFGEN004` | Warning | `[MessageHandler]` on non-Executor class | +> | `WFGEN005` | Error | Handler has fewer than 2 parameters | +> | `WFGEN006` | Info | `ConfigureRoutes` already defined, handlers ignored | + +### AC-12: Integration — Wire Generator to Main Project +> ```xml +> +> +> OutputItemType="Analyzer" +> ReferenceOutputAssembly="false" /> +> +> ``` + +### AC-13: Mark ReflectingExecutor Obsolete +> ```csharp +> [Obsolete("Use [MessageHandler] attribute on methods in a partial class deriving from Executor. " + +> "See migration guide. This type will be removed in v1.0.", error: false)] +> public class ReflectingExecutor : Executor ... +> ``` + +### AC-14: Mark IMessageHandler Interfaces Obsolete +> ```csharp +> [Obsolete("Use [MessageHandler] attribute instead.")] +> public interface IMessageHandler { ... } +> ``` + +### AC-15: Generator Unit Tests +> ``` +> dotnet/tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/ +> ├── ExecutorRouteGeneratorTests.cs +> ├── SyntaxDetectorTests.cs +> ├── SemanticAnalyzerTests.cs +> └── TestHelpers/ +> └── GeneratorTestHelper.cs +> ``` +> +> Test cases: +> - Simple single handler +> - Multiple handlers on one class +> - Handlers with different signatures (void, ValueTask, ValueTask) +> - Nested classes +> - Generic executors +> - Inheritance chains (Executor -> CustomBase -> Concrete) +> - Class-level `[SendsMessage]`/`[YieldsMessage]` attributes +> - Manual `ConfigureRoutes` present (should skip generation) +> - Invalid signatures (should produce diagnostics) + +### AC-16: Integration Tests +> - Port existing `ReflectingExecutor` test cases to use `[MessageHandler]` +> - Verify generated routes match reflection-discovered routes + +### AC-17: Files to Create +> | Path | Purpose | +> |------|---------| +> | `dotnet/src/Microsoft.Agents.AI.Workflows.Generators/Microsoft.Agents.AI.Workflows.Generators.csproj` | Generator project | +> | `dotnet/src/Microsoft.Agents.AI.Workflows.Generators/ExecutorRouteGenerator.cs` | Main generator | +> | `dotnet/src/Microsoft.Agents.AI.Workflows.Generators/Models/ExecutorInfo.cs` | Data model | +> | `dotnet/src/Microsoft.Agents.AI.Workflows.Generators/Models/HandlerInfo.cs` | Data model | +> | `dotnet/src/Microsoft.Agents.AI.Workflows.Generators/Analysis/SyntaxDetector.cs` | Syntax analysis | +> | `dotnet/src/Microsoft.Agents.AI.Workflows.Generators/Analysis/SemanticAnalyzer.cs` | Semantic analysis | +> | `dotnet/src/Microsoft.Agents.AI.Workflows.Generators/Generation/SourceBuilder.cs` | Code gen | +> | `dotnet/src/Microsoft.Agents.AI.Workflows.Generators/Diagnostics/DiagnosticDescriptors.cs` | Diagnostics | +> | `dotnet/src/Microsoft.Agents.AI.Workflows/Attributes/MessageHandlerAttribute.cs` | Handler attribute | +> | `dotnet/src/Microsoft.Agents.AI.Workflows/Attributes/SendsMessageAttribute.cs` | Class-level send | +> | `dotnet/src/Microsoft.Agents.AI.Workflows/Attributes/YieldsMessageAttribute.cs` | Class-level yield | +> | `dotnet/tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/*.cs` | Generator tests | + +### AC-18: Files to Modify +> | Path | Changes | +> |------|---------| +> | `dotnet/src/Microsoft.Agents.AI.Workflows/Microsoft.Agents.AI.Workflows.csproj` | Add generator reference | +> | `dotnet/src/Microsoft.Agents.AI.Workflows/Reflection/ReflectingExecutor.cs` | Add `[Obsolete]` | +> | `dotnet/src/Microsoft.Agents.AI.Workflows/Reflection/IMessageHandler.cs` | Add `[Obsolete]` | +> | `dotnet/Microsoft.Agents.sln` | Add new projects | + +### AC-19: Example Usage (End State) +> ```csharp +> [SendsMessage(typeof(PollToken))] +> public partial class MyChatExecutor : ChatProtocolExecutor +> { +> [MessageHandler] +> private async ValueTask HandleQueryAsync( +> ChatQuery query, IWorkflowContext ctx, CancellationToken ct) +> { +> // Return type automatically inferred as output +> return new ChatResponse(...); +> } +> +> [MessageHandler(Yield = [typeof(StreamChunk)], Send = [typeof(InternalMessage)])] +> private void HandleStream(StreamRequest req, IWorkflowContext ctx) +> { +> // Explicit Yield/Send for complex handlers +> } +> } +> ``` +> +> Generated: +> ```csharp +> partial class MyChatExecutor +> { +> protected override RouteBuilder ConfigureRoutes(RouteBuilder routeBuilder) +> { +> routeBuilder = base.ConfigureRoutes(routeBuilder); +> return routeBuilder +> .AddHandler(this.HandleQueryAsync) +> .AddHandler(this.HandleStream); +> } +> +> protected override ISet ConfigureSentTypes() +> { +> var types = base.ConfigureSentTypes(); +> types.Add(typeof(PollToken)); +> types.Add(typeof(InternalMessage)); // From handler attribute +> return types; +> } +> +> protected override ISet ConfigureYieldTypes() +> { +> var types = base.ConfigureYieldTypes(); +> types.Add(typeof(ChatResponse)); // From return type +> types.Add(typeof(StreamChunk)); // From handler attribute +> return types; +> } +> } +> ``` + +## Constraints (verbatim) + +### C-1: Target Framework +> Target `netstandard2.0` + +### C-2: Roslyn Version +> Reference `Microsoft.CodeAnalysis.CSharp` 4.8.0+ + +### C-3: Analyzer Packaging +> Set `IsRoslynComponent=true`, `EnforceExtendedAnalyzerRules=true` +> Package as analyzer in `analyzers/dotnet/cs` + +### C-4: Migration Strategy +> Migration: Clean break - requires direct `Executor` inheritance (not `ReflectingExecutor`) + +### C-5 [inferred] +> From: "Handler accessibility: Any (private, protected, internal, public)" +> +> Inference: The source generator must discover handler methods regardless of accessibility level. + +### C-6 [inferred] +> From: "Class has `partial` modifier" (detection criteria) +> +> Inference: The containing executor class MUST be declared `partial` for the source generator to emit code into it. + +## Additional Context (verbatim) +``` +## Design Decisions (Confirmed) + +- **Attribute syntax**: Inline properties on `[MessageHandler(Yield=[...], Send=[...])]` +- **Class-level attributes**: Generate `ConfigureSentTypes()`/`ConfigureYieldTypes()` from `[SendsMessage]`/`[YieldsMessage]` +- **Migration**: Clean break - requires direct `Executor` inheritance (not `ReflectingExecutor`) +- **Handler accessibility**: Any (private, protected, internal, public) +``` + +## Sign-off +- [x] Every AC from the source is quoted above under its own heading. +- [x] Every constraint from the source is quoted or marked [inferred] with its source text. +- [x] Nothing in this file has been summarized or paraphrased. diff --git a/.designs/1/synthesis-checklist.md b/.designs/1/synthesis-checklist.md new file mode 100644 index 0000000000..4efc0fc692 --- /dev/null +++ b/.designs/1/synthesis-checklist.md @@ -0,0 +1,50 @@ +# synthesis-checklist.md — Pre-Synthesis Source Re-grounding + +Re-grounded from source.md at 2026-05-12T03:58Z. + +## AC Re-grounding Table + +| ID | Verbatim text (RE-COPY from source.md) | Clauses (enumerated) | Each clause satisfied by which component? | +|-------|---------------------------------------|----------------------|-------------------------------------------| +| AC-1 | "dotnet/src/Microsoft.Agents.AI.Workflows.Generators/ ├── Microsoft.Agents.AI.Workflows.Generators.csproj ├── ExecutorRouteGenerator.cs ├── Models/ │ ├── ExecutorInfo.cs │ └── HandlerInfo.cs ├── Analysis/ │ ├── SyntaxDetector.cs │ └── SemanticAnalyzer.cs ├── Generation/ │ └── SourceBuilder.cs └── Diagnostics/ └── DiagnosticDescriptors.cs" | (i) .csproj exists (ii) ExecutorRouteGenerator.cs exists (iii) Models/ExecutorInfo.cs exists (iv) Models/HandlerInfo.cs exists (v) Analysis/SyntaxDetector.cs exists (vi) Analysis/SemanticAnalyzer.cs exists (vii) Generation/SourceBuilder.cs exists (viii) Diagnostics/DiagnosticDescriptors.cs exists | (i) Generator project — EXISTS (ii) Generator project — EXISTS (iii) Generator project — EXISTS (iv) Generator project — EXISTS (v) Generator project — DOES NOT EXIST (detection inline in ExecutorRouteGenerator via ForAttributeWithMetadataName) (vi) Generator project — EXISTS (vii) Generator project — EXISTS (viii) Generator project — EXISTS | +| AC-2 | "Target netstandard2.0, Reference Microsoft.CodeAnalysis.CSharp 4.8.0+, Set IsRoslynComponent=true, EnforceExtendedAnalyzerRules=true, Package as analyzer in analyzers/dotnet/cs" | (i) TFM is netstandard2.0 (ii) Roslyn reference >= 4.8.0 (iii) IsRoslynComponent=true (iv) EnforceExtendedAnalyzerRules=true (v) analyzer pack path | (i) .csproj line 5 — YES (ii) .csproj line 49 — DEVIATION: uses 4.4.0 not 4.8.0 (deliberate for broader SDK compatibility) (iii) .csproj line 16 — YES (iv) .csproj line 17 — YES (v) .csproj line 55 — YES | +| AC-3 | "[AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = false)] public sealed class MessageHandlerAttribute : Attribute { public Type[]? Yield { get; set; } public Type[]? Send { get; set; } }" | (i) AttributeTargets.Method (ii) AllowMultiple = false (iii) Inherited = false (iv) sealed class (v) Yield property Type[]? (vi) Send property Type[]? | (i) MessageHandlerAttribute.cs:48 — YES (ii) MessageHandlerAttribute.cs:48 — YES (iii) MessageHandlerAttribute.cs:48 — YES (iv) MessageHandlerAttribute.cs:49 — YES (v) MessageHandlerAttribute.cs:59 — YES (vi) MessageHandlerAttribute.cs:69 — YES | +| AC-4 | "[AttributeUsage(AttributeTargets.Class, AllowMultiple = true, Inherited = true)] public sealed class SendsMessageAttribute : Attribute { public Type Type { get; } public SendsMessageAttribute(Type type) => this.Type = type; }" | (i) AttributeTargets.Class (ii) AllowMultiple = true (iii) Inherited = true (iv) sealed class (v) Type property (vi) Constructor takes Type | (i) SendsMessageAttribute.cs:32 — DEVIATION: actual is Class\|Method (broader) (ii) SendsMessageAttribute.cs:32 — YES (iii) SendsMessageAttribute.cs:32 — YES (iv) SendsMessageAttribute.cs:33 — YES (v) SendsMessageAttribute.cs:38 — YES (vi) SendsMessageAttribute.cs:45 — YES | +| AC-5 | "[AttributeUsage(AttributeTargets.Class, AllowMultiple = true, Inherited = true)] public sealed class YieldsMessageAttribute : Attribute { public Type Type { get; } public YieldsMessageAttribute(Type type) => this.Type = type; }" | (i) AttributeTargets.Class (ii) AllowMultiple = true (iii) Inherited = true (iv) sealed class (v) Type property (vi) Constructor takes Type (vii) Named YieldsMessageAttribute | (i) YieldsOutputAttribute.cs:32 — DEVIATION: actual is Class\|Method (broader) (ii) YieldsOutputAttribute.cs:32 — YES (iii) YieldsOutputAttribute.cs:32 — YES (iv) YieldsOutputAttribute.cs:33 — YES (v) YieldsOutputAttribute.cs:35 — YES (vi) YieldsOutputAttribute.cs:42 — YES (vii) DEVIATION: actual name is YieldsOutputAttribute, not YieldsMessageAttribute | +| AC-6 | "Class has partial modifier. Class has at least one method with [MessageHandler] attribute" | (i) partial modifier detection (ii) [MessageHandler] method detection | (i) SemanticAnalyzer.cs:367-380 (IsPartialClass) — YES (ii) ExecutorRouteGenerator.cs:33-37 (ForAttributeWithMetadataName predicate) — YES | +| AC-7 | "Class derives from Executor (directly or transitively). Class does NOT already define ConfigureRoutes with a body. Handler method has valid signature: (TMessage, IWorkflowContext[, CancellationToken]). Handler returns void, ValueTask, or ValueTask" | (i) Executor derivation check (ii) ConfigureRoutes not already defined (iii) Valid handler signature (iv) Valid return type | (i) SemanticAnalyzer.cs:385-400 (DerivesFromExecutor) — YES (ii) SemanticAnalyzer.cs:406-418 (HasConfigureProtocolDefined) — DEVIATION: checks ConfigureProtocol not ConfigureRoutes (iii) SemanticAnalyzer.cs:462-533 (AnalyzeHandler) — YES (iv) SemanticAnalyzer.cs:539-569 (GetSignatureKind) — YES | +| AC-8 | "void Handler(T, IWorkflowContext) → AddHandler(this.Handler) ... ValueTask Handler(T, IWorkflowContext, CT) → AddHandler(this.Handler)" | (i) void sync → AddHandler (ii) void sync+CT → AddHandler (iii) ValueTask → AddHandler (iv) ValueTask+CT → AddHandler (v) TResult → AddHandler (vi) ValueTask+CT → AddHandler | (i) SourceBuilder.cs:178-189 (AppendHandlerGenericArgs) + HandlerSignatureKind.VoidSync — YES (ii) Same with CT detection — YES (iii) HandlerSignatureKind.VoidAsync — YES (iv) Same with CT — YES (v) HandlerSignatureKind.ResultSync — YES (vi) HandlerSignatureKind.ResultAsync — YES | +| AC-9 | "protected override RouteBuilder ConfigureRoutes(RouteBuilder routeBuilder) { ... } protected override ISet ConfigureSentTypes() { ... } protected override ISet ConfigureYieldTypes() { ... }" | (i) Generated ConfigureRoutes override (ii) Generated ConfigureSentTypes override (iii) Generated ConfigureYieldTypes override (iv) auto-generated header (v) #nullable enable | (i) DEVIATION: generates ConfigureProtocol(ProtocolBuilder) not ConfigureRoutes(RouteBuilder); route config via nested .ConfigureRoutes() callback (ii) DEVIATION: generates .SendsMessage() fluent calls on ProtocolBuilder, not separate method (iii) DEVIATION: generates .YieldsOutput() fluent calls on ProtocolBuilder, not separate method (iv) SourceBuilder.cs:32 — YES (v) SourceBuilder.cs:33 — YES | +| AC-10 | "Directly extends Executor → No base call. Extends executor with [MessageHandler] methods → routeBuilder = base.ConfigureRoutes(routeBuilder);. Extends executor with manual ConfigureRoutes → routeBuilder = base.ConfigureRoutes(routeBuilder);" | (i) Direct Executor extension: no base call (ii) Extension of attributed executor: base call (iii) Extension of manual override: base call | (i) SourceBuilder.cs:84 (no base call when BaseHasConfigureProtocol=false) — YES (ii) SourceBuilder.cs:79 (base.ConfigureProtocol call when BaseHasConfigureProtocol=true) — YES (iii) SemanticAnalyzer.cs:424-448 (BaseHasConfigureProtocol detection) — YES | +| AC-11 | "WFGEN001 Error Handler missing IWorkflowContext parameter ... WFGEN006 Info ConfigureRoutes already defined, handlers ignored" | (i) Handler missing IWorkflowContext → Error (ii) Invalid return type → Error (iii) Must be partial → Error (iv) Non-Executor class → Warning (v) Insufficient parameters → Error (vi) Already defined → Info | (i) DiagnosticDescriptors.cs:34-40 MAFGENWF001 — YES (ID differs) (ii) DiagnosticDescriptors.cs:45-51 MAFGENWF002 — YES (ID differs) (iii) DiagnosticDescriptors.cs:56-62 MAFGENWF003 — YES (ID differs) (iv) DiagnosticDescriptors.cs:67-73 MAFGENWF004 — YES (ID differs) (v) DiagnosticDescriptors.cs:78-84 MAFGENWF005 — YES (ID differs) (vi) DiagnosticDescriptors.cs:89-95 MAFGENWF006 — YES (ID differs, checks ConfigureProtocol not ConfigureRoutes) + EXTRA: MAFGENWF007 for static handlers | +| AC-12 | "" | (i) ProjectReference to generator (ii) OutputItemType="Analyzer" (iii) ReferenceOutputAssembly="false" | (i) Microsoft.Agents.AI.Workflows.csproj:35-39 — YES (ii) .csproj:37 — YES (iii) .csproj:38 — YES | +| AC-13 | "[Obsolete(\"Use [MessageHandler] attribute on methods in a partial class deriving from Executor. See migration guide. This type will be removed in v1.0.\", error: false)] public class ReflectingExecutor : Executor ..." | (i) [Obsolete] attribute present (ii) Message references [MessageHandler] (iii) error: false | (i) ReflectingExecutor.cs:21-22 — YES (ii) YES — message says "Use [MessageHandler] attribute..." (iii) DEVIATION: actual uses default error (false), message says "will be removed in a future version" not "v1.0" | +| AC-14 | "[Obsolete(\"Use [MessageHandler] attribute instead.\")] public interface IMessageHandler { ... }" | (i) [Obsolete] attribute on IMessageHandler (ii) [Obsolete] on IMessageHandler | (i) IMessageHandler.cs:17-18 — YES (ii) IMessageHandler.cs:42-43 — YES | +| AC-15 | "dotnet/tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/ ├── ExecutorRouteGeneratorTests.cs ├── SyntaxDetectorTests.cs ├── SemanticAnalyzerTests.cs └── TestHelpers/ └── GeneratorTestHelper.cs Test cases: Simple single handler, Multiple handlers, different signatures, Nested classes, Generic executors, Inheritance chains, Class-level attributes, Manual ConfigureRoutes, Invalid signatures" | (i) ExecutorRouteGeneratorTests.cs exists (ii) SyntaxDetectorTests.cs exists (iii) SemanticAnalyzerTests.cs exists (iv) GeneratorTestHelper.cs exists (v) Single handler test (vi) Multiple handlers test (vii) Signature variants (viii) Nested classes (ix) Generic executors (x) Inheritance chains (xi) Class-level attributes (xii) Manual override skip (xiii) Invalid signature diagnostics | (i) YES — exists (~37KB) (ii) DOES NOT EXIST (iii) DOES NOT EXIST (iv) YES — exists (v-xiii) ExecutorRouteGeneratorTests.cs covers these cases — YES (33 test methods [corrected from 67]) | +| AC-16 | "Port existing ReflectingExecutor test cases to use [MessageHandler]. Verify generated routes match reflection-discovered routes" | (i) Ported test cases (ii) Behavioral equivalence verification | (i) [UNVERIFIED — cannot run dotnet test to confirm] (ii) No behavioral equivalence tests found; all generator tests are structural (source-text verification). GAP identified by six-sigma analysis | +| AC-17 | "Files to Create: [12 files listed]" | (i-xii) Each file exists | (i) .csproj — YES (ii) ExecutorRouteGenerator.cs — YES (iii) ExecutorInfo.cs — YES (iv) HandlerInfo.cs — YES (v) SyntaxDetector.cs — DOES NOT EXIST (inline) (vi) SemanticAnalyzer.cs — YES (vii) SourceBuilder.cs — YES (viii) DiagnosticDescriptors.cs — YES (ix) MessageHandlerAttribute.cs — YES (x) SendsMessageAttribute.cs — YES (xi) YieldsMessageAttribute.cs — DEVIATION: named YieldsOutputAttribute.cs (xii) Test files — YES | +| AC-18 | "Files to Modify: Workflows.csproj (generator ref), ReflectingExecutor.cs ([Obsolete]), IMessageHandler.cs ([Obsolete]), Microsoft.Agents.sln (add projects)" | (i) .csproj has generator reference (ii) ReflectingExecutor.cs has [Obsolete] (iii) IMessageHandler.cs has [Obsolete] (iv) Solution includes projects | (i) .csproj:35-39 — YES (ii) ReflectingExecutor.cs:21-22 — YES (iii) IMessageHandler.cs:17-18, 42-43 — YES (iv) Solution is .slnx not .sln — YES (present) | +| AC-19 | "[SendsMessage(typeof(PollToken))] public partial class MyChatExecutor : ChatProtocolExecutor { [MessageHandler] ... }" | (i) Class-level SendsMessage works (ii) Return type inferred as yield (iii) Explicit Yield/Send on handler works (iv) Generated output matches expected pattern | (i) ExecutorRouteGenerator pipeline 2 — YES (ii) HandlerInfo.HasOutput + SourceBuilder — YES (iii) ExecutorRouteGenerator pipeline 1 with Send/Yield extraction — YES (iv) DEVIATION: generated code uses ConfigureProtocol not ConfigureRoutes; uses fluent .SendsMessage()/.YieldsOutput() not separate methods | + +## Fidelity Corrections Addendum + +The following corrections from verification-report.md MUST be used in design-doc.md: + +| # | Inaccurate Claim | Correction | +|---|-----------------|------------| +| 1 | "15 ADRs found" | Actual: 28 ADR files. Conclusion (none related to source generators) remains valid. | +| 2 | "ReflectingExecutor.cs is 77 lines, total ~414+" | Actual: 76 lines; total reflection subsystem is 487 lines across 6 files. | +| 3 | "67 test methods" | Actual: 33 test methods (33 [Fact] attributes). | +| 4 | "BaseHasConfigureProtocol at line 25" | Actual: line 26. | +| 5 | "2 sort locations" | Actual: 4 sort locations (lines 316, 317, 629, 661). | + +## Deviation Summary + +Key plan-vs-implementation deviations (all deliberate architectural decisions): +1. `YieldsMessageAttribute` → `YieldsOutputAttribute` (naming consistency with YieldOutputAsync) +2. `ConfigureRoutes(RouteBuilder)` → `ConfigureProtocol(ProtocolBuilder)` (unified API) +3. Separate `ConfigureSentTypes()`/`ConfigureYieldTypes()` → fluent `.SendsMessage()`/`.YieldsOutput()` on ProtocolBuilder +4. `WFGEN001-006` → `MAFGENWF001-007` (framework naming convention + extra static check) +5. `SyntaxDetector.cs` separate file → inline detection via `ForAttributeWithMetadataName` +6. `AttributeTargets.Class` → `AttributeTargets.Class | AttributeTargets.Method` (broader) +7. Roslyn 4.8.0+ → 4.4.0 (broader SDK compatibility) diff --git a/.designs/1/ux.md b/.designs/1/ux.md new file mode 100644 index 0000000000..aed13d6ca2 --- /dev/null +++ b/.designs/1/ux.md @@ -0,0 +1,122 @@ +# D3: User Experience + +## Dimension Summary + +User experience for a source generator is mediated through: (1) the attribute-based declaration API that developers write, (2) compile-time diagnostics that guide correct usage, (3) the migration path from `ReflectingExecutor` to the source-generated approach, and (4) IDE integration (IntelliSense, error squiggles, generated code visibility). + +--- + +## Option U1: Current UX with Comprehensive Diagnostics (RECOMMENDED) + +The existing implementation provides a full diagnostic suite (7 diagnostics, MAFGENWF001-007) covering all validation scenarios. Users get immediate feedback in their IDE when handler signatures are wrong, classes are not partial, or classes do not derive from Executor. + +**Happy path workflow:** +1. User creates a `partial class` extending `Executor` +2. User adds `[MessageHandler]` to handler methods +3. Source generator runs automatically during build/IDE typing +4. Generated `ConfigureProtocol` override appears in IDE (can be inspected via "Go to Generated Files") +5. If handler has wrong signature -> red squiggle with MAFGENWF001/002/005/007 +6. If class not partial -> red squiggle with MAFGENWF003 +7. If class not an Executor -> warning squiggle with MAFGENWF004 + +**Failure path UX:** +- MAFGENWF001 (Error): "Method 'X' marked with [MessageHandler] must have IWorkflowContext as the second parameter" -- actionable, tells user exactly what parameter to add +- MAFGENWF002 (Error): "Method 'X' marked with [MessageHandler] must return void, ValueTask, or ValueTask" -- actionable, lists valid return types +- MAFGENWF003 (Error): "Class 'X' contains [MessageHandler] methods but is not declared as partial" -- actionable, add `partial` +- MAFGENWF004 (Warning): "Method 'X' is marked with [MessageHandler] but class 'Y' does not derive from Executor" -- warning because the attribute can exist on non-executors without harm +- MAFGENWF005 (Error): "Method 'X' marked with [MessageHandler] must have at least 2 parameters (message and IWorkflowContext)" -- actionable +- MAFGENWF006 (Info): "Class 'X' already defines ConfigureProtocol; [MessageHandler] methods will be ignored" -- informational, no action needed +- MAFGENWF007 (Error): "Method 'X' marked with [MessageHandler] cannot be static" -- actionable, remove `static` + +**Trade-offs:** +- (+) Complete diagnostic coverage for all AC-7 validation rules plus the additional static check +- (+) Error messages are actionable and include the offending method/class name +- (+) MAFGENWF006 gracefully handles the coexistence scenario (manual + attributed) +- (+) Works with any handler accessibility level (private, protected, internal, public) per C-5 +- (-) No code fix providers (IDE quick-fixes like "make class partial") -- users must apply fixes manually + +**Reversibility:** Easy. + +**Constraint compliance:** +- C-4 (clean break): The `[Obsolete]` on `ReflectingExecutor` (ReflectingExecutor.cs:21-22) warns users during migration +- C-5 (any accessibility): Verified -- SemanticAnalyzer does not check accessibility modifiers +- C-6 (partial required): MAFGENWF003 enforces this + +**Dependencies produced:** Diagnostic message text consumed by D5 (security -- information leakage review). +**Dependencies required:** Attribute shapes from D1 (API), validation rules from D2 (data model). + +**Risks:** +- Severity: Low. No code fix providers means slightly higher friction during migration. +- Mitigation: The `[Obsolete]` message on ReflectingExecutor already points users to the migration path. + +--- + +## Option U2: Add Roslyn Code Fix Providers + +Implement `CodeFixProvider` classes for the most common diagnostics: +- MAFGENWF003: Auto-add `partial` modifier +- MAFGENWF005: Auto-add `IWorkflowContext` parameter stub +- MAFGENWF007: Auto-remove `static` modifier + +**Trade-offs:** +- (+) One-click fixes in IDE reduce migration friction +- (+) Standard Roslyn practice for analyzer+generator packages +- (-) Code fix providers are a separate assembly that must also target netstandard2.0 +- (-) Increases scope: new project, new tests, packaging complexity +- (-) Not in any AC or constraint -- pure scope expansion + +**Reversibility:** Easy -- code fixes are additive. + +**Constraint compliance:** C-3 applies (must package as analyzer). + +**Dependencies produced:** Code fix assembly consumed by D6 (integration -- NuGet packaging). +**Dependencies required:** Diagnostic descriptors from D1 (API). + +**Risks:** +- Severity: Low. Adds scope not in the plan. +- Mitigation: Could be a follow-up iteration. + +--- + +## Option U3: Migration Guide Documentation + +Create a migration guide document that walks users through converting `ReflectingExecutor` subclasses to source-generated executors. + +**Trade-offs:** +- (+) Reduces confusion during the clean-break migration (C-4) +- (+) The `[Obsolete]` message on ReflectingExecutor.cs:21-22 says "See migration guide" -- so a guide is implicitly promised +- (-) Documentation is out of scope for this design (no AC covers it) +- (-) Docs can become stale + +**Reversibility:** Easy. + +**Constraint compliance:** All constraints met. + +**Dependencies produced:** Documentation consumed by D6 (integration). +**Dependencies required:** Final attribute API from D1. + +**Risks:** +- Severity: Low. + +--- + +## Option U4: Add Source Generator Telemetry/Logging + +Add diagnostic logging to the source generator to help debug issues when generation silently fails or produces unexpected output. + +**REJECTED: violates C-1 (netstandard2.0 restrictions)** + +Source generators running in the Roslyn compiler host have very limited access to logging/telemetry infrastructure. The netstandard2.0 constraint (C-1) and the analyzer sandbox further restrict available APIs. Roslyn's recommended approach is diagnostics (which the generator already uses via MAFGENWF001-007). + +**Trade-offs:** +- (+) Would help debug complex inheritance chains +- (-) No reliable logging channel available in analyzer context +- (-) Adds dependency on logging framework incompatible with analyzer hosting + +**Reversibility:** Moderate. + +--- + +## Summary + +**Recommended option: U1** -- The current UX with 7 diagnostics provides comprehensive coverage. Code fix providers (U2) and migration guides (U3) are valuable follow-ups but are out of scope for the current design. diff --git a/.designs/1/verification-report.md b/.designs/1/verification-report.md new file mode 100644 index 0000000000..a2d773b4b5 --- /dev/null +++ b/.designs/1/verification-report.md @@ -0,0 +1,158 @@ +# Fidelity Verification Report + +## Summary +- Total claims checked: 98 +- Verified: 86 +- Inaccurate: 12 (with corrections) +- Unverifiable: 0 (excluded) + +## Claim Details + +| # | Claim | Source File | Classification | Correction (if inaccurate) | +|---|-------|------------|----------------|---------------------------| +| 1 | `Executor` is at `Executor.cs:164` with signature `public abstract class Executor : IIdentified` | codebase-snapshot.md | VERIFIED | Confirmed at line 164 | +| 2 | `ConfigureProtocol` abstract method at `Executor.cs:216` | codebase-snapshot.md | VERIFIED | Confirmed at line 216: `protected abstract ProtocolBuilder ConfigureProtocol(ProtocolBuilder protocolBuilder);` | +| 3 | Executor constructor at line 179 with signature `protected Executor(string id, ExecutorOptions? options = null, bool declareCrossRunShareable = false)` | codebase-snapshot.md | VERIFIED | Confirmed at line 179 | +| 4 | `Executor` at line 382, `Executor` at line 407 | codebase-snapshot.md | VERIFIED | `Executor` at line 382, `Executor` at line 407 | +| 5 | `ReflectingExecutor` at `ReflectingExecutor.cs:23-27` | codebase-snapshot.md | VERIFIED | Lines 23-27 match: `public class ReflectingExecutor<...> : Executor where TExecutor : ReflectingExecutor` | +| 6 | `ReflectingExecutor` already marked `[Obsolete]` at lines 21-22 | codebase-snapshot.md | VERIFIED | Lines 21-22: `[Obsolete("Use [MessageHandler] attribute on methods in a partial class deriving from Executor. " + "This type will be removed in a future version.")]` | +| 7 | Obsolete message: `"Use [MessageHandler] attribute on methods in a partial class deriving from Executor. This type will be removed in a future version."` | codebase-snapshot.md | VERIFIED | Exact match at lines 21-22 | +| 8 | `ReflectingExecutor` overrides `ConfigureProtocol(ProtocolBuilder)` (NOT `ConfigureRoutes`) | codebase-snapshot.md | VERIFIED | Line 36: `protected override ProtocolBuilder ConfigureProtocol(ProtocolBuilder protocolBuilder)` | +| 9 | `RouteBuilder` at `RouteBuilder.cs`, `AddHandlerInternal(Type, MessageHandlerF, Type?, bool)` at line 51-89 | codebase-snapshot.md | VERIFIED | Line 51: `internal RouteBuilder AddHandlerInternal(Type messageType, MessageHandlerF handler, Type? outputType, bool overwrite = false)` through line 89 | +| 10 | 8 public `AddHandler` overloads on RouteBuilder | codebase-snapshot.md | VERIFIED | Verified by reading RouteBuilder.cs -- public overloads exist for Action/Func combinations with/without CT and with/without TResult | +| 11 | `ProtocolBuilder` at `ProtocolBuilder.cs` with methods `SendsMessage()`, `YieldsOutput()`, `ConfigureRoutes(Action)`, `RouteBuilder` property | codebase-snapshot.md | VERIFIED | `SendsMessage()` at line 88, `YieldsOutput()` at line 117, `ConfigureRoutes` at line 150, `RouteBuilder` property at line 143 | +| 12 | `IWorkflowContext` at `IWorkflowContext.cs` with methods `SendMessageAsync()`, `YieldOutputAsync()` | codebase-snapshot.md | VERIFIED | `SendMessageAsync` at line 35, `YieldOutputAsync` at line 49 | +| 13 | `MessageHandlerAttribute` at `Attributes/MessageHandlerAttribute.cs:48-70` with `AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = false)` | codebase-snapshot.md | VERIFIED | Line 48: `[AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = false)]`, line 49: `public sealed class MessageHandlerAttribute : Attribute`, properties Yield at line 59, Send at line 69 | +| 14 | `SendsMessageAttribute` at `Attributes/SendsMessageAttribute.cs:32-49` with `AttributeTargets.Class \| AttributeTargets.Method` | codebase-snapshot.md | VERIFIED | Line 32: `[AttributeUsage(AttributeTargets.Class \| AttributeTargets.Method, AllowMultiple = true, Inherited = true)]`, line 33: `public sealed class SendsMessageAttribute : Attribute` | +| 15 | `YieldsOutputAttribute` at `Attributes/YieldsOutputAttribute.cs:32-49` with `AttributeTargets.Class \| AttributeTargets.Method` | codebase-snapshot.md | VERIFIED | Line 32: `[AttributeUsage(AttributeTargets.Class \| AttributeTargets.Method, AllowMultiple = true, Inherited = true)]`, line 33: `public sealed class YieldsOutputAttribute : Attribute` | +| 16 | `IMessageHandler` at `Reflection/IMessageHandler.cs` already marked `[Obsolete]` | codebase-snapshot.md | VERIFIED | Lines 17-18 and 42-43 both have `[Obsolete]` | +| 17 | `ExecutorRouteGenerator` at `ExecutorRouteGenerator.cs:22-161` with `[Generator] public sealed class ExecutorRouteGenerator : IIncrementalGenerator` | codebase-snapshot.md | VERIFIED | Line 21-22: `[Generator] public sealed class ExecutorRouteGenerator : IIncrementalGenerator`, file ends at line 161 | +| 18 | Three pipelines in ExecutorRouteGenerator | codebase-snapshot.md | VERIFIED | Pipeline 1 at lines 32-37 (MessageHandler), Pipeline 2 at lines 40-45 (SendsMessage), Pipeline 3 at lines 48-53 (YieldsOutput) | +| 19 | Diagnostic IDs `MAFGENWF001-007` with specific severities and conditions | codebase-snapshot.md | VERIFIED | All 7 diagnostics confirmed in DiagnosticDescriptors.cs with matching IDs, severities, and conditions | +| 20 | `MAFGENWF001` (Error) at DiagnosticDescriptors.cs:34-40 | api.md, audit.md | VERIFIED | Lines 34-40 match exactly | +| 21 | `MAFGENWF002` (Error) at DiagnosticDescriptors.cs:45-51 | api.md, audit.md | VERIFIED | Lines 45-51 match exactly | +| 22 | `MAFGENWF003` (Error) at DiagnosticDescriptors.cs:56-62 | api.md, audit.md | VERIFIED | Lines 56-62 match exactly | +| 23 | `MAFGENWF004` (Warning) at DiagnosticDescriptors.cs:67-73 | api.md, audit.md | VERIFIED | Lines 67-73 match exactly | +| 24 | `MAFGENWF005` (Error) at DiagnosticDescriptors.cs:78-84 | api.md, audit.md | VERIFIED | Lines 78-84 match exactly | +| 25 | `MAFGENWF006` (Info) at DiagnosticDescriptors.cs:89-95 | api.md, audit.md | VERIFIED | Lines 89-95 match exactly | +| 26 | `MAFGENWF007` (Error) at DiagnosticDescriptors.cs (not in plan) | api.md, audit.md | VERIFIED | Lines 100-106: `HandlerCannotBeStatic` with id "MAFGENWF007" | +| 27 | `SemanticAnalyzer.cs:462` (`AnalyzeHandler`) does not check accessibility (C-5 claim) | api.md | VERIFIED | `AnalyzeHandler` method at line 462 -- checks static, parameter count, IWorkflowContext, CT, return type. No accessibility check. | +| 28 | `SemanticAnalyzer.cs:367-380` (`IsPartialClass`) checks for `partial` | api.md | VERIFIED | Lines 367-380: `IsPartialClass` method iterates `DeclaringSyntaxReferences`, checks `SyntaxKind.PartialKeyword` | +| 29 | `SemanticAnalyzer.cs:539-568` (`GetSignatureKind`) | api.md | VERIFIED | Lines 539-569: `GetSignatureKind` method | +| 30 | `SourceBuilder.cs:178-189` (`AppendHandlerGenericArgs`) | api.md | INACCURATE | Method `AppendHandlerGenericArgs` is at lines 178-189 in the file. However, the method name claim is correct. The line range is exactly correct. VERIFIED on re-check. | +| 31 | `HandlerInfo.cs:10` for `VoidSync` in `HandlerSignatureKind` enum | api.md | VERIFIED | Line 10: `VoidSync` (within comments/docs, actual enum value. The enum `HandlerSignatureKind` starts at line 8, `VoidSync` at line 11). Close enough within ~1 line. | +| 32 | `HandlerInfo.cs:34` for record type | api.md, data.md | VERIFIED | Line 34: `internal sealed record HandlerInfo(` | +| 33 | `ExecutorInfo.cs:18` for record type | api.md, data.md | VERIFIED | Line 18: `internal sealed record ExecutorInfo(` | +| 34 | Solution file is `dotnet/agent-framework-dotnet.slnx` | codebase-snapshot.md, integration.md | VERIFIED | Confirmed by examining the codebase | +| 35 | Test file `ExecutorRouteGeneratorTests.cs` is ~37KB | codebase-snapshot.md, multiple files | VERIFIED | Actual size: 37498 bytes (~37KB) | +| 36 | Test files: `GeneratorTestHelper.cs` and `SyntaxTreeFluentExtensions.cs` exist | codebase-snapshot.md | VERIFIED | Both files confirmed present | +| 37 | `SyntaxDetector.cs` does NOT exist as separate file | codebase-snapshot.md, data.md, audit.md | VERIFIED | Confirmed -- no such file in the generator project | +| 38 | `ForAttributeWithMetadataName` used at `ExecutorRouteGenerator.cs:33-53` | data.md, scale.md | VERIFIED | Three `ForAttributeWithMetadataName` calls at lines 33, 41, 49 | +| 39 | `.csproj:5` targets `netstandard2.0` | data.md, audit.md | VERIFIED | Line 5: `netstandard2.0` | +| 40 | `.csproj:49` references Roslyn `4.4.0` | data.md, audit.md, conflicts.md | VERIFIED | Line 49: `VersionOverride="4.4.0"` | +| 41 | `.csproj` comment at lines 45-48 explains the 4.4.0 choice | data.md, conflicts.md | VERIFIED | Lines 45-48 contain the rationale comment | +| 42 | `InjectIsExternalInitOnLegacy` in .csproj | data.md | VERIFIED | Line 13: `true` | +| 43 | Model files: `MethodAnalysisResult.cs`, `ClassProtocolInfo.cs`, `AnalysisResult.cs`, `DiagnosticInfo.cs`, `DiagnosticLocationInfo.cs`, `ProtocolAttributeKind.cs`, `ImmutableEquatableArray.cs`, `EquatableArray.cs` all exist in `Models/` | data.md | VERIFIED | All files confirmed present via `find` | +| 44 | `HandlerSignatureKind` enum at `HandlerInfo.cs:8-21` | data.md | VERIFIED | Enum `HandlerSignatureKind` at lines 8-21 | +| 45 | `ExecutorInfo.cs:25` for `BaseHasConfigureProtocol` | data.md | INACCURATE | `BaseHasConfigureProtocol` is at line 26 (record parameter), not line 25. Line 25 is `ImmutableEquatableArray Handlers,` | +| 46 | `SemanticAnalyzer.cs:110-189` for `CombineHandlerMethodResults` | data.md, scale.md | VERIFIED | Method starts at line 110, ends at line 189 | +| 47 | `SemanticAnalyzer.cs:385-400` for `DerivesFromExecutor` | audit.md | VERIFIED | Lines 385-400: `DerivesFromExecutor` method | +| 48 | `SemanticAnalyzer.cs:406-418` for `HasConfigureProtocolDefined` | audit.md | VERIFIED | Lines 406-418: `HasConfigureProtocolDefined` method | +| 49 | `SemanticAnalyzer.cs:424-448` for `BaseHasConfigureProtocol` | data.md | VERIFIED | Lines 424-448: `BaseHasConfigureProtocol` method | +| 50 | `SemanticAnalyzer.cs:462-533` for handler parameter validation | audit.md | VERIFIED | `AnalyzeHandler` at lines 462-533 | +| 51 | `SourceBuilder.cs:32` for `// ` | audit.md, security.md | VERIFIED | Line 32: `sb.AppendLine("// ");` | +| 52 | `SourceBuilder.cs:33` for `#nullable enable` | audit.md | VERIFIED | Line 33: `sb.AppendLine("#nullable enable");` | +| 53 | `SourceBuilder.cs:66` for `partial class MyExecutor` | audit.md | VERIFIED | Line 66: `sb.AppendLine($"{indent}partial class {info.ClassName}{info.GenericParameters}");` | +| 54 | `SourceBuilder.cs:72` for generated `ConfigureProtocol` override | audit.md, api.md | VERIFIED | Line 72: `sb.AppendLine($"{memberIndent}protected override ProtocolBuilder ConfigureProtocol(ProtocolBuilder protocolBuilder)");` | +| 55 | `SourceBuilder.cs:79` for `base.ConfigureProtocol(protocolBuilder)` | audit.md | VERIFIED | Line 79: `sb.Append($"{bodyIndent}return base.ConfigureProtocol(protocolBuilder)");` | +| 56 | `SourceBuilder.cs:84` for `return protocolBuilder` | audit.md | VERIFIED | Line 84: `sb.Append($"{bodyIndent}return protocolBuilder");` | +| 57 | `SourceBuilder.cs:134` for `.ConfigureRoutes` callback | audit.md | VERIFIED | Line 134: `sb.AppendLine(".ConfigureRoutes(ConfigureRoutes);");` | +| 58 | `SourceBuilder.cs:149` and `161` for `this.MethodName` | security.md | VERIFIED | Line 149: `sb.AppendLine($"(this.{handler.MethodName});");`, Line 161: `sb.Append($"(this.{handler.MethodName})");` -- actually 161, not exactly 149/161 but within range | +| 59 | `SourceBuilder.cs:206` for `.SendsMessage()` | audit.md | INACCURATE | `.SendsMessage()` is emitted at line 206 of SourceBuilder.cs. Let me check: `GenerateConfigureSentTypes` method starts at line 198. Line 206: `sb.AppendLine($".SendsMessage<{type}>()");`. Correct. | +| 60 | `SourceBuilder.cs:235` for `.YieldsOutput()` | audit.md | INACCURATE | `GenerateConfigureYieldTypes` method starts at line 227. `.YieldsOutput()` is at line 235: `sb.AppendLine($".YieldsOutput<{type}>()");`. Correct. | +| 61 | `Microsoft.Agents.AI.Workflows.csproj:35-39` for generator ProjectReference | integration.md, audit.md | VERIFIED | Lines 35-39 contain the generator ProjectReference with `OutputItemType="Analyzer"`, `ReferenceOutputAssembly="false"`, `GlobalPropertiesToRemove="TargetFramework"` | +| 62 | `Microsoft.Agents.AI.Workflows.csproj:36` for ProjectReference Include | audit.md | VERIFIED | Line 36 is the `` | +| 66 | `.csproj:16` for `IsRoslynComponent=true` | integration.md, audit.md | VERIFIED | Line 16 | +| 67 | `.csproj:17` for `EnforceExtendedAnalyzerRules=true` | integration.md, audit.md | VERIFIED | Line 17 | +| 68 | `.csproj:55` for `PackagePath="analyzers/dotnet/cs"` | integration.md, security.md | VERIFIED | Line 55 | +| 69 | `.csproj:41` for `DevelopmentDependency=true` | security.md | VERIFIED | Line 41 | +| 70 | `.csproj:21` for `SuppressDependenciesWhenPacking=true` | security.md | VERIFIED | Line 21 | +| 71 | `INamedTypeSymbol.ToDisplayString(SymbolDisplayFormat.FullyQualifiedFormat)` at `SemanticAnalyzer.cs:234, 507, 518` | security.md | VERIFIED | Line 234: `typeSymbol.ToDisplayString(SymbolDisplayFormat.FullyQualifiedFormat)`, Line 506-507: `inputType.ToDisplayString(SymbolDisplayFormat.FullyQualifiedFormat)`, Line 518: `namedReturn.TypeArguments[0].ToDisplayString(SymbolDisplayFormat.FullyQualifiedFormat)` | +| 72 | 15 ADRs found when searching `find docs/decisions -name "0*.md"` | codebase-snapshot.md | INACCURATE | Actual count is 28 ADR files matching `0*.md`, not 15 | +| 73 | No prior designs found; only `.designs/1/` exists | codebase-snapshot.md | VERIFIED | `ls .designs/` shows only `1` | +| 74 | `SemanticAnalyzer.cs:316, 629` for sorted type arrays | scale.md | INACCURATE | Line 316 corresponds to `sendTypes.Sort(StringComparer.Ordinal);` (correct). Line 629 corresponds to `builder.Sort(StringComparer.Ordinal);` in `ExtractTypeArray` (correct). However, scale.md says `SemanticAnalyzer.cs:316, 629` as two sort locations. There is also sorting at line 661. The claim mentions only two; there are actually three sort sites (lines 316, 317, 629, 661). Lines 316-317 are for CombineOutputOnlyResults send/yield types. | +| 75 | `SemanticAnalyzer.cs:47-104` for `AnalyzeHandlerMethod` | scale.md, elevation_assessment.md | VERIFIED | Method starts at line 47, ends at line 104 | +| 76 | `ExecutorRouteGenerator.cs:57-65` for `Collect().Combine()` merge step | scale.md | INACCURATE | The `Collect().Combine()` patterns are at lines 57-59 (protocol combining) and 62-65 (method+protocol combining). The claim says 57-65, which is a span covering both -- this is approximately correct. | +| 77 | `ExecutorRouteGenerator.cs:94-126` for `CombineAllResults` | scale.md | VERIFIED | `CombineAllResults` method at lines 94-126 | +| 78 | `ExecutorRouteGenerator.cs:33-37` for ForAttributeWithMetadataName predicate filtering `MethodDeclarationSyntax` | elevation_assessment.md | VERIFIED | Lines 33-37: ForAttributeWithMetadataName with `node is MethodDeclarationSyntax` predicate | +| 79 | `ReflectingExecutor.cs:36-75` for ConfigureProtocol override | elevation_assessment.md | VERIFIED | Lines 36-75: `ConfigureProtocol` override method through closing brace. Actual: lines 36-75 is the full method including the handler iteration. | +| 80 | `RouteBuilderExtensions.cs:47-78` for `GetHandlerInfos` | elevation_assessment.md | VERIFIED | Lines 47-78: `GetHandlerInfos` method | +| 81 | Reflection file line counts: ReflectingExecutor.cs (77 lines), IMessageHandler.cs (55 lines), RouteBuilderExtensions.cs (79 lines), MessageHandlerInfo.cs (149 lines), ReflectionExtensions.cs (54 lines), ValueTaskTypeErasure.cs (claimed to exist) | elevation_assessment.md | INACCURATE | Actual: ReflectingExecutor.cs=76 (not 77), IMessageHandler.cs=55 (correct), RouteBuilderExtensions.cs=79 (correct), MessageHandlerInfo.cs=149 (correct), ReflectionExtensions.cs=54 (correct), ValueTaskTypeErasure.cs=74 (exists). Total is 487, not "~414+". Elevation says "~6 files, ~414+ lines removed" but actual total is 487 lines. | +| 82 | `MessageHandlerInfo.cs:16-149` for `MessageHandlerInfo` struct | elevation_assessment.md | VERIFIED | Line 16: `internal readonly struct MessageHandlerInfo`, line count 149. | +| 83 | `ExecutorRouteGenerator.cs:24-26` for hardcoded fully-qualified attribute names | dependencies.md | VERIFIED | Lines 24-26: three `const string` definitions for attribute names | +| 84 | `ReflectingExecutor` ConfigureProtocol lines 48-58 for auto send/yield option checks | six_sigma_gaps.md | INACCURATE | The auto-send check `Options.AutoSendMessageHandlerResultObject` is at line 50, auto-yield check is at line 55. The range 48-58 covers lines from the `if (handlerInfo.OutType != null)` block which starts at line 48. However, the claim says "lines 48-58" -- let me recheck. The actual code at lines 48-58 in ReflectingExecutor.cs covers the conditional output type registration block. This is correct. VERIFIED on re-check. | +| 85 | `ProtocolBuilder.Build()` at line 156-173 with AutoYield at lines 167-169 | six_sigma_gaps.md | VERIFIED | Build method at lines 156-173. `AutoYieldOutputHandlerResultObject` check at lines 167-169: `if (options.AutoYieldOutputHandlerResultObject) { yieldTypes.UnionWith(router.DefaultOutputTypes); }` | +| 86 | `ExecutorInfo.cs:37` for `ShouldGenerateYieldedOutputRegistrations` | six_sigma_gaps.md | VERIFIED | Line 37: `public bool ShouldGenerateYieldedOutputRegistrations => !this.ClassYieldTypes.IsEmpty \|\| this.HasHandlerWithYieldTypes;` | +| 87 | `ExecutorInfo` lines 72-75 for HasOutput check within HasHandlerWithYieldTypes | six_sigma_gaps.md | VERIFIED | Lines 72-75 within `HasHandlerWithYieldTypes`: `if (handler.HasOutput) { return true; }` | +| 88 | Test file has 67 test methods | six_sigma_gaps.md | INACCURATE | Actual count: 33 `[Fact]` attributes and 0 `[Theory]` attributes = 33 test methods, not 67 | +| 89 | `ReflectionSmokeTest.cs` exists at `dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/ReflectionSmokeTest.cs` | six_sigma_gaps.md | VERIFIED | File exists at that path | +| 90 | `SemanticAnalyzer.AnalyzeHandler` at line 477 validates `Parameters.Length >= 2` | six_sigma_gaps.md | VERIFIED | Line 477: `if (methodSymbol.Parameters.Length < 2)` | +| 91 | `SemanticAnalyzer` line 492-493 for hasCancellationToken check | six_sigma_gaps.md | VERIFIED | Lines 492-493: `bool hasCancellationToken = methodSymbol.Parameters.Length >= 3 && methodSymbol.Parameters[2].Type.ToDisplayString() == CancellationTokenTypeName;` | +| 92 | `ExecutorRouteGenerator.GetHintName` at line 153-155 with `info.GenericParameters!.Length - 2` | six_sigma_gaps.md | INACCURATE | `GetHintName` method starts at line 131. The `Length - 2` line is at line 154. The claim says "line 153-155" which is correct for the relevant span. However, the claim describes the line numbering as "ExecutorRouteGenerator.cs line 153-155" which is accurate. VERIFIED on re-check. | +| 93 | Executor at lines 382-423 of Executor.cs already overrides ConfigureProtocol | six_sigma_gaps.md | VERIFIED | `Executor` at line 382, its `ConfigureProtocol` at line 386-393. `Executor` at line 407, its `ConfigureProtocol` at line 412-418. Full range to line 423 covers both. | +| 94 | Pipelines 2 and 3 use `predicate: static (node, _) => node is ClassDeclarationSyntax` at lines 40-53 | six_sigma_gaps.md | VERIFIED | Line 43: `predicate: static (node, _) => node is ClassDeclarationSyntax`, Line 51: same predicate | +| 95 | `SourceBuilder.cs:27-127` for `Generate()` method | elevation_assessment.md | VERIFIED | Method starts at line 27 and ends at line 127 | +| 96 | `DiagnosticDescriptors.cs:34-107` for 7 descriptors | elevation_assessment.md | VERIFIED | First descriptor at line 34, last (`HandlerCannotBeStatic`) ends at line 106. The claim says 107 which includes the closing brace of the class -- close enough. | +| 97 | `RouteBuilder.cs:51-89` for `AddHandlerInternal` | six_sigma_gaps.md | VERIFIED | Line 51: `AddHandlerInternal` method definition, through line 89 | +| 98 | `IMessageHandler.cs:19,44` for the two interface declarations | elevation_assessment.md | VERIFIED | Line 19: `public interface IMessageHandler`, Line 44: `public interface IMessageHandler` | + +## Inaccuracy Details + +### Inaccuracy 1: ADR Count +- **Claim**: "15 ADRs found" (codebase-snapshot.md) +- **Actual**: 28 ADR files match `find docs/decisions -name "0*.md"` +- **Impact**: Low. The conclusion ("none related to source generators") remains valid regardless of count. + +### Inaccuracy 2: Reflection File Total Line Count +- **Claim**: "~6 files, ~414+ lines removed" (elevation_assessment.md) +- **Actual**: 6 files, 487 total lines. ReflectingExecutor.cs is 76 lines (claimed 77). +- **Impact**: Low. The conclusion (subtraction gate passes) is strengthened by the higher actual count. + +### Inaccuracy 3: Test Method Count +- **Claim**: "67 test methods" (six_sigma_gaps.md) +- **Actual**: 33 test methods (33 `[Fact]` attributes, 0 `[Theory]` attributes) +- **Impact**: Medium. The claim of 67 test methods significantly overstates the actual test count. + +### Inaccuracy 4: ExecutorInfo.cs:25 for BaseHasConfigureProtocol +- **Claim**: "`BaseHasConfigureProtocol` boolean flag on ExecutorInfo (ExecutorInfo.cs:25)" (data.md) +- **Actual**: `BaseHasConfigureProtocol` is at line 26 in the record parameter list. Line 25 is `ImmutableEquatableArray Handlers,`. +- **Impact**: Negligible. Off by 1 line. + +### Inaccuracy 5: Sort location count +- **Claim**: "sorted type arrays (SemanticAnalyzer.cs:316, 629)" implies two sort locations (scale.md) +- **Actual**: There are four sort operations: lines 316, 317 (in CombineOutputOnlyResults), 629 (in ExtractTypeArray), and 661 (in GetClassLevelTypes). The claim mentions only two representative locations. +- **Impact**: Low. The claim is directionally correct but incomplete. + +### Inaccuracy 6: ReflectingExecutor.cs line count +- **Claim**: "ReflectingExecutor.cs (77 lines)" (elevation_assessment.md) +- **Actual**: 76 lines +- **Impact**: Negligible. Off by 1 line. + +### Inaccuracy 7: SourceBuilder.cs:206 claim +- **Re-verified**: Line 206 is `sb.AppendLine($".SendsMessage<{type}>()");` which IS correct. Reclassified as VERIFIED. + +### Inaccuracy 8: SourceBuilder.cs:235 claim +- **Re-verified**: Line 235 is `sb.AppendLine($".YieldsOutput<{type}>()");` which IS correct. Reclassified as VERIFIED. + +## Revised Summary After Re-verification +- Total claims checked: 98 +- Verified: 93 +- Inaccurate: 5 (with corrections below) + 1. ADR count: 28, not 15 (codebase-snapshot.md) + 2. Reflection files total: 487 lines, not "~414+" (elevation_assessment.md); ReflectingExecutor.cs is 76 lines, not 77 + 3. Test method count: 33, not 67 (six_sigma_gaps.md) + 4. ExecutorInfo.cs: BaseHasConfigureProtocol is at line 26, not 25 (data.md) -- negligible + 5. Sort location count: 4 locations, not 2 (scale.md) -- low impact, directionally correct diff --git a/.designs/1/verification.md b/.designs/1/verification.md new file mode 100644 index 0000000000..abf58b15a8 --- /dev/null +++ b/.designs/1/verification.md @@ -0,0 +1,36 @@ +# verification.md — AC and Constraint Verification Tables + +## Table A — Constraint Verification + +| ID | Verbatim text (copy from source.md) | My interpretation | Prohibits | Requires | Dimension owner(s) | Relaxation impact | +|------|-------------------------------------|-------------------|-----------|----------|---------------------|-------------------| +| C-1 | "Target `netstandard2.0`" | Generator must compile against netstandard2.0 to run in all Roslyn host environments (VS, dotnet build, Rider) | Targeting net6.0+ or other TFMs for the generator assembly | netstandard2.0 TFM in generator .csproj | Integration | Generator would not load in older VS or .NET SDK versions | +| C-2 | "Reference `Microsoft.CodeAnalysis.CSharp` 4.8.0+" | Must use Roslyn 4.8+ APIs for incremental generator support | Using older Roslyn APIs that lack ForAttributeWithMetadataName | PackageReference to Microsoft.CodeAnalysis.CSharp >= 4.8.0 | Integration | Older SDK versions would not support the generator | +| C-3 | "Set `IsRoslynComponent=true`, `EnforceExtendedAnalyzerRules=true` Package as analyzer in `analyzers/dotnet/cs`" | Generator project must be packaged as a Roslyn analyzer component | Packaging as a normal library reference | Correct MSBuild properties and pack path | Integration | NuGet consumers would not get the generator | +| C-4 | "Migration: Clean break - requires direct `Executor` inheritance (not `ReflectingExecutor`)" | Users must migrate from ReflectingExecutor to direct Executor inheritance with [MessageHandler] | Supporting both ReflectingExecutor and source-generated routes on the same class | Executor (or derived) as base class, partial modifier | API | Users who don't migrate would lose functionality when ReflectingExecutor is removed | +| C-5 [inferred] | "Handler accessibility: Any (private, protected, internal, public)" | Generator must discover handlers regardless of access modifier | Restricting to only public methods | Syntax/semantic analysis that ignores accessibility | API | Some handler patterns would stop working | +| C-6 [inferred] | "Class has `partial` modifier" (detection criteria) | Executor class must be declared partial for generator to emit into it | Using non-partial classes with [MessageHandler] | MAFGENWF003 diagnostic for non-partial classes | API | Generator cannot emit code into non-partial classes | + +## Table B — AC Verification + +| ID | Verbatim text (copy from source.md) | My interpretation | What evidence proves it works? | Dimension owner(s) | +|-------|-------------------------------------|-------------------|-------------------------------|---------------------| +| AC-1 | "dotnet/src/Microsoft.Agents.AI.Workflows.Generators/ ├── Microsoft.Agents.AI.Workflows.Generators.csproj ├── ExecutorRouteGenerator.cs # Main incremental generator ├── Models/ │ ├── ExecutorInfo.cs │ └── HandlerInfo.cs ├── Analysis/ │ ├── SyntaxDetector.cs │ └── SemanticAnalyzer.cs ├── Generation/ │ └── SourceBuilder.cs └── Diagnostics/ └── DiagnosticDescriptors.cs" | Generator project must have specified directory/file structure | `find src/Microsoft.Agents.AI.Workflows.Generators -type f` matches structure; note SyntaxDetector.cs does not exist as separate file (detection is inline) | Data | +| AC-2 | "Target `netstandard2.0` Reference `Microsoft.CodeAnalysis.CSharp` 4.8.0+ Set `IsRoslynComponent=true`, `EnforceExtendedAnalyzerRules=true` Package as analyzer in `analyzers/dotnet/cs`" | Project file must have correct TFM, Roslyn reference, and analyzer packaging config | Inspect .csproj for TargetFramework, PackageReference, IsRoslynComponent, EnforceExtendedAnalyzerRules, pack path | Integration | +| AC-3 | "[AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = false)] public sealed class MessageHandlerAttribute : Attribute { public Type[]? Yield { get; set; } public Type[]? Send { get; set; } }" | MessageHandlerAttribute must exist with Yield and Send properties | Read MessageHandlerAttribute.cs and verify signature matches | API | +| AC-4 | "[AttributeUsage(AttributeTargets.Class, AllowMultiple = true, Inherited = true)] public sealed class SendsMessageAttribute : Attribute { public Type Type { get; } public SendsMessageAttribute(Type type) => this.Type = type; }" | SendsMessageAttribute must exist with Type constructor parameter; note actual also targets Method | Read SendsMessageAttribute.cs; verify signature; note AttributeTargets difference (actual: Class\|Method) | API | +| AC-5 | "[AttributeUsage(AttributeTargets.Class, AllowMultiple = true, Inherited = true)] public sealed class YieldsMessageAttribute : Attribute { public Type Type { get; } public YieldsMessageAttribute(Type type) => this.Type = type; }" | A yields-output attribute must exist; note actual name is YieldsOutputAttribute not YieldsMessageAttribute | Read YieldsOutputAttribute.cs; verify functional equivalence despite name difference | API | +| AC-6 | "Class has `partial` modifier Class has at least one method with `[MessageHandler]` attribute" | Generator syntax detection must check for partial modifier and MessageHandler attribute on methods | Unit test: class without partial → MAFGENWF003 error; class with [MessageHandler] method detected | Data | +| AC-7 | "Class derives from `Executor` (directly or transitively) Class does NOT already define `ConfigureRoutes` with a body Handler method has valid signature: `(TMessage, IWorkflowContext[, CancellationToken])` Handler returns `void`, `ValueTask`, or `ValueTask`" | Semantic validation must check Executor inheritance, existing ConfigureProtocol (evolved from ConfigureRoutes), handler signature, and return type | Unit tests for each validation rule; MAFGENWF001-007 diagnostics fire for violations | Data | +| AC-8 | "void Handler(T, IWorkflowContext) → AddHandler(this.Handler) ... ValueTask Handler(T, IWorkflowContext, CT) → AddHandler(this.Handler)" | All 6 handler signature patterns must map to correct AddHandler generic overloads | Unit tests for each signature variant; verify generated AddHandler vs AddHandler calls | API | +| AC-9 | "partial class MyExecutor { protected override RouteBuilder ConfigureRoutes(RouteBuilder routeBuilder) { ... return routeBuilder .AddHandler(this.Handler1) .AddHandler(this.Handler2); } protected override ISet ConfigureSentTypes() { ... } protected override ISet ConfigureYieldTypes() { ... } }" | Generated code must override protocol configuration with handler registrations and type declarations; note actual generates ConfigureProtocol with fluent ProtocolBuilder, not separate methods | SourceBuilder.Generate() output matches expected pattern; unit tests verify generated code compiles and registers handlers | API / Integration | +| AC-10 | "Directly extends Executor → No base call (abstract) Extends executor with [MessageHandler] methods → routeBuilder = base.ConfigureRoutes(routeBuilder); Extends executor with manual ConfigureRoutes → routeBuilder = base.ConfigureRoutes(routeBuilder);" | Generated code must correctly chain base class calls based on inheritance; actual uses ConfigureProtocol not ConfigureRoutes | Unit tests for inheritance chains; verify BaseHasConfigureProtocol flag drives base call generation | Data / Integration | +| AC-11 | "WFGEN001 Error Handler missing IWorkflowContext parameter WFGEN002 Error Handler has invalid return type WFGEN003 Error Executor with [MessageHandler] must be partial WFGEN004 Warning [MessageHandler] on non-Executor class WFGEN005 Error Handler has fewer than 2 parameters WFGEN006 Info ConfigureRoutes already defined, handlers ignored" | Diagnostics must exist for all 6 validation rules; actual uses MAFGENWF001-007 IDs (7 diagnostics, includes static handler check) | DiagnosticDescriptors.cs contains all descriptors; unit tests verify each diagnostic fires | Observability | +| AC-12 | "" | Workflows project must reference generator as an analyzer, not a library | Inspect Microsoft.Agents.AI.Workflows.csproj for analyzer ProjectReference | Integration | +| AC-13 | "[Obsolete(\"Use [MessageHandler] attribute on methods in a partial class deriving from Executor. See migration guide. This type will be removed in v1.0.\", error: false)] public class ReflectingExecutor : Executor ..." | ReflectingExecutor must be marked Obsolete with migration guidance | Read ReflectingExecutor.cs line 21-22; confirmed [Obsolete] present | API | +| AC-14 | "[Obsolete(\"Use [MessageHandler] attribute instead.\")] public interface IMessageHandler { ... }" | IMessageHandler interfaces must be marked Obsolete | Read IMessageHandler.cs; confirmed [Obsolete] present | API | +| AC-15 | "dotnet/tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/ ├── ExecutorRouteGeneratorTests.cs ├── SyntaxDetectorTests.cs ├── SemanticAnalyzerTests.cs └── TestHelpers/ └── GeneratorTestHelper.cs Test cases: Simple single handler, Multiple handlers, Handlers with different signatures, Nested classes, Generic executors, Inheritance chains, Class-level attributes, Manual ConfigureRoutes present, Invalid signatures" | Comprehensive unit tests must exist covering all listed scenarios | Test project exists; ExecutorRouteGeneratorTests.cs (~37KB) covers listed scenarios | Data / Integration | +| AC-16 | "Port existing ReflectingExecutor test cases to use [MessageHandler] Verify generated routes match reflection-discovered routes" | Integration tests must verify source-generated routes match reflection-discovered routes | Check for integration test project or test cases comparing generated vs reflected routes | Integration | +| AC-17 | "Files to Create: [12 files listed across generator, attributes, and tests]" | All listed files must exist in the codebase | `ls` each path; all generator and attribute files exist; SyntaxDetector.cs is inline not separate | Data | +| AC-18 | "Files to Modify: Microsoft.Agents.AI.Workflows.csproj (Add generator reference), ReflectingExecutor.cs (Add [Obsolete]), IMessageHandler.cs (Add [Obsolete]), Microsoft.Agents.sln (Add new projects)" | Listed files must contain the specified modifications | Verify each modification; all confirmed present in current codebase | Integration | +| AC-19 | "[SendsMessage(typeof(PollToken))] public partial class MyChatExecutor : ChatProtocolExecutor { [MessageHandler] private async ValueTask HandleQueryAsync(...) { ... } [MessageHandler(Yield = [typeof(StreamChunk)], Send = [typeof(InternalMessage)])] private void HandleStream(StreamRequest req, IWorkflowContext ctx) { ... } }" | The end-state usage pattern must work: class-level and handler-level attributes, return type inference, explicit Yield/Send | Compile-time verification via generator tests; generated ConfigureProtocol matches expected output | API / Integration |