feat: LLM based prose format conistency workflow

[hpnyaggerman]

This workflow is an implementation of an idea I described in #55. It is not perfect but it works and it has a shape I believe is viable. Up to you @OrbFrontend whether you want to merge it but I believe no one will be at a loss with this workflow in the default Orb selection.

[OrbFrontend]

@hpnyaggerman I'll give this a spin later. I actually trained a very small pov/tense classifier here as I said I would - miliseconds on a CPU for 256 tokens: https://huggingface.co/chartreuse-verte/ettin-povtense-17m, it's already being used for auto POV in image gen workflow.

I'll see if it can be integrated into the workflow to assist the agent model.

[OrbFrontend]

@hpnyaggerman

1. Can you fix the modal UI so it looks presentable?

2. What's the add element function for? Are there other use cases besides narration, speech, inner monologue, quotation that I can't think of?

3. Incompatibility with existing algormith-based format consistency:

INFO:backend.workflows.format_consistency.hooks:format-consistency: normalized draft (dialogue=quoted, narration=asterisk -> dialogue=bare, narration=asterisk)
INFO:backend.inference.client:LLM complete: model=default, tools=["prose_format_report"], tool_choice={'type': 'function', 'function': {'name': 'prose_format_report'}}
INFO:httpx:HTTP Request: POST http://localhost:5000/v1/chat/completions "HTTP/1.1 200 OK"
INFO:backend.inference.client:LLM complete: assembled keys=['tool_calls', 'finish_reason'], has_tool_calls=True, content_len=null, usage={'completion_tokens': 157, 'prompt_tokens': 556, 'total_tokens': 713, 'prompt_tokens_details': {'cached_tokens': 236}}
INFO:backend.inference.client:LLM complete: model=default, tools=["prose_format_patch"], tool_choice={'type': 'function', 'function': {'name': 'prose_format_patch'}}
INFO:httpx:HTTP Request: POST http://localhost:5000/v1/chat/completions "HTTP/1.1 200 OK"
INFO:backend.inference.client:LLM complete: assembled keys=['tool_calls', 'finish_reason'], has_tool_calls=True, content_len=null, usage={'completion_tokens': 257, 'prompt_tokens': 655, 'total_tokens': 912, 'prompt_tokens_details': {'cached_tokens': 221}}
INFO:backend.inference.client:LLM complete: model=default, tools=["prose_format_report"], tool_choice={'type': 'function', 'function': {'name': 'prose_format_report'}}
INFO:httpx:HTTP Request: POST http://localhost:5000/v1/chat/completions "HTTP/1.1 200 OK"
INFO:backend.inference.client:LLM complete: assembled keys=['tool_calls', 'finish_reason'], has_tool_calls=True, content_len=null, usage={'completion_tokens': 157, 'prompt_tokens': 556, 'total_tokens': 713, 'prompt_tokens_details': {'cached_tokens': 551}}
INFO:backend.inference.kv_tracker:KV cache report  (provider = truth; local = msgs-prefix + tools-match, template-dependent):
  director:direct_scene         provider: cached=2198/2743 tok (80.1%) [prompt_tokens_details.cached_tokens]  |  local: msgs_overlap=6602/8852c (74.6%) vs prev-turn 'director:direct_scene'; tools_MATCH
  writer                        provider: cached=2359/2507 tok (94.1%) [prompt_tokens_details.cached_tokens]  |  local: msgs_overlap=7251/7945c (91.3%) vs 'director:direct_scene'; tools_MATCH
  forced:prose_format_report    provider: cached=236/556 tok (42.4%) [prompt_tokens_details.cached_tokens]  |  local: msgs_overlap=22/1852c (1.2%) vs 'writer'; tools_DIFFER (prev=2414c, this=669c)
  forced:prose_format_patch     provider: cached=221/655 tok (33.7%) [prompt_tokens_details.cached_tokens]  |  local: msgs_overlap=35/2265c (1.5%) vs 'forced:prose_format_report'; tools_DIFFER (prev=669c, this=641c)
  forced:prose_format_report    provider: cached=551/556 tok (99.1%) [prompt_tokens_details.cached_tokens]  |  local: msgs_overlap=35/1852c (1.9%) vs 'forced:prose_format_patch'; tools_DIFFER (prev=641c, this=669c)

Re-judge's prompt is identical to judge, meaning the enforcer emitted 257 tokens that did nothing - because the algorithm-based one already fixed everything. Still the judge flagged it anyway. This is Gemma 4 31B.

KV-cache report above is another eyebrow-raising thing. modern providers have different lanes for different prefix hashes but some might configure their engine to only have a single lane per hash, and the hash can be derived from user id (This is my speculation, I know it's possible to be stingy and "tweak" by disabling reuse and limit allocated memory pages, but I can't say if anyone actually does it). This is acceptable for now, maybe I'll add a kv cache bust warning to the user and leave them a hint what's breaking in the future, and they can switch between minimal and full modes if not compatible.

[hpnyaggerman]

@OrbFrontend Will look into it tomorrow or slightly later.