[Bug] Wrong SQL gets auto-saved to Tool Memory without validation — Poisoned Memory problem

[SardaKartik]

Bug Description

When the LLM generates SQL and executes it successfully, save_question_tool_args
is called automatically — even if the result is semantically wrong.

A query can execute without any SQL error but still return incorrect data
(e.g. summing order_count instead of revenue). This incorrect behavior
gets saved to Tool Memory and reused for all future similar questions,
reinforcing errors over time.

Steps to Reproduce

1. Ask a question that generates new SQL
2. The SQL executes without error but returns incorrect data
3. save_question_tool_args is called automatically
4. Ask a similar question → the wrong SQL is retrieved and reused

Expected Behavior

SQL should only be saved to Tool Memory after a human confirms the result is correct.

Proposed Fix

Add an optional require_human_approval_for_memory flag.
When enabled, show a "👍 Save to memory" button and only persist
the SQL after explicit user approval.

[viphonestsong]

That's true. Theoretically, correct SQL execution does not mean the query aligns with the business logic, and I have also identified this issue. I suggest designing adjustments for this in future versions.

[SardaKartik]

@viphonestsong
Totally agree! I've already put together a fix for this in PR #1102 that adds a require_human_approval_for_memory flag — when enabled, SQL is only saved to memory after explicit user approval (👍 button), so wrong queries can never silently poison future responses. Would love any feedback on the approach!