Skip to content

Code review improvements#175

Open
mernst wants to merge 40 commits into
mainfrom
review
Open

Code review improvements#175
mernst wants to merge 40 commits into
mainfrom
review

Conversation

@mernst

@mernst mernst commented Apr 26, 2026

Copy link
Copy Markdown
Collaborator

No description provided.

@mernst
mernst requested a review from jyoo980 April 26, 2026 17:28
@coderabbitai

coderabbitai Bot commented Apr 26, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The pull request documents DefaultLlmBackend routing, retry, compaction, return, and error behavior, and adjusts final exception handling. It replaces broad Ruff suppressions with targeted rules and moves the CBMC Meta import behind TYPE_CHECKING. Line-number mismatch errors now include supplied input lines. Syntax-fixer tests update one expected mapping and reformat two test cases without changing their asserted outputs.

Possibly related PRs

Suggested reviewers: jyoo980

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 56.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (3)
translation/ast/cbmc_ast.py (2)

568-595: ⚠️ Potential issue | 🟡 Minor

Companion docstring also references ValueError — verify it's still accurate.

_validate_frees_target ultimately delegates to _validate_side_effect_free, which now raises TypeError for CallOp. The docstring at lines 569 and 581 still claims ValueError. If you update the sibling method, please update this one too so the documented contract matches what callers actually receive.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@translation/ast/cbmc_ast.py` around lines 568 - 595, Update the docstring for
_validate_frees_target to reflect the actual exception type raised by its
helpers: change the "Raises" section (and any inline mention of ValueError) to
TypeError since _validate_side_effect_free now raises TypeError for CallOp;
ensure the wording matches the current behavior of _validate_side_effect_free
and adjust any references in the docstring to "_validate_side_effect_free" or
"TypeError" so callers see the correct contract for _validate_frees_target.

529-545: ⚠️ Potential issue | 🟡 Minor

Update docstring to reflect TypeError instead of ValueError.

The method currently raises TypeError (line 544) but the docstring (line 530) and Raises: block (line 540) document ValueError. Update both the summary and exception documentation to match the implementation.

📝 Proposed docstring fix
     def _validate_side_effect_free(self, expr: Any) -> None:
-        """Raise ValueError if an expression contains a function call.
+        """Raise TypeError if an expression contains a function call.

         This is a best-effort attempt to validate that an expression is side-effect free by checking
         for the presence of function calls in an expression. Some functions are obviously
         side-effect free, but this information requires a more complicated analysis to obtain.

         Args:
             expr (Any): The expression to validate.

         Raises:
-            ValueError: Raised when a function call appears in the expression.
+            TypeError: Raised when a function call appears in the expression.
         """

Callers are not impacted: the method is used internally within the AST transformer, and Lark's transformer automatically wraps any raised exception in a VisitError. Downstream exception handlers that catch ValueError remain compatible because the exception is further converted to ValueError by CBMC_PARSER.parse() when caught as VisitError.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@translation/ast/cbmc_ast.py` around lines 529 - 545, The docstring for
_validate_side_effect_free is inconsistent: it states it raises ValueError but
the implementation raises TypeError; update the summary and the "Raises:"
section in the docstring to document TypeError instead of ValueError so the
documentation matches the actual behavior of _validate_side_effect_free (which
raises TypeError when encountering a CallOp).
verification/verification_input.py (1)

19-22: ⚠️ Potential issue | 🟡 Minor

Inconsistent docstring type — same defect being fixed at line 67.

While line 67 was corrected from strCFunction, the VerificationContext class docstring at line 20 still describes callee_specs as dict[str, FunctionSpecification], but the actual annotation at line 24 is dict[CFunction, FunctionSpecification]. Recommend fixing this in the same PR for consistency.

📝 Proposed fix
     Attributes:
-        callee_specs (dict[str, FunctionSpecification]): The specs for a function's callees.
+        callee_specs (dict[CFunction, FunctionSpecification]): The specs for a function's callees.
         global_variable_specs (dict[str, str]): The specs for global program variables.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@verification/verification_input.py` around lines 19 - 22, Docstring for
VerificationContext is inconsistent: update the callee_specs type annotation in
the class docstring from dict[str, FunctionSpecification] to dict[CFunction,
FunctionSpecification] to match the actual annotation and the change made at
line 67; ensure you mention CFunction and FunctionSpecification and also keep
the description of global_variable_specs as dict[str, str] unchanged in the same
docstring.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@models/default_llm_backend.py`:
- Around line 195-197: The Returns docstring only documents the
tuple[ConversationMessage, ...] case but the function signature and
implementation also return None; update the Returns section to list both
outcomes: a tuple[ConversationMessage, ...] when compaction succeeds and None
when no compaction was performed (matching the signature
tuple[ConversationMessage, ...] | None and the places where the function returns
None), mentioning the conditions when None is returned so the docstring matches
the implementation.

In `@test/translation/test_cbmc_parser.py`:
- Line 449: Replace the invalid noqa rule on the pytest.raises context: change
the comment `# noqa: assert-raises-exception` to the correct Ruff rule `# noqa:
PT011` on the line containing `with pytest.raises((ValueError, VisitError))` in
test_cbmc_parser.py so the pytest-raises warning is silenced with the proper
code.

---

Outside diff comments:
In `@translation/ast/cbmc_ast.py`:
- Around line 568-595: Update the docstring for _validate_frees_target to
reflect the actual exception type raised by its helpers: change the "Raises"
section (and any inline mention of ValueError) to TypeError since
_validate_side_effect_free now raises TypeError for CallOp; ensure the wording
matches the current behavior of _validate_side_effect_free and adjust any
references in the docstring to "_validate_side_effect_free" or "TypeError" so
callers see the correct contract for _validate_frees_target.
- Around line 529-545: The docstring for _validate_side_effect_free is
inconsistent: it states it raises ValueError but the implementation raises
TypeError; update the summary and the "Raises:" section in the docstring to
document TypeError instead of ValueError so the documentation matches the actual
behavior of _validate_side_effect_free (which raises TypeError when encountering
a CallOp).

In `@verification/verification_input.py`:
- Around line 19-22: Docstring for VerificationContext is inconsistent: update
the callee_specs type annotation in the class docstring from dict[str,
FunctionSpecification] to dict[CFunction, FunctionSpecification] to match the
actual annotation and the change made at line 67; ensure you mention CFunction
and FunctionSpecification and also keep the description of global_variable_specs
as dict[str, str] unchanged in the same docstring.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: eccdb65e-d659-41f7-8f45-74b00dcd49b1

📥 Commits

Reviewing files that changed from the base of the PR and between 88fda72 and b798933.

📒 Files selected for processing (24)
  • eval/get_verification_summary.py
  • eval/insert_verified_specs.py
  • models/__init__.py
  • models/default_llm_backend.py
  • specifications/__init__.py
  • specifications/llm_specification_generator.py
  • test/translation/test_cbmc_parser.py
  • translation/__init__.py
  • translation/ast/__init__.py
  • translation/ast/cbmc_ast.py
  • translation/normalization.py
  • util/__init__.py
  • util/c_function.py
  • util/execution/__init__.py
  • util/file_util.py
  • util/rust/__init__.py
  • util/spec_conversation.py
  • util/specification/__init__.py
  • util/text_util.py
  • util/tree_sitter_util.py
  • verification/__init__.py
  • verification/avocado_stub_util.py
  • verification/proof_state.py
  • verification/verification_input.py
💤 Files with no reviewable changes (2)
  • util/tree_sitter_util.py
  • verification/proof_state.py

Comment thread models/default_llm_backend.py
Comment thread test/translation/test_cbmc_parser.py Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
models/default_llm_backend.py (2)

163-172: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Fix Ruff warnings RET504 and TRY300.

As per coding guidelines, all code must pass linting with make checks yielding 0 errors or warnings. Returning the result of completion directly eliminates the unnecessary local assignment (RET504), and appending # noqa: TRY300 suppresses the try-block return warning.

🛠️ Proposed fix
         while True:
             try:
-                response = completion(
-                    model=self.model,
-                    messages=[message.to_dict() for message in messages],
-                    temperature=temperature,
-                    api_key=self.api_key,
-                    vertex_credentials=self.vertex_credentials,
-                    max_tokens=self.max_tokens,
-                    **kwargs,
-                )
-                return response
+                return completion(
+                    model=self.model,
+                    messages=[message.to_dict() for message in messages],
+                    temperature=temperature,
+                    api_key=self.api_key,
+                    vertex_credentials=self.vertex_credentials,
+                    max_tokens=self.max_tokens,
+                    **kwargs,
+                )  # noqa: TRY300
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@models/default_llm_backend.py` around lines 163 - 172, Update the completion
call in the surrounding method to return completion(...) directly instead of
assigning it to response first, resolving RET504; add the targeted # noqa:
TRY300 suppression to that return statement so make checks reports no warnings.

Sources: Coding guidelines, Linters/SAST tools


144-149: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Suppress Ruff dynamically typed expressions warnings (ANN401).

As per coding guidelines, all code must pass linting with make checks yielding 0 errors or warnings. Ruff flagged the use of Any for **kwargs and the return type. Suppressing ANN401 resolves these warnings since litellm natively utilizes dynamic types here.

🛠️ Proposed fix
     def _send_with_retry(
         self,
         messages: tuple[ConversationMessage, ...],
         temperature: float,
-        **kwargs: Any,
-    ) -> Any:
+        **kwargs: Any,  # noqa: ANN401
+    ) -> Any:  # noqa: ANN401
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@models/default_llm_backend.py` around lines 144 - 149, Add a targeted Ruff
`ANN401` suppression to `_send_with_retry`, covering both its `**kwargs: Any`
parameter and `Any` return annotation while preserving the existing signature
and litellm integration.

Sources: Coding guidelines, Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@models/default_llm_backend.py`:
- Line 39: Update the __init__ method in the default LLM backend to declare an
explicit None return type and add a targeted noqa suppression for Ruff’s FBT001
boolean-positional-argument warning, preserving the existing signature behavior
for callers.

---

Outside diff comments:
In `@models/default_llm_backend.py`:
- Around line 163-172: Update the completion call in the surrounding method to
return completion(...) directly instead of assigning it to response first,
resolving RET504; add the targeted # noqa: TRY300 suppression to that return
statement so make checks reports no warnings.
- Around line 144-149: Add a targeted Ruff `ANN401` suppression to
`_send_with_retry`, covering both its `**kwargs: Any` parameter and `Any` return
annotation while preserving the existing signature and litellm integration.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 431c465f-f0fc-425f-a38c-cdf586ff994b

📥 Commits

Reviewing files that changed from the base of the PR and between b798933 and 19c3b08.

📒 Files selected for processing (4)
  • models/default_llm_backend.py
  • test/util/test_spec_syntax_fixer.py
  • translation/ast/cbmc_ast.py
  • util/text_util.py

vertex_credentials (str | None): JSON credentials for Vertex AI; `None` otherwise.
"""

def __init__(self, model: str, use_vertex_api: bool):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Fix Ruff warnings by adding a return type and suppressing FBT001.

As per coding guidelines, all code must pass linting with make checks yielding 0 errors or warnings. Ruff flagged a missing return type (ANN204) and a boolean positional argument (FBT001). Adding -> None and # noqa: FBT001 resolves this without breaking existing callers.

🛠️ Proposed fix
-    def __init__(self, model: str, use_vertex_api: bool):
+    def __init__(self, model: str, use_vertex_api: bool) -> None:  # noqa: FBT001
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
def __init__(self, model: str, use_vertex_api: bool):
def __init__(self, model: str, use_vertex_api: bool) -> None: # noqa: FBT001
🧰 Tools
🪛 Ruff (0.15.21)

[warning] 39-39: Missing return type annotation for special method __init__

Add return type annotation: None

(ANN204)


[warning] 39-39: Boolean-typed positional argument in function definition

(FBT001)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@models/default_llm_backend.py` at line 39, Update the __init__ method in the
default LLM backend to declare an explicit None return type and add a targeted
noqa suppression for Ruff’s FBT001 boolean-positional-argument warning,
preserving the existing signature behavior for callers.

Sources: Coding guidelines, Linters/SAST tools

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants