Skip to content

Flm with gemma 4 has very poor ability to call tools #721

Description

@tidotua

I am using flm backend for lemonade server on Linux with Gemma 4 e4b.
And it's support for tools is very poor.

AMD Ryzen AI 9 HX PRO 375 w/ Radeon 890M × 12
Linux Mint 22.3 - Cinnamon 64-bit
7.0.0-31-generic
lemonade version 11.9.0
flm npu v1.0.3
gemma4-it-e4b-FLM 9.10GB


User

Please write a python script that writes json with "name" and "description" keys into out.txt file. Please save this python script into scrypt.py


Assistant (Coder · gemma4-it-e4b-FLM (local) · 11.9s)

Tool: write

Input:

{
  "content": "\"\"\"import json\n\ndata = {\n    \"name\": \"Example Item\",\n    \"description\": \"This is a sample description.\"\n}\n\nwith open('out.txt', 'w') as f:\n    json.dump(data, f)\n\"\"",
  "filePath": "\"scrypt.py\""
}

Output:

Wrote file successfully.

It wrote file with ["scrypt.py"] name and following invalid content:

"""import json

data = {
"name": "Example Item",
"description": "This is a sample description."
}

with open('out.txt', 'w') as f:
json.dump(data, f)
"

And later just stuck with tool calls:

Tool: write

Input:

{
  "content": "import json\n\ndata = {\n    \"name\": \"Example Item",
  "description": "This is a sample description."
}

Error:

The write tool was called with invalid arguments: SchemaError(Missing key
  at ["filePath"]).
Please rewrite the input so it satisfies the expected schema.

Assistant (Coder · gemma4-it-e4b-FLM (local) · 13.5s)

Tool: write

Input:

{
  "content": "import json\n\ndata = {\n    \"name\": \"Example Item",
  "description": "This is a sample description."
}

Error:

The write tool was called with invalid arguments: SchemaError(Missing key
  at ["filePath"]).
Please rewrite the input so it satisfies the expected schema.

response:write{value:<|"|>Wrote file successfully.<|"|>}<tool_call|>


Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions