Skip to content

docs: explain local model context sizing - #913

Open
ousamabenyounes wants to merge 2 commits into
usestrix:mainfrom
ousamabenyounes:docs/issue-286-local-context
Open

docs: explain local model context sizing#913
ousamabenyounes wants to merge 2 commits into
usestrix:mainfrom
ousamabenyounes:docs/issue-286-local-context

Conversation

@ousamabenyounes

Copy link
Copy Markdown
Contributor

Summary

  • document practical local-model context window sizes for quick, standard, and deep scans
  • add Ollama Modelfile, LM Studio, and llama.cpp context configuration guidance
  • list common context clipping symptoms and the Strix context knobs users can tune

Validation

  • git diff --check -> passed
  • docs-only change; no runtime tests run

Closes #286

@greptile-apps

greptile-apps Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This documentation update explains local-model context sizing and management.

  • Adds practical context-window recommendations for quick, standard, and deep scans.
  • Documents Ollama, LM Studio, and llama.cpp context configuration.
  • Lists clipping symptoms and the Strix environment variables used to tune compaction and tool-output retention.

Confidence Score: 3/5

The documentation should be corrected before merging because its custom 64k Ollama example can cause Strix to budget against a 200k fallback and compact too late.

The newly recommended custom model name may not resolve through model metadata, while the documented configuration leaves Strix's 200k fallback unchanged despite configuring Ollama for only 65,536 tokens.

Files Needing Attention: docs/llm-providers/local.mdx

Important Files Changed

Filename Overview
docs/advanced/configuration.mdx Documents the implemented context-management environment variables and their matching defaults.
docs/llm-providers/local.mdx Adds useful local-context guidance, but the custom Ollama model example can leave Strix using a 200k fallback for a 64k runtime window.
Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
docs/llm-providers/local.mdx:61-65
**Fallback exceeds Ollama context**

When the custom `qwen3-vl-64k` name is not present in LiteLLM's model metadata, Strix budgets against its 200,000-token fallback rather than Ollama's 65,536-token window, delaying compaction until requests exceed the server limit and produce context-overflow or lost-state behavior.

```suggestion
```bash
ollama create qwen3-vl-64k -f Modelfile
export STRIX_LLM="ollama/qwen3-vl-64k"
export LLM_API_BASE="http://localhost:11434"
export STRIX_CONTEXT_FALLBACK_TOKENS="65536"
```
```

Reviews (1): Last reviewed commit: "docs: explain local model context sizing" | Re-trigger Greptile

Comment thread docs/llm-providers/local.mdx
@ousamabenyounes

Copy link
Copy Markdown
Contributor Author

Fixed Greptile feedback in b2e7441.

Change:

  • Added STRIX_CONTEXT_FALLBACK_TOKENS="65536" to the Ollama 64k example.
  • Added guidance to keep Strix fallback context aligned with custom local model server context.

Validation:

  • git diff --check -> passed
  • docs-only change; no runtime tests run

@ousamabenyounes
ousamabenyounes force-pushed the docs/issue-286-local-context branch from b2e7441 to 88c9465 Compare July 27, 2026 20:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Docs: Prevent context clipping issues with local models

1 participant