Skip to content

feat: replace DuckDuckGo web search with Exa/Parallel MCP providers#3

Open
andyzengmath wants to merge 2 commits into
samarth777:mainfrom
andyzengmath:feat/mcp-websearch
Open

feat: replace DuckDuckGo web search with Exa/Parallel MCP providers#3
andyzengmath wants to merge 2 commits into
samarth777:mainfrom
andyzengmath:feat/mcp-websearch

Conversation

@andyzengmath

Copy link
Copy Markdown

Problem

DuckDuckGo Lite web search has become unreliable. DDG now aggressively detects and blocks automated requests with CAPTCHAs, causing web search to silently return empty results. The proxy logs show:

⚠ DDG Lite returned CAPTCHA
⚠ All search providers failed

This affects all users who don't have a Brave API key configured, since DDG Lite was the default (and only free) provider.

Solution

Replace DuckDuckGo with Exa and Parallel MCP endpoints as the primary search backend — the same approach used by OpenCode (184k stars).

These providers:

  • Use the standard JSON-RPC 2.0 MCP protocol (tools/call method)
  • Work without API keys out of the box
  • Are purpose-built for AI agent search queries
  • Have no CAPTCHA or bot detection issues

Provider Priority Chain

1. Brave Search API    (if BRAVE_API_KEY set — unchanged)
2. Exa / Parallel MCP  (NEW — free, no key, 50/50 split with cross-fallback)
3. DuckDuckGo Lite     (demoted to fallback — may hit CAPTCHAs)
4. DuckDuckGo Instant  (last resort — limited to knowledge graph)

How It Works

  • Traffic is split 50/50 between Exa and Parallel based on a deterministic hash of the search query
  • If one provider fails, it automatically tries the other before falling through to DDG
  • 25-second timeout per MCP request
  • Response parsing handles both Exa's plain-text format and Parallel's JSON format

New Environment Variables (all optional)

Variable Description
WEBSEARCH_PROVIDER Force exa or parallel (default: auto 50/50 split)
EXA_API_KEY Optional API key for Exa
PARALLEL_API_KEY Optional Bearer token for Parallel

Testing

Verified working with claude-sonnet-4.5 through the proxy:

🔍 Web search: "latest version of Node.js"
✓ Exa returned 3 results

No configuration needed — just restart the proxy and web search works immediately.

DuckDuckGo Lite has become unreliable due to aggressive CAPTCHA/bot
detection on automated requests. This causes web search to silently
fail with empty results.

Replace with Exa and Parallel MCP endpoints (same approach used by
OpenCode) as the primary search backend:

- Uses JSON-RPC 2.0 MCP protocol (tools/call method)
- Traffic split 50/50 between Exa and Parallel by default
- Automatic cross-fallback (if one fails, tries the other)
- Works without API keys out of the box
- 25-second timeout per request

Provider priority chain:
  1. Brave Search API (if BRAVE_API_KEY set)
  2. Exa/Parallel MCP (primary, free, no key needed)
  3. DuckDuckGo Lite (fallback, may hit CAPTCHAs)
  4. DuckDuckGo Instant Answer (last resort)

New env vars (all optional):
  - WEBSEARCH_PROVIDER: force "exa" or "parallel"
  - EXA_API_KEY: optional key for Exa
  - PARALLEL_API_KEY: optional key for Parallel
Log model mapping, stream mode, token usage, and stop reason
for each request to match the detail level of the original proxy.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant