Financial API Solution Builder

A LangChain Deep Agent that helps developers work with a financial-API marketplace catalog end-to-end: discover the right APIs, design a recommended integration architecture, and stand up a working sandbox application with starter code.

Built as a reference template for evaluating LangChain's Deep Agents library alongside LangSmith for tracing, evaluation, and hosted deployment. Everything here is meant to be cloned, customized, and adapted to your own API catalog.

What this demonstrates

• A supervisor + three scoped subagents built with deepagents, each owning the tool set needed for one workflow.
• A virtual filesystem that lets the agent persist generated artifacts (code snippets, test plans, READMEs) into per-thread state — visible in the UI, downloadable, no real disk writes.
• End-to-end LangSmith integration: tracing on every turn, a pre-wired evaluation pipeline with both code-based and LLM-as-judge evaluators, and a langgraph.json ready for LangSmith Deployments.
• A simple Streamlit UI with a chat pane, a browsable catalog, and an artifacts panel — small enough to read in one sitting and easy to fork.

The three workflows

1. Natural-language search — "Which API should I use for instant 24/7 payments?" → ranked list of matches with tier and prerequisites.
2. Solution-to-API mapping — "I want to build a neobank with onboarding, accounts, payments, and reconciliation." → recommended APIs, prerequisite chain, build order, and tier callouts.
3. Guided implementation — "Create a sandbox app, subscribe to these APIs, and give me a Python snippet." → mock app + credentials, subscription confirmation, starter code, test plan, and README written to the virtual filesystem.

Architecture

        ┌────────────────────────┐
        │   Supervisor (router)  │
        └──────────┬─────────────┘
                   │  task(subagent, …)
   ┌───────────────┼───────────────────┐
   ▼               ▼                   ▼
┌─────────┐  ┌──────────────┐  ┌────────────────────┐
│ search_ │  │ solution_    │  │ implementation_    │
│ agent   │  │ architect_   │  │ agent              │
│         │  │ agent        │  │                    │
│ • search_  │ • search_   │  │ • get_api_detail   │
│   catalog  │   catalog   │  │ • create_app       │
│ • get_api_ │ • get_api_  │  │ • request_creds    │
│   detail   │   detail    │  │ • subscribe_apis   │
│         │  │              │  │ • generate_snippet │
│         │  │              │  │ + write_file / ls  │
└─────────┘  └──────────────┘  └────────────────────┘
                   │
                   ▼
        ┌────────────────────────┐
        │ utils/catalog.json     │
        │  (mock API catalog)    │
        └────────────────────────┘

The supervisor has no domain tools — it just routes. Each subagent gets its own scoped tool list so it can't reach into another subagent's responsibilities. The implementation subagent additionally inherits the filesystem middleware tools (write_file, read_file, ls) from deepagents, enabling artifact persistence.

Quickstart

Prerequisites

• Python 3.11+
• uv for dependency management
• An Anthropic API key
• A LangSmith API key

Setup

git clone <your-fork-url>
cd financial-api-solution-builder

uv sync
cp .env.example .env

Edit .env:

ANTHROPIC_API_KEY=sk-ant-...
LANGSMITH_API_KEY=lsv2_...
LANGSMITH_PROJECT=financial-api-solution-builder
LANGSMITH_TRACING=true

Run

Mode	Command	Use when
Streamlit UI	.venv/bin/streamlit run app.py	Demoing; you want chat + catalog browser + artifacts panel
CLI	.venv/bin/python main.py	Quickly iterating on prompts or tools
LangGraph Studio	.venv/bin/langgraph dev	Visualizing the graph and inspecting per-step state

Example queries

Each one targets a different subagent:

Search:        Which API should I use for instant 24/7 payments?

Solution map:  I want to build a neobank with onboarding, accounts,
               payments, and reconciliation. What APIs do I need?

Build it:      Build me a real-time payments integration. App name
               "demo-rtp", sandbox env, subscribe to accounts.core
               and payments.rtp, Python snippet for sending an RTP
               payment. Save everything to the virtual filesystem.

The third prompt fills the artifacts panel with downloadable files.

Tracing with LangSmith

When LANGSMITH_TRACING=true and LANGSMITH_API_KEY are set, every invocation is captured. Open your LangSmith project to see:

• The supervisor's routing decision (which subagent it dispatched and why).
• The subagent's tool calls in order, with arguments, results, latency, and token usage.
• The full agent state at each step — including the virtual filesystem.

Common uses: debugging unexpected behavior, saving production traces as eval dataset examples, comparing experiments across prompt or model variants.

Evaluation

A minimal pipeline is included:

.venv/bin/python evals/create_dataset.py    # one-time: creates a dataset with 1 example per workflow
.venv/bin/python evals/run_experiment.py    # runs the agent over the dataset

Two evaluators ship out of the box:

• mentions_expected_apis — code-based check that the response includes the expected API ids.
• helpfulness_judge — an LLM-as-judge using a rubric stored on each example.

Results appear under Experiments in your LangSmith project.

Deployment

A langgraph.json is included so the agent can be deployed to LangSmith Deployments. See the LangSmith Deployments documentation for the current setup steps.

Repository layout

main.py                 Supervisor + three subagents; exports `agent`.
app.py                  Streamlit UI (chat, catalog browser, artifacts panel).
langgraph.json          LangSmith Deployments / `langgraph dev` config.
utils/
  prompts.py            Supervisor + subagent system prompts.
  tools.py              Tool definitions (catalog search, mock app/creds, snippet generation).
  catalog.json          Mock API catalog (~15 entries).
evals/
  create_dataset.py     Builds the LangSmith eval dataset.
  evaluators.py         Code-based + LLM-as-judge evaluators.
  run_experiment.py     Runs the agent against the dataset.

Customizing for your catalog

This template is meant to be forked and adapted. The most common swap-in points:

What	Where	Notes
API catalog	utils/catalog.json	Replace entries with your own. Each entry needs id, name, category, product_family, tier, description, when_to_use, prerequisites, sample_request.
App / credential / subscription endpoints	utils/tools.py (create_app, request_credentials, subscribe_apis)	Currently return hardcoded mocks; swap to real HTTP calls against your marketplace.
Snippet base URL and templates	utils/tools.py (API_BASE_URL, _http_to_curl, _http_to_python)	Change https://api.example.com and snippet shapes to match your API.
Agent prompts / persona	utils/prompts.py	Adjust tone, guardrails, or workflow steps.
Eval examples and rubrics	evals/create_dataset.py	Tailor to the integrations your customers actually care about.

Troubleshooting

Symptom	Most likely cause
Agent doesn't respond locally	ANTHROPIC_API_KEY missing or invalid in .env
No traces in LangSmith	LANGSMITH_TRACING=true not set, or LANGSMITH_API_KEY invalid
Artifacts panel never appears	The implementation workflow hasn't run yet — try the "Build it" example query