DichVariables API

FastAPI service for analyzing dichotomous (0/1) survey data: case segmentation, hierarchical clustering (dendrograms), brand association matrices, and distance graphs. Includes a small web UI and is deployable to Vercel.

Production URL: https://dichvariables-api.vercel.app

Quick start

Prepare an Arrow IPC file (.arrow) with binary columns and a case-weight column.
POST a JSON payload and the file to /api/v1/analyze.
Read JSON results; dendrograms and the associations matrix include PNGs as base64.

curl -X POST "https://dichvariables-api.vercel.app/api/v1/analyze" \
  -F "payload=@payload.json" \
  -F "dataframe=@data.arrow"

Web UI: open / in a browser when the server is running.

API usage

Endpoints

Method	Path	Description
`GET`	`/health`	Health check `{"status":"ok"}`
`GET`	`/`	Web UI
`POST`	`/api/v1/analyze`	Run analysis

`POST /api/v1/analyze`

Content type: multipart/form-data

Field	Type	Description
`payload`	string (JSON)	Analysis configuration (see below)
`dataframe`	file	Arrow IPC stream/file with 0/1 data

Requirements:

At least 2 rows (cases/observations).
All values in entity columns are treated as binary (non-zero → 1).
Case weights must be non-negative and not all zero.

Authentication

Set API_KEY in the server environment (e.g. Vercel env vars) to require a shared secret on analysis requests.

Header	Value
`X-API-Key`	Same value as the server `API_KEY` env var

POST /api/v1/analyze — requires the header when API_KEY is configured (returns 401 if missing or wrong).
GET /health, GET / — remain public.

If API_KEY is not set, the API stays open (useful for local development).

curl -X POST "https://dichvariables-api.vercel.app/api/v1/analyze" \
  -H "X-API-Key: your-secret-key" \
  -F "payload=@payload.json" \
  -F "dataframe=@data.arrow"

The web UI includes an API key field (stored in session storage for the browser session).

Validation errors

When columns are missing or misnamed, the API returns 422 with structured detail:

{
  "detail": {
    "message": "Missing columns for pairs: ...",
    "columns_in_file": ["IM1_11_2", "wrakin2"],
    "found_pairs": [[2, 11]],
    "found_variable_ids": [2],
    "found_item_ids": [11],
    "missing_pairs": [[2, 12]]
  }
}

Use found_* fields to compare your file against the payload.

Analysis errors (e.g. dendrogram requested with too few entities) also return 422 with a plain-text detail message.

Single mode vs multiple mode

Single mode (`mode: "single"`)

Each column is one variable. Each row is one case.

Concept	Description
Columns	`VAR_{id}` (e.g. `VAR_1`, `VAR_2`)
Rows	Cases / respondents
Weight	One column (e.g. `wrakin1`), set via `weight_column` or last non-entity column
Dendrogram (`dendrogram`)	Clusters variables

Multiple mode (`mode: "multiple"`)

Each column is one variable × item pair (e.g. brand × trait). Each row is one case.

Concept	Description
Columns	`{prefix}_{item_id}_{variable_id}` (e.g. `IM1_11_2`)
`column_prefix`	Required prefix without underscores (e.g. `IM1`)
`variables`	Trait definitions (IDs used in column names)
`items`	Brand/subject definitions (IDs used in column names)
Rows	Cases / respondents
Weight	One column (e.g. `wrakin2`)
`dendrogram`	Clusters items (brands)
`dendrogram_variables`	Clusters variables (traits)
`associations_matrix`	Weighted brand × trait association chart

Column rule (multiple mode):

{column_prefix}_{item_id}_{variable_id}

Example: item 11 (T-2), variable 2 (zaupanja vreden), prefix IM1 → column IM1_11_2.

The API expects the full cross product of all variables × all items — one column per pair.

One brand in multiple mode

These outputs work with a single item:

segmentation, graph, associations_matrix, dendrogram_variables

Special cases:

dendrogram only with one brand → 422 (needs ≥2 brands to cluster).
dendrogram + dendrogram_variables with one brand → item dendrogram is skipped; variables dendrogram is returned.

Request payload

Top-level fields

Field	Required	Description
`variables`	always	Map of variable ID → `{short_description, long_description, group_id?}`
`mode`	always	`"single"` or `"multiple"`
`items`	multiple mode	Map of item ID → descriptions
`column_prefix`	multiple mode	Prefix for entity columns (no `_`)
`groups`	if `group_id` used	Map of group ID → descriptions
`weight_column`	optional	Name of weight column; default = last non-entity column
`outputs`	always	At least one output block (see below)

Outputs

Output key	Applies to	Description
`segmentation`	both modes	Cluster cases into `num_segments` groups
`dendrogram`	single: variables; multiple: items	Hierarchical clustering + PNG + group summaries
`dendrogram_variables`	multiple mode	Hierarchical clustering of variables + PNG + group summaries
`associations_matrix`	multiple mode	Horizontal bar matrix: variables × brands
`graph`	both modes	Pairwise case distances (nodes = rows)

Dendrogram config

{
  "distance": "jaccard",
  "grouping": "average",
  "num_groups": 14,
  "image_width": 1200,
  "image_height": 800,
  "image_dpi": 150
}

Field	Description
`distance`	`"jaccard"` or `"simpson"`
`grouping`	`"ward"`, `"complete"`, or `"average"` (linkage method)
`num_groups`	Number of flat clusters for coloring, assignments, and cut line
`image_width`	Optional PNG width in pixels (default: 2800 at 200 dpi)
`image_height`	Optional PNG height; auto-scaled from label count if omitted
`image_dpi`	PNG resolution, 72–600 (default: 200)

Dendrogram PNG features:

Cluster-colored leaf labels and background bands
Dashed vertical cut line at the linkage distance for num_groups
Bottom-right caption: similarity, linkage, groups, Elements (variable count), Mode, and Brands (multiple mode)

When both image_width and image_height are set, the PNG matches those dimensions exactly.

Associations matrix config

{
  "sort_by_item_id": 101,
  "image_width": 1400,
  "image_height": 900,
  "image_dpi": 200
}

Field	Description
`sort_by_item_id`	Optional brand ID used to sort variable rows descending. Default: brand with highest mean association
`image_width` / `image_height` / `image_dpi`	Optional PNG sizing (same pattern as dendrogram)

Chart layout:

Rows = variables (traits), sorted descending by the chosen brand column
Columns = brands (items), sorted left-to-right by sum of association values in each column
Horizontal bars with percentage labels; variable labels in a dedicated left column

All config fields except sort_by_item_id are optional; {} is valid.

Example — multiple mode

{
  "variables": {
    "2": {"short_description": "Trait A", "long_description": "Trait A long"},
    "5": {"short_description": "Trait B", "long_description": "Trait B long"}
  },
  "mode": "multiple",
  "column_prefix": "IM1",
  "items": {
    "11": {"short_description": "Brand X", "long_description": "Brand X long"},
    "12": {"short_description": "Brand Y", "long_description": "Brand Y long"}
  },
  "weight_column": "wrakin2",
  "outputs": {
    "dendrogram": {
      "distance": "jaccard",
      "grouping": "ward",
      "num_groups": 14
    },
    "dendrogram_variables": {
      "distance": "jaccard",
      "grouping": "ward",
      "num_groups": 14
    },
    "associations_matrix": {
      "sort_by_item_id": 11
    }
  }
}

Required columns: IM1_11_2, IM1_11_5, IM1_12_2, IM1_12_5, wrakin2.

Response fields

Field	Content
`segmentation`	`{num_segments, assignments}` — row index → segment ID
`dendrogram`	Clustering of items (multiple) or variables (single): config, `cluster_assignments`, `groups`, PNG metadata, `image_png_base64`
`dendrogram_variables`	Variable clustering (multiple mode): same shape as `dendrogram`
`associations_matrix`	`{variable_ids, item_ids, sort_by_item_id, values, image_* , image_png_base64}`
`graph`	`{distance, nodes, edges}` — edges are pairwise case distances

Dendrogram result shape

{
  "distance": "jaccard",
  "grouping": "ward",
  "num_groups": 14,
  "color_threshold": 0.38,
  "cluster_assignments": {"1": 3, "2": 1},
  "groups": {
    "1": {
      "summary": "Network leadership",
      "items": ["najboljše omrežje", "najhitrejše omrežje", "vodilen"]
    },
    "3": {
      "summary": "User care",
      "items": ["skrbi za svoje uporabnike"]
    }
  },
  "image_width": 2800,
  "image_height": 2200,
  "image_dpi": 200,
  "image_png_base64": "..."
}

cluster_assignments: entity ID → cluster ID
groups: cluster ID → {summary, items} (see below)

Associations matrix result shape

values: 2D matrix of weighted association rates (0–1), rows = variable_ids, columns = item_ids (both in display order)
Each cell is the weighted share of cases where that brand–trait pair is coded 1

PNG images are UTF-8 base64 in image_png_base64.

Dendrogram groups (AI summaries)

Each dendrogram includes a groups dictionary keyed by cluster ID.

Field	Description
`summary`	Max 4 words. AI-generated when the cluster has 2+ variables/items; otherwise the single label (truncated)
`items`	List of variable or brand label strings in that cluster

Environment variables (production)

Set on Vercel (or locally) for AI summaries on multi-item clusters:

Variable	Required	Description
`OPENAI_API_KEY`	recommended	Enables OpenAI summaries for clusters with 2+ members
`OPENAI_MODEL`	optional	Default: `gpt-4o-mini`

Without OPENAI_API_KEY, multi-item clusters use a simple fallback (first label, truncated to 4 words).

Note: Cursor Composer runs inside the Cursor IDE and is not available as an API for this service. Use OpenAI or another LLM provider with a standard API key.

Similarity and distance calculations

See docs/STATISTICS.md for:

When to use Jaccard vs Simpson (decision guide and use cases)
How each metric is calculated (formulas, weighted examples)
Worked examples with step-by-step element distances in single mode (variables) and multiple mode (items + variables)
How similarities apply to cases, variables, and items/brands

Short guide:

Metric	Use when
Jaccard	Symmetric comparison; profiles have similar “density”; penalize differences in both presence and absence. Default choice.
Simpson	One profile is often a subset of another; you care how well the smaller profile is contained in the larger.

Both return distance = 1 − similarity and respect case weights.

Python examples

Call API and save dendrogram images + groups

import base64
import json
from pathlib import Path

import requests

API_URL = "https://dichvariables-api.vercel.app/api/v1/analyze"

payload = {
    "variables": {
        "2": {"short_description": "A", "long_description": "Trait A"},
        "5": {"short_description": "B", "long_description": "Trait B"},
    },
    "mode": "multiple",
    "column_prefix": "IM1",
    "items": {
        "11": {"short_description": "X", "long_description": "Brand X"},
        "12": {"short_description": "Y", "long_description": "Brand Y"},
    },
    "weight_column": "wrakin2",
    "outputs": {
        "dendrogram": {"distance": "jaccard", "grouping": "ward", "num_groups": 14},
        "dendrogram_variables": {"distance": "jaccard", "grouping": "ward", "num_groups": 14},
        "associations_matrix": {},
    },
}

with open("data.arrow", "rb") as f:
    r = requests.post(
        API_URL,
        headers={"X-API-Key": "your-secret-key"},  # omit if API_KEY is not configured
        data={"payload": json.dumps(payload)},
        files={"dataframe": ("data.arrow", f, "application/vnd.apache.arrow.stream")},
        timeout=300,
    )
r.raise_for_status()
result = r.json()

out = Path("images")
out.mkdir(exist_ok=True)

for key, filename in [
    ("dendrogram", "dendrogram_items.png"),
    ("dendrogram_variables", "dendrogram_variables.png"),
    ("associations_matrix", "associations_matrix.png"),
]:
    if key in result:
        (out / filename).write_bytes(
            base64.b64decode(result[key]["image_png_base64"])
        )

if "dendrogram_variables" in result:
    print(json.dumps(result["dendrogram_variables"]["groups"], indent=2, ensure_ascii=False))

Build a minimal Arrow file (Polars)

import io
from pathlib import Path

import polars as pl

df = pl.DataFrame({
    "IM1_11_2": [1, 0, 1],
    "IM1_12_2": [0, 1, 0],
    "wrakin2": [1.0, 2.0, 1.5],
})
buf = io.BytesIO()
df.write_ipc(buf)
Path("data.arrow").write_bytes(buf.getvalue())

Local development

pip install -r requirements.txt uvicorn polars
export OPENAI_API_KEY=sk-...   # optional, for dendrogram group summaries
export API_KEY=your-secret     # optional, enables API key auth
uvicorn backend.main:app --reload --port 8000
python smoke_test.py

Regenerate embedded frontend assets after editing backend/public/*:

python scripts/generate_frontend_assets.py

Project layout

backend/
  main.py              FastAPI app
  auth.py              Optional API key check
  models.py            Request/response schemas
  dataframe.py         Arrow parsing & column validation
  pipeline.py          Analysis orchestration
  services/
    segmentation.py
    dendrogram.py
    associations_matrix.py
    group_summarizer.py   AI cluster summaries (OpenAI)
    graph.py
    distances.py
  public/              Web UI source
docs/
  STATISTICS.md        Similarity & distance formulas
smoke_test.py          Integration tests

Name		Name	Last commit message	Last commit date
Latest commit History 40 Commits
backend		backend
docs		docs
scripts		scripts
.gitignore		.gitignore
.python-version		.python-version
README.md		README.md
pyproject.toml		pyproject.toml
requirements.txt		requirements.txt
smoke_test.py		smoke_test.py

Folders and files

Latest commit

History

Repository files navigation

DichVariables API

Contents

Quick start

API usage

Endpoints

POST /api/v1/analyze

Authentication

Validation errors

Single mode vs multiple mode

Single mode (mode: "single")

Multiple mode (mode: "multiple")

One brand in multiple mode

Request payload

Top-level fields

Outputs

Dendrogram config

Associations matrix config

Example — multiple mode

Response fields

Dendrogram result shape

Associations matrix result shape

Dendrogram groups (AI summaries)

Environment variables (production)

Similarity and distance calculations

Python examples

Call API and save dendrogram images + groups

Build a minimal Arrow file (Polars)

Local development

Project layout

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

`POST /api/v1/analyze`

Single mode (`mode: "single"`)

Multiple mode (`mode: "multiple"`)

Packages