Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
3d6996a
feat(gateway): scaffold service + /health + auth shims + deploy/CI (#59)
qiuethan Jul 5, 2026
41ced60
feat(gateway): api_keys store (in-memory + postgres) + migration (#59)
qiuethan Jul 5, 2026
5016b95
feat(gateway): platform_auth wiring for external keys (#59)
qiuethan Jul 5, 2026
6189bb7
feat(gateway): gateway-keys CLI (issue/list/revoke) (#59)
qiuethan Jul 5, 2026
cb37093
feat(gateway): directory http client (github lookup + identifiers) (#59)
qiuethan Jul 5, 2026
429b864
feat(gateway): /v1/resolve/discord endpoint (#59)
qiuethan Jul 5, 2026
52e86bc
chore(gateway): remove accidentally committed __pycache__
qiuethan Jul 5, 2026
e56b6e4
feat(gateway): per-key rate limiting (#59)
qiuethan Jul 5, 2026
656e30a
fix(gateway): audit outermost so 429s are logged (#59)
qiuethan Jul 5, 2026
18c5818
docs(gateway): document external gateway + first endpoint (#59)
qiuethan Jul 5, 2026
099a4d9
harden(gateway): rate-limit eviction + login url-encoding + 503 loggi…
qiuethan Jul 5, 2026
1998b15
fix(gateway): no env-bootstrap admin key on the public door (#59)
qiuethan Aug 17, 2026
61b8fe5
fix(gateway): bound the rate limiter and meter issued keys, not heade…
qiuethan Aug 17, 2026
33e00a4
fix(gateway): stop the resolver's 404s revealing directory membership…
qiuethan Aug 17, 2026
537abfa
chore(gateway): pool the outbound client, and tidy the rough edges (#59)
qiuethan Aug 17, 2026
84e651f
docs(gateway): say what the public door actually does (#59)
qiuethan Aug 17, 2026
65a85e8
chore: give services/gateway its own ownership zone (#59)
qiuethan Aug 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@
/packages/auth/ @qiuethan
/services/connectors/ @qiuethan
/services/documentation-system/ @qiuethan
/services/gateway/ @qiuethan
/services/llm/ @qiuethan
/services/meeting/ @qiuethan
/services/team-tracking/ @qiuethan
Expand Down
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ label-consistency.yml fails the build if you miss one — docs/CODE-OWNERSHIP.md
Full list, and what each zone covers: docs/CODE-OWNERSHIP.md
-->

`discord-bot` · `packages/auth` · `services/connectors` · `services/documentation-system` · `services/llm` · `services/meeting` · `services/team-tracking` · `services/verification` · `docs` · `scripts` · `.github` · `root`
`discord-bot` · `packages/auth` · `services/connectors` · `services/documentation-system` · `services/gateway` · `services/llm` · `services/meeting` · `services/team-tracking` · `services/verification` · `docs` · `scripts` · `.github` · `root`

<!-- If this spans more than one zone, explain why it can't be split: -->

Expand Down
6 changes: 6 additions & 0 deletions .github/labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,11 @@
- any-glob-to-any-file:
- services/documentation-system/**

"zone: services/gateway":
- changed-files:
- any-glob-to-any-file:
- services/gateway/**

"zone: services/llm":
- changed-files:
- any-glob-to-any-file:
Expand Down Expand Up @@ -74,6 +79,7 @@
- services/**
- "!services/connectors/**"
- "!services/documentation-system/**"
- "!services/gateway/**"
- "!services/llm/**"
- "!services/meeting/**"
- "!services/team-tracking/**"
Expand Down
22 changes: 22 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,24 @@ jobs:
- name: Ruff format check
run: uv run ruff format --check .

gateway-test:
runs-on: ubuntu-latest
defaults:
run:
working-directory: services/gateway
steps:
- uses: actions/checkout@v4

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win

Tighten checkout credentials and job permissions.

This job doesn’t need a persisted write token. Set read-only permissions for the job and disable credential persistence on checkout.

🔐 Suggested workflow hardening
 gateway-test:
   runs-on: ubuntu-latest
+  permissions:
+    contents: read
   defaults:
     run:
       working-directory: services/gateway
   steps:
     - uses: actions/checkout@v4
+      with:
+        persist-credentials: false
     - name: Install uv
       uses: astral-sh/setup-uv@v5
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- uses: actions/checkout@v4
gateway-test:
runs-on: ubuntu-latest
permissions:
contents: read
defaults:
run:
working-directory: services/gateway
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- name: Install uv
uses: astral-sh/setup-uv@v5
🧰 Tools
🪛 zizmor (1.26.1)

[warning] 80-80: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false

(artipacked)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/ci.yml at line 80, The checkout step currently persists
the GitHub token and the job lacks explicit least-privilege permissions. Update
the workflow job that uses actions/checkout@v4 to set read-only job permissions
and configure the checkout action to disable credential persistence, so the
token is not stored for later git operations.

Source: Linters/SAST tools

- name: Install uv
uses: astral-sh/setup-uv@v5
- name: Install dependencies
run: uv sync --extra dev
- name: Run tests
run: uv run pytest
- name: Ruff check
run: uv run ruff check .
- name: Ruff format check
run: uv run ruff format --check .

node-test:
runs-on: ubuntu-latest
defaults:
Expand Down Expand Up @@ -253,6 +271,10 @@ jobs:
run: docker build -f services/llm/Dockerfile -t llm:ci .
- name: Smoke-test llm image (imports resolve at boot)
run: docker run --rm llm:ci python -c "import src.api.app"
- name: Build gateway image
run: docker build -f services/gateway/Dockerfile -t gateway:ci .
- name: Smoke-test gateway image (imports incl. contracts/ resolve at boot)
run: docker run --rm gateway:ci python -c "import src.api.app"
- name: Build discord-bot image
run: docker build -t discord-bot:ci ./discord-bot
- name: Smoke-test discord-bot image (source parses)
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/pr-zone-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ jobs:
packages/*) echo packages/other ;;
services/connectors/*) echo services/connectors ;;
services/documentation-system/*) echo services/documentation-system ;;
services/gateway/*) echo services/gateway ;;
services/llm/*) echo services/llm ;;
services/meeting/*) echo services/meeting ;;
services/team-tracking/*) echo services/team-tracking ;;
Expand Down
13 changes: 9 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ Every domain has a first-class HTTP API — build your own dashboard, sync job,

- **[team-tracking](services/team-tracking/README.md)** — 26 endpoints across `people`, `teams`, `role_kinds`, `team_memberships`, `providers`, `person_identifiers`, `api_keys`. Full point-in-time roster queries. Scoped API keys, per-request audit log. **Actively consumed** by the Discord bot in production.
- **[documentation-system](services/documentation-system/README.md)** — endpoints over `docs` and `sources`; ingest a URL and it's normalized, dedup'd, fetched (title + snapshot for supported sources), and owner-validated against team-tracking. Ownership degrades gracefully if the directory is unreachable. **Consumed** by the Discord bot's `/doc` command group (`add`, `list`, `show`, `remove`).
- **[gateway](services/gateway/README.md)** — the one **public** service. A narrow, scoped, rate-limited door onto the directory for external consumers (e.g. a GitHub Action) that shouldn't hold an internal team-tracking key. First endpoint: `GET /v1/resolve/discord/{github_login}`, returning only a Discord id.

Every service speaks OpenAPI. Point Swagger UI or codegen at them. (`meeting`'s WebSocket route isn't representable in OpenAPI — its wire format is documented in [`services/meeting/README.md`](services/meeting/README.md).)

Expand All @@ -64,6 +65,7 @@ The other four are internal-facing: **[llm](services/llm/README.md)** (`POST /ch
| [`services/verification/`](services/verification/README.md) | Email verification: request a one-time code and confirm it, linking a subject (e.g. `discord:<id>`) to a verified email; requires the `verification:write` scope | **Deployed** (staging + prod). |
| [`services/meeting/`](services/meeting/README.md) | Meeting recording: transcribes a Discord voice session (Amazon Transcribe) and returns LLM-generated minutes as a branded PDF; no DB, nothing persisted | **Deployed** (staging). Consumed by the bot's `/record` command group; requires the `meetings` scope. |
| [`services/connectors/`](services/connectors/README.md) | Stateless outbound adapter: fetches document content (Google Docs/Sheets/Slides/Drive) on behalf of internal consumers via a service account; no DB | **Deployed** (staging). Consumed by documentation-system's Google source fetches; requires the `fetch` scope. |
| [`services/gateway/`](services/gateway/README.md) | The one **public** service — a scoped, rate-limited external gateway onto the directory, with its own external key registry and one internal team-tracking key | Built. First endpoint: `GET /v1/resolve/discord/{github_login}` (used by #34's reviewer-ping GitHub Action). |
| [`discord-bot/`](discord-bot/README.md) | Discord slash-command frontend + a browser-based "web playground" for iterating on commands without a Discord token | **Deployed** (staging + prod). All slash commands are stable and registered globally; 0 beta. |
| Search / retrieval | Full-text + semantic search over the catalog's snapshots | Deferred (not built) |

Expand Down Expand Up @@ -123,14 +125,17 @@ Misty/
│ ├── llm/ Bedrock /chat proxy — 8002, NO database
│ ├── meeting/ Live meeting transcription — 8004, NO database,
│ │ stateful (in-memory sessions)
│ └── connectors/ Google source fetch adapter — 8005, NO database
│ ├── connectors/ Google source fetch adapter — 8005, NO database
│ └── gateway/ External API gateway — 8006, own Postgres (external
│ key registry). The one PUBLIC service: scoped,
│ rate-limited, curated read surface
│ (every service above has the same docs/ set:
│ API.md, ARCHITECTURE.md, CONTRIBUTING.md, DEPLOYMENT.md)
├── packages/
│ └── auth/ platform_auth — shared API-key auth lib (argon2 hashing,
│ scopes, FastAPI deps, audit middleware); a pure leaf
│ consumed by all six services via thin shims
│ consumed by all seven services via thin shims
├── discord-bot/ Discord frontend + web playground
│ ├── src/ Node.js + discord.js
Expand All @@ -149,7 +154,7 @@ Misty/
├── PULL_REQUEST_TEMPLATE.md Zone, verification steps, deployment notes
├── ISSUE_TEMPLATE/ Bug / feature / epic issue forms (Blocked by + Zone fields)
└── workflows/
├── ci.yml Tests + lint + Docker builds on every PR (10 jobs)
├── ci.yml Tests + lint + Docker builds on every PR (11 jobs)
├── main-source-guard.yml Enforces "PRs to main come from staging"
├── pr-zone-check.yml Warns on PRs spanning multiple CODEOWNERS zones
├── label-consistency.yml Fails when the zone or area list drifts (runs check-labels.mjs)
Expand All @@ -161,7 +166,7 @@ Misty/
└── blocked-ready-automation.yml Syncs blocked/ready issue labels
```

Each service is self-contained: its own tests, its own docs, and its own database *if it needs one* — `llm`, `meeting`, and `connectors` deliberately have none. Dependencies are managed as one uv workspace rooted at this repo's `pyproject.toml`/`uv.lock`, and all six services share one leaf, `packages/auth` (`platform_auth`), for API-key auth — a shared *library* dependency, not a dependency between services, which remain independent of each other. Add a new service by dropping it in `services/` following the same shape (and adding its CI job in the same PR).
Each service is self-contained: its own tests, its own docs, and its own database *if it needs one* — `llm`, `meeting`, and `connectors` deliberately have none. Dependencies are managed as one uv workspace rooted at this repo's `pyproject.toml`/`uv.lock`, and all seven services share one leaf, `packages/auth` (`platform_auth`), for API-key auth — a shared *library* dependency, not a dependency between services, which remain independent of each other. Add a new service by dropping it in `services/` following the same shape (and adding its CI job in the same PR).

---

Expand Down
41 changes: 41 additions & 0 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,47 @@ describes how it applies them concretely.
driven by an actor supplied via `X-On-Behalf-Of` rather than by the key alone.
No other service needs this today.

## Access architecture: two doors, not one gateway

The platform has exactly two ways in, and they're deliberately asymmetric — there is
**no internal gateway**. Internal services trust each other directly; only external
callers go through a gateway at all.

- **Internal door.** team-tracking, documentation-system, and the gateway's own
outbound call to team-tracking all authenticate the same way: a per-consumer key,
argon2-hashed, scoped, issued by the target service's own CLI (`team-tracking-keys`,
`doc-keys`). The machinery is the shared [`packages/auth`](../packages/auth)
(`platform_auth`) library described above — each internal service is its own
authority over its own keys. There is no shared internal proxy standing in front of
them; adding a service that trusts another means issuing it a key on that service,
nothing more.
- **External door — [`services/gateway/`](../services/gateway/README.md), implemented.**
Callers outside the org's trust boundary (a GitHub Action, a future third-party
integration) never get a team-tracking key. Instead they hold a key issued by the
gateway's own external registry (`gateway-keys`, scoped e.g. `resolve:discord`), and
the gateway holds exactly **one** internal team-tracking key
(`identifiers:read`) for its own outbound calls. It composes and curates — it never
passes an internal response straight through — and adds the protections an
externally-facing surface needs that internal services don't: a public Railway
domain, per-key rate limiting behind a per-IP flood guard, and audit logging of
every external request.

The asymmetry runs one level deeper than the key registries. Every internal
service also accepts an env-bootstrap key (`API_KEY`), which `platform_auth`
resolves to the wildcard `admin` scope — the grace path you use to reach the
admin API that issues the first real key, and safe because only the private
network can reach it. **The gateway does not.** It has no admin API to
bootstrap (`gateway-keys` writes to its database directly) and it is the one
service on the public internet, so it passes `get_env_key=lambda: None` and
every caller must present an issued, scoped key. When adding an
externally-reachable service, copy that, not the internal shim.

The gateway's first (and so far only) endpoint is the resolver,
`GET /v1/resolve/discord/{github_login}` — it turns a GitHub login into the Discord id
linked to the same person in the directory, and nothing else, for #34's reviewer-ping
GitHub Action. New external use cases get new narrow endpoints on the gateway, not
broader access to team-tracking itself.

## Why the directory is built first

The build order — **directory → docs catalog → search** — isn't arbitrary. It follows
Expand Down
1 change: 1 addition & 0 deletions docs/CODE-OWNERSHIP.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ Fourteen buckets. Every tracked file lands in exactly one.
| `packages/other` | `packages/*` | `/packages/` | @qiuethan |
| `services/connectors` | `services/connectors/*` | `/services/connectors/` | @qiuethan |
| `services/documentation-system` | `services/documentation-system/*` | `/services/documentation-system/` | @qiuethan |
| `services/gateway` | `services/gateway/*` | `/services/gateway/` | @qiuethan |
| `services/llm` | `services/llm/*` | `/services/llm/` | @qiuethan |
| `services/meeting` | `services/meeting/*` | `/services/meeting/` | @qiuethan |
| `services/team-tracking` | `services/team-tracking/*` | `/services/team-tracking/` | @qiuethan |
Expand Down
5 changes: 5 additions & 0 deletions services/gateway/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
.venv
__pycache__
*.pyc
.env
.pytest_cache
16 changes: 16 additions & 0 deletions services/gateway/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
DATABASE_URL=postgresql+psycopg://gateway:dev_password@localhost:5435/gateway

# No API_KEY. Unlike the internal services, the gateway has no env-bootstrap
# admin key — inbound callers must present a scoped key issued by `gateway-keys`
# and stored in the api_keys table. See src/api/auth.py.

# Outbound: the gateway's own team-tracking key, scoped identifiers:read.
DIRECTORY_BASE_URL=http://localhost:8000
DIRECTORY_API_KEY=dev-api-key-change-me

GATEWAY_ENV=local

# Set to true wherever a proxy terminates TLS in front of the gateway (Railway
# does). Off by default so a directly-exposed deploy can't be handed a spoofed
# X-Forwarded-For and slip the per-IP rate limit.
TRUST_PROXY_HEADERS=false
11 changes: 11 additions & 0 deletions services/gateway/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
FROM python:3.11-slim
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
WORKDIR /app
COPY pyproject.toml uv.lock ./
COPY packages/ ./packages/
COPY services/gateway/ ./services/gateway/
RUN uv sync --frozen --no-dev --package gateway
ENV PATH="/app/.venv/bin:$PATH"
WORKDIR /app/services/gateway
EXPOSE 8000
CMD ["uvicorn", "src.api.app:app", "--host", "0.0.0.0", "--port", "8000"]
Comment on lines +1 to +11

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Run the gateway as a non-root user.

The container currently starts Uvicorn as root. For a public-facing gateway, that’s unnecessary risk.

🔧 Suggested Dockerfile hardening
 FROM python:3.11-slim
 COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
 WORKDIR /app
 COPY pyproject.toml uv.lock ./
 COPY packages/ ./packages/
 COPY services/gateway/ ./services/gateway/
 RUN uv sync --frozen --no-dev --package gateway
 ENV PATH="/app/.venv/bin:$PATH"
+RUN addgroup --system gateway && adduser --system --ingroup gateway gateway
+USER gateway
 WORKDIR /app/services/gateway
 EXPOSE 8000
 CMD ["uvicorn", "src.api.app:app", "--host", "0.0.0.0", "--port", "8000"]
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
FROM python:3.11-slim
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
WORKDIR /app
COPY pyproject.toml uv.lock ./
COPY packages/ ./packages/
COPY services/gateway/ ./services/gateway/
RUN uv sync --frozen --no-dev --package gateway
ENV PATH="/app/.venv/bin:$PATH"
WORKDIR /app/services/gateway
EXPOSE 8000
CMD ["uvicorn", "src.api.app:app", "--host", "0.0.0.0", "--port", "8000"]
FROM python:3.11-slim
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
WORKDIR /app
COPY pyproject.toml uv.lock ./
COPY packages/ ./packages/
COPY services/gateway/ ./services/gateway/
RUN uv sync --frozen --no-dev --package gateway
ENV PATH="/app/.venv/bin:$PATH"
RUN addgroup --system gateway && adduser --system --ingroup gateway gateway
USER gateway
WORKDIR /app/services/gateway
EXPOSE 8000
CMD ["uvicorn", "src.api.app:app", "--host", "0.0.0.0", "--port", "8000"]
🧰 Tools
🪛 Trivy (0.69.3)

[error] 1-1: Image user should not be 'root'

Specify at least 1 USER command in Dockerfile with non-root user as argument

Rule: DS-0002

Learn more

(IaC/Dockerfile)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@services/gateway/Dockerfile` around lines 1 - 11, The gateway container
currently runs Uvicorn as root, so harden the Dockerfile by running the final
image under a dedicated non-root user. Update the service image setup around the
existing COPY/uv sync/ENV/CMD steps to create or switch to a low-privilege user
before starting uvicorn, and ensure the app files and working directory used by
services/gateway are accessible to that user. Keep the startup command unchanged
except for executing it as the non-root user.

Source: Linters/SAST tools

Loading
Loading