Skip to content

Feat/posthog integration#2

Merged
maxjneto merged 6 commits into
mainfrom
feat/posthog-integration
Jun 1, 2026
Merged

Feat/posthog integration#2
maxjneto merged 6 commits into
mainfrom
feat/posthog-integration

Conversation

@maxjneto

@maxjneto maxjneto commented Jun 1, 2026

Copy link
Copy Markdown
Owner

Summary

  • Product Analytics — Frontend and backend event pipeline wired into a single PostHog project, keyed by Clerk user ID. Covers document creation, phase progression, save/restore actions in both the free editor and refinement workspace, and a first_document_completed milestone event.
  • Session Replay — Records user sessions with document content masked at the SDK level (.editor-content, .markdown-preview, data-ph-mask). Gated behind cookie consent (opt_out_capturing_by_default: true).
  • LLM Analytics — Every Azure OpenAI call traced via PostHog's wrapped client, grouped by document_id. Per-call context includes phase, section type, document type, and retry flag. Prompt bodies redacted via POSTHOG_REDACT_PROMPTS.
  • Surveys — Five automatic surveys configured in the PostHog dashboard (no survey IDs in code). Triggered by existing analytics events (first_document_completed, phase_completed, document_exported, document_opened).
  • MCP bugfix — Fixed UnboundLocalError in _ApiKeyMiddleware when a request arrives without a Bearer token (e.g. the initial GET /mcp handshake from MCP clients).
  • Event naming cleanup — Removed the misfiring refinement_local_changes_saved auto-save event from EditorLayout. Replaced with editor_changes_saved (explicit save, free editor) and refinement_changes_saved (explicit save, refinement workspace). Auto-save debounce removed from EditorLayout entirely.

New events

Event Source Trigger
document_opened Frontend Document page mount
phase_completed Frontend Phase transition
first_document_completed Frontend auditcompleted
editor_changes_saved Frontend Explicit save in free editor
editor_version_restored Frontend Version restore in free editor
refinement_changes_saved Frontend Explicit save in refinement workspace
refinement_version_restored Frontend Version restore in refinement workspace

Test plan

  • Open a document → document_opened fires in PostHog live events
  • Advance a phase → phase_completed + phase_entered both fire
  • Complete a full guided document → first_document_completed fires once
  • In free editor: type content, click "Save changes" → editor_changes_saved fires; no event fires from typing alone
  • In refinement: click "Save changes" → refinement_changes_saved fires
  • In refinement: restore a version → refinement_version_restored fires
  • Connect an MCP client → no 500 on GET /mcp handshake

maxjneto and others added 6 commits May 31, 2026 12:17
…d analytics foundation

Backend: wire PostHog posthog-python client; instrument every phase function
(discovery, alignment, generation, refinement, audit, completion) with
$ai_trace / $ai_span events via a new services/observability.py helper.
Guardrails also capture span events on violations.

Frontend: initialise posthog-js (opt-out by default), add CookieConsentBanner
that drives opt-in/opt-out, wire Clerk user identity to PostHog, add initial
product events (phase_entered, document_created) and session-recording mask
selectors for editor content.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…CP events

- Add capture_event helper to observability service for custom PostHog events
- Fire alignment_section_reopened per rejected section in alignment phase
- Fire mcp_document_created, mcp_section_written, mcp_snapshot_saved, mcp_version_activated at HTTP layer via api_key_id detection
- Add document_opened, phase_completed, phase_entered events in DocumentPage
- Add discovery_question_answered/skipped in FollowUpQuestions
- Add alignment_section_reopened in AlignmentLayout
- Add refinement_edit_requested, refinement_question_asked, refinement_section_finalized in workspace store
- Add refinement_version_branched in TimelinePanel
- Add refinement_local_changes_saved, document_exported in EditorLayout
- Add editor_version_created, editor_version_restored in EditorVersionPanel
- Add document_exported in CompletedLayout
- Add data-ph-mask to SummaryCard, AuditProblems, CompletedLayout textareas/findings
- Guard posthog.identify behind consent in useClerkPosthogIdentity

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…_document_completed

- Remove auto-save debounce from EditorLayout (was misfiring refinement_local_changes_saved)
- Add editor_changes_saved to EditorVersionPanel.handleSaveChanges (explicit save only)
- Add refinement_version_restored to TimelinePanel.handleRestore
- Add refinement_changes_saved + Save changes button to TimelinePanel
- Add saveActiveContent action to workspace store
- Add first_document_completed event on audit → completed phase transition

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
key was only assigned inside the if block, crashing on unauthenticated
requests like the initial GET /mcp handshake from MCP clients.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@maxjneto
maxjneto merged commit 4a24408 into main Jun 1, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant