[codex] Add AST analysis and agent navigation maps

CHANGELOG.md

@@ -10,6 +10,19 @@ All notable changes to DevMap are documented in this file.
 - Public benchmark results
 - Feedback-driven fixes from the `0.1.0` beta
 
+### Added
+
+- `ts-morph` analysis for JavaScript and TypeScript behind a normalized
+  analyzer registry with heuristic and fallback analyzers
+- Lightweight `.devmap/index.json` and per-feature navigation maps for agents
+
+### Changed
+
+- Feature detection now separates documentation, landing UI, CLI commands,
+  analysis, snapshot, and AI roles before assigning technical features
+- Generated agent guidance now uses index-first navigation and treats the full
+  snapshot as a last-resort archive
+
 ## [0.1.0] - 2026-06-15
 
 Initial early beta release.

PRD.md

@@ -317,6 +317,7 @@ Run static analysis, generate project snapshot, and output a readable project ov
 - Detect database/schema
 - Detect entry points
 - Identify critical files
+- Generate a lightweight agent index and per-feature navigation maps
 - Generate architecture overview
 - Save snapshot to `.devmap/snapshot.json`
 
@@ -335,9 +336,11 @@ devmap analyze --deep
 | AI usage | Lower | Higher |
 | Best for | Quick mapping | Large/unfamiliar projects |
 
-**Generated file:**
+**Generated files:**
 
 ```txt
+.devmap/index.json
+.devmap/features/*.json
 .devmap/snapshot.json
 ```
 
@@ -493,6 +496,23 @@ Recommended header:
 
 ---
 
+### `.devmap/index.json` and `.devmap/features/*.json`
+
+Generated by `devmap analyze` as the lightweight navigation layer for AI
+agents.
+
+Preferred reading order:
+
+1. `.devmap/index.json`
+2. the relevant `.devmap/features/*.json` map
+3. source files listed in `sourcePriority`
+4. `.devmap/snapshot.json` only when more detail is required
+
+The index must remain short and must not duplicate full dependency or change
+impact data.
+
+---
+
 ### `.devmap/snapshot.json`
 
 Generated by `devmap analyze`.
@@ -503,6 +523,7 @@ Generated by `devmap analyze`.
 - Source of truth for `devmap ask`
 - Reusable context for AI tools
 - Regenerated every time the project is re-analyzed
+- Full analysis archive and backward-compatible source for DevMap commands
 
 **Important:**
 
@@ -675,6 +696,11 @@ Static analysis detects:
 - Entry points
 - Critical files
 
+JavaScript and TypeScript files use a normalized `ts-morph` analyzer for
+imports, exports, symbols, and function metadata. Other source languages keep
+the heuristic analyzer, with a low-confidence fallback for unknown file types.
+The analyzer registry preserves the extension point for future parsers.
+
 Results are converted into compact structured data before AI interpretation.
 
 ### MVP Principle
@@ -704,8 +730,8 @@ interface DevMapSnapshot {
   version: string;
   generatedAt: string;
   agentInstructions: {
-    navigationPolicy: "snapshot-first";
-    defaultMode: "minimal-exploration";
+    navigationPolicy: "index-first";
+    defaultMode: "feature-map-first";
     maxInitialFiles: number;
     missingSnapshotAction: "run-devmap-analyze";
     staleSnapshotAction: "run-devmap-analyze-fresh";
@@ -775,6 +801,8 @@ interface DevMapSnapshot {
   impact map for future generated docs and safer AI-assisted edits.
 - Snapshot should include compact machine-readable agent instructions. The
   complete agent navigation contract belongs in generated `DEVMAP.md`.
+- `.devmap/index.json` and feature maps are derived navigation artifacts, not a
+  replacement for the full versioned snapshot.
 - AI-generated file purpose and search terms must be batched and optional.
   Analyze must continue if enrichment fails.
 

README.md

@@ -45,7 +45,7 @@ The snapshot contains:
 * Minimal high-confidence feature and request flows
 * Feature entry points and lightweight business flows
 * Onboarding path and file-level change impact
-* Snapshot-first agent navigation policy
+* Index-first agent navigation policy with focused feature maps
 * Project relationships
 
 One analysis. Reusable context. Any codebase.
@@ -104,7 +104,9 @@ snapshot.json
 | ----------------------- | -------------------- |
 | `DEVMAP.md`             | DevMap instructions  |
 | `AGENTS.md`             | AI agent entry point |
-| `.devmap/snapshot.json` | Core project context |
+| `.devmap/index.json`    | Lightweight agent navigation |
+| `.devmap/features/*.json` | Focused feature maps |
+| `.devmap/snapshot.json` | Full project context archive |
 | `ONBOARDING.md`         | Optional onboarding guide |
 
 The snapshot is the primary output of DevMap.
@@ -185,6 +187,10 @@ Snapshot saved:
 
 ## For AI Agents
 
+Agents should read `.devmap/index.json` first, open the relevant feature map,
+and inspect its `sourcePriority` files. `.devmap/snapshot.json` is the full
+archive for cases where the lightweight navigation layer is insufficient.
+
 If you use Claude Code, OpenAI Codex, Gemini CLI, Cursor, Windsurf, Aider, GitHub Copilot, or Amazon Q — DevMap provides reusable project context that works across all of them.
 
 Without DevMap:

docs/architecture.md

@@ -128,8 +128,24 @@ The analyzer extracts useful structure from scanned files.
 * Detect external services
 * Detect database usage
 * Detect entry points
-* Detect critical files
-* Detect common features
+* Detect critical files
+* Detect common features
+
+### Analyzer Registry
+
+Scanned files pass through a normalized analyzer registry before `ProjectMap`
+is built:
+
+```txt
+Scanner -> TsMorphAnalyzer | HeuristicAnalyzer | FallbackAnalyzer -> FileAnalysis
+```
+
+- `.ts`, `.tsx`, `.js`, and `.jsx` use `ts-morph` with high confidence.
+- Other recognized source files keep regex/heuristic extraction with medium
+  confidence.
+- Unknown file types receive a low-confidence fallback result.
+- Existing snapshot fields remain available; AST metadata enriches imports,
+  exports, symbols, and top functions for JavaScript and TypeScript.
 
 ---
 
@@ -402,6 +418,10 @@ Feature metadata also stores a primary `entryPoint` and a short
 gives future generated docs a human-oriented path through the feature, not only
 a list of files.
 
+Structural feature flows describe behavior such as scanning, analyzer
+selection, project-map construction, snapshot persistence, and navigation-file
+generation. They must not duplicate the feature file list as a second list.
+
 ### Onboarding and Change Impact
 
 Snapshot schema includes two lightweight navigation aids:
@@ -418,9 +438,9 @@ full symbol graph.
 ### Agent Contract
 
 Generated `DEVMAP.md` contains the complete agent navigation contract. It tells
-agents to use `.devmap/snapshot.json` before broad repository exploration, to
-prefer feature entry points and flows, and to run `devmap analyze` when the
-snapshot is missing.
+agents to read `.devmap/index.json`, open the relevant feature map, and inspect
+its `sourcePriority` files before broad repository exploration. The full
+`.devmap/snapshot.json` is used only when the lightweight maps are insufficient.
 
 The snapshot also stores a compact `agentInstructions` object for machine
 readers. This is intentionally small: policy fields live in JSON, while the
@@ -709,9 +729,11 @@ Stores:
 └── snapshot.json
 ```
 
-Stores:
-
-* latest project snapshot
+Stores:
+
+* latest project snapshot
+* lightweight agent index in `.devmap/index.json`
+* focused maps in `.devmap/features/*.json`
 
 Future:
 
@@ -728,9 +750,11 @@ Future:
 DevMap may generate or update:
 
 ```txt
-DEVMAP.md
-AGENTS.md
-.devmap/snapshot.json
+DEVMAP.md
+AGENTS.md
+.devmap/index.json
+.devmap/features/*.json
+.devmap/snapshot.json
 ```
 
 Detailed generated file behavior is documented in:

docs/commands.md

@@ -146,12 +146,15 @@ devmap analyze --deep
 * Detect database usage
 * Detect entry points
 * Detect critical files
+* Analyze JS/TS imports, exports, symbols, and functions with `ts-morph`
+* Keep heuristic and fallback analysis for other file types
 * Build a compact file index with purpose, scope, top functions/code symbols,
   search terms, feature references, and importance
 * Generate minimal high-confidence feature and request/API flows
 * Infer feature entry points and short business flows where possible
 * Build a lightweight onboarding path and file-level change impact map
 * Generate architecture overview
+* Generate `.devmap/index.json` and `.devmap/features/*.json` for agents
 * Save snapshot to `.devmap/snapshot.json`
 
 ### Internal Flow
@@ -172,11 +175,13 @@ Snapshot
 Terminal Output
 ```
 
-### Generated File
-
-```txt
-.devmap/snapshot.json
-```
+### Generated Files
+
+```txt
+.devmap/index.json
+.devmap/features/*.json
+.devmap/snapshot.json
+```
 
 ### Output Example
 
@@ -241,6 +246,8 @@ Shared utilities, database access, authentication logic, and helpers.
 * Do not send the entire project source to AI
 * Snapshot must be regenerated after analyze
 * Snapshot must remain compact and deterministic
+* Agent index must remain small and must not duplicate full change-impact or
+  dependency data
 * AI metadata enrichment must be batched and optional
 * Analyze must continue if purpose or search-term enrichment fails
 * Raw provider errors must not be shown directly to users

docs/development-testing.md

@@ -1,5 +1,23 @@
 # DevMap Development Testing
 
+## Analyzer Registry And Agent Navigation
+
+Focused verification:
+
+```bash
+pnpm --filter devmap exec tsx --test test/file-analyzers.test.ts test/agent-navigation.test.ts test/analyzers.test.ts test/analyze-ai.test.ts
+```
+
+After `devmap analyze --fresh`, verify:
+
+- JS/TS `fileIndex` entries use `analyzer: "ts-morph"` and high confidence;
+- non-JS source keeps heuristic analysis and unknown files use fallback;
+- `.devmap/index.json` is short and contains no full `changeImpact` map;
+- every index feature points to a readable `.devmap/features/*.json` file;
+- docs and landing UI do not become evidence for Authentication or other
+  technical backend features;
+- generated agent instructions use index-first navigation.
+
 ## Agent JSON Output
 
 Packaged-command verification should include machine-readable output:

docs/for-me-personal/DEBUG.md

@@ -962,3 +962,40 @@ pnpm --filter devmap exec tsx --test test/analyzers.test.ts
 Folder dotfile yang dipakai alat development dapat berisi kata kunci AI, auth,
 atau service. Scanner harus membedakan metadata development dari source project
 agar snapshot tetap merepresentasikan aplikasi yang dianalisis.
+
+## 19. Authentication Palsu Dari Prompt Dan Dokumentasi Source
+
+**Tanggal:** 2026-06-20
+**Status:** Selesai
+
+### Gejala
+
+Setelah role filtering pertama, snapshot DevMap sendiri masih mendeteksi
+Authentication dari `contextBuilder.ts`, `snapshotEnrichment.ts`, onboarding,
+dan generated instructions. File tersebut hanya menyebut contoh auth di string.
+
+### Akar Masalah
+
+Fallback semantic auth role membaca seluruh content dan menganggap kombinasi
+kata `auth`, `session`, `middleware`, atau `guard` sebagai runtime behavior.
+Prompt dan dokumentasi embedded memenuhi pola itu tanpa implementasi auth.
+
+### Solusi
+
+- Technical features tidak memakai documentation, landing UI, atau test files.
+- Auth consumer membutuhkan bukti path, import, atau symbol.
+- Guard membutuhkan bukti auth dan guard pada path/symbol/import, bukan content
+  bebas.
+- Provider membutuhkan auth import atau symbol yang kuat.
+- Regression fixture memasukkan prompt-like strings agar bug tidak kembali.
+
+### Verifikasi
+
+Focused analyzer tests lulus dan fresh analysis pada root DevMap menghasilkan
+AI Integration, Analysis Engine, CLI Commands, Documentation, Snapshot Engine,
+serta Web Landing tanpa Authentication.
+
+### Pelajaran
+
+Kata teknis di prompt, docs, dan detector source bukan bukti capability runtime.
+Feature attribution harus bertumpu pada struktur kode dan ownership file.

docs/for-me-personal/PROGRESS.md

@@ -1,6 +1,37 @@
 # Progress DevMap
 
-Terakhir diperbarui: 2026-06-19
+Terakhir diperbarui: 2026-06-20
+
+## Update 2026-06-20
+
+### AST Analyzer Dan Agent Navigation
+
+- Menambahkan analyzer registry dengan output `FileAnalysis` yang konsisten.
+- `.ts`, `.tsx`, `.js`, dan `.jsx` sekarang dianalisis memakai `ts-morph`
+  untuk imports, exports, symbols, line number, exported state, dan async state.
+- File source lain tetap memakai heuristic analyzer; tipe yang tidak dikenal
+  memakai fallback low-confidence.
+- Snapshot v1 tetap mempertahankan field lama dan menambah analyzer id,
+  analysis confidence, serta symbol metadata.
+- `devmap analyze` sekarang menulis `.devmap/index.json` dan satu feature map
+  per fitur di `.devmap/features/`, termasuk saat snapshot cache dipakai ulang.
+- Generated `DEVMAP.md` dan block `AGENTS.md` memakai urutan index, feature map,
+  source priority, lalu full snapshot sebagai last resort.
+- Feature detector memisahkan documentation, web landing, CLI commands,
+  analysis engine, snapshot engine, dan AI integration sebelum technical
+  feature attribution.
+- False-positive Authentication pada source DevMap sendiri dihapus dengan
+  mensyaratkan bukti path, import, atau symbol, bukan sekadar kata di content.
+- Validasi manual pada root DevMap menghasilkan enam feature tanpa
+  Authentication palsu; index berukuran sekitar 4.5 KB.
+- Structural flows sekarang menjelaskan aksi nyata seperti scan, analyzer
+  selection, ProjectMap build, snapshot persistence, dan index generation,
+  bukan mengulang daftar dependency.
+- Critical files pada index memprioritaskan executable entry point, feature
+  entry point, dan behavioral support files; `ai/types.ts` tidak lagi masuk
+  hanya karena import count tinggi.
+- Full CLI suite lulus 96/96, TypeScript typecheck dan production build lulus,
+  serta packed tarball E2E lulus untuk fixture Next.js dan Express.
 
 ## Update 2026-06-19
 
@@ -960,9 +991,10 @@ Tahap berikutnya menambahkan primary feature entry point, business flow ringkas,
 shallow agar snapshot lebih memahami project tanpa masuk ke symbol graph penuh.
 
 Generated `DEVMAP.md` sekarang memiliki Agent Navigation Contract yang meminta
-agent memakai snapshot-first, menjalankan `devmap analyze` saat snapshot hilang,
-dan meminta user menjalankan `devmap init` jika DevMap belum terkonfigurasi.
-Snapshot juga menyimpan `agentInstructions` kecil untuk machine reader.
+agent memakai snapshot-first pada implementasi saat itu. Kontrak ini kemudian
+diganti pada 2026-06-20 menjadi index-first dan feature-map-first, dengan full
+snapshot sebagai fallback. Agent tetap menjalankan `devmap analyze` saat output
+hilang dan meminta user menjalankan `devmap init` jika belum terkonfigurasi.
 
 Verifikasi: