feat: detect page/websocket kinds across 9 framework extractors by jorgeraad · Pull Request #9 · pensarai/surface

jorgeraad · 2026-04-29T20:20:45Z

Closes #7. Adds page/websocket kind detection across 9 framework extractors (everything in the issue checklist).

Validated against 60 real-world benchmark apps (argus-validation-benchmarks):

53 endpoints reclassified from api → page (Flask render_template, Express res.render, Django TemplateView, etc.)
3 new websocket endpoints detected that the old extractors missed entirely
No regressions: same 735 endpoints still detected, plus the 3 new ones (738 total)

Each framework lands as its own commit with a fixture-driven vitest test under src/extractors/fixtures//. Also ports nextjs.test.ts from bun:test to vitest so the suite runs through one runner.

Note

Medium Risk
Classification is heuristic static analysis across many frameworks, so mis-labels are possible, but behavior is fixture-tested and mainly affects metadata rather than route discovery.

Overview
Adds endpoint kind classification (api, page, websocket) across Django, Express, FastAPI, Flask, Gin/Go, Laravel, NestJS, Rails, and Spring so mapped routes reflect how they behave, not just that a route exists.

Each extractor gains framework-specific heuristics: e.g. Django resolves views per app (including nested views/ packages) and walks CBV inheritance for template bases; Express balances route call parens and only treats real socket.io instances as WS; NestJS scopes @Controller / @WebSocketGateway per class (with comment-safe parsing); Rails walks ActionController::Base vs ::API ancestry and tracks namespace block depth; Spring distinguishes @Controller from @RestController and picks up STOMP @MessageMapping handlers.

Testing and tooling: Vitest replaces Bun for unit tests (test / test:watch), with per-framework fixtures under src/extractors/__fixtures__/ and matching *.test.ts files. NestJS/gRPC fixtures move out of scripts/fixtures; ESLint/tsconfig exclude the new fixture trees.

^{Reviewed by Cursor Bugbot for commit 3cb7430. Bugbot is set up for automated code reviews on this repo. Configure here.}

…do correctly Address Cursor Bugbot review on #9: - nestjs findClasses: attribute only the immediately-preceding decorator block to each class, so a @Controller/@WebSocketGateway in an earlier class in the same file no longer bleeds onto later classes. Add a multi-controller regression fixture + test. - rails: process 'end' (close + namespace pop) before 'do' (open) so a combined 'end; scope do' line pops namespaces at the correct depth.

…rt tests to vitest Rebased onto feat/grpc-discovery (#14, stacked). Integrate #14's code-first gRPC detection into #9's class-aware nestjs extractor: - @GrpcMethod/@GrpcStreamMethod handling is now a third match loop inside the main per-file loop, reusing findClasses/classAt so the owning class name is resolved consistently (correct even in multi-controller files). Broaden the file filter to include @grpc so gRPC-only controllers are scanned. - gRPC endpoints keep kind:"api" with transport:"grpc"; server/bidi streaming is modeled via grpc.streamingType, not a websocket kind. Documented the decision in grpc.ts. - Port #14's grpc.test.ts and nestjs gRPC tests from bun:test to vitest; move grpc/nestjs-grpc fixtures under src/extractors/__fixtures__/. No bun:test references remain.

…do correctly Address Cursor Bugbot review on #9: - nestjs findClasses: attribute only the immediately-preceding decorator block to each class, so a @Controller/@WebSocketGateway in an earlier class in the same file no longer bleeds onto later classes. Add a multi-controller regression fixture + test. - rails: process 'end' (close + namespace pop) before 'do' (open) so a combined 'end; scope do' line pops namespaces at the correct depth.

…rt tests to vitest Rebased onto feat/grpc-discovery (#14, stacked). Integrate #14's code-first gRPC detection into #9's class-aware nestjs extractor: - @GrpcMethod/@GrpcStreamMethod handling is now a third match loop inside the main per-file loop, reusing findClasses/classAt so the owning class name is resolved consistently (correct even in multi-controller files). Broaden the file filter to include @grpc so gRPC-only controllers are scanned. - gRPC endpoints keep kind:"api" with transport:"grpc"; server/bidi streaming is modeled via grpc.streamingType, not a websocket kind. Documented the decision in grpc.ts. - Port #14's grpc.test.ts and nestjs gRPC tests from bun:test to vitest; move grpc/nestjs-grpc fixtures under src/extractors/__fixtures__/. No bun:test references remain.

Address Cursor Bugbot findings on #9's kind-detection heuristics: - django: resolve CBV page detection through the full inheritance chain so a view that extends a template base indirectly (HomeView -> SiteBaseView -> TemplateView) is still a page. - django: scope the view/function registry per app directory (with a global fallback) so two apps defining the same view name don't clobber each other's page-vs-api kind. - django: capture single-line function-view bodies (def view(request): return render(...)) and an optional -> ReturnType annotation, so they aren't misclassified as api. - go: scope websocket-handler names per directory (package) so an ordinary HTTP route isn't mislabeled websocket just because an unrelated package defines a same-named .Upgrade()-ing handler. Add regression fixtures + vitest cases for each.

…do correctly Address Cursor Bugbot review on #9: - nestjs findClasses: attribute only the immediately-preceding decorator block to each class, so a @Controller/@WebSocketGateway in an earlier class in the same file no longer bleeds onto later classes. Add a multi-controller regression fixture + test. - rails: process 'end' (close + namespace pop) before 'do' (open) so a combined 'end; scope do' line pops namespaces at the correct depth.

…rt tests to vitest Rebased onto feat/grpc-discovery (#14, stacked). Integrate #14's code-first gRPC detection into #9's class-aware nestjs extractor: - @GrpcMethod/@GrpcStreamMethod handling is now a third match loop inside the main per-file loop, reusing findClasses/classAt so the owning class name is resolved consistently (correct even in multi-controller files). Broaden the file filter to include @grpc so gRPC-only controllers are scanned. - gRPC endpoints keep kind:"api" with transport:"grpc"; server/bidi streaming is modeled via grpc.streamingType, not a websocket kind. Documented the decision in grpc.ts. - Port #14's grpc.test.ts and nestjs gRPC tests from bun:test to vitest; move grpc/nestjs-grpc fixtures under src/extractors/__fixtures__/. No bun:test references remain.

Address Cursor Bugbot findings on #9's kind-detection heuristics: - django: resolve CBV page detection through the full inheritance chain so a view that extends a template base indirectly (HomeView -> SiteBaseView -> TemplateView) is still a page. - django: scope the view/function registry per app directory (with a global fallback) so two apps defining the same view name don't clobber each other's page-vs-api kind. - django: capture single-line function-view bodies (def view(request): return render(...)) and an optional -> ReturnType annotation, so they aren't misclassified as api. - go: scope websocket-handler names per directory (package) so an ordinary HTTP route isn't mislabeled websocket just because an unrelated package defines a same-named .Upgrade()-ing handler. Add regression fixtures + vitest cases for each.

Rebased onto feat/grpc-discovery @ b14fcc4, which removed the buggy short-name dropBareGrpcAliases in favor of an identical-wire-path collapse (proto wins via isProtoGrpc). Reconcile #9's kind-detection layer with the new base: - adopt b14fcc4's nestjs-grpc-proto fixture (package-qualified hero.HeroesService decorators + a BillingController @GrpcMethod() no-arg case). - replace the stale 'proto wins over bare decorator' / 'dedup lives in mapRaw' tests with b14fcc4's 'proto tiebreak + owning-class service resolution' cases (ported to vitest); drop the now-unused mapRaw import. The nestjs extractor already resolves the @GrpcMethod fallback service from the owning class via classAt, matching #14.

Sets up vitest as the test runner and adds the first per-extractor test using the FastAPI fixture (which already emits kind: "websocket" for @app.websocket routes). Establishes the fixture-driven pattern that subsequent per-framework tests will follow.

…ls) kinds Class-based views deriving from TemplateView/ListView/DetailView and FBVs calling render() are now emitted as kind: "page". websocket_urlpatterns entries (Django Channels) emit kind: "websocket". Strips .as_view/.as_asgi from handler names so view classes can be looked up by name.

Replaced the lazy single-line route regex with a balanced-paren walker so multi-line arrow handler bodies can be inspected. Page detection looks for res.render(...) or res.sendFile(...) in the handler body. Websocket detection covers app.ws() (express-ws), io.on('connection')/io.of() (socket.io), and new WebSocketServer (ws library).

A route is page-shaped when its decorator args set response_class=HTMLResponse or its function body returns an HTMLResponse(...) or .TemplateResponse(...). Body window is bounded to the next top-level decorator/def to avoid bleeding between adjacent routes.

A handler whose body contains render_template() or render_template_string() emits kind: "page". Body window is bounded to the next top-level def/class/ decorator at column 0 so adjacent handlers do not leak into each other's classification.

A route handler emits kind: "websocket" when its file imports github.com/gorilla/websocket and the handler body contains an .Upgrade( call. Applies uniformly across gin, echo, fiber, and net/http via a shared findWebsocketHandlers helper. Also tightened the handler-name regex so trailing handler args resolve correctly.

Routes defined in routes/web.php emit kind: "page" (session/CSRF/Blade view convention); routes from routes/api.php remain kind: "api". The file-of-origin distinction is reused from the existing /api prefix logic.

… kinds Methods carrying @Render('view') emit kind: "page". Classes annotated with @WebSocketGateway() emit each @SubscribeMessage('event') method as a websocket endpoint with method WS. Methods are now associated with their owning class via a per-class scan so controller vs gateway is disambiguated.

Routes are classified by walking the controller's inheritance chain on disk: ActionController::Base ancestor → kind: "page"; ActionController::API ancestor → kind: "api". Resolution memoizes per file with a cycle guard. Also fixes a pre-existing bug where namespace blocks leaked across the whole routes.rb file — namespaces are now tracked block-by-block.

@controller

…g) kinds Methods on a class annotated with @controller (without @RestController) emit kind: "page"; @RestController stays kind: "api". @MessageMapping and @SubscribeMapping methods emit a separate websocket endpoint regardless of class kind. Class context is resolved via a back-scan from each mapping match, so inner classes and multi-class files are handled.

@ts-nocheck

Fixture directories under src/extractors/__fixtures__/ contain synthetic test inputs that mimic external frameworks (e.g. @nestjs/common imports without the package being installed). They're scanned as raw source text by the extractors, never compiled or imported. Excluding them from both the TypeScript program and the ESLint config keeps tooling clean without needing per-file @ts-nocheck escape hatches.

The rest of the test suite uses vitest (introduced for the per-extractor fixture tests). Unify on one runner so 'bun run test' covers all suites. Replaces import.meta.dir (bun-only) with the standard ESM fileURLToPath(import.meta.url) pattern.

@get

…r stack The previous heuristic checked a 400-char backward window and a 400-char forward window for @Render. In a controller with multiple methods, that easily included @Render decorators belonging to a sibling method, so a plain @get sitting near a @Render-decorated handler was misclassified as a page. Replace both windows with a precise line-by-line walk that captures only the contiguous decorator stack of the current method — stopping at any line that ends a previous statement (}, ;) or otherwise looks like code. Add a regression test where a @get("/data") sibling sits directly below a @Render-decorated method and asserts its kind is "api".

…do correctly Address Cursor Bugbot review on #9: - nestjs findClasses: attribute only the immediately-preceding decorator block to each class, so a @Controller/@WebSocketGateway in an earlier class in the same file no longer bleeds onto later classes. Add a multi-controller regression fixture + test. - rails: process 'end' (close + namespace pop) before 'do' (open) so a combined 'end; scope do' line pops namespaces at the correct depth.

…rt tests to vitest Rebased onto feat/grpc-discovery (#14, stacked). Integrate #14's code-first gRPC detection into #9's class-aware nestjs extractor: - @GrpcMethod/@GrpcStreamMethod handling is now a third match loop inside the main per-file loop, reusing findClasses/classAt so the owning class name is resolved consistently (correct even in multi-controller files). Broaden the file filter to include @grpc so gRPC-only controllers are scanned. - gRPC endpoints keep kind:"api" with transport:"grpc"; server/bidi streaming is modeled via grpc.streamingType, not a websocket kind. Documented the decision in grpc.ts. - Port #14's grpc.test.ts and nestjs gRPC tests from bun:test to vitest; move grpc/nestjs-grpc fixtures under src/extractors/__fixtures__/. No bun:test references remain.

… routes Phase 2 kind-coverage audit: confirm #12's re_path()/raw-string routes flow through the same page-vs-api classification as plain path() routes (no undefined kind). Audited all extractors — nextjs already assigns page/api correctly (and is tested), server-actions emits kind:action, grpc/connect stays api with transport+streamingType, and actix/sst/openapi emit only REST/spec routes so the api default is correct.

Rebased onto the updated feat/grpc-discovery (per-proto Connect scoping + gRPC dedup moved into mapRaw). Preserve its new tests under the vitest port: - move grpc-mixed monorepo fixtures under src/extractors/__fixtures__/ and keep the connect-vs-plain-per-package test (as vitest it()). - re-add the 'dedup lives in mapRaw so impact() sees it too' nestjs test.

Address Cursor Bugbot findings on #9's kind-detection heuristics: - django: resolve CBV page detection through the full inheritance chain so a view that extends a template base indirectly (HomeView -> SiteBaseView -> TemplateView) is still a page. - django: scope the view/function registry per app directory (with a global fallback) so two apps defining the same view name don't clobber each other's page-vs-api kind. - django: capture single-line function-view bodies (def view(request): return render(...)) and an optional -> ReturnType annotation, so they aren't misclassified as api. - go: scope websocket-handler names per directory (package) so an ordinary HTTP route isn't mislabeled websocket just because an unrelated package defines a same-named .Upgrade()-ing handler. Add regression fixtures + vitest cases for each.

Rebased onto feat/grpc-discovery @ b14fcc4, which removed the buggy short-name dropBareGrpcAliases in favor of an identical-wire-path collapse (proto wins via isProtoGrpc). Reconcile #9's kind-detection layer with the new base: - adopt b14fcc4's nestjs-grpc-proto fixture (package-qualified hero.HeroesService decorators + a BillingController @GrpcMethod() no-arg case). - replace the stale 'proto wins over bare decorator' / 'dedup lives in mapRaw' tests with b14fcc4's 'proto tiebreak + owning-class service resolution' cases (ported to vitest); drop the now-unused mapRaw import. The nestjs extractor already resolves the @GrpcMethod fallback service from the owning class via classAt, matching #14.

Address Cursor Bugbot finding: the function-view regex only matched sync `def`, so `async def` template views were never indexed (misclassified api) and a following async view could be absorbed into the preceding sync view's body (bleeding its render() and mismarking a JSON route as page). funcRe now matches an optional `async` prefix and treats `async def` as a body terminator. Added a regression fixture (async_page + a sync json_only right before it) + test.

Address Cursor Bugbot round 3: - django: resolve the page/api registry within the whole app subtree (the dir containing urls.py), not just the immediate parent, so views in a nested package (myapp/views/*.py) still resolve to their owning app and same-named views across apps keep the right kind. Add a blogapp fixture with a views/ package colliding by name with an otherapp APIView. - express: only treat .of()/.on("connection") as socket.io when the file actually imports socket.io AND the call is on a confirmed server instance (new Server(...), require("socket.io")(...), or a factory binding). Unrelated .of() chains are no longer emitted as websocket endpoints. Add a fixture with a genuine io.of("/rooms") plus an unrelated registry.of("/plugins"). Also format two fixture package.json files that CI's prettier --check flagged.

cursor

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

^{Bugbot Autofix is ON. A cloud agent has been kicked off to fix the reported issues.}

^{Reviewed by Cursor Bugbot for commit 70617ad. Configure here.}

@controller

…platform Address Cursor Bugbot round 5 on the rebased-onto-main head: - nestjs: findClasses now detects class boundaries on a comment-masked copy of the file (offsets preserved), so a `class` token inside a // or /* */ comment no longer creates a phantom class that steals later handlers and drops the @controller prefix. Add a fixture with `// class Helper` inside a controller. - django & go: dirOf now normalizes \\ to / and guards the no-separator case, so per-app view scoping and per-package websocket scoping work on Windows scan paths (which use \\) instead of corrupting the path via lastIndexOf('/')=-1.

cursor Bot reviewed Apr 29, 2026

View reviewed changes

Comment thread src/extractors/nestjs.ts

jorgeraad assigned kylejryan Apr 29, 2026

cursor Bot reviewed Apr 29, 2026

View reviewed changes

Comment thread src/extractors/nestjs.ts Outdated

Comment thread src/extractors/rails.ts

jorgeraad force-pushed the feat/kind-detection-frameworks branch from 712b724 to 5ac6b19 Compare July 9, 2026 18:16

cursor Bot reviewed Jul 9, 2026

View reviewed changes

Comment thread src/extractors/django.ts

jorgeraad force-pushed the feat/kind-detection-frameworks branch from 58998e5 to 1cc30f0 Compare July 9, 2026 18:31

cursor Bot reviewed Jul 9, 2026

View reviewed changes

Comment thread src/extractors/django.ts

Comment thread src/extractors/django.ts

Comment thread src/extractors/go.ts

jorgeraad changed the base branch from main to feat/grpc-discovery July 9, 2026 18:35

jorgeraad force-pushed the feat/kind-detection-frameworks branch from 1cc30f0 to 406c721 Compare July 9, 2026 18:35

cursor Bot reviewed Jul 9, 2026

View reviewed changes

Comment thread src/extractors/django.ts Outdated

jorgeraad force-pushed the feat/kind-detection-frameworks branch from a675110 to bad7fbb Compare July 9, 2026 19:07

cursor Bot reviewed Jul 9, 2026

View reviewed changes

Comment thread src/extractors/django.ts Outdated

Comment thread src/extractors/express.ts

Test added 8 commits July 9, 2026 21:13

feat(laravel): detect page kind for routes/web.php

3e2e5b2

Routes defined in routes/web.php emit kind: "page" (session/CSRF/Blade view convention); routes from routes/api.php remain kind: "api". The file-of-origin distinction is reused from the existing /api prefix logic.

Test and others added 14 commits July 9, 2026 21:13

chore: apply prettier formatting to nestjs and rails extractors

07e8a8d

jorgeraad changed the base branch from feat/grpc-discovery to main July 9, 2026 21:14

jorgeraad force-pushed the feat/kind-detection-frameworks branch from 3e2ae5c to 70617ad Compare July 9, 2026 21:14

cursor Bot reviewed Jul 9, 2026

View reviewed changes

Comment thread src/extractors/nestjs.ts

Comment thread src/extractors/django.ts Outdated

jorgeraad merged commit df140d3 into main Jul 9, 2026
2 checks passed

Navigation Menu

feat: detect page/websocket kinds across 9 framework extractors - #9

feat: detect page/websocket kinds across 9 framework extractors#9
jorgeraad merged 23 commits into
mainfrom
feat/kind-detection-frameworks

jorgeraad commented Apr 29, 2026 •

edited by cursor Bot

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

jorgeraad commented Apr 29, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jorgeraad commented Apr 29, 2026 •

edited by cursor Bot

Loading