Skip to content

feat: add Katana crawler component#316

Open
OWConnoi wants to merge 1 commit into
ShipSecAI:mainfrom
OWConnoi:codex/add-katana-component
Open

feat: add Katana crawler component#316
OWConnoi wants to merge 1 commit into
ShipSecAI:mainfrom
OWConnoi:codex/add-katana-component

Conversation

@OWConnoi
Copy link
Copy Markdown

Adds a ProjectDiscovery Katana wrapper component for security workflows.

What changed:

  • Registers shipsec.katana.crawl in the worker component registry.
  • Runs Katana through the Docker runner with isolated target-list input volume.
  • Adds configurable crawl depth, strategy, scope, JS/form/XHR extraction, known files, rate/timeout/retry options, filtering, headers, proxy, and custom flags.
  • Normalizes Katana JSONL and plain URL output into structured endpoints, URL lists, raw output, effective options, and analytics-ready results.
  • Adds unit coverage for registration, CLI arg building, JSONL/plain output parsing, mocked runner normalization, empty-target skip, and non-zero exit handling.

Verification:

  • npx -y bun@1.1.20 test src/components/security/__tests__/katana.test.ts from worker/
  • npx -y bun@1.1.20 run lint from worker/
  • npx -y bun@1.1.20 run typecheck from repo root
  • git diff --check

Closes #12.

Signed-off-by: OWConnoi <123777754+OWConnoi@users.noreply.github.com>
@chatgpt-codex-connector
Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create Katana Component

1 participant