Skip to content

Scope the capabilities page and add Cloud availability per row - #1364

Open
hmishra2250 wants to merge 3 commits into
mainfrom
docs/capabilities-and-licensing
Open

Scope the capabilities page and add Cloud availability per row#1364
hmishra2250 wants to merge 3 commits into
mainfrom
docs/capabilities-and-licensing

Conversation

@hmishra2250

@hmishra2250 hmishra2250 commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Two things to know before reviewing.

Licensing is deliberately excluded from this PR, pending Legal. The page was originally
capabilities-and-licensing.mdx; the ## Licensing section, the licence-dependent misconception
row, and all three [LEGAL REVIEW: confirm] placeholders are removed from the published page and
ship nowhere in this repo. The draft is held outside the repo for Legal and must not be added back
in any form, navigated or not, until Legal signs off — Mintlify has no proven mechanism in this
repo to keep a committed file out of the public site.

The Browser Sandbox navigation addition is included. docs.json nests features/browser
under a new "Interact / Browser Sandbox" nav group and renames the API-reference group to
"Interact / Browser Sandbox Endpoints". If features/browser was a deliberate soft launch, drop
that part of the docs.json change and the rest of this PR still stands.

Summary

Third-party pages assert Firecrawl capability facts that Firecrawl's own snapshots contradict, and
the Interact and Browser Sandbox surfaces were not addressable at all: features/browser was absent
from navigation. This branch adds a canonical capabilities.mdx with a Cloud versus self-hosted
availability column repeated inline in every section, puts Browser Sandbox in the nav, and removes
the licensing section from the published page pending Legal. It also fixes five occurrences of a
broken anchor the original branch was shipping.

Why (evidence)

Source reports: Weekly Deep Insights DI-2026-09-03-WEEKLY (2026-09-03) and DI-2026-08-30-WEEKLY
(2026-08-30); Developer-bank positioning read
(analysis/loop/run/DEVELOPER-POSITIONING-READ.txt, 2026-08-31).

Findings: F5-CORRECTION-QUEUE, thesis "current third-party pages assert Firecrawl capability and
licence facts that other current snapshots contradict". Grade C plus, verdict file
verdicts/deep-F5-corrections.md, where only 2 of 7 targets were double-cited but those two are the
ones the branch acts on. OB-03-interactive-capability-not-addressable, grade B in the
2026-09-03 review: "documented but not addressable: it lives inside extraction pages behind a bounty
and a Preview label." Developer read hypothesis H3: the Interact sandbox is invisible to the
models reasoning about Firecrawl, with 0 of 140 developer answers mentioning Interact or Browser
Sandbox and 4 mentioning the legacy actions array; adjudications D2, D4, D12 and D13 are STALE.

Verified evidence: aimultiple.com asserts "Node.js only (no Python bindings)" against snapshots
showing from firecrawl import FirecrawlApp; dev.to asserts "Apache 2.0 licence" and "103,000
GitHub stars" against a README snapshot showing AGPL-3.0 core with MIT SDKs and 175.7k stars;
knowledgesdk.com asserts "Firecrawl cannot fill out a form." against docs/features/interact.
Developer traces VDEV-202, VDEV-222, VDEV-247 and VDEV-953 reason about actions as if it were the
whole product.

Availability facts are sourced to contributing/open-source-or-cloud.mdx:30 ("Agent, Browser,
managed dashboards, enhanced proxy paths, and enterprise controls are delivered through Cloud") and
the self-hosted feature-support table at contributing/self-host.mdx:175-180.

Final verdict: APPROVE per the resolution addendum to
review-grounding/FINAL-REVIEW-docs-mcp.md. The review body records REQUEST CHANGES; that blocker
is described and resolved below.

Changes

  • capabilities-and-licensing.mdx becomes capabilities.mdx, git-tracked as a rename, 192 lines.
    Sections: opening paragraph, ## Capabilities at a glance, 10 capability sections with runnable
    examples, ## Common misconceptions, GitHub-figures note.
  • The ## Capabilities at a glance table carries Capability | Availability | Documented in.
    Behavior contract: every capability section repeats its availability inline, so the fact survives
    being retrieved in isolation, which was the original defect. Interact, Browser Sandbox, login and
    session reuse, and screenshots and page actions are marked Cloud only, citing
    /contributing/self-host#self-hosted-feature-support and
    /contributing/open-source-or-cloud#use-firecrawl-cloud-when. JavaScript rendering, PDF parsing,
    and Search are Cloud and self-hosted; structured JSON extraction on self-hosted requires an
    OpenAI-compatible provider or Ollama. Two rows go beyond the brief (screenshots and page actions;
    the provider requirement for structured extraction) because the same self-host table states them
    and their absence would reproduce the same false-composite defect.
  • The opening paragraph drops the unconditional SDK-licence sentence and states Search correctly on
    the first screen: "returns titles, descriptions, and URLs, and full page content for every result
    when scrapeOptions is supplied."
  • The Search row and section state the default (titles, descriptions, URLs plus query-relevant
    Highlights) and the scrapeOptions / scrape_options contract, per features/search.mdx:32 and
    :34.
  • The self-host snippet is now git clone, cd firecrawl, git checkout v2.11.162, with prose
    pointing at the mandatory .env step and the docker compose invocation at
    contributing/self-host.mdx#configure-the-evaluation-deployment. The previous two-liner ran
    docker compose in the parent directory and created no .env.
  • The ## Licensing section (8-row table plus AGPL Note), the licence-dependent misconception row,
    and all three [LEGAL REVIEW: confirm] placeholders are removed from the published page. The
    misconceptions table keeps 5 rows: the licence row is replaced by a self-hosting-parity row.
  • docs.json: features/browser nested under a new "Interact / Browser Sandbox" nav group; the
    API-reference group renamed from "Interact Endpoints" to "Interact / Browser Sandbox Endpoints";
    the Get Started entry retargeted from capabilities-and-licensing to capabilities.
  • features/interact.mdx gains the jobs-to-be-done opener from OB-03.
  • Anchor bug fixed: the OB-03 opener and the capability page both linked
    #persistent-profiles-with-scrape--interact with a double dash. The live slug is
    #persistent-profiles-with-scrape-+-interact, verified against
    https://docs.firecrawl.dev/features/interact. Five occurrences corrected. This was shipping
    broken in the original branch.

Blocker resolved in commit 4d8be852. The branch had preserved the licensing content at
drafts/licensing-DRAFT.md on the claim that an unnavigated .md file has no URL. That claim is
false: https://docs.firecrawl.dev/reconciliation/INITIAL_DRIFT_REPORT returns 200 with a full
rendered Mintlify page, a title and an OG image, and so does .../CLAUDE, while
.../reconciliation/THIS_DOES_NOT_EXIST_XYZ returns 404 as a control. Absence from docs.json
removes the sidebar link, not the route, so the branch would have published a live public page whose
first line reads "DRAFT — Licensing (BLOCKED ON LEGAL — DO NOT PUBLISH)". No provably unrouted
in-repo location exists: docs.json has no ignore or exclude key of any kind (its only
top-level keys are $schema, colors, contextual, favicon, footer, integrations, logo, name, navbar, navigation, redirects, seo, theme), and the one live 404 found, docs.firecrawl.dev/README, is
Mintlify special-casing that literal root filename with no second README.md in the tree to confirm
it generalizes. The file was therefore removed with git rm; the draft content is preserved for
Legal in scratchpad/product-prs/D2-capabilities-licensing-v2.md and ships nowhere.

Diffstat: 3 files changed, 210 insertions, 2 deletions.

Verification

docs.json parses as valid JSON.

Nav resolution: 1,772 nav entries, 1,430 unique, 0 missing files. This is the check that catches the
rename regression.

Internal links on the touched files: all ](/…) targets in capabilities.mdx and
features/interact.mdx exist as .mdx files, 0 broken.

Anchors verified against the live rendered page rather than a local slug guess, which is how the
double-dash bug surfaced: #persistent-profiles-with-scrape-+-interact confirmed present in the
served HTML from curl https://docs.firecrawl.dev/features/interact, plus
#use-firecrawl-cloud-when (:27), #self-hosted-feature-support (:170),
#configure-the-evaluation-deployment (:73), #interact-via-prompting (:105),
#running-code (:126), #session-lifecycle (:228), and /features/search-highlights. HTTP 200
on all pages checked.

Negative control on published content: grep across all *.mdx excluding drafts/ returns zero hits
for LEGAL REVIEW and zero for SDKs are MIT. The only AGPL and MIT strings in published .mdx
are pre-existing ones in v1/introduction.mdx, its localizations, and
learn/guide/firecrawl-ui-template.mdx, none touched by this branch. No dangling reference to
capabilities-and-licensing remains in any non-localized .mdx or .json.

Diff audit for stray unnavigated content: git diff --name-status origin/main...HEAD lists only
capabilities.mdx (new, navigated), docs.json, features/interact.mdx and the removed
drafts/licensing-DRAFT.md.

mint broken-links under Node 22.23.2: 208 broken links in 95 files, identical to baseline; neither
capabilities.mdx nor features/interact.mdx appears. mint validate: the same 17 pre-existing
locale snippet warnings; no touched file appears.

Localized trees untouched, per the repo CLAUDE.md.

Not in this PR

  • Legal sign-off on the licensing section. The full draft, including the licence table, the
    AGPL-obligations placeholder, the removed misconception row and the "what must not be said" list,
    lives only in scratchpad/product-prs/D2-capabilities-licensing-v2.md. It must not be added back
    to the docs repo in any form, navigated or not, until Legal signs off, because Mintlify has no
    proven mechanism in this repo to keep a committed file out of the public site. Two things block
    publication: Legal must confirm the SDK licence position, and Core must fix the underlying
    declarations.
  • A Core PR, drafted but not filed. Core was not modified. Title:
    chore(licensing): fix stale GPLv3 declaration and add missing SDK LICENSE files. Item 1:
    apps/python-sdk/setup.py:67 declares license="GNU General Public License v3 (GPLv3)",
    contradicting apps/python-sdk/LICENSE (MIT) and apps/python-sdk/pyproject.toml:22
    (license = {text = "MIT License"}); that string is what lands in PyPI package metadata for any
    build still going through setup.py. Item 2: apps/go-sdk, apps/rust-sdk, apps/php-sdk,
    apps/java-sdk and apps/dot-net-sdk ship no LICENSE file, while README.md:905 describes the
    SDKs collectively as MIT, which is not a per-package grant under an AGPL-3.0 repository.
  • An Agent row in the capabilities table. Agent is covered in the intro paragraph and the
    self-hosting row but has no row of its own, so its Cloud-only status does not survive isolated
    retrieval the way every other capability's does. Optional nit from the review.

Links

Files changed:

  • capabilities.mdx (renamed from capabilities-and-licensing.mdx)
  • docs.json
  • features/interact.mdx

Evidence packet:
agent-experience-deepinsights-cleanroom/artifacts/deep-insights-sep3-verification-20260904/

Licensing draft for Legal:
scratchpad/product-prs/D2-capabilities-licensing-v2.md, section "Licensing draft (for Legal; not
in repo)".

hmishra2250 and others added 3 commits September 4, 2026 18:33
@hmishra2250 @claude
docs: add canonical capabilities-and-licensing page; make Interact/Br…
5128dc8 
…owser Sandbox addressable

Third-party pages assert capability and licence facts about Firecrawl that our
own artifacts contradict. Deep Insights Sep-3 verification (F5-CORRECTION-QUEUE,
grade C+, both-sides-cited rows) found:

- "Node.js only (no Python bindings)" — refuted; `from firecrawl import Firecrawl`
  appears in multiple snapshots and in /sdks/python.
- "Apache 2.0 licence" — refuted; the core repository LICENSE is AGPL-3.0 with
  MIT SDKs.
- "Firecrawl cannot fill out a form" — contradicted by /features/interact.

The verifier's one engineering fix is a canonical, extractable page whose first
~500 tokens answer each denied capability as a standalone declarative sentence
with a runnable example, and state the licence split plainly, so agents and
reviewers retrieve the canonical answer instead of the third-party one.

Adds capabilities-and-licensing.mdx:
- "Capabilities at a glance": ten declarative-sentence headings, each citing the
  in-repo page that documents it and carrying the smallest example already
  published in these docs.
- "Licensing": transcribed from LICENSE/README/package manifests in the
  firecrawl and firecrawl-mcp-server repositories. Facts not verifiable from a
  local repo file carry [LEGAL REVIEW: confirm] placeholders instead of an
  assertion, including the AGPL-obligation-trigger wording.
- "Common misconceptions": each denied capability mapped to the correct
  statement and its documentation link, neutral, no vendor names.

Also addresses OB-03 (Aug-30 wave, verified): the interact/authenticated
capability is documented but not addressable — features/browser was absent from
navigation, agents never named "Interact"/"Browser Sandbox" (0 of 140 developer
answers) and reasoned from the legacy `actions` array instead. This commit adds
an "Interact / Browser Sandbox" nav group carrying both features/interact and
features/browser, renames the API-reference group to "Interact / Browser Sandbox
Endpoints", and gives features/interact a job-to-be-done opener (scrape behind a
login; click through pagination; fill and submit a form; reuse an authenticated
session; start a session without scraping) linking only to sections that already
exist.

Evidence: Deep Insights F5-CORRECTION-QUEUE (2026-09-03) and OB-03 (2026-08-30).
No localized files or navigation blocks were modified.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@hmishra2250
docs: scope page to capabilities; split licensing out pending Legal
fcbf396 
Review outcome on docs/capabilities-and-licensing: the capability/
addressability half ships, the licensing half is blocked on Legal and
must not ship as canonical.

- Rename capabilities-and-licensing.mdx -> capabilities.mdx and update
  the Get Started nav entry.
- Remove the Licensing section, the AGPL obligations note, the
  licence-dependent misconception row, and every [LEGAL REVIEW: confirm]
  placeholder from the published page. Nothing in the shipped page now
  depends on an unresolved licence fact.
- Preserve the removed draft verbatim at drafts/licensing-DRAFT.md — a
  plain .md file in a non-navigated directory, so Mintlify does not build
  it as a page (same pattern as reconciliation/INITIAL_DRIFT_REPORT.md).
  It carries the blockers and the proposed Core PR for Legal to pick up.
- Opening paragraph no longer claims all SDKs are MIT. That claim is
  contradicted by apps/python-sdk/setup.py:67 (GPLv3) and by five SDKs
  that ship no LICENSE file at all.
- Add an Availability column (Cloud / self-hosted) to a new
  capabilities-at-a-glance table, plus an inline availability line on each
  capability section. Agent, Browser and Interact are Cloud-delivered per
  contributing/open-source-or-cloud.mdx:30 and
  contributing/self-host.mdx:180; screenshots and page actions likewise;
  structured JSON extraction needs an OpenAI-compatible provider or Ollama
  when self-hosted.
- Search: the first-screen sentence and the section heading now state the
  real contract — titles, descriptions and URLs by default, full page
  content when scrapeOptions is supplied (features/search.mdx:32).
- Self-host snippet is runnable: adds `cd firecrawl` and the pinned
  `git checkout v2.11.162`, and points at the required .env step rather
  than implying `docker compose up` is the next command
  (contributing/self-host.mdx:66-89).
- Fix the anchor introduced with the OB-03 opener: the live slug is
  #persistent-profiles-with-scrape-+-interact, not the double-dash form.

The OB-03 fix is kept intact: features/browser is addressable in nav, the
API-reference group is renamed to Interact / Browser Sandbox Endpoints,
and features/interact.mdx opens with jobs-to-be-done.

Deep Insights: F5 (capability claims not addressable from a canonical
page), OB-03 (Browser Sandbox unreachable from navigation).
@hmishra2250
fix(docs): remove licensing draft from repo, it is not provably unrouted
4d8be85 
Review found the D2 blocker: Mintlify routes unnavigated .md files as
live public pages (docs.firecrawl.dev/reconciliation/INITIAL_DRIFT_REPORT
-> 200), so drafts/licensing-DRAFT.md, absent only from docs.json nav,
would have deployed at /drafts/licensing-DRAFT with its own first line
reading "DO NOT PUBLISH".

- git rm drafts/licensing-DRAFT.md. Checked for a genuinely non-deployed
  location: docs.json has no ignore/exclude key of any kind, and the
  root README.md's 404 is Mintlify special-casing that literal filename,
  not a property of being unnavigated -- there is no second README.md in
  the tree to confirm it generalizes to a subdirectory. No provably
  unrouted location exists in this repo, so the draft is kept out
  entirely rather than relocated in-repo.
- Preserved the draft's full content (licence table, AGPL note, removed
  misconception row, "what must not be said" list, proposed Core PR) in
  the review report at
  scratchpad/product-prs/D2-capabilities-licensing-v2.md under
  "Licensing draft (for Legal; not in repo)".
- Grepped the branch diff (git diff --name-status origin/main...HEAD):
  the only other added file is capabilities.mdx, which is navigated in
  docs.json. No other unnavigated content added.
- docs.json parses; mint broken-links (Node 22.23.2): 208/95 files,
  byte-identical to origin/main baseline, neither touched file appears;
  mint validate: same pre-existing 17 warnings, none on touched files.
@mintlify

mintlify Bot commented Sep 4, 2026 •
edited
Loading

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
firecrawl 🟢 Ready View Preview Sep 4, 2026, 4:56 PM

💡 Tip: Enable Automations to automatically generate PRs for you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@hmishra2250