From 491ce01fb9c23d200a557ef242ad7d39e3122ad8 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 28 Aug 2026 13:22:48 +0000 Subject: [PATCH] Add agent quickstart docs for all SDK languages One-file-per-language canonical quickstart covering search, scrape, and interact for Node.js, Python, Rust, Java, and Elixir. Generated from SDK source and OpenAPI spec with full parameter documentation. Co-Authored-By: Claude Opus 4.6 (1M context) Claude-Session: https://claude.ai/code/session_01C2VosNMvZAudGEHJLVCgLE --- agent-quickstart/elixir.mdx | 241 ++++++++++++++++++++++++++++++++ agent-quickstart/java.mdx | 265 ++++++++++++++++++++++++++++++++++++ agent-quickstart/node.mdx | 236 ++++++++++++++++++++++++++++++++ agent-quickstart/python.mdx | 228 +++++++++++++++++++++++++++++++ agent-quickstart/rust.mdx | 260 +++++++++++++++++++++++++++++++++++ 5 files changed, 1230 insertions(+) create mode 100644 agent-quickstart/elixir.mdx create mode 100644 agent-quickstart/java.mdx create mode 100644 agent-quickstart/node.mdx create mode 100644 agent-quickstart/python.mdx create mode 100644 agent-quickstart/rust.mdx diff --git a/agent-quickstart/elixir.mdx b/agent-quickstart/elixir.mdx new file mode 100644 index 000000000..7dca4490d --- /dev/null +++ b/agent-quickstart/elixir.mdx @@ -0,0 +1,241 @@ +--- +title: "Elixir Agent Quickstart" +description: "Canonical Firecrawl Elixir quickstart for external agents using search, scrape, and interact." +--- + +# Firecrawl Elixir Agent Quickstart + +This file is the canonical quickstart for external agents integrating Firecrawl with Elixir. It is generated from SDK source (`:firecrawl` hex package v1.10.0) and the Firecrawl OpenAPI spec. + +The Elixir client is auto-generated from the OpenAPI spec. Function names follow the OpenAPI operation IDs rather than short aliases. + +## Install + +Add to your `mix.exs` dependencies: + +```elixir +defp deps do + [ + {:firecrawl, "~> 1.10"} + ] +end +``` + +## Authenticate + +```elixir +# Set the API key in application config +config :firecrawl, api_key: "fc-YOUR_API_KEY" + +# Or pass per-request +Firecrawl.search_and_scrape([query: "test"], api_key: "fc-YOUR_API_KEY") +``` + +Per-request options override the application config. The API key is optional: scrape, search, and interact fall back to the keyless free tier (rate-limited per IP). + +All functions accept a trailing keyword list `opts` that can include: +- `:api_key` -- override the API key per-request. +- `:base_url` -- override the base URL (default: `"https://api.firecrawl.dev/v2"`). +- Any other keys are passed through to `Req`. + +## When To Use What + +- **`search_and_scrape`**: Use when you start with a query and need discovery. Returns web, news, and/or image results, optionally with scraped content. +- **`scrape_and_extract_from_url`**: Use when you already have a URL and want page content in markdown, HTML, JSON, or other formats. +- **`interact_with_scrape_browser_session`**: Use when the page needs clicks, form fills, or post-scrape browser actions. Runs code in a browser sandbox tied to a scrape job. + +## Search + +### Why use it + +Search the web programmatically and optionally scrape each result page in a single call. Use it for discovery when you do not yet have specific URLs. + +### Preferred SDK function + +```elixir +Firecrawl.search_and_scrape(params \\ [], opts \\ []) +Firecrawl.search_and_scrape!(params \\ [], opts \\ []) +``` + +The bang variant (`!`) raises on error instead of returning `{:error, ...}`. + +### Example + +```elixir +{:ok, response} = Firecrawl.search_and_scrape( + query: "firecrawl web scraping API", + limit: 5, + scrape_options: [formats: ["markdown"]] +) + +response.body["data"]["web"] +|> Enum.each(fn item -> + IO.puts("#{item["url"]} #{item["title"]}") +end) +``` + +### Parameters + +Passed as a keyword list. The `query` key is required. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `query` | `:string` | **yes** | The search query. | +| `sources` | `{:list, :any}` | no | Sources: `"web"`, `"news"`, `"images"`. Default: `["web"]`. | +| `categories` | `{:list, :any}` | no | Filter: `"github"`, `"research"`, `"pdf"`, `"developer"`. Default: `[]`. | +| `include_domains` | `{:list, :string}` | no | Restrict results to these domains. Cannot be used with `exclude_domains`. | +| `exclude_domains` | `{:list, :string}` | no | Exclude results from these domains. | +| `limit` | `:integer` | no | Maximum results per source type. | +| `tbs` | `:string` | no | Time-based filter (e.g. `"qdr:d"` for past day, `"sbd:1,qdr:w"` for sorted by date, past week). | +| `location` | `:string` | no | Location for geo-targeted results (e.g. `"San Francisco,California,United States"`). | +| `country` | `:string` | no | ISO country code (e.g. `"US"`). | +| `ignore_invalid_urls` | `:boolean` | no | Exclude URLs invalid for other Firecrawl endpoints. | +| `timeout` | `:integer` | no | Timeout in milliseconds. | +| `highlights` | `:boolean` | no | Generate query-relevant highlights. Default: `true`. | +| `scrape_options` | `:keyword_list` | no | Options for scraping search results. | +| `enterprise` | `{:list, :string}` | no | Enterprise ZDR options: `["zdr"]` or `["anon"]`. | + +### Response + +Returns `{:ok, %Req.Response{}}` with `body["data"]` containing: +- `"web"`: list of web results (with full document fields when `scrape_options` provided). +- `"news"`: list of news results. +- `"images"`: list of image results. + +## Scrape + +### Why use it + +Retrieve page content from a known URL. Returns markdown by default, with options for HTML, JSON extraction, screenshots, and more. + +### Preferred SDK function + +```elixir +Firecrawl.scrape_and_extract_from_url(params \\ [], opts \\ []) +Firecrawl.scrape_and_extract_from_url!(params \\ [], opts \\ []) +``` + +### Example + +```elixir +{:ok, response} = Firecrawl.scrape_and_extract_from_url( + url: "https://example.com", + formats: ["markdown", "links"], + only_main_content: true +) + +IO.puts(response.body["data"]["markdown"]) +IO.inspect(response.body["data"]["links"]) +``` + +### Parameters + +Passed as a keyword list. The `url` key is required. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `url` | `:string` | **yes** | The URL to scrape. | +| `formats` | `{:list, :any}` | no | Output formats: `"markdown"`, `"html"`, `"rawHtml"`, `"links"`, `"images"`, `"screenshot"`, `"summary"`, `"json"`, `"audio"`, `"video"`, or format objects. Default: `["markdown"]`. | +| `only_main_content` | `:boolean` | no | Exclude headers, navs, footers. | +| `include_tags` | `{:list, :string}` | no | Only include content from these HTML tags. | +| `exclude_tags` | `{:list, :string}` | no | Exclude content from these HTML tags. | +| `headers` | `:any` | no | Custom HTTP headers. | +| `timeout` | `:integer` | no | Timeout in ms. Min: 1000, default: 60000, max: 300000. | +| `wait_for` | `:integer` | no | Delay in ms before scraping. | +| `mobile` | `:boolean` | no | Emulate mobile device. | +| `actions` | `{:list, :any}` | no | Browser actions before scraping. | +| `location` | `:keyword_list` | no | Geo settings (e.g. `[country: "US", languages: ["en-US"]]`). | +| `parsers` | `{:list, :any}` | no | Parser config (e.g. `[%{type: "pdf", mode: "auto"}]`). | +| `skip_tls_verification` | `:boolean` | no | Skip TLS verification. | +| `remove_base64_images` | `:boolean` | no | Remove base64 images. | +| `block_ads` | `:boolean` | no | Block ads and cookie popups. | +| `proxy` | `:basic \| :enhanced \| :auto` | no | Proxy type. | +| `max_age` | `:integer` | no | Use cached result if younger than this (ms). | +| `min_age` | `:integer` | no | Only check cache, never trigger fresh scrape. | +| `store_in_cache` | `:boolean` | no | Cache the result. | +| `lockdown` | `:boolean` | no | Only serve cached results. | +| `redact_pii` | `:boolean` | no | Redact PII from markdown. | +| `profile` | `:keyword_list` | no | Persistent browser profile: `[name: "my-profile", save_changes: true]`. | +| `audit_metadata` | `:keyword_list` | no | SIEM logging: `[username: "user@example.com"]`. | +| `zero_data_retention` | `:boolean` | no | Enable zero data retention. | + +### Response + +Returns `{:ok, %Req.Response{}}` with `body["data"]` containing document fields matching the requested formats. + +## Interact + +### Why use it + +Run code in the browser sandbox of an existing scrape job. Use it for post-scrape browser automation: filling forms, clicking buttons, extracting dynamic content. + +### Preferred SDK function + +```elixir +Firecrawl.interact_with_scrape_browser_session(job_id, params \\ [], opts \\ []) +Firecrawl.interact_with_scrape_browser_session!(job_id, params \\ [], opts \\ []) +``` + +### Example + +```elixir +{:ok, response} = Firecrawl.interact_with_scrape_browser_session( + "job-id-from-scrape", + code: "document.title", + language: :node, + timeout: 30 +) + +IO.puts(response.body["stdout"]) + +# When done, stop the browser session +Firecrawl.stop_interactive_scrape_browser_session("job-id-from-scrape") +``` + +### Parameters + +The first argument is the `job_id` (String). Remaining parameters are passed as a keyword list. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `job_id` | `String.t()` | **yes** | The scrape job ID (first positional argument). | +| `code` | `:string` | **yes** | Code to execute in the browser sandbox. | +| `language` | `:python \| :node \| :bash` | no | Runtime language. Default: `:node`. | +| `timeout` | `:integer` | no | Execution timeout in seconds. | +| `origin` | `:string` | no | Origin label for telemetry. | + +### Response + +Returns `{:ok, %Req.Response{}}` with body containing: +- `"success"`: boolean +- `"cdpUrl"`: CDP WebSocket URL. +- `"liveViewUrl"`: read-only live view URL. +- `"interactiveLiveViewUrl"`: interactive live view URL. +- `"stdout"`, `"result"`: standard output. +- `"stderr"`: standard error. +- `"exitCode"`: process exit code. +- `"killed"`: whether killed due to timeout. +- `"error"`: error message. + +### Companion function + +```elixir +Firecrawl.stop_interactive_scrape_browser_session(job_id, opts \\ []) +Firecrawl.stop_interactive_scrape_browser_session!(job_id, opts \\ []) +``` + +Stops the browser session (sends `DELETE /scrape/{jobId}/interact`). + +## Notes + +- **OpenAPI-shaped client**: The Elixir client is auto-generated from the OpenAPI spec. Function names follow the `operationId` (e.g. `search_and_scrape`, `scrape_and_extract_from_url`, `interact_with_scrape_browser_session`). +- **Naming**: All parameter keys use snake_case atoms (e.g. `:only_main_content`, `:include_tags`). The SDK converts to camelCase JSON keys. +- **No prompt support**: Unlike the JS, Python, and Rust SDKs, the Elixir SDK `interact_with_scrape_browser_session` requires `code` and does not support a `prompt` parameter for natural-language browser agent instructions. +- **Bang variants**: Every function has a `!` variant that raises `Firecrawl.Error` instead of returning `{:error, ...}`. +- **NimbleOptions validation**: All parameters are validated at the SDK level before the request is sent. + +## Source Of Truth + +- `firecrawl/apps/elixir-sdk/lib/firecrawl.ex` +- `firecrawl/apps/elixir-sdk/mix.exs` +- `firecrawl-docs/api-reference/v2-openapi.json` diff --git a/agent-quickstart/java.mdx b/agent-quickstart/java.mdx new file mode 100644 index 000000000..18d14a33d --- /dev/null +++ b/agent-quickstart/java.mdx @@ -0,0 +1,265 @@ +--- +title: "Java Agent Quickstart" +description: "Canonical Firecrawl Java quickstart for external agents using search, scrape, and interact." +--- + +# Firecrawl Java Agent Quickstart + +This file is the canonical quickstart for external agents integrating Firecrawl with Java. It is generated from SDK source (`com.firecrawl:firecrawl-java` v1.16.0) and the Firecrawl OpenAPI spec. + +## Install + +**Gradle:** + +```groovy +implementation("com.firecrawl:firecrawl-java:1.16.0") +``` + +**Maven:** + +```xml + + com.firecrawl + firecrawl-java + 1.16.0 + +``` + +Requires Java 11+. + +## Authenticate + +```java +import com.firecrawl.client.FirecrawlClient; + +// Builder pattern +FirecrawlClient client = FirecrawlClient.builder() + .apiKey("fc-YOUR_API_KEY") + .build(); + +// With options +FirecrawlClient client = FirecrawlClient.builder() + .apiKey("fc-YOUR_API_KEY") + .apiUrl("https://api.firecrawl.dev") // optional, for self-hosted + .timeoutMs(300_000L) // default 300000 (5 min) + .maxRetries(3) // default 3 + .backoffFactor(0.5) // default 0.5 + .build(); + +// From environment (reads FIRECRAWL_API_KEY env var or firecrawl.apiKey system property) +FirecrawlClient client = FirecrawlClient.fromEnv(); +``` + +The API key falls back to `FIRECRAWL_API_KEY` env var, then `firecrawl.apiKey` system property. A null/blank key enables the keyless free tier (rate-limited per IP). + +## When To Use What + +- **`search`**: Use when you start with a query and need discovery. Returns web, news, and/or image results, optionally with scraped content. +- **`scrape`**: Use when you already have a URL and want page content in markdown, HTML, JSON, or other formats. +- **`interact`**: Use when the page needs clicks, form fills, or post-scrape browser actions. Runs code in a browser sandbox tied to a scrape job. + +## Search + +### Why use it + +Search the web programmatically and optionally scrape each result page in a single call. Use it for discovery when you do not yet have specific URLs. + +### Preferred SDK method + +```java +SearchData search(String query) +SearchData search(String query, SearchOptions options) +``` + +### Example + +```java +import com.firecrawl.models.SearchOptions; +import com.firecrawl.models.SearchData; +import com.firecrawl.models.ScrapeOptions; + +SearchData results = client.search("firecrawl web scraping API", + SearchOptions.builder() + .limit(5) + .scrapeOptions(ScrapeOptions.builder().formats(List.of("markdown")).build()) + .build()); + +for (Map item : results.getWeb()) { + System.out.println(item.get("url") + " " + item.get("title")); +} +``` + +### Parameters + +`SearchOptions` -- all fields nullable, built via `SearchOptions.builder()`. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `query` | `String` | The search query (required, first argument to `search()`). | +| `sources` | `List` | Sources: `"web"`, `"news"`, `"images"` as strings or `{type: "web"}` maps. Default: `["web"]`. | +| `categories` | `List` | Filter: `"github"`, `"research"`, `"pdf"`. | +| `includeDomains` | `List` | Restrict results to these domains. Cannot be used with `excludeDomains`. | +| `excludeDomains` | `List` | Exclude results from these domains. | +| `limit` | `Integer` | Maximum results per source type. | +| `tbs` | `String` | Time-based search filter (e.g. `"qdr:d"` for past day). | +| `location` | `String` | Location for geo-targeted results (e.g. `"US"`). | +| `ignoreInvalidURLs` | `Boolean` | Exclude URLs invalid for other Firecrawl endpoints. | +| `timeout` | `Integer` | Timeout in milliseconds. | +| `highlights` | `Boolean` | Generate query-relevant highlights. Default: `true`. | +| `scrapeOptions` | `ScrapeOptions` | Options applied to each search result page. | +| `integration` | `String` | Integration identifier. | + +### Response + +Returns `SearchData`: +- `web`: `List>` -- web results. +- `news`: `List>` -- news results. +- `images`: `List>` -- image results. + +## Scrape + +### Why use it + +Retrieve page content from a known URL. Returns markdown by default, with options for HTML, JSON extraction, screenshots, and more. + +### Preferred SDK method + +```java +Document scrape(String url) +Document scrape(String url, ScrapeOptions options) +``` + +### Example + +```java +import com.firecrawl.models.ScrapeOptions; +import com.firecrawl.models.Document; + +Document doc = client.scrape("https://example.com", + ScrapeOptions.builder() + .formats(List.of("markdown", "links")) + .onlyMainContent(true) + .build()); + +System.out.println(doc.getMarkdown()); +System.out.println(doc.getLinks()); +``` + +### Parameters + +`ScrapeOptions` -- all fields nullable, built via `ScrapeOptions.builder()`. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `url` | `String` | The URL to scrape (required, first argument to `scrape()`). | +| `formats` | `List` | Output formats. String values: `"markdown"`, `"html"`, `"rawHtml"`, `"links"`, `"images"`, `"screenshot"`, `"summary"`, `"json"`, `"audio"`, `"video"`. Also accepts typed objects: `JsonFormat`, `QuestionFormat`, `HighlightsFormat`. Default: `["markdown"]`. | +| `onlyMainContent` | `Boolean` | Exclude headers, navs, footers. Default: `true`. | +| `includeTags` | `List` | Only include content from these HTML tags. | +| `excludeTags` | `List` | Exclude content from these HTML tags. | +| `headers` | `Map` | Custom HTTP headers. | +| `timeout` | `Integer` | Timeout in milliseconds. Min: 1000, max: 300000. Default: 60000. | +| `waitFor` | `Integer` | Delay in ms before scraping. | +| `mobile` | `Boolean` | Emulate a mobile device. | +| `actions` | `List>` | Browser actions before scraping. | +| `location` | `LocationConfig` | Geo settings: `LocationConfig(country, languages)`. | +| `parsers` | `List` | Parser config. Use `PdfParser` for PDF options: `mode` (`"fast"`, `"auto"`, `"ocr"`), `maxPages`, `pages`, `blocks`, `pageMarkers`. | +| `skipTlsVerification` | `Boolean` | Skip TLS verification. | +| `removeBase64Images` | `Boolean` | Remove base64 images. | +| `blockAds` | `Boolean` | Block ads and cookie popups. | +| `proxy` | `String` | Proxy type: `"basic"`, `"stealth"`, `"enhanced"`, `"auto"`. | +| `maxAge` | `Long` | Use cached result if younger than this many ms. | +| `storeInCache` | `Boolean` | Cache the result. | +| `lockdown` | `Boolean` | Only serve cached results. | +| `redactPII` | `Boolean` | Redact PII from markdown. | +| `auditMetadata` | `AuditMetadata` | SIEM logging attribution (enterprise). | +| `integration` | `String` | Integration identifier. | + +### Response + +Returns `Document` with getters for: `markdown`, `html`, `rawHtml`, `json`, `summary`, `links`, `images`, `screenshot`, `audio`, `video`, `attributes`, `actions`, `answer`, `highlights`, `warning`, `changeTracking`, `branding`, `product`, `menu`, `pages`, `blocks`, `metadata`. + +## Interact + +### Why use it + +Run code in the browser sandbox of an existing scrape job. Use it for post-scrape browser automation: filling forms, clicking buttons, extracting dynamic content. + +### Preferred SDK method + +```java +BrowserExecuteResponse interact(String jobId, String code) +BrowserExecuteResponse interact(String jobId, String code, String language, Integer timeout) +BrowserExecuteResponse interact(String jobId, String code, String language, Integer timeout, String origin) +``` + +### Example + +```java +import com.firecrawl.models.BrowserExecuteResponse; + +BrowserExecuteResponse result = client.interact("job-id-from-scrape", "document.title"); +System.out.println(result.getStdout()); + +// With language and timeout +BrowserExecuteResponse result = client.interact( + "job-id-from-scrape", + "document.title", + "node", + 30 +); + +// When done, stop the browser session +client.stopInteractiveBrowser("job-id-from-scrape"); +``` + +### Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| `jobId` | `String` | The scrape job ID (required). | +| `code` | `String` | Code to execute in the browser sandbox (required). Max: 100000 chars. | +| `language` | `String` | Runtime language: `"python"`, `"node"`, `"bash"`. Default: `"node"`. | +| `timeout` | `Integer` | Execution timeout in seconds. Min: 1, max: 300. Default: 30. | +| `origin` | `String` | Origin label for telemetry. Auto-set to `"java-sdk@{version}"` if null. | + +### Response + +Returns `BrowserExecuteResponse`: +- `success`: `boolean` +- `stdout`: Standard output. +- `result`: Standard output (alias for stdout). +- `stderr`: Standard error output. +- `exitCode`: Process exit code. +- `killed`: Whether the process was killed due to timeout. +- `error`: Error message if execution failed. + +### Companion method + +```java +BrowserDeleteResponse stopInteractiveBrowser(String jobId) +``` + +Stops the browser session and returns `sessionDurationMs` and `creditsBilled`. + +### Async variants + +All methods have async counterparts returning `CompletableFuture`: +- `scrapeAsync(url, options)` +- `searchAsync(query, options)` +- `interactAsync(jobId, code)` / `interactAsync(jobId, code, language, timeout)` / `interactAsync(jobId, code, language, timeout, origin)` +- `stopInteractiveBrowserAsync(jobId)` + +## Notes + +- **Naming**: All parameters use camelCase (e.g. `onlyMainContent`, `includeTags`, `skipTlsVerification`). +- **Deprecated aliases**: `scrapeExecute()` (all overloads) is a deprecated alias for `interact()`. `deleteScrapeBrowser()` is a deprecated alias for `stopInteractiveBrowser()`. Use the preferred methods. +- **No prompt support**: Unlike the JS, Python, and Rust SDKs, the Java SDK `interact` method requires `code` and does not support a `prompt` parameter for natural-language browser agent instructions. +- **Overloads**: The `interact` method has three overloads with increasing parameter specificity. The simplest form uses `"node"` language and server-default timeout. + +## Source Of Truth + +- `firecrawl/apps/java-sdk/src/main/java/com/firecrawl/client/FirecrawlClient.java` +- `firecrawl/apps/java-sdk/src/main/java/com/firecrawl/models/ScrapeOptions.java` +- `firecrawl/apps/java-sdk/src/main/java/com/firecrawl/models/SearchOptions.java` +- `firecrawl/apps/java-sdk/src/main/java/com/firecrawl/models/Document.java` +- `firecrawl-docs/api-reference/v2-openapi.json` diff --git a/agent-quickstart/node.mdx b/agent-quickstart/node.mdx new file mode 100644 index 000000000..c643759c9 --- /dev/null +++ b/agent-quickstart/node.mdx @@ -0,0 +1,236 @@ +--- +title: "Node.js Agent Quickstart" +description: "Canonical Firecrawl Node.js/TypeScript quickstart for external agents using search, scrape, and interact." +--- + +# Firecrawl Node.js Agent Quickstart + +This file is the canonical quickstart for external agents integrating Firecrawl with Node.js or TypeScript. It is generated from SDK source (`@mendable/firecrawl-js` v4.37.0) and the Firecrawl OpenAPI spec. + +## Install + +```bash +npm install @mendable/firecrawl-js +``` + +## Authenticate + +```typescript +import Firecrawl from "@mendable/firecrawl-js"; + +// With an API key +const app = new Firecrawl("fc-YOUR_API_KEY"); + +// Or with options +const app = new Firecrawl({ + apiKey: "fc-YOUR_API_KEY", + apiUrl: "https://api.firecrawl.dev", // optional, for self-hosted +}); + +// Keyless (free tier, rate-limited per IP) +const app = new Firecrawl(); +``` + +The API key falls back to the `FIRECRAWL_API_KEY` environment variable. The API URL falls back to `FIRECRAWL_API_URL` or `https://api.firecrawl.dev`. + +## When To Use What + +- **`search`**: Use when you start with a query and need discovery. Returns web, news, and/or image results, optionally with scraped content. +- **`scrape`**: Use when you already have a URL and want page content in markdown, HTML, JSON, or other formats. +- **`interact`**: Use when the page needs clicks, form fills, or post-scrape browser actions. Runs code or a natural-language prompt in a browser sandbox tied to a scrape job. + +## Search + +### Why use it + +Search the web programmatically and optionally scrape each result page in a single call. Use it for discovery when you do not yet have specific URLs. + +### Preferred SDK method + +```typescript +app.search(query: string, options?: Omit): Promise +``` + +### Example + +```typescript +const results = await app.search("firecrawl web scraping API", { + limit: 5, + scrapeOptions: { formats: ["markdown"] }, +}); + +for (const item of results.web ?? []) { + console.log(item.url, item.title); +} +``` + +### Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| `query` | `string` | The search query (required, first positional argument). | +| `sources` | `Array<"web" \| "news" \| "images" \| { type: ... }>` | Sources to search. Determines which arrays appear in the response. Default: `["web"]`. | +| `categories` | `Array<"github" \| "research" \| "pdf" \| "developer" \| { type: ... }>` | Filter results by category. Default: `[]` (no filter). | +| `includeDomains` | `string[]` | Restrict results to these domains. Cannot be used with `excludeDomains`. | +| `excludeDomains` | `string[]` | Exclude results from these domains. Cannot be used with `includeDomains`. | +| `limit` | `number` | Maximum number of results per source type. | +| `tbs` | `string` | Time-based search filter. Examples: `"qdr:d"` (past day), `"qdr:w"` (past week), `"sbd:1,qdr:w"` (sorted by date, past week). | +| `location` | `string` | Location string for geo-targeted results (e.g. `"San Francisco,California,United States"`). | +| `ignoreInvalidURLs` | `boolean` | Exclude URLs invalid for other Firecrawl endpoints. Default: `false`. | +| `timeout` | `number` | Timeout in milliseconds. | +| `highlights` | `boolean` | Generate query-relevant highlights. Default: `true`. | +| `scrapeOptions` | `ScrapeOptions` | Options applied to each search result page. Pass this to get full page content. | +| `enterprise` | `Array<"default" \| "anon" \| "zdr">` | Enterprise Zero Data Retention options. | +| `threatProtection` | `ThreatProtectionOptions` | Per-request threat protection override (enterprise). | +| `integration` | `string` | Integration identifier for tracking. | +| `origin` | `string` | Origin label for request attribution. | + +### Response + +Returns `SearchData` with optional arrays: +- `web`: `Array` -- web results, with full `Document` fields when `scrapeOptions` is provided. +- `news`: `Array` -- news results. +- `images`: `Array` -- image results. + +## Scrape + +### Why use it + +Retrieve page content from a known URL. Returns markdown by default, with options for HTML, JSON extraction, screenshots, and more. + +### Preferred SDK method + +```typescript +app.scrape(url: string, options?: ScrapeCallOptions): Promise +``` + +### Example + +```typescript +const doc = await app.scrape("https://example.com", { + formats: ["markdown", "links"], + onlyMainContent: true, +}); + +console.log(doc.markdown); +console.log(doc.links); +``` + +### Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| `url` | `string` | The URL to scrape (required, first positional argument). | +| `formats` | `FormatOption[]` | Output formats. String values: `"markdown"`, `"html"`, `"rawHtml"`, `"links"`, `"images"`, `"screenshot"`, `"summary"`, `"changeTracking"`, `"json"`, `"attributes"`, `"branding"`, `"product"`, `"menu"`, `"audio"`, `"video"`. Also accepts objects for `"json"`, `"screenshot"`, `"changeTracking"`, `"question"`, `"highlights"` with extra config. Default: `["markdown"]`. | +| `onlyMainContent` | `boolean` | Exclude headers, navs, footers. Default: `true`. | +| `includeTags` | `string[]` | Only include content from these HTML tags. | +| `excludeTags` | `string[]` | Exclude content from these HTML tags. | +| `headers` | `Record` | Custom HTTP headers (cookies, user-agent, etc.). | +| `timeout` | `number` | Timeout in milliseconds. Min: 1000, max: 300000. Default: 60000. | +| `waitFor` | `number` | Additional delay in ms before scraping (for JS rendering). | +| `mobile` | `boolean` | Emulate a mobile device. Default: `false`. | +| `actions` | `ActionOption[]` | Browser actions to perform before scraping (`wait`, `click`, `write`, `press`, `scroll`, `screenshot`, `scrape`, `executeJavascript`, `pdf`). | +| `location` | `LocationConfig` | Geo settings: `{ country?: string, languages?: string[] }`. | +| `parsers` | `Array` | Parser config. PDFParser: `{ type: "pdf", mode?: "fast" \| "auto" \| "ocr", maxPages?: number, pages?: boolean, blocks?: boolean, pageMarkers?: boolean }`. | +| `skipTlsVerification` | `boolean` | Skip TLS certificate verification. | +| `removeBase64Images` | `boolean` | Remove base64 images from output. Default: `true`. | +| `fastMode` | `boolean` | Faster scraping with reduced accuracy. | +| `blockAds` | `boolean` | Block ads and cookie popups. Default: `true`. | +| `proxy` | `"basic" \| "stealth" \| "enhanced" \| "auto" \| string` | Proxy type. Default: `"auto"`. | +| `maxAge` | `number` | Use cached result if younger than this many ms. Default: 172800000 (2 days). | +| `minAge` | `number` | Only check cache, never trigger fresh scrape. | +| `storeInCache` | `boolean` | Cache the result. Default: `true`. | +| `lockdown` | `boolean` | Only serve cached results, never make outbound requests. | +| `redactPII` | `boolean \| RedactPIIOptions` | Redact PII from markdown. Options: `{ mode?: "accurate" \| "aggressive" \| "fast", entities?: Array<"PERSON" \| "EMAIL" \| "PHONE" \| "LOCATION" \| "FINANCIAL" \| "SECRET">, replaceStyle?: "tag" \| "mask" \| "remove" }`. | +| `threatProtection` | `ThreatProtectionOptions` | Threat protection override (enterprise). | +| `profile` | `{ name: string, saveChanges?: boolean }` | Persistent browser profile for maintaining state across scrapes. | +| `auditMetadata` | `AuditMetadata` | User attribution for SIEM logging (enterprise). | +| `integration` | `string` | Integration identifier. | +| `origin` | `string` | Origin label for attribution. | +| `autoResume` | `boolean` | SDK-only. Auto-retry on large documents (e.g. big PDFs). Default: `true`. | + +### Response + +Returns a `Document` with fields matching the requested formats: `markdown`, `html`, `rawHtml`, `json`, `summary`, `links`, `images`, `screenshot`, `audio`, `video`, `answer`, `highlights`, `changeTracking`, `branding`, `product`, `menu`, `pages`, `blocks`, `metadata`, `warning`. + +## Interact + +### Why use it + +Run code or a natural-language prompt in the browser sandbox of an existing scrape job. Use it for post-scrape browser automation: filling forms, clicking buttons, extracting dynamic content. + +### Preferred SDK method + +```typescript +app.interact(jobId: string, args: ScrapeExecuteRequest): Promise +``` + +### Example + +```typescript +// First, scrape a page with actions to keep the browser alive +const doc = await app.scrape("https://example.com", { + formats: ["markdown"], +}); +const jobId = doc.metadata?.sourceURL; // use the job ID from the scrape response + +// Execute code in the browser +const result = await app.interact(jobId, { + code: "document.title", + language: "node", + timeout: 30, +}); + +console.log(result.stdout); + +// When done, stop the browser session +await app.stopInteraction(jobId); +``` + +### Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| `jobId` | `string` | The scrape job ID (required, first positional argument). | +| `code` | `string` | Code to execute in the browser sandbox. At least one of `code` or `prompt` is required. Max: 100000 chars. | +| `prompt` | `string` | Natural-language instruction for the browser agent. At least one of `code` or `prompt` is required. | +| `language` | `"python" \| "node" \| "bash"` | Runtime language for code execution. Default: `"node"`. | +| `timeout` | `number` | Execution timeout in seconds. Min: 1, max: 300. Default: 30. | +| `origin` | `string` | Origin label for telemetry. | + +### Response + +Returns `ScrapeExecuteResponse`: +- `success`: `boolean` +- `cdpUrl`: Raw CDP WebSocket URL for direct Playwright/Puppeteer connection. +- `liveViewUrl`: Read-only live view URL. +- `interactiveLiveViewUrl`: Interactive live view URL. +- `output`: AI agent response (only when using `prompt`). +- `stdout`, `result`: Standard output from executed code. +- `stderr`: Standard error output. +- `exitCode`: Process exit code. +- `killed`: Whether the process was killed due to timeout. +- `error`: Error message if execution failed. + +### Companion method + +```typescript +app.stopInteraction(jobId: string): Promise +``` + +Stops the interactive browser session and returns billing info (`sessionDurationMs`, `creditsBilled`). + +## Notes + +- **Naming**: All parameters use camelCase (e.g. `onlyMainContent`, `includeTags`, `skipTlsVerification`). +- **Deprecated aliases**: `scrapeUrl()` is a deprecated alias for `scrape()`. `scrapeExecute()` is a deprecated alias for `interact()`. `stopInteractiveBrowser()` and `deleteScrapeBrowser()` are deprecated aliases for `stopInteraction()`. Use the preferred methods. +- **Auto-resume**: The `scrape` method automatically retries on `processing_continues` signals for large documents (at most 5 resumes / 20 minutes). Set `autoResume: false` to disable. +- **Search validation**: `includeDomains` and `excludeDomains` cannot both be set. `limit` must be positive. `timeout` must be positive. + +## Source Of Truth + +- `firecrawl/apps/js-sdk/firecrawl/src/v2/client.ts` +- `firecrawl/apps/js-sdk/firecrawl/src/v2/types.ts` +- `firecrawl/apps/js-sdk/firecrawl/src/v2/methods/scrape.ts` +- `firecrawl/apps/js-sdk/firecrawl/src/v2/methods/search.ts` +- `firecrawl-docs/api-reference/v2-openapi.json` diff --git a/agent-quickstart/python.mdx b/agent-quickstart/python.mdx new file mode 100644 index 000000000..d4ab9002c --- /dev/null +++ b/agent-quickstart/python.mdx @@ -0,0 +1,228 @@ +--- +title: "Python Agent Quickstart" +description: "Canonical Firecrawl Python quickstart for external agents using search, scrape, and interact." +--- + +# Firecrawl Python Agent Quickstart + +This file is the canonical quickstart for external agents integrating Firecrawl with Python. It is generated from SDK source (`firecrawl-py` v4.40.0) and the Firecrawl OpenAPI spec. + +## Install + +```bash +pip install firecrawl-py +``` + +## Authenticate + +```python +from firecrawl import Firecrawl + +# With an API key +app = Firecrawl(api_key="fc-YOUR_API_KEY") + +# With options +app = Firecrawl( + api_key="fc-YOUR_API_KEY", + api_url="https://api.firecrawl.dev", # optional, for self-hosted + timeout=300, # default request timeout in seconds + max_retries=3, # default 3 + backoff_factor=0.5 # default 0.5 +) + +# Keyless (free tier, rate-limited per IP) +app = Firecrawl() +``` + +The API key falls back to the `FIRECRAWL_API_KEY` environment variable. + +## When To Use What + +- **`search`**: Use when you start with a query and need discovery. Returns web, news, and/or image results, optionally with scraped content. +- **`scrape`**: Use when you already have a URL and want page content in markdown, HTML, JSON, or other formats. +- **`interact`**: Use when the page needs clicks, form fills, or post-scrape browser actions. Runs code or a natural-language prompt in a browser sandbox tied to a scrape job. + +## Search + +### Why use it + +Search the web programmatically and optionally scrape each result page in a single call. Use it for discovery when you do not yet have specific URLs. + +### Preferred SDK method + +```python +app.search(query, *, sources=None, categories=None, ...) -> SearchData +``` + +### Example + +```python +results = app.search("firecrawl web scraping API", limit=5, scrape_options=ScrapeOptions(formats=["markdown"])) + +for item in results.web or []: + print(item.url, item.title) +``` + +### Parameters + +All parameters after `query` are keyword-only. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `query` | `str` | The search query (required, positional). | +| `sources` | `list[str \| Source]` | Sources to search: `"web"`, `"news"`, `"images"`. Default: `["web"]`. | +| `categories` | `list[str \| Category]` | Filter by category: `"github"`, `"research"`, `"pdf"`, `"developer"`. | +| `include_domains` | `list[str]` | Restrict results to these domains. Cannot be used with `exclude_domains`. | +| `exclude_domains` | `list[str]` | Exclude results from these domains. Cannot be used with `include_domains`. | +| `limit` | `int` | Maximum results per source type. Default: 5. | +| `tbs` | `str` | Time-based search filter. Examples: `"qdr:d"` (past day), `"qdr:w"` (past week). | +| `location` | `str` | Location for geo-targeted results (e.g. `"San Francisco,California,United States"`). | +| `ignore_invalid_urls` | `bool` | Exclude URLs invalid for other Firecrawl endpoints. Default: `False`. | +| `timeout` | `int` | Timeout in milliseconds. Default: 300000. | +| `highlights` | `bool` | Generate query-relevant highlights. Default: `True`. | +| `scrape_options` | `ScrapeOptions` | Options applied to each search result page. Pass this to get full page content. | +| `integration` | `str` | Integration identifier for tracking. | +| `enterprise` | `list[str]` | Enterprise ZDR options: `["zdr"]` or `["anon"]`. | +| `threat_protection` | `ThreatProtectionOptions` | Per-request threat protection override (enterprise). | + +### Response + +Returns `SearchData` with optional fields: +- `web`: `list[SearchResultWeb | Document]` -- web results, with full `Document` fields when `scrape_options` is provided. +- `news`: `list[SearchResultNews | Document]` -- news results. +- `images`: `list[SearchResultImages]` -- image results. + +## Scrape + +### Why use it + +Retrieve page content from a known URL. Returns markdown by default, with options for HTML, JSON extraction, screenshots, and more. + +### Preferred SDK method + +```python +app.scrape(url, *, formats=None, headers=None, ...) -> Document +``` + +### Example + +```python +doc = app.scrape("https://example.com", formats=["markdown", "links"], only_main_content=True) + +print(doc.markdown) +print(doc.links) +``` + +### Parameters + +All parameters after `url` are keyword-only. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `url` | `str` | The URL to scrape (required, positional). | +| `formats` | `list[FormatOption]` | Output formats. String values: `"markdown"`, `"html"`, `"rawHtml"`, `"links"`, `"images"`, `"screenshot"`, `"summary"`, `"changeTracking"`, `"json"`, `"attributes"`, `"branding"`, `"product"`, `"menu"`, `"audio"`, `"video"`. Also accepts typed format objects. Default: `["markdown"]`. | +| `only_main_content` | `bool` | Exclude headers, navs, footers. Default: `True`. | +| `include_tags` | `list[str]` | Only include content from these HTML tags. | +| `exclude_tags` | `list[str]` | Exclude content from these HTML tags. | +| `headers` | `dict[str, str]` | Custom HTTP headers (cookies, user-agent, etc.). | +| `timeout` | `int` | Timeout in milliseconds. Min: 1000, max: 300000. Default: 60000. | +| `wait_for` | `int` | Additional delay in ms before scraping. | +| `mobile` | `bool` | Emulate a mobile device. Default: `False`. | +| `actions` | `list[ActionOption]` | Browser actions to perform before scraping (`WaitAction`, `ClickAction`, `WriteAction`, `PressAction`, `ScrollAction`, `ScreenshotAction`, `ScrapeAction`, `ExecuteJavascriptAction`, `PDFAction`). | +| `location` | `Location` | Geo settings: `Location(country="US", languages=["en-US"])`. | +| `parsers` | `list[str \| PDFParser]` | Parser config. `PDFParser(type="pdf", mode="auto", max_pages=None, pages=False, blocks=False, page_markers=False)`. | +| `skip_tls_verification` | `bool` | Skip TLS certificate verification. | +| `remove_base64_images` | `bool` | Remove base64 images from output. Default: `True`. | +| `fast_mode` | `bool` | Faster scraping with reduced accuracy. | +| `block_ads` | `bool` | Block ads and cookie popups. Default: `True`. | +| `proxy` | `str` | Proxy type: `"basic"`, `"stealth"`, `"enhanced"`, `"auto"`. Default: `"auto"`. | +| `max_age` | `int` | Use cached result if younger than this many ms. Default: 172800000 (2 days). | +| `store_in_cache` | `bool` | Cache the result. Default: `True`. | +| `lockdown` | `bool` | Only serve cached results, never make outbound requests. | +| `threat_protection` | `ThreatProtectionOptions` | Threat protection override (enterprise). | +| `profile` | `dict[str, Any]` | Persistent browser profile: `{"name": "my-profile", "saveChanges": True}`. | +| `audit_metadata` | `AuditMetadata` | User attribution for SIEM logging (enterprise). | +| `integration` | `str` | Integration identifier. | +| `auto_resume` | `bool` | SDK-only. Auto-retry on large documents (e.g. big PDFs). | + +### Response + +Returns a `Document` with fields matching the requested formats: `markdown`, `html`, `raw_html`, `json`, `summary`, `links`, `images`, `screenshot`, `audio`, `video`, `answer`, `highlights`, `change_tracking`, `branding`, `product`, `menu`, `pages`, `blocks`, `metadata`, `warning`. + +## Interact + +### Why use it + +Run code or a natural-language prompt in the browser sandbox of an existing scrape job. Use it for post-scrape browser automation: filling forms, clicking buttons, extracting dynamic content. + +### Preferred SDK method + +```python +app.interact(job_id, code=None, *, prompt=None, language="node", timeout=None, origin=None) -> BrowserExecuteResponse +``` + +### Example + +```python +# First, scrape a page +doc = app.scrape("https://example.com", formats=["markdown"]) +job_id = doc.metadata.source_url # use the job ID from the scrape response + +# Execute code in the browser +result = app.interact(job_id, code="document.title", language="node", timeout=30) +print(result.stdout) + +# Or use a natural-language prompt +result = app.interact(job_id, prompt="Click the login button and fill in the email field with test@example.com") +print(result.output) + +# When done, stop the browser session +app.stop_interaction(job_id) +``` + +### Parameters + +| Parameter | Type | Description | +|-----------|------|-------------| +| `job_id` | `str` | The scrape job ID (required, positional). | +| `code` | `str \| None` | Code to execute in the browser sandbox. At least one of `code` or `prompt` is required. | +| `prompt` | `str \| None` | Natural-language instruction for the browser agent. At least one of `code` or `prompt` is required. Keyword-only. | +| `language` | `"python" \| "node" \| "bash"` | Runtime language for code execution. Default: `"node"`. Keyword-only. | +| `timeout` | `int \| None` | Execution timeout in seconds. Min: 1, max: 300. Keyword-only. | +| `origin` | `str \| None` | Origin label for telemetry. Keyword-only. | + +### Response + +Returns `BrowserExecuteResponse`: +- `success`: `bool` +- `cdp_url`: Raw CDP WebSocket URL for direct connection. +- `live_view_url`: Read-only live view URL. +- `interactive_live_view_url`: Interactive live view URL. +- `output`: AI agent response (only when using `prompt`). +- `stdout`, `result`: Standard output from executed code. +- `stderr`: Standard error output. +- `exit_code`: Process exit code. +- `killed`: Whether the process was killed due to timeout. +- `error`: Error message if execution failed. + +### Companion method + +```python +app.stop_interaction(job_id: str) -> BrowserDeleteResponse +``` + +Stops the interactive browser session and returns billing info (`session_duration_ms`, `credits_billed`). + +## Notes + +- **Naming**: All parameters use snake_case (e.g. `only_main_content`, `include_tags`, `skip_tls_verification`). The SDK handles conversion to camelCase for the API. +- **Deprecated aliases**: `FirecrawlApp` is a deprecated alias for `Firecrawl`. `scrape_url()` is a deprecated alias for `scrape()`. `scrape_execute()` is a deprecated alias for `interact()`. `stop_interactive_browser()` and `delete_scrape_browser()` are deprecated aliases for `stop_interaction()`. Use the preferred methods. +- **Async support**: An `AsyncFirecrawl` client is available for async/await usage with the same API surface. +- **Search validation**: `include_domains` and `exclude_domains` cannot both be set. `limit` must be positive and <= 100. `timeout` must be positive and <= 300000. + +## Source Of Truth + +- `firecrawl/apps/python-sdk/firecrawl/client.py` +- `firecrawl/apps/python-sdk/firecrawl/v2/client.py` +- `firecrawl/apps/python-sdk/firecrawl/v2/types.py` +- `firecrawl-docs/api-reference/v2-openapi.json` diff --git a/agent-quickstart/rust.mdx b/agent-quickstart/rust.mdx new file mode 100644 index 000000000..019be777c --- /dev/null +++ b/agent-quickstart/rust.mdx @@ -0,0 +1,260 @@ +--- +title: "Rust Agent Quickstart" +description: "Canonical Firecrawl Rust quickstart for external agents using search, scrape, and interact." +--- + +# Firecrawl Rust Agent Quickstart + +This file is the canonical quickstart for external agents integrating Firecrawl with Rust. It is generated from SDK source (`firecrawl` crate v2.17.0) and the Firecrawl OpenAPI spec. + +## Install + +Add to your `Cargo.toml`: + +```toml +[dependencies] +firecrawl = "2.17" +tokio = { version = "1", features = ["full"] } +``` + +## Authenticate + +```rust +use firecrawl::Client; + +// With an API key +let app = Client::new("fc-YOUR_API_KEY")?; + +// For self-hosted (API key is optional) +let app = Client::new_selfhosted("https://your-instance.com", Some("fc-YOUR_API_KEY"))?; + +// Keyless (free tier, rate-limited per IP) +let app = Client::new_selfhosted("https://api.firecrawl.dev", None::)?; +``` + +## When To Use What + +- **`search`**: Use when you start with a query and need discovery. Returns web, news, and/or image results, optionally with scraped content. +- **`scrape`**: Use when you already have a URL and want page content in markdown, HTML, JSON, or other formats. +- **`interact`**: Use when the page needs clicks, form fills, or post-scrape browser actions. Runs code or a natural-language prompt in a browser sandbox tied to a scrape job. + +## Search + +### Why use it + +Search the web programmatically and optionally scrape each result page in a single call. Use it for discovery when you do not yet have specific URLs. + +### Preferred SDK method + +```rust +app.search(query, options) -> Result +``` + +### Example + +```rust +use firecrawl::{Client, SearchOptions, ScrapeOptions}; + +let app = Client::new("fc-YOUR_API_KEY")?; + +let results = app.search("firecrawl web scraping API", SearchOptions { + limit: Some(5), + scrape_options: Some(ScrapeOptions::default()), + ..Default::default() +}).await?; + +if let Some(web) = &results.data.web { + for item in web { + println!("{:?}", item); + } +} +``` + +### Parameters + +`SearchOptions` -- all fields are `Option`, struct derives `Default`. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `query` | `impl AsRef` | The search query (required, first argument). | +| `limit` | `Option` | Maximum results. Default: 5, max: 20. | +| `sources` | `Option>` | Sources: `Web`, `News`, `Images`. Default: `[Web]`. | +| `categories` | `Option>` | Filter: `Github`, `Research`, `Pdf`. | +| `include_domains` | `Option>` | Restrict results to these domains. Cannot be used with `exclude_domains`. | +| `exclude_domains` | `Option>` | Exclude results from these domains. | +| `tbs` | `Option` | Time-based search filter (e.g. `"qdr:d"`). | +| `location` | `Option` | Geographic location string. | +| `ignore_invalid_urls` | `Option` | Exclude URLs invalid for other Firecrawl endpoints. | +| `timeout` | `Option` | Timeout in milliseconds. | +| `highlights` | `Option` | Generate query-relevant highlights. Default: `true`. | +| `scrape_options` | `Option` | Options applied to each result page. | +| `integration` | `Option` | Integration identifier. | +| `origin` | `Option` | Origin label. Auto-set to `"rust-sdk@{version}"` if `None`. | + +### Response + +Returns `SearchResponse` containing: +- `success`: `bool` +- `data`: `SearchData` with `web: Option>`, `news: Option>`, `images: Option>`. +- `warning`: `Option` + +`SearchResultOrDocument` is an enum: `WebResult(SearchResultWeb)` or `Document(Document)`. + +## Scrape + +### Why use it + +Retrieve page content from a known URL. Returns markdown by default, with options for HTML, JSON extraction, screenshots, and more. + +### Preferred SDK method + +```rust +app.scrape(url, options) -> Result +``` + +### Example + +```rust +use firecrawl::{Client, ScrapeOptions, Format}; + +let app = Client::new("fc-YOUR_API_KEY")?; + +let doc = app.scrape("https://example.com", ScrapeOptions { + formats: Some(vec![Format::Markdown, Format::Links]), + only_main_content: Some(true), + ..Default::default() +}).await?; + +println!("{:?}", doc.markdown); +println!("{:?}", doc.links); +``` + +### Parameters + +`ScrapeOptions` -- all fields are `Option`, struct derives `Default`. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `url` | `impl AsRef` | The URL to scrape (required, first argument). | +| `formats` | `Option>` | Output formats: `Markdown`, `Html`, `RawHtml`, `Links`, `Images`, `Screenshot`, `Summary`, `ChangeTracking`, `Json`, `Attributes`, `Branding`, `Product`, `Menu`, `Audio`, `Video`, `Question(QuestionFormat)`, `Highlights(HighlightsFormat)`. Default: `[Markdown]`. | +| `only_main_content` | `Option` | Exclude headers, navs, footers. Default: `true`. | +| `include_tags` | `Option>` | Only include content from these HTML tags. | +| `exclude_tags` | `Option>` | Exclude content from these HTML tags. | +| `headers` | `Option>` | Custom HTTP headers. | +| `timeout` | `Option` | Timeout in milliseconds. | +| `wait_for` | `Option` | Delay in ms before scraping. | +| `mobile` | `Option` | Emulate mobile device. | +| `actions` | `Option>` | Browser actions before scraping. | +| `location` | `Option` | Geo settings: `LocationConfig { country, languages }`. | +| `parsers` | `Option>` | Parser config (e.g. PDF: `ParserConfig::Pdf { mode, max_pages, pages, blocks, page_markers }`). | +| `skip_tls_verification` | `Option` | Skip TLS verification. | +| `remove_base64_images` | `Option` | Remove base64 images. | +| `fast_mode` | `Option` | Faster scraping. | +| `block_ads` | `Option` | Block ads and cookie popups. | +| `proxy` | `Option` | Proxy: `Basic`, `Stealth`, `Enhanced`, `Auto`. | +| `max_age` | `Option` | Use cached result if younger than this (seconds). | +| `min_age` | `Option` | Only check cache, never trigger fresh scrape. | +| `store_in_cache` | `Option` | Cache the result. | +| `lockdown` | `Option` | Only serve cached results. | +| `redact_pii` | `Option` | Redact PII from markdown. | +| `audit_metadata` | `Option` | SIEM logging attribution (enterprise). | +| `profile` | `Option` | Persistent browser profile: `ProfileConfig { name, save_changes }`. | +| `integration` | `Option` | Integration identifier. | +| `json_options` | `Option` | JSON extraction: `{ schema, system_prompt, prompt, check_prompt_injection }`. | +| `screenshot_options` | `Option` | Screenshot config: `{ full_page, quality, viewport }`. | +| `change_tracking_options` | `Option` | Change tracking config. | +| `attribute_selectors` | `Option>` | Attribute selectors for extraction. | +| `origin` | `Option` | Origin label. Auto-set to `"rust-sdk@{version}"` if `None`. | + +### Response + +Returns `Document` with fields: `markdown`, `html`, `raw_html`, `json`, `summary`, `links`, `images`, `screenshot`, `audio`, `video`, `metadata`, `actions`, `answer`, `highlights`, `warning`, `change_tracking`, `branding`, `product`, `menu`, `pages`, `blocks`. + +### Convenience method + +```rust +app.scrape_with_schema(url, schema, prompt) -> Result +``` + +Shortcut for JSON extraction with a schema. + +## Interact + +### Why use it + +Run code or a natural-language prompt in the browser sandbox of an existing scrape job. Use it for post-scrape browser automation: filling forms, clicking buttons, extracting dynamic content. + +### Preferred SDK method + +```rust +app.interact(job_id, options) -> Result +``` + +### Example + +```rust +use firecrawl::{Client, ScrapeExecuteOptions, ScrapeExecuteLanguage}; + +let app = Client::new("fc-YOUR_API_KEY")?; + +let result = app.interact("job-id-from-scrape", ScrapeExecuteOptions { + code: Some("document.title".into()), + language: Some(ScrapeExecuteLanguage::Node), + timeout: Some(30), + ..Default::default() +}).await?; + +println!("{:?}", result.stdout); + +// When done, stop the browser session +app.stop_interaction("job-id-from-scrape").await?; +``` + +### Parameters + +`ScrapeExecuteOptions` -- all fields are `Option`, struct derives `Default`. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `job_id` | `impl AsRef` | The scrape job ID (required, first argument). | +| `code` | `Option` | Code to execute. At least one of `code` or `prompt` is required. | +| `prompt` | `Option` | Natural-language instruction for the browser agent. At least one of `code` or `prompt` is required. | +| `language` | `Option` | Runtime: `Python`, `Node`, `Bash`. Default: `Node`. | +| `timeout` | `Option` | Execution timeout in seconds. Min: 1, max: 300. | +| `origin` | `Option` | Origin label. Auto-set if `None`. | + +### Response + +Returns `ScrapeExecuteResponse`: +- `success`: `bool` +- `live_view_url`: `Option` +- `interactive_live_view_url`: `Option` +- `output`: `Option` -- AI agent response (only with `prompt`). +- `stdout`, `result`: `Option` -- standard output. +- `stderr`: `Option` -- standard error. +- `exit_code`: `Option` +- `killed`: `Option` +- `error`: `Option` + +### Companion method + +```rust +app.stop_interaction(job_id) -> Result +``` + +Stops the browser session and returns `session_duration_ms` and `credits_billed`. + +## Notes + +- **Naming**: All struct fields use snake_case. The SDK handles serialization to camelCase for the API via serde. +- **Deprecated aliases**: `scrape_execute()` is a deprecated alias for `interact()`. `stop_interactive_browser()` and `delete_scrape_browser()` are deprecated aliases for `stop_interaction()`. Use the preferred methods. +- **Async**: All methods are async and require a Tokio runtime. +- **Origin**: The SDK automatically sets the `origin` field to `"rust-sdk@{version}"` on `scrape`, `search`, and `interact` if not explicitly provided. + +## Source Of Truth + +- `firecrawl/apps/rust-sdk/src/client.rs` +- `firecrawl/apps/rust-sdk/src/scrape.rs` +- `firecrawl/apps/rust-sdk/src/search.rs` +- `firecrawl/apps/rust-sdk/src/types.rs` +- `firecrawl-docs/api-reference/v2-openapi.json`