Skip to content

docs: add agent quickstart files for all SDK languages #1344

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
ericciarla wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

docs: add agent quickstart files for all SDK languages #1344

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
179 changes: 179 additions & 0 deletions agent-quickstart/elixir.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
---
title: "Elixir Agent Quickstart"
description: "Canonical Firecrawl Elixir quickstart for external agents using search, scrape, and interact."
---

# Firecrawl Elixir Agent Quickstart

Canonical quickstart for external agents. Generated from the `firecrawl` Elixir SDK source and the v2 OpenAPI spec. Function names match the auto-generated module in `lib/firecrawl.ex`.

## Install

Add to `mix.exs`:

```elixir
{:firecrawl, "~> 1.11"}
```

## Authenticate

```elixir
# config/runtime.exs or config.exs
config :firecrawl, api_key: System.get_env("FIRECRAWL_API_KEY")

# Or pass api_key per call:
{:ok, res} = Firecrawl.search_and_scrape(
[query: "firecrawl webhooks"],
api_key: "fc-your-api-key"
)
```

Every function accepts `base_url` in opts (defaults to `"https://api.firecrawl.dev/v2"`). Additional opts are passed through to `Req`.

## When To Use What

- **search**: use when you start with a query and need discovery. Returns relevant pages you can then scrape or interact with.
- **scrape**: use when you already have a URL and want structured page content (markdown, HTML, JSON extraction, screenshots, etc.).
- **interact**: use when the page needs clicks, form fills, or other browser actions after a scrape has created a session. Requires a scrape job ID from a prior scrape.

## Search

### Why use it

Discover relevant pages from a query. Constrain results to a site with `site:` in the query string (e.g. `site:docs.firecrawl.dev webhooks`).

### Preferred SDK method

`Firecrawl.search_and_scrape(params \\ [], opts \\ [])`

### Example

```elixir
{:ok, res} = Firecrawl.search_and_scrape(query: "site:docs.firecrawl.dev webhook retries")
```

Every function has a bang variant (`search_and_scrape!`) that raises on error.

### Parameters

Keyword list `params`:

| Parameter | Type | Required | Description |
|---|---|---|---|
| `query` | `string` | **yes** | Search query. Use `site:example.com` to scope to a domain. |
| `sources` | `list` | no | Which sources to search: `:web`, `:news`, `:images` (atoms or strings). |
| `categories` | `list` | no | Filter by category: `:developer`, `:research`, `:pdf` (atoms or strings). |
| `include_domains` | `list(string)` | no | Restrict results to these domains. |
| `exclude_domains` | `list(string)` | no | Exclude results from these domains. |
| `limit` | `integer` | no | Maximum number of results. |
| `tbs` | `string` | no | Time-based filter (e.g. `"qdr:d"` for past day). |
| `location` | `string` | no | Localized results. |
| `country` | `string` | no | ISO 3166-1 alpha-2 country code (e.g. `"US"`). |
| `ignore_invalid_urls` | `boolean` | no | Drop URLs that cannot be scraped. |
| `timeout` | `integer` | no | Request timeout in milliseconds. |
| `highlights` | `boolean` | no | Generate query-relevant highlights. |
| `scrape_options` | `keyword list` | no | Scrape each search result. See Scrape parameters. |
| `enterprise` | `list(string)` | no | Enterprise controls: `"zdr"` (zero data retention), `"anon"` (anonymized). |

## Scrape

### Why use it

Retrieve structured content from a URL in one or more formats: markdown, HTML, JSON extraction, screenshots, and more.

### Preferred SDK method

`Firecrawl.scrape_and_extract_from_url(params \\ [], opts \\ [])`

### Example

```elixir
{:ok, res} = Firecrawl.scrape_and_extract_from_url(
url: "https://example.com",
formats: ["markdown"],
only_main_content: true
)
```

### Parameters

Keyword list `params`:

| Parameter | Type | Required | Description |
|---|---|---|---|
| `url` | `string` | **yes** | The URL to scrape. |
| `formats` | `list` | no | Output formats. Strings: `"markdown"`, `"html"`, `"rawHtml"`, `"links"`, `"images"`, `"screenshot"`, `"summary"`, `"changeTracking"`, `"json"`, `"branding"`, `"audio"`, `"video"`. Maps: `%{type: "json", prompt: ..., schema: ...}`, `%{type: "screenshot", fullPage: true}`, `%{type: "changeTracking", modes: ["git-diff"]}`, `%{type: "attributes", selectors: [...]}`. |
| `headers` | `map` | no | Custom HTTP headers sent with the request. |
| `include_tags` | `list(string)` | no | Only include content from these HTML tags. |
| `exclude_tags` | `list(string)` | no | Exclude content from these HTML tags. |
| `only_main_content` | `boolean` | no | Strip nav, footer, and other boilerplate. |
| `timeout` | `integer` | no | Request timeout in milliseconds. |
| `wait_for` | `integer` | no | Wait for the page to render (milliseconds). |
| `mobile` | `boolean` | no | Emulate a mobile viewport. |
| `parsers` | `list` | no | File parsing controls: `"pdf"` or `%{type: "pdf", mode: "fast" \| "auto" \| "ocr", maxPages: n}`. |
| `actions` | `list(map)` | no | Browser actions before scraping: `%{type: "click", selector: ...}`, `%{type: "wait", milliseconds: ...}`, `%{type: "write", text: ...}`, `%{type: "press", key: ...}`, `%{type: "scroll", direction: "up" \| "down"}`, `%{type: "scrape"}`, `%{type: "executeJavascript", script: ...}`, `%{type: "pdf"}`. |
| `location` | `keyword list` | no | Geo targeting: `[country: "US", languages: ["en-US"]]`. |
| `skip_tls_verification` | `boolean` | no | Skip TLS certificate verification. |
| `remove_base64_images` | `boolean` | no | Drop base64 images from markdown output. |
| `block_ads` | `boolean` | no | Block ads and cookie popups. |
| `proxy` | `:basic \| :enhanced \| :auto` | no | Proxy mode. |
| `max_age` | `integer` | no | Use cached content if younger than this (milliseconds). |
| `min_age` | `integer` | no | Use cached content only if at least this old (milliseconds). |
| `store_in_cache` | `boolean` | no | Cache the scrape result. |
| `lockdown` | `boolean` | no | Serve only cached results; no outbound requests. |
| `redact_pii` | `boolean` | no | Redact personally identifiable information. |
| `audit_metadata` | `keyword list` | no | User attribution for SIEM logging: `[username: "..."]`. |
| `profile` | `keyword list` | no | Persistent browser profile: `[name: "...", save_changes: true]`. |
| `zero_data_retention` | `boolean` | no | End-to-end zero data retention. |

## Interact

### Why use it

Control the browser session tied to a prior scrape. Use for code execution in the browser session. Requires a scrape job ID from a prior scrape response.

### Preferred SDK method

`Firecrawl.interact_with_scrape_browser_session(job_id, params \\ [], opts \\ [])`

### Example

```elixir
{:ok, scrape_res} = Firecrawl.scrape_and_extract_from_url(
url: "https://example.com",
formats: ["markdown"]
)

job_id = get_in(scrape_res.body, ["data", "metadata", "scrapeId"])

{:ok, result} = Firecrawl.interact_with_scrape_browser_session(job_id,
code: "console.log(await page.title());",
language: :node
)
```

To stop the session: `Firecrawl.stop_interactive_scrape_browser_session(job_id)`

### Parameters

| Parameter | Type | Required | Description |
|---|---|---|---|
| `job_id` | `string` | **yes** | Scrape job ID (path parameter). |
| `code` | `string` | **yes** | Code to execute in the browser session. |
| `language` | `:python \| :node \| :bash` | no | Runtime for code execution. |
| `timeout` | `integer` | no | Execution timeout in seconds. |

## Notes

- Function names are **auto-generated from the OpenAPI spec** and follow a verbose pattern: `scrape_and_extract_from_url`, `search_and_scrape`, `interact_with_scrape_browser_session`. Use them exactly as named.
- Every function has a **bang variant** (e.g. `search_and_scrape!`) that raises `Firecrawl.Error` on failure instead of returning `{:error, ...}`.
- Parameter names in the keyword list use **snake_case** (e.g. `only_main_content`, `scrape_options`), which the SDK converts to camelCase JSON keys before sending.
- The Elixir SDK's `interact_with_scrape_browser_session` only exposes the `code` parameter, not `prompt`. For natural-language browser instructions, use the HTTP API directly or another SDK.
- No deprecated aliases exist — the SDK is auto-generated from the OpenAPI spec.
- Proxy values use atoms (`:basic`, `:enhanced`, `:auto`) rather than strings.
- All functions accept `api_key` and `base_url` in the trailing `opts` keyword list.

## Source Of Truth

- `firecrawl/apps/elixir-sdk/lib/firecrawl.ex`
- `firecrawl-docs/api-reference/v2-openapi.json`
210 changes: 210 additions & 0 deletions agent-quickstart/java.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,210 @@
---
title: "Java Agent Quickstart"
description: "Canonical Firecrawl Java quickstart for external agents using search, scrape, and interact."
---

# Firecrawl Java Agent Quickstart

Canonical quickstart for external agents. Generated from the `firecrawl-java` SDK source and the v2 OpenAPI spec. Method names and parameter types match the `FirecrawlClient` public API.

## Install

Maven:

```xml
<dependency>
<groupId>com.firecrawl</groupId>
<artifactId>firecrawl-java</artifactId>
<version>1.17.0</version>
</dependency>
```

Gradle:

```gradle
implementation("com.firecrawl:firecrawl-java:1.17.0")
```

Requires Java 11+.

## Authenticate

```java
import com.firecrawl.client.FirecrawlClient;

FirecrawlClient client = FirecrawlClient.builder()
.apiKey(System.getenv("FIRECRAWL_API_KEY"))
.build();

// Or from environment: FirecrawlClient.fromEnv()
```

Builder options: `apiKey` (falls back to `FIRECRAWL_API_KEY` env or `firecrawl.apiKey` system property), `apiUrl` (defaults to `https://api.firecrawl.dev`), `timeoutMs` (default `300000`), `maxRetries` (default `3`), `backoffFactor` (default `0.5`), `asyncExecutor` (default `ForkJoinPool.commonPool()`), `httpClient` (custom OkHttp instance).

## When To Use What

- **search**: use when you start with a query and need discovery. Returns relevant pages you can then scrape or interact with.
- **scrape**: use when you already have a URL and want structured page content (markdown, HTML, JSON extraction, screenshots, etc.).
- **interact**: use when the page needs clicks, form fills, or other browser actions after a scrape has created a session. Requires a `scrapeId` from a prior scrape.

## Search

### Why use it

Discover relevant pages from a query. Constrain results to a site with `site:` in the query string (e.g. `site:docs.firecrawl.dev webhooks`).

### Preferred SDK method

- `client.search(query)` → `SearchData`
- `client.search(query, options)` → `SearchData`

### Example

```java
import com.firecrawl.models.SearchData;
import java.util.List;
import java.util.Map;

SearchData results = client.search("site:docs.firecrawl.dev webhook retries");
List<Map<String, Object>> web = results.getWeb();
```

**Important:** `search()` returns `SearchData`. Access results via `getWeb()`, `getNews()`, `getImages()` — each returns `List<Map<String, Object>>` (may be null).

### Parameters

`SearchOptions` built via `SearchOptions.builder()`:

| Parameter | Type | Description |
|---|---|---|
| `query` | `String` | Search query (first positional argument). Use `site:example.com` to scope. |
| `sources` | `List<Object>` | Which sources to search: `"web"`, `"news"`, `"images"`. |
| `categories` | `List<Object>` | Filter by category: `"github"`, `"research"`, `"pdf"`. |
| `includeDomains` | `List<String>` | Restrict results to these domains. |
| `excludeDomains` | `List<String>` | Exclude results from these domains. |
| `limit` | `Integer` | Maximum number of results. |
| `tbs` | `String` | Time-based filter (e.g. `"qdr:d"` for past day). |
| `location` | `String` | Localized results. |
| `ignoreInvalidURLs` | `Boolean` | Drop URLs that cannot be scraped. |
| `timeout` | `Integer` | Request timeout in milliseconds. |
| `highlights` | `Boolean` | Generate query-relevant highlights. Defaults to `true` server-side. |
| `scrapeOptions` | `ScrapeOptions` | Scrape each search result. See Scrape parameters. |
| `integration` | `String` | Integration identifier for server-side tracking. |

## Scrape

### Why use it

Retrieve structured content from a URL in one or more formats: markdown, HTML, JSON extraction, screenshots, and more.

### Preferred SDK method

- `client.scrape(url)` → `Document`
- `client.scrape(url, options)` → `Document`

### Example

```java
import com.firecrawl.models.ScrapeOptions;
import com.firecrawl.models.Document;

Document doc = client.scrape("https://example.com",
ScrapeOptions.builder()
.formats(List.of("markdown"))
.onlyMainContent(true)
.build()
);

System.out.println(doc.getMarkdown());
```

Async variant: `client.scrapeAsync(url, options)` returns `CompletableFuture<Document>`.

### Parameters

`ScrapeOptions` built via `ScrapeOptions.builder()`:

| Parameter | Type | Description |
|---|---|---|
| `url` | `String` | The URL to scrape (first positional argument). |
| `formats` | `List<Object>` | Output formats. Strings: `"markdown"`, `"html"`, `"rawHtml"`, `"links"`, `"screenshot"`, `"json"`, `"audio"`, `"video"`. Objects: `JsonFormat`, `QuestionFormat`, `HighlightsFormat`. |
| `headers` | `Map<String, String>` | Custom HTTP headers sent with the request. |
| `includeTags` | `List<String>` | Only include content from these HTML tags. |
| `excludeTags` | `List<String>` | Exclude content from these HTML tags. |
| `onlyMainContent` | `Boolean` | Strip nav, footer, and other boilerplate. |
| `timeout` | `Integer` | Request timeout in milliseconds. |
| `waitFor` | `Integer` | Wait for the page to render (milliseconds). |
| `mobile` | `Boolean` | Emulate a mobile viewport. |
| `parsers` | `List<Object>` | File parsing controls (e.g. `"pdf"` or `PdfParser` with `maxPages`). |
| `actions` | `List<Map<String, Object>>` | Browser actions before scraping. |
| `location` | `LocationConfig` | Geo targeting with `country` and `languages`. |
| `skipTlsVerification` | `Boolean` | Skip TLS certificate verification. |
| `removeBase64Images` | `Boolean` | Drop base64 images from markdown output. |
| `blockAds` | `Boolean` | Block ads and cookie popups. |
| `proxy` | `String` | Proxy mode: `"basic"`, `"stealth"`, `"enhanced"`, `"auto"`, or custom URL. |
| `maxAge` | `Long` | Use cached content if younger than this (milliseconds). |
| `storeInCache` | `Boolean` | Cache the scrape result. |
| `lockdown` | `Boolean` | Serve only cached results; no outbound requests. |
| `redactPII` | `Boolean` | Redact personally identifiable information. |
| `auditMetadata` | `AuditMetadata` | User attribution for SIEM logging (has `username` field). |
| `integration` | `String` | Integration identifier for server-side tracking. |

## Interact

### Why use it

Control the browser session tied to a prior scrape (via `metadata.scrapeId`). Use for code execution in the browser session. The Java SDK exposes `code`-based interaction.

### Preferred SDK method

- `client.interact(jobId, code)` → `BrowserExecuteResponse`
- `client.interact(jobId, code, language, timeout)` → `BrowserExecuteResponse`

### Example

```java
import com.firecrawl.models.Document;
import com.firecrawl.models.BrowserExecuteResponse;

Document doc = client.scrape("https://example.com",
ScrapeOptions.builder().formats(List.of("markdown")).build()
);

String jobId = (String) doc.getMetadata().get("scrapeId");

BrowserExecuteResponse result = client.interact(jobId,
"console.log(await page.title());",
"node",
60
);

System.out.println(result.getStdout());
```

To stop the session: `client.stopInteractiveBrowser(jobId)`

### Parameters

| Parameter | Type | Description |
|---|---|---|
| `jobId` | `String` | Scrape job ID from `document.metadata.scrapeId`. |
| `code` | `String` | Code to execute in the browser session (e.g. Playwright `page` usage). |
| `language` | `String` | Runtime: `"python"`, `"node"`, `"bash"`. Defaults to `"node"`. |
| `timeout` | `Integer` | Execution timeout in seconds (1-300). |

Async variants: `client.interactAsync(...)` returns `CompletableFuture<BrowserExecuteResponse>`.

## Notes

- All parameter names use **camelCase** (e.g. `onlyMainContent`, `scrapeOptions`, `ignoreInvalidURLs`).
- Options use the **builder pattern**: `ScrapeOptions.builder().formats(...).build()`.
- The Java SDK `interact` method exposes the `code` parameter directly. For natural-language prompts, use the HTTP API directly or another SDK.
- Deprecated aliases: `scrapeExecute` → `interact`, `deleteScrapeBrowser` → `stopInteractiveBrowser`. Always use the modern names.
- Every sync method has an `Async` variant returning `CompletableFuture`.

## Source Of Truth

- `firecrawl/apps/java-sdk/src/main/java/com/firecrawl/client/FirecrawlClient.java`
- `firecrawl/apps/java-sdk/src/main/java/com/firecrawl/models/ScrapeOptions.java`
- `firecrawl/apps/java-sdk/src/main/java/com/firecrawl/models/SearchOptions.java`
- `firecrawl-docs/api-reference/v2-openapi.json`
Loading