Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
549 changes: 379 additions & 170 deletions README.md

Large diffs are not rendered by default.

200 changes: 200 additions & 0 deletions example_notebook/Deletion and Opt-Out (DSR) Example Notebook.ipynb

Large diffs are not rendered by default.

164 changes: 164 additions & 0 deletions example_notebook/First Party Data (1PD) Example Notebook.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": "# First Party Data (1PD) Example Notebook\n\nSend your own customer audiences from Databricks to The Trade Desk for targeting.\n\n## Is this the right notebook for you?\n\n| | |\n|---|---|\n| **Who Should Use This?** | Advertisers uploading their own customer data, and onboarders acting on an advertiser's behalf |\n| **What Does the Notebook Do?** | Adds identities to a first-party audience segment so you can target them |\n| **What Data Does It Send?** | One row per identity per segment |\n| **Destination Trade Desk Endpoint** | `POST /data/advertiser` (via the `AdvertiserContext` class of the ttd-databricks SDK) |\n| **Relevant OpenTTD API Documentation** | [First-party data](https://open.thetradedesk.com/advertiser/docsApp/GuidesAdvertiser/data/doc/post-data-advertiser-firstparty) \u00b7 [External provider](https://open.thetradedesk.com/provider/docsApp/GuidesProvider/audience/doc/post-data-advertiser-external) |\n\nOnboarding conversion events instead? Use [Offline Conversion Data (CAPI) Example Notebook](https://github.com/thetradedesk/ttd-databricks-python/blob/add-easy-start-examples-per-usecase-to-sdk-docs/example_notebook/Offline%20Conversion%20Data%20%28CAPI%29%20Example%20Notebook.ipynb).\nRemoving users? Use [Deletion and Opt-Out (DSR) Example Notebook](https://github.com/thetradedesk/ttd-databricks-python/blob/add-easy-start-examples-per-usecase-to-sdk-docs/example_notebook/Deletion%20and%20Opt-Out%20%28DSR%29%20Example%20Notebook.ipynb).\n\n## What you need\n\n- [ ] A TTD Platform API token \u2014 see [Create an API token](https://open.thetradedesk.com/advertiser/docsApp/GuidesAdvertiser/data/doc/DataApiCallsAdvertiser#ui-method-create)\n- [ ] Your advertiser ID\n- [ ] (Optional) A data provider ID \u2014 only if you are an external provider acting for the advertiser\n- [ ] (Optional) A UID2 operator base URL, API key and client secret \u2014 to send raw email addresses or phone numbers"
},
{
"cell_type": "markdown",
"metadata": {},
"source": "## Step 1: Install the SDK"
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": "%pip install ttd-databricks\n\ndbutils.library.restartPython()"
},
{
"cell_type": "markdown",
"metadata": {},
"source": "## Step 2: Configure credentials\n\nFor a first run you can paste values inline. In production, read them from Databricks Secrets:\n\n```python\nAPI_TOKEN = dbutils.secrets.get(scope=\"ttd\", key=\"api-token\")\nADVERTISER_ID = dbutils.secrets.get(scope=\"ttd\", key=\"advertiser-id\")\n```\n\n> **Note:** Authenticate with a Platform API token, sent as the `TTD-Auth` header. Generate one\n> in the OpenTTD Access Management app \u2014 see [Create an API token](https://open.thetradedesk.com/advertiser/docsApp/GuidesAdvertiser/data/doc/DataApiCallsAdvertiser#ui-method-create). Secret keys and\n> `TtdSignature` headers are a legacy method and are not supported by this SDK."
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": "API_TOKEN = \"<your-ttd-auth-token>\"\nADVERTISER_ID = \"<your-advertiser-id>\"\nDATA_PROVIDER_ID = None # optional; set only if you are an external provider"
},
{
"cell_type": "markdown",
"metadata": {},
"source": "## Step 3: Create the client and context"
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": "from ttd_databricks_python.ttd_databricks import (\n AdvertiserContext,\n TTDEndpoint,\n TtdDatabricksClient,\n get_ttd_input_schema,\n)\n\n# Two ways to create the client \u2014 this notebook uses (ii) everywhere below.\n# i) Dependency injection \u2014 you build the DataClient and pass it in:\n# from ttd_data import DataClient\n# client = TtdDatabricksClient(data_api_client=DataClient(ttd_auth=API_TOKEN))\n# ii) Factory \u2014 from_params builds the DataClient for you:\nclient = TtdDatabricksClient.from_params(api_token=API_TOKEN)\n\ncontext = AdvertiserContext(\n advertiser_id=ADVERTISER_ID,\n data_provider_id=DATA_PROVIDER_ID,\n)\n\nprint(f\"Context: {context}\")"
},
{
"cell_type": "markdown",
"metadata": {},
"source": "## Step 4: Inspect the required input schema\n\nMandatory columns: `id_type`, `id_value`, `segment_name`\n\nOptional columns: `cookie_mapping_partner_id`, `timestamp_utc`, `ttl_in_minutes`, `base_bid_cpm`, `base_bid_cpm_metadata`, `bid_factor`"
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": "from ttd_databricks_python.ttd_databricks import TTDEndpoint, get_ttd_input_schema\nfrom ttd_databricks_python.ttd_databricks.schemas import get_required_column_names\n\ninput_schema = get_ttd_input_schema(TTDEndpoint.ADVERTISER)\n\nprint(\"Mandatory columns:\", get_required_column_names(TTDEndpoint.ADVERTISER))\nprint(\"\\nFull input schema:\")\nfor field in input_schema.fields:\n print(f\" {field.name}: {field.dataType.simpleString()} (nullable={field.nullable})\")"
},
{
"cell_type": "markdown",
"metadata": {},
"source": "## Step 5: Prepare your input DataFrame\n\nOne row per identity per segment. `segment_name` is the audience the identity joins, and\n`ttl_in_minutes` controls how long it stays targetable (default 90 days, maximum 180 days).\n\n> **Tip:** Start with a handful of rows. Confirming the end-to-end flow on a small sample is much easier to troubleshoot than a full load."
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": "from datetime import datetime, timezone\n\nrows = [\n {\"id_type\": \"TDID\", \"id_value\": \"123e4567-e89b-12d3-a456-426652340000\",\n \"segment_name\": \"my_first_segment\", \"ttl_in_minutes\": 43200},\n {\"id_type\": \"DAID\", \"id_value\": \"a9342d1f-69f1-4bf8-bc2b-1f20eb451f21\",\n \"segment_name\": \"my_first_segment\", \"ttl_in_minutes\": 43200},\n {\"id_type\": \"UID2\", \"id_value\": \"48MjlfIUZpOKNAm9nod7/jCLAXUYsnE1tpVHQSDS0uo=\",\n \"segment_name\": \"my_first_segment\", \"ttl_in_minutes\": 43200},\n {\"id_type\": \"RampID\", \"id_value\": \"XY1005wXyWPB1SgpMUKIpzA0I3UaLEz-2lg0wFAr1PWK7FMhs\",\n \"segment_name\": \"my_first_segment\", \"ttl_in_minutes\": 43200,\n \"timestamp_utc\": datetime(2026, 1, 15, 10, 0, 0, tzinfo=timezone.utc)},\n]\n\ninput_df = spark.createDataFrame(rows, schema=input_schema)\ndisplay(input_df)"
},
{
"cell_type": "markdown",
"metadata": {},
"source": "## Step 6: Sending Data\n\nThere are two ways to send your data. Pick one \u2014 you do not need both.\n\n| | Ad hoc | Batch processing |\n|---|---|---|\n| **State management** | None. Every call sends every row you give it. | Provided. A metadata table records progress, so each run sends only rows added since the last one. |\n| **Input** | A DataFrame you build in the notebook | A Delta input table |\n| **Output** | Returned inline as a DataFrame | Written to a Delta output table |\n| **Best for** | One-off loads and first tests | Recurring pipelines |"
},
{
"cell_type": "markdown",
"metadata": {},
"source": "### Step 6a: Ad Hoc Usage (No State Management Provided)\n\n`push_data` sends your DataFrame straight to the Data API and returns the input columns\nenriched with per-row status. The SDK keeps no record of what it has already sent, so\nre-running this cell sends every row again.\n\nIt does not raise on API or row-level failures \u2014 every outcome is reported inline."
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": "result_df = client.push_data(df=input_df, context=context, batch_size=1600)\n\ndisplay(result_df)"
},
{
"cell_type": "markdown",
"metadata": {},
"source": "`push_data` adds these columns to your input:\n\n| Column | Meaning |\n|---|---|\n| `success` | `True` if the row was accepted |\n| `error_code` | Failure category, `null` on success |\n| `error_message` | Human-readable reason, `null` on success |\n| `processed_timestamp` | When the row was submitted |\n| `uid2_resolutions` | Raw identifier \u2192 UID2 mapping, empty unless `uid2_config` was set |"
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": "from pyspark.sql.functions import col\n\ntotal = result_df.count()\nsucceeded = result_df.filter(col(\"success\")).count()\n\nprint(f\"Total: {total} | Succeeded: {succeeded} | Failed: {total - succeeded}\")\n\nfailed_df = result_df.filter(~col(\"success\"))\nif failed_df.count():\n display(failed_df.select(\"error_code\", \"error_message\"))"
},
{
"cell_type": "markdown",
"metadata": {},
"source": "### Step 6b: Batch Processing (State Management Provided)\n\n`batch_process` reads from a Delta input table, writes results to an output table, and\nrecords how far it got in a metadata table. With `process_new_records_only=True`, each run\npicks up only the rows added since the last successful run, so you can schedule it without\nre-sending history.\n\nUsers must create a separate set of these tables for each advertiser or data\nprovider you send data for.\n\n**One time steps:** Create the three Delta tables. Every future run reuses these same tables \u2014\nthe metadata table is what remembers your progress, so do not drop or recreate it between\nruns. The `setup_*` methods are safe to re-run: they return the existing table if it is\nalready there."
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": "from ttd_databricks_python.ttd_databricks import TTDEndpoint\n\ninput_table = client.setup_input_table(endpoint=TTDEndpoint.ADVERTISER)\noutput_table = client.setup_output_table(endpoint=TTDEndpoint.ADVERTISER)\nmetadata_table = client.setup_metadata_table()\n\nprint(f\"Input table: {input_table}\")\nprint(f\"Output table: {output_table}\")\nprint(f\"Metadata table: {metadata_table}\")"
},
{
"cell_type": "markdown",
"metadata": {},
"source": "**Every run:**\n\n**1. Append new rows to the input table.** In production this is your upstream pipeline's job.\nThis example writes into the Delta table from a DataFrame, but you are free to use any\nother method. `batch_process` is indifferent to how the data arrives in the input table."
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": "from pyspark.sql import functions as F\n\n(\n spark.createDataFrame(rows, schema=input_schema)\n .withColumn(\"updated_at\", F.current_timestamp())\n .write.format(\"delta\").mode(\"append\").saveAsTable(input_table)\n)\n\ndisplay(spark.table(input_table))"
},
{
"cell_type": "markdown",
"metadata": {},
"source": "**2. Call `batch_process`.** Re-running it picks up only rows appended since the last run."
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": "client.batch_process(\n context=context,\n input_table=input_table,\n output_table=output_table,\n metadata_table=metadata_table,\n process_new_records_only=True, # incremental; set False to reprocess every row\n batch_size=1600, # rows per API request\n)\n\ndisplay(spark.table(output_table))\ndisplay(spark.table(metadata_table))"
},
{
"cell_type": "markdown",
"metadata": {},
"source": "## (Optional) Sending email addresses or phone numbers (UID2)\n\nSkip this section if you are sending device IDs or UID2s you have already resolved.\n\nPass a `uid2_config` when you create the client. It is the same call as Step 3 with one\nextra argument. Then set `id_type` to `Email`, `Phone`,\n`HashedEmail`, or `HashedPhone` on your rows. From there `push_data` and `batch_process` take them exactly like any other\nidentifier type, and nothing else about Step 6a or Step 6b changes.\n\nThe SDK resolves each identifier to a UID2 (or EUID) using your operator before any\nrequest is made to The Trade Desk's Data APIs, so The Trade Desk never receives the raw\nemail or phone number. The raw identifier is sent to the UID2 operator configured in\n`uid2_config` in order to be resolved, so choose that operator accordingly. The mapping\ncomes back in the `uid2_resolutions` column."
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": "from ttd_data.uid2 import IdentityScope, UID2Config\n\nfrom ttd_databricks_python.ttd_databricks import TtdDatabricksClient\n\nuid2_client = TtdDatabricksClient.from_params(\n api_token=API_TOKEN,\n uid2_config=UID2Config(\n base_url=\"<your-uid2-operator-url>\",\n api_key=\"<your-uid2-api-key>\",\n client_secret=\"<your-uid2-client-secret>\",\n identity_scope=IdentityScope.UID2, # use IdentityScope.EUID for European identities\n ),\n)\n\nuid2_data = [\n {\"id_type\": \"Email\", \"id_value\": \"user@example.com\",\n \"segment_name\": \"my_first_segment\", \"ttl_in_minutes\": 43200},\n {\"id_type\": \"HashedEmail\", \"id_value\": \"tMmiiTI7IaAcPpQPFQ65uMVCWH8av9jw4cwf/F5HVRQ=\",\n \"segment_name\": \"my_first_segment\", \"ttl_in_minutes\": 43200},\n]\n\nuid2_result_df = uid2_client.push_data(\n df=spark.createDataFrame(uid2_data, schema=input_schema),\n context=context,\n)\n\ndisplay(uid2_result_df.select(\"id_type\", \"success\", \"error_message\", \"uid2_resolutions\"))"
},
{
"cell_type": "markdown",
"metadata": {},
"source": "## Next steps\n\n- **Other use cases** \u2014 one notebook per use case:\n [First Party Data (1PD) Example Notebook](https://github.com/thetradedesk/ttd-databricks-python/blob/add-easy-start-examples-per-usecase-to-sdk-docs/example_notebook/First%20Party%20Data%20%281PD%29%20Example%20Notebook.ipynb),\n [Third Party Data (3PD) Example Notebook](https://github.com/thetradedesk/ttd-databricks-python/blob/add-easy-start-examples-per-usecase-to-sdk-docs/example_notebook/Third%20Party%20Data%20%283PD%29%20Example%20Notebook.ipynb),\n [Offline Conversion Data (CAPI) Example Notebook](https://github.com/thetradedesk/ttd-databricks-python/blob/add-easy-start-examples-per-usecase-to-sdk-docs/example_notebook/Offline%20Conversion%20Data%20%28CAPI%29%20Example%20Notebook.ipynb),\n [Deletion and Opt-Out (DSR) Example Notebook](https://github.com/thetradedesk/ttd-databricks-python/blob/add-easy-start-examples-per-usecase-to-sdk-docs/example_notebook/Deletion%20and%20Opt-Out%20%28DSR%29%20Example%20Notebook.ipynb).\n- **Full reference** \u2014 [README](https://github.com/thetradedesk/ttd-databricks-python/blob/add-easy-start-examples-per-usecase-to-sdk-docs/README.md) covers authentication, error handling,\n UID2 support, custom HTTP clients, and server URL overrides.\n- **Server URLs** \u2014 you do not need to configure one. Each endpoint already targets its\n own default server, and it can be overridden with a preferred server if you need one."
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.10.0"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
Loading
Loading