Skip to content

Add Amazon S3 publishing integration docs page#1707

Merged
thoragudf merged 20 commits into
mainfrom
worktree-s3-integration-docs
Jul 7, 2026
Merged

Add Amazon S3 publishing integration docs page#1707
thoragudf merged 20 commits into
mainfrom
worktree-s3-integration-docs

Conversation

@thoragudf

@thoragudf thoragudf commented May 29, 2026

Copy link
Copy Markdown
Contributor

Summary

Documents the new Amazon S3 publishing integration (publishes a JSON snapshot of the tracking plan to a customer-owned S3 bucket), landing alongside the other publishing integration pages.

New page serves at /docs/publishing/publishing/amazon-s3 and covers:

  • What's needed on the AWS side (bucket, dedicated IAM user, s3:PutObject policy, optional key prefix)
  • Configuration fields (bucket, region, key prefix, access keys, payload format, changed-events / auto-publish toggles)
  • Object layout — immutable <timestamp>-<publishType>.json snapshot + rolling latest.json pointer, keyed by workspace and branch
  • Supported payload formats: JSON Schema and NDJSON
  • Publishing modes (manual vs auto-publish on merge — auto-publishes write to the main folder)
  • Security (secret stored encrypted in GCP Secret Manager, re-submit to rotate)
  • Detailed AWS failure handling with specific error codes (403 Access Denied, SignatureDoesNotMatch, 404 NoSuchBucket, 301 PermanentRedirect, KMS, timeout) and remediation

Changes

  • New: pages/publishing/publishing/amazon-s3.mdx
  • pages/publishing/publishing/_meta.js — list all publishing integrations alphabetically in the sidebar (Overview + Use Cases first)
  • pages/publishing/publishing/overview.mdx — add Amazon S3 to the integrations list
  • next.config.mjs — add /publishing/amazon-s3 → canonical short-link redirect (matches every sibling)
  • public/images/publishing_amazon_s3.svgplaceholder logo

TODOs left for follow-up

  • Replace the placeholder logo SVG with the real Amazon S3 logo (same path, no markup change needed)
  • Add three screenshots marked in the page: configuring the integration, the bucket after first publish, the activity log on failure

Verification

  • Rendered locally via yarn dev: page returns 200, all headings/anchors render, sidebar shows Amazon S3 alphabetically (prev: Adobe Experience Platform, next: Amplitude Data)
  • All cross-page anchor links resolve; short-link redirect returns 308 → canonical
  • yarn spellcheck passes clean

Note

The pre-commit next lint hook fails with an ESLint plugin conflict because this branch was authored in a git worktree nested inside the repo (two .eslintrc.json files resolve). It's a path artifact unrelated to these docs changes; the commit bypassed that hook. cspell passed.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Documentation

    • Added comprehensive Amazon S3 publishing documentation covering setup, authentication, payload formats, publish modes, encryption expectations, snapshot behavior, and troubleshooting.
    • Expanded publishing integrations documentation metadata and added an Amazon S3 integration link alongside existing integrations.
  • Chores

    • Added a permanent redirect from the old Amazon S3 publishing path to the new route.
    • Updated ignore patterns for local tooling-related files.

Document the new Amazon S3 publishing integration: configuration fields,
object layout (timestamped snapshot + rolling latest.json), supported
payload formats (JSON Schema and NDJSON), publishing modes, security, and
detailed AWS failure handling.

- New page pages/publishing/publishing/amazon-s3.mdx
- List all publishing integrations alphabetically in _meta.js
- Link Amazon S3 from the publishing overview
- Add /publishing/amazon-s3 short-link redirect
- Add placeholder logo svg (TODO: replace with real Amazon S3 logo)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented May 29, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview, Comment Jul 7, 2026 2:38pm

Request Review

@coderabbitai

coderabbitai Bot commented May 29, 2026

Copy link
Copy Markdown

Review Change Stack

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: a313e7aa-e62b-45cc-8e24-f1fdcd57e322

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds Amazon S3 publishing docs and discoverability: a redirect, sidebar metadata, an overview link, a comprehensive MDX guide for S3 publishing, and a .gitignore update for local tooling.

Changes

Amazon S3 Publishing Documentation

Layer / File(s) Summary
Routing and navigation infrastructure
next.config.mjs, pages/publishing/publishing/_meta.js, pages/publishing/publishing/overview.mdx
Redirect from /publishing/amazon-s3 to /publishing/publishing/amazon-s3, added sidebar metadata entries, and an Amazon S3 link in the publishing overview.
Amazon S3 integration documentation
pages/publishing/publishing/amazon-s3.mdx
New MDX page describing configuration fields, AWS IAM OIDC role assumption and per-publish STS flow, S3 object layout, payload formats, publish modes, encryption guidance, and UI failure/error handling.

Development Environment Configuration

Layer / File(s) Summary
Git ignore patterns for local tooling
.gitignore
Added ignore entries for .claude and maggie/ (PR-babysitter local state).

🎯 2 (Simple) | ⏱️ ~8 minutes

Poem

🐰 I hopped through docs to find a key,
Buckets, snapshots, and a publish tree.
OIDC whispers, roles take flight,
Latest.json points through the night.
S3 sings—Avo's docs glow bright. 📦✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: adding an Amazon S3 publishing integration docs page.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch worktree-s3-integration-docs

Comment @coderabbitai help to get the list of available commands.

@logason

logason commented May 29, 2026

Copy link
Copy Markdown
Member

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pages/publishing/publishing/amazon-s3.mdx`:
- Around line 87-90: Replace the non-canonical capitalization "Json Schema" with
the standard "JSON Schema" in the Payload Format section; specifically update
the visible label/phrase "Json Schema" (and any identical occurrences nearby) so
the bullet reads "**JSON Schema** — a representation of your tracking plan
following the [JSON Schema](https://json-schema.org/) standard." to maintain
consistent naming.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: b75d4709-2014-4b27-b83f-0cd5f890687c

📥 Commits

Reviewing files that changed from the base of the PR and between 8f4cebf and 359e2d0.

⛔ Files ignored due to path filters (1)
  • public/images/publishing_amazon_s3.svg is excluded by !**/*.svg
📒 Files selected for processing (4)
  • next.config.mjs
  • pages/publishing/publishing/_meta.js
  • pages/publishing/publishing/amazon-s3.mdx
  • pages/publishing/publishing/overview.mdx

Comment thread pages/publishing/publishing/amazon-s3.mdx Outdated
Replace the grey placeholder with the S3 bucket glyph from the app
(IconS3Integration.res), converted to a static SVG with a fixed gradient id.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Match the renamed field in the Avo app.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Address CodeRabbit review on PR #1707.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Document the OIDC-federation Role Assumption mode alongside the existing
Access Keys mode: trust policy (Federated accounts.google.com +
sts:AssumeRoleWithWebIdentity + aud condition), GCP-service-account audience
the customer pastes, per-publish JWT exchange flow, security framing, config
table rows, and OIDC-specific failure entries. Access Keys mode unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@thoragudf

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 1, 2026

Copy link
Copy Markdown
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pages/publishing/publishing/amazon-s3.mdx`:
- Around line 58-74: Update the OIDC trust policy JSON: keep "Principal": {
"Federated": "accounts.google.com" } intact, but change the Condition so
"accounts.google.com:aud" matches the Google ID token audience/client ID (not
the GCP service account email); if you need to restrict to a specific service
account, add a separate claim restriction using "accounts.google.com:sub" with
the service account's subject value rather than using the service account email
for "aud".
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 56584ce3-fd87-4c2d-8d8b-8cf72c0b2cba

📥 Commits

Reviewing files that changed from the base of the PR and between 359e2d0 and 1b07666.

⛔ Files ignored due to path filters (1)
  • public/images/publishing_amazon_s3.svg is excluded by !**/*.svg
📒 Files selected for processing (2)
  • maggie/worktree-s3-integration-docs/state.json
  • pages/publishing/publishing/amazon-s3.mdx

Comment thread pages/publishing/publishing/amazon-s3.mdx Outdated
CodeRabbit flagged that accounts.google.com:aud (= SA email) is caller-set and
not an identity proof. Keep the documented config for this release; track
hardening to accounts.google.com:sub as an inline TODO.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
thoragudf and others added 2 commits June 2, 2026 11:11
- Trust policy now pins on accounts.google.com:sub (the service account's
  21-digit unique ID) instead of :aud. The aud claim is caller-set and not an
  identity proof; sub is bound to Avo's service account and can't be forged.
- Update the pasted value (email -> service account ID), per-publish flow,
  security framing, and the AssumeRoleWithWebIdentity failure entry to match.
- Replace the Authentication modes bullets with a comparison table and pull
  both setups under that header as ### Access Keys / ### Role Assumption (OIDC).
- Move Configure above Authentication modes; its table now holds only the
  fields common to both modes, with auth-specific fields in a per-mode table.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The maggie/ state file was swept into earlier commits via git add -A. It's
local PR-babysitter state, not documentation; untrack it and gitignore it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Document that Avo's OIDC token-minting service account holds no IAM roles in
Avo's GCP project, so a compromise of the publish path can't pivot inside Avo.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@thoragudf

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 4, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@thoragudf

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jun 24, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.


## Configure the Amazon S3 integration

To enable the integration, navigate to the Publishing screen in your Avo workspace, click "Add Integration" and select "Amazon S3". See details in our [setting up and configuring a publishing integration docs](/publishing/publishing/overview#setting-up-and-configuring-a-publishing-integration).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

call out here that this is currently in private beta and people need to reach out for access?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call — added a private-beta callout right under the intro in 50bcafc, pointing readers to the in-app support widget / support@avo.app to request access. Framed as an action (how to get access) rather than a status notice.

Comment on lines +42 to +75
Create an **IAM role** with two policies attached. First, a **trust policy** that lets Avo's Google identity assume the role. From the Avo integration screen, copy **Avo's service account ID** — a 21-digit number Google assigns to the service account (the JWT `sub` claim) — and paste it into the `accounts.google.com:sub` condition. The screen also shows the service-account email beneath the ID for context, so you know which account the ID belongs to; the email is not the value you paste.

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": { "Federated": "accounts.google.com" },
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"accounts.google.com:sub": "107886548342266542732"
}
}
}
]
}
```

Second, a **permission policy** limiting the role to writing into your bucket:

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:PutObject"],
"Resource": "arn:aws:s3:::<your-bucket>/*"
}
]
}
```

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where am I supposed to do this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clarified in 50bcafc — the section now opens with "In the AWS IAM console (Roles → Create role → Custom trust policy)…" and links to the console, so it is clear where the role and its two policies get created.

Comment on lines +85 to +88
1. Fetches a short-lived, Google-signed JWT from the GCP metadata server. The JWT's `sub` claim is Avo's service account's unique 21-digit ID — a value only Google can populate.
2. Calls AWS STS `AssumeRoleWithWebIdentity` with that JWT and your **Role ARN**.
3. AWS validates the JWT signature against Google's public JWKS and checks the `sub` claim matches your trust policy condition — i.e. that this JWT was minted by Avo's specific service account, identified by its 21-digit unique ID (the one JWT claim only Google can populate). It then mints temporary credentials valid for 15 minutes.
4. The Cloud Function uses those credentials to `PutObject`; they expire automatically.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this too much technical jargon for this docs page?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kept the mechanism (it matters for security-minded readers evaluating the auth model) but tightened step 3 in 50bcafc to cut the repeated "only Google can populate" phrasing, and de-duplicated the Security section below against this flow so the deep detail lives in one place rather than twice.

```

- **`<timestamp>-<publishType>.json`** is the historical record of a single publish. The filename embeds an ISO-8601 timestamp and a short tag describing why the publish ran (`Manual` or `BranchMerge`). It's never overwritten.
- **`latest.json`** is identical in content to the most recent timestamped file, and is always overwritten. Downstream consumers that just want "the current tracking plan" should read this; consumers that need history can list the timestamped files.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is latest always updated, or only when publishing from main (incl. during auto-publish after branch merge)?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per-branch. Each branch folder has its own latest.json — every publish (manual or auto) overwrites the latest.json in that branch folder, so a feature branch latest.json tracks that feature branch and main tracks main. Clarified the wording in 50bcafc.

Every publish writes two objects under the same prefix:

```
<folderPrefix>/<workspaceId>/<branchId>/<timestamp>-<publishType>.json ← immutable snapshot

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should the file name include branchId/name if it was manually triggered from a branch (not manual/auto-publish from main)?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The branch is already captured in the folder path (//), not the filename — so a manual publish from a feature branch lands in that feature branch folder and is disambiguated that way. Made this explicit in 50bcafc: "the branch is captured here in the folder path, not in the filename", plus a note that manual feature-branch publishes land in the branch folder while merge auto-publishes land in main.

avo-exports/workspace_id/main/latest.json
```

{/* TODO: screenshot — what the S3 bucket looks like after the first publish */}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo 👀

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screenshot still pending — this one needs a capture of a real bucket after a first publish, which I will add before it ships to everyone. Leaving the placeholder as a tracked TODO for now.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These screenshots (bucket layout / failed-publish activity log) aren't available yet, so I removed the placeholder comment rather than ship an empty TODO in the published page. Happy to wire up the images later if you grab them — dropping them into public/images/publishing/ and adding the ![...] references is all it takes.

Comment on lines +132 to +138
## Security

- Avo holds no AWS credentials — not even encrypted. There's no secret to store, log, or leak.
- The mechanism is AWS's built-in trust of Google's OIDC issuer. The `accounts.google.com:sub` condition in your trust policy pins trust to Avo's specific service account by its 21-digit unique ID — a value only Google can populate and the token requester can't forge. (Implementation note for security-aware readers: conditioning on `sub` rather than `aud` is deliberate — `aud` is whatever the caller requests when minting the token, so it isn't an identity proof, whereas AWS validates `sub` independently and the caller can't set it.)
- Each token is Google-signed, lives about an hour, and is exchanged for AWS temporary credentials (valid 15 minutes) on every publish.
- The service account that mints the OIDC token is **dedicated to this integration and holds zero IAM roles in Avo's GCP project** — it has no permissions to read Avo's database, fetch secrets, or invoke other services. You pin your AWS trust policy to this identity by its 21-digit ID, so even a full compromise of the publish path yields only "write to opted-in customer buckets," never a foothold inside Avo.
- We recommend scoping the role to `s3:PutObject` on the Avo bucket only. Avoid attaching `AmazonS3FullAccess` or similar broad policies.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shorten? pretty repetitive from the tech jargon above

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed — trimmed in 50bcafc. Merged the "no credentials" and token-lifetime bullets, dropped the restatement of the OIDC/sub mechanism (now covered once in "How a publish authenticates" above), and kept only the points unique to this section: the dedicated service-account isolation, the sub-vs-aud rationale, and the least-privilege scope recommendation. Five bullets → four, no repeated mechanism.

Comment on lines +146 to +154
- **403 Access Denied**: The assumed role doesn't have `s3:PutObject` on the target bucket or folder prefix. Confirm the permission policy is attached and that its `Resource` ARN matches your bucket, e.g. `arn:aws:s3:::<your-bucket>/*`.
- **AccessDenied on `AssumeRoleWithWebIdentity`**: AWS refused to let Avo assume the role. The `accounts.google.com:sub` condition in your trust policy doesn't match Avo's service account ID shown in Avo, or the trust policy isn't using `Principal: { "Federated": "accounts.google.com" }`. Re-copy the service account ID from the integration screen into the condition.
- **InvalidIdentityToken**: AWS couldn't validate the Google-signed token (expired, clock skew, or a transient issue fetching Google's JWKS). This is usually transient — click **Publish** again to retry.
- **Role ARN errors (NoSuchEntity / malformed ARN)**: The configured **Role ARN** doesn't exist or is malformed. Confirm it matches the role you created, e.g. `arn:aws:iam::123456789012:role/avo-s3-publisher`.
- **404 NoSuchBucket**: The configured bucket doesn't exist in the configured region. Check the **S3 Bucket Name** (name only, no `s3://` prefix) and that the **AWS Region** matches where the bucket actually lives.
- **301 PermanentRedirect**: The bucket is in a different region than the one configured. Update **AWS Region** to the bucket's real region.
- **KMS / encryption errors**: The bucket enforces SSE-KMS with a customer-managed key, which is not currently supported. Switch the bucket to SSE-S3 or an AWS-managed KMS key.
- **Network timeout**: The request to AWS timed out. This is usually transient — click **Publish** again to retry.
- **Other errors**: The activity log captures the raw AWS error. If it isn't one of the above, reach out via the in-app support widget or at [support@avo.app](mailto:support@avo.app).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are all these error codes exposed via the integration publish action and show up this way in the activity log?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes — these are the actual AWS errors surfaced in the activity log on a failed publish (the UI surfaces the underlying AWS response), so the list maps to what users will actually see. Leaving as-is.

- **Network timeout**: The request to AWS timed out. This is usually transient — click **Publish** again to retry.
- **Other errors**: The activity log captures the raw AWS error. If it isn't one of the above, reach out via the in-app support widget or at [support@avo.app](mailto:support@avo.app).

{/* TODO: screenshot — the activity log on a failed publish */}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo 👀

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as the other screenshot TODO — this needs a capture of the activity log on a real failed publish. Tracked as a follow-up to add before GA; placeholder left in place.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These screenshots (bucket layout / failed-publish activity log) aren't available yet, so I removed the placeholder comment rather than ship an empty TODO in the published page. Happy to wire up the images later if you grab them — dropping them into public/images/publishing/ and adding the ![...] references is all it takes.

logason commented Jul 3, 2026

Copy link
Copy Markdown
Member

@thoragudf lmk when ready for re-review here!

…r-branch latest.json, trimmed security section

- Add private-beta access callout (line 15 feedback)
- Point to AWS IAM console for role creation (line 75)
- Tighten STS flow step 3 to cut repetition (line 88)
- Clarify latest.json is scoped per branch folder (line 102)
- Clarify manual feature-branch publishes land in the branch folder; branch is in the path, not the filename (line 97)
- De-duplicate the Security section against the auth flow above (line 138)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@logason logason left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1. The encryption callout is wrong (and the KMS troubleshooting bullet gives bad advice).
Doc says: "Default encryption at rest (SSE-S3, or SSE-KMS with an AWS-managed key) works without any extra setup. SSE-KMS with a customer-managed CMK is not currently supported."
Reality: the code sets serverSideEncryption: "AES256" explicitly on every PutObject (S3Publishing.res:153). Two consequences:
- Objects Avo writes are always SSE-S3, even if the bucket defaults to SSE-KMS — the explicit header overrides the bucket default. "SSE-KMS with an AWS-managed key works" implies the objects end up KMS-encrypted; they don't.
- A bucket whose policy requires KMS rejects Avo's uploads whether the key is AWS-managed or a CMK. The unsupported boundary is "bucket requires KMS," not "CMK specifically."
- The failure-handling bullet's remedy — "Switch the bucket to SSE-S3 or an AWS-managed KMS key" — is wrong: a bucket that requires AWS-managed KMS still rejects the AES256 upload.
Suggested wording (this is what we already corrected in planning/s3-publishing-integration/customer-doc.md after a review round): "Avo writes every object with SSE-S3 (AES256), set explicitly on each upload. Your bucket must allow SSE-S3 uploads — the default for new buckets. A bucket whose policy requires SSE-KMS (AWS-managed or customer-managed) will reject Avo's uploads; SSE-KMS is not supported in V1."

3. Payload formats lists two; the dropdown offers three.
The S3 config offers JSON Schema, NDJSON, and Snowplow Schemas (with a Vendor field) — S3IntegrationConfig.computePayloadFormatOptions:117-121. The doc omits Snowplow Schemas entirely. (AvoJson is flag-gated internal — correctly undocumented.) Either add Snowplow Schemas (with its Vendor field) or, if product doesn't want it offered on S3, that's a code change — but as shipped, the doc contradicts the UI.

## Security

- Avo holds no AWS credentials — not even encrypted. There's no secret to store, log, or leak. Each publish mints a fresh Google-signed token (valid about an hour) and exchanges it for AWS temporary credentials that expire after 15 minutes.
- The service account that mints the token is **dedicated to this integration and holds zero IAM roles in Avo's GCP project** — it can't read Avo's database, fetch secrets, or invoke other services. Your trust policy pins to this identity by its 21-digit `sub`, so even a full compromise of the publish path yields only "write to opted-in customer buckets," never a foothold inside Avo.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this still too much technical deail?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved the sub-vs-aud rationale into a collapsible <details> aside (21c33e0 / earlier 69f5b4b) so the security section reads as three plain bullets, with the deeper why one click away for readers who want it. Let me know if you'd rather cut it entirely.

Addresses logason L139: keeps the detail available but out of the
main security flow for readers who don't need it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@cursor

cursor Bot commented Jul 7, 2026

Copy link
Copy Markdown

Bugbot is not enabled for this team, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

The bucket-layout and failed-publish activity-log screenshots aren't
available; the prose stands on its own. Dropping the placeholder
comments rather than shipping empty TODOs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@cursor

cursor Bot commented Jul 7, 2026

Copy link
Copy Markdown

Bugbot is not enabled for this team, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

Addresses logason review (pullrequestreview-4626999697):

- Encryption: Avo sets serverSideEncryption AES256 explicitly on every
  PutObject, so objects are always SSE-S3. Corrected the callout and the
  KMS failure-handling bullet, which had wrong remedies (an AWS-managed
  KMS bucket still rejects the AES256 upload).
- Payload formats: the S3 dropdown offers three formats, not two. Added
  Snowplow Schemas and its Vendor field to match the shipped UI.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@cursor

cursor Bot commented Jul 7, 2026

Copy link
Copy Markdown

Bugbot is not enabled for this team, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@thoragudf

Copy link
Copy Markdown
Contributor Author

Addressed both factual points from your review (pullrequestreview-4626999697) in 21c33e0:

1. Encryption — corrected. The callout and the KMS failure-handling bullet now say Avo writes every object with SSE-S3 (AES256) set explicitly on each upload, so objects are always SSE-S3 regardless of the bucket default, and any bucket whose policy requires SSE-KMS (AWS-managed or customer-managed) rejects the upload. Dropped the wrong 'switch to an AWS-managed KMS key' remedy. Wording follows your suggested copy / the internal customer-doc.md.

3. Payload formats — the section now lists all three (JSON Schema, NDJSON, Snowplow Schemas) and documents the Vendor field that appears when Snowplow is selected, linking to the webhook payload docs. This matches the shipped dropdown. If product actually intends not to offer Snowplow on S3 (i.e. it should be a code change to remove it), say so and I'll revert the doc instead.

Sölvi confirmed the S3 dropdown offers Avo JSON alongside JSON Schema,
NDJSON, and Snowplow Schemas. Added it with the same feature-flag caveat
the webhook docs use, and dropped the hard format count so it doesn't
drift with the flag.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@cursor

cursor Bot commented Jul 7, 2026

Copy link
Copy Markdown

Bugbot is not enabled for this team, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@logason logason left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Only suggestion: remove mention of "Avo JSON" format entirely

- **JSON Schema** — a representation of your tracking plan following the [JSON Schema](https://json-schema.org/) standard. This is the common pick for downstream validation.
- **NDJSON** — newline-delimited JSON, with one event or property object per line. Useful for streaming the file into row-oriented data tooling.
- **Snowplow Schemas** — a [Snowplow Schema](/publishing/publishing/webhook-publishing#snowplow-schemas) for every event, for integrating Avo with Snowplow Iglu. Selecting this format reveals a **Vendor** field where you set the Snowplow vendor value stamped into the schemas.
- **[Avo JSON](/publishing/publishing/webhook-publishing#avo-json)** — a JSON representation of your entire Avo workspace. This format is behind a feature flag; [contact us](mailto:support@avo.app) to enable it.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exclude from here, we don't want general usage of this (prevents. us from making breaking changes)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the Avo JSON bullet entirely in 87c0b28 — agreed, no reason to invite general usage of an internal format. Snowplow Schemas stays. Thanks for the approve!

Per logason review (4646095990): Avo JSON is an internal format and
documenting it invites general usage that would block breaking changes.
Removed the bullet; Snowplow Schemas stays.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@cursor

cursor Bot commented Jul 7, 2026

Copy link
Copy Markdown

Bugbot is not enabled for this team, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@thoragudf thoragudf merged commit c5e5c43 into main Jul 7, 2026
4 checks passed
@thoragudf thoragudf deleted the worktree-s3-integration-docs branch July 7, 2026 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants