fix(api): backup producers drop config and crash uploads with multiline metadata

[bchoor]

This PR fixes two distinct bugs in the backup pipeline that stack on top of each other: custom_command backups first fail because the orchestrator drops the producer's config keys (Bug 1), and once that is fixed they fail again because the artifact's multiline metadata is sent as illegal S3 object-metadata headers (Bug 2). Both fixes are needed before a custom_command backup — which is what every mail-server backup policy is — can complete. One commit per bug.

Bug 1: produceCommand dropped from payloadConfig

Every backup policy using the custom_command producer fails at run time with:

custom_command producer requires `produceCommand` in policy payload config

BackupOrchestrator builds the options passed to the producer as an allowlist of exactly three keys — sourceIds, command, exclude — pulled off policy.payloadConfig:

const producerOpts = {
  sourceIds: (policy.payloadConfig as { sourceIds?: string[] })?.sourceIds,
  command: (policy.payloadConfig as { command?: string })?.command,
  exclude: (policy.payloadConfig as { exclude?: string[] })?.exclude,
};

Any other field stored in payloadConfig — notably produceCommand, restoreCommand, and artifactName, which the custom_command producer requires — is silently dropped before it ever reaches producer.produce(...). Since a custom_command policy's entire configuration lives in those producer-specific keys, this allowlist breaks the producer outright. In practice this means every backup policy of kind custom_command fails, which includes all mail-server backup policies (they are configured exclusively as custom_command).

Fix 1

Spread the full policy.payloadConfig into producerOpts first, before the three explicit keys. The explicit sourceIds/command/exclude assignments are left in place unchanged, so any policy that only relies on those keys keeps behaving exactly as before — the spread only adds back the keys that were previously being thrown away.

Bug 2: artifact metadata crashes the upload with an invalid header

This second bug is only reachable once Bug 1 is fixed and a custom_command producer actually runs. The producer's artifact carries its produceCommand/restoreCommand text in artifact.metadata (needed later for restore). uploadArtifact forwards that metadata object as-is into the object-store put call:

await destination.put(key, hasher, {
  size: artifact.sizeHint,
  contentType: "application/octet-stream",
  metadata: artifact.metadata as Record<string, string>,
});

For an S3-compatible destination, each metadata key becomes an x-amz-meta-* HTTP header, and HTTP header values must be printable ASCII with no newlines. produceCommand/restoreCommand are multiline shell scripts, so the upload dies with:

Invalid character in header content ["x-amz-meta-producecommand"]

Fix 2

Filter the metadata passed to the object-store put call down to header-safe values only (typeof v === "string" and printable-ASCII-only), and pass that filtered object as metadata instead of the raw one. The unfiltered artifact.metadata is unaffected elsewhere: it is still recorded verbatim on the backup run's DB record, which is what the restore path reads restoreCommand from — restore never reads object-store headers, so this filtering has no effect on restore behavior.

Verification

On a self-hosted v0.3.0 instance managing a mail server, a custom_command backup policy was re-run after each fix in turn:

• Before either fix: failed immediately with the produceCommand error above.
• After Bug 1's fix alone: the producer ran, then failed uploading its artifact with the invalid-header error above.
• After both fixes: the run completed successfully — it transitioned through snapshotting → uploading → verifying, its artifact was uploaded to an S3-compatible backup destination, and the run's manifest was written alongside it.

The run history telling the whole story — bottom to top: the Bug 1 failure (produceCommand error), the Bug 2 failure after Fix 1 alone (metadata header error), and the same policy succeeding after both fixes:

🤖 Generated with Claude Code