diff --git a/Dockerfile b/Dockerfile index 4a66c60..c735ca1 100644 --- a/Dockerfile +++ b/Dockerfile @@ -19,6 +19,7 @@ ENV S3_BUCKET=**None** ENV S3_REGION us-west-1 ENV S3_PATH 'backup' ENV S3_ENDPOINT=**None** +ENV S3_MULTIPART_THRESHOLD 5GB ENV S3_S3V4=no ENV SCHEDULE=**None** ENV SUCCESS_WEBHOOK=**None** diff --git a/README.md b/README.md index b9e6675..7edbfd9 100644 --- a/README.md +++ b/README.md @@ -42,6 +42,17 @@ postgres-backup: SUCCESS_WEBHOOK: https://sb-ping.ru/8pp9RGwDDPzTL2R8MRb8Ae ``` +### Buckets with object lock (WORM / compliance lock) + +Some S3 implementations reject `CompleteMultipartUpload` with `AccessDenied` once object lock is enabled on the bucket — completing a multipart upload has to clean up the uploaded parts, and the lock forbids that. To stay clear of it, the dump is uploaded in a single request unless it is larger than `S3_MULTIPART_THRESHOLD` (`5GB` by default, the maximum size of a single S3 upload). Set it to a lower value (e.g. `8MB`, the AWS CLI default) to get the usual multipart behaviour back: + +```yaml +environment: + S3_MULTIPART_THRESHOLD: 8MB +``` + +Also make sure `S3_REGION` matches the region of your bucket: with a wrong region every upload wastes a round-trip on an `AuthorizationHeaderMalformed` error before the AWS CLI retries with the right one. + ### Choose the right version We publish multiple builds targeting specific PostgreSQL versions (now its 16, 17 and 18). While you can always use the latest version, it's recommended to choose the build that matches your server's PostgreSQL version. This prevents compatibility issues where you might create a backup that can't be restored on your current server. For example, if you're running PostgreSQL 16, use tag `1.4.0-pg16` instead of `1.4.0` or `latest`. diff --git a/backup.sh b/backup.sh index 6b05931..a302507 100644 --- a/backup.sh +++ b/backup.sh @@ -52,6 +52,11 @@ else AWS_ARGS="--endpoint-url ${S3_ENDPOINT}" fi +# Buckets with object lock (WORM, compliance lock) may reject +# CompleteMultipartUpload, so upload the dump in a single request whenever it is +# small enough. Dumps above the threshold still fall back to a multipart upload. +aws configure set default.s3.multipart_threshold "${S3_MULTIPART_THRESHOLD:-5GB}" + # env vars needed for aws tools export AWS_ACCESS_KEY_ID="${S3_ACCESS_KEY_ID}" export AWS_SECRET_ACCESS_KEY="${S3_SECRET_ACCESS_KEY}"