Skip to content

fix: drop the /etc/hosts requirement by setting S3_PROXY_URL - #44

Open
EtienneProthon wants to merge 2 commits into
pyronear:mainfrom
EtienneProthon:fix/remove-etc-hosts-requirement
Open

fix: drop the /etc/hosts requirement by setting S3_PROXY_URL#44
EtienneProthon wants to merge 2 commits into
pyronear:mainfrom
EtienneProthon:fix/remove-etc-hosts-requirement

Conversation

@EtienneProthon

@EtienneProthon EtienneProthon commented Aug 10, 2026

Copy link
Copy Markdown

Problem

Setting up this env requires adding 127.0.0.1 minio to the machine's hosts file. It needs sudo, it hijacks the name minio for every project on that machine, and it is especially awkward on Windows.

pyro-api builds presigned image URLs from S3_ENDPOINT_URL (http://minio:9000). That host only resolves inside the compose network, but the URLs are opened by the browser on the host — so without the hosts entry the alert images fail with ERR_NAME_NOT_RESOLVED.

Fix

pyro-api already supports S3_PROXY_URL, which rewrites the host of presigned URLs (storage.py). This repo simply never passed it. Setting it to http://localhost:9000 — the port MinIO already publishes — makes presigned URLs loadable from the host, and the hosts entry can go.

Internal consumers (pyro_api, temporal_model_api) keep talking to minio:9000 over the compose network; only browser-facing URLs are rewritten.

No new service, no extra port.

Why this works (and when it would stop)

boto3 presigns with SigV2 here (AWSAccessKeyId=...&Signature=...), and SigV2 does not cover the Host header, so rewriting the host keeps the signature valid. Measured on the same object:

Signature version Host rewrite
default (what pyro-api uses) → SigV2 200
forced SigV2 200
forced SigV4 403 SignatureDoesNotMatch

MinIO is not being lenient — it validates correctly (unsigned and tampered requests both return 403). This is noted in .env.test: if the S3 backend or boto3's default ever requires SigV4, this needs a Host-rewriting reverse proxy instead. Worth keeping in mind for a move off MinIO, whose repo is now archived — Garage, for instance, only accepts SigV4.

Also in here

tests/test_media.py runs on the host and had the same dependency, so it prefers the public endpoint too.

Its bucket assertions were stale: they expected a literal admin bucket, from before the SERVER_NAME prefix. Real buckets are {SERVER_NAME}-alert-api-{org_id}. This failure is pre-existing and endpoint-independent — it just was never reached, since the test failed earlier at connection time. Assertions rewritten against the actual naming.

Verification

Run with no minio entry in /etc/hosts (ping miniocannot resolve minio):

  • make run → all services healthy, init reports 0 errors
  • upload a detection, then GET /api/v1/detections/{id}/url200, URL host is localhost:9000
  • fetching that URL from the host → 200, bytes identical to the uploaded image
  • the frontend bundle contains no hardcoded S3 host, so it uses the URL the API returns
  • pytest tests/test_media.py → passes

Not covered: tests/test_alerts.py could not run on my machine — a local PostgreSQL occupies 127.0.0.1:5432 ahead of the Docker binding, so it queries the wrong database. Unrelated to S3 and untouched here.

Upgrading

The 127.0.0.1 minio line is harmless if left in place; existing setups keep working either way.

🤖 Generated with Claude Code

Presigned image URLs were built from S3_ENDPOINT_URL (http://minio:9000).
That host only resolves inside the compose network, but the URLs are opened
by the browser on the host — hence the `127.0.0.1 minio` prerequisite.

pyro-api already supports S3_PROXY_URL, which rewrites the host of presigned
URLs; this repo just never passed it. Set it to http://localhost:9000 (the
port MinIO already publishes) and drop the hosts entry.

tests/test_media.py runs on the host and had the same dependency, so it now
prefers the public endpoint too. Its bucket assertions were stale — they
expected an `admin` bucket, from before the SERVER_NAME prefix — and only
surfaced once the test could connect at all; rewritten against the actual
{SERVER_NAME}-alert-api-{org_id} naming.

Verified end to end with no hosts entry: upload a detection, GET its /url,
and load the returned URL from the host (200, matching bytes).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VkLLGbSBX8pMthZzfmTdsj

@fe51 fe51 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @EtienneProthon thank you for this first PR here, will make it smoother for next contributors !

Verified on my side with no minio entry in /etc/hosts: presigned URLs come back with host localhost:9000 and load fine from the browser.

One heads-up, not a change request: make init doesn't touch an existing .env, so anyone upgrading needs to add S3_PROXY_URL=http://localhost:9000 to theirs manually, otherwise removing the hosts entry breaks their images (happened to me while testing).

@fe51 fe51 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@EtienneProthon thanks a lot !

Ruff is not happy (see CI), could you have a look and then we will be good to go :)

tests/test_media.py:33:89: E501 Line too long (90 > 88 characters)
Found 1 error.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VkLLGbSBX8pMthZzfmTdsj
@EtienneProthon
EtienneProthon requested a review from fe51 August 11, 2026 20:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants