feat: support remote compilation over SSH tunnels by wmann-celonis · Pull Request #103 · celonis/homcc

Willi Mann (wmann-celonis) · 2026-07-10T08:44:43Z

Summary

Adds SSH as a first-class connection type alongside TCP. Hosts written as @HOST / USER@HOST are served by tunneling to an already running homccd: a multiplexed OpenSSH master connection (ControlMaster/ControlPersist) local-forwards a port to the daemon's loopback port, and the existing homcc protocol runs over that port.

This model was chosen to satisfy three constraints:

The server keeps enforcing its global connection/compilation limit — every tunneled connection reaches the same daemon as direct TCP clients, so the existing limit covers both transports.
TCP and SSH work in parallel — one daemon serves both.
Low connection latency — the SSH master is established once per host and reused, so per-job cost stays close to a plain TCP connect.

Changes

Transport abstraction (homcc/client/client.py): extract RemoteCompilationClient base holding all protocol logic; TCPClient becomes a thin subclass.
SSH transport (homcc/client/ssh.py, new): SSHTunnel (master lifecycle, local port-forward, per-host flock so the many concurrent homcc processes a build spawns converge on one tunnel) and SSHClient.
Dispatch (homcc/client/compilation.py): create_remote_client picks the transport from host.type.
Host parsing (homcc/common/host.py): SSH hosts accept an optional remote daemon port — @host:port, user@host:port, @[ipv6]:port.
Errors (homcc/common/errors.py): SSHError(ConnectionError) so tunnel failures fall through to the next host / local fallback like any lost connection.
Config (homcc/client/config.py): ssh_executable, ssh_control_persist, ssh_options via env vars and homcc.conf.
Tests: tests/client/ssh_test.py (tunnel logic, dispatch, error paths), SSH port-parsing tests, config tests, and an opt-in e2e test behind --runssh.
Docs: README hosts/config/deployment sections and CLAUDE.md.

No server changes are required.

Testing

Unit tests pass (parsing, ssh, compilation, messages).
The generated ssh command line was validated against the real ssh binary (fails on connection, not usage → syntax correct).
SSHClient shares the entire protocol path with TCPClient (only _open_connection differs, connecting to a local port), which is covered by the existing TCP e2e tests.
The full SSH round-trip runs under pytest --runssh in an environment with an sshd reachable on the local host.

Notes

Relies on the developer's existing SSH setup (keys/agent); homcc does not manage SSH credentials.
SSH hosts default the remote daemon port to 3126 (matching TCP) when unspecified.

🤖 Generated with Claude Code

Copilot

Copilot wasn't able to review this pull request because all changed files are excluded by your Copilot content exclusion settings. See Excluding content from Copilot for details.

Oliver Layer (OliLay)

Left some minor comments. I like the approach since we do not teach homccd how to handle SSH. 👍

Oliver Layer (OliLay) · 2026-07-14T13:35:02Z

 ## Deployment hints
 Things to keep in mind when deploying `homccd`:
- `homcc` currently does not support any transport encryption such as TLS, so source files would get transmitted over the internet in plain text if not using a VPN.
+- `homcc` does not support built-in transport encryption such as TLS: plain TCP hosts transmit source files unencrypted, so a VPN is required over untrusted networks. Alternatively, use an SSH host (`@HOST`/`USER@HOST`) to tunnel the connection through an encrypted, authenticated SSH channel to a `homccd` bound to the remote loopback interface.


I think we can remove this now?

Left it — it's not just about TLS, the other two bullets (no per-client connection limit, no restriction on docker/chroot selection) still hold regardless of transport. Reworded it instead so it doesn't imply VPN is the only mitigation now that SSH tunneling is an option too: 9102cd4

Oliver Layer (OliLay) · 2026-07-14T13:48:45Z

+    @classmethod
+    def get_ssh_options(cls) -> Optional[List[str]]:
+        if (ssh_options := os.getenv(cls.HOMCC_SSH_OPTIONS_ENV_VAR)) is not None:
+            return ssh_options.split()


I think this doesn't work with spaces in SSH options, does it? Maybe https://docs.python.org/3/library/shlex.html#shlex.split could work here?

Good catch, fixed with shlex.split in both the env var and config-file parsing paths. 9102cd4.

Simon Pirsch (spirsch) · 2026-07-15T10:01:00Z

+    def close(self):
+        """Tear down the multiplexed master connection. Idle masters are otherwise reaped via ControlPersist."""
+        subprocess.run(  # noqa: PLW1510
+            [*self._control_args(), "-O", "exit", self.target],
+            stdout=subprocess.DEVNULL,
+            stderr=subprocess.DEVNULL,
+            check=False,
+        )
+        self._port_file.unlink(missing_ok=True)


I couldn't quite figure out how this method is used. Where is it called? How is this triggered?

Traced it — it's actually dead code, nothing calls it (masters are reaped by ControlPersist instead). Removed it in 9102cd4. Shout if there was an intended caller I'm missing (e.g. explicit teardown on client exit) and I'll bring it back.

@host

Add SSH as a first-class connection type alongside TCP. SSH hosts (@host / USER@HOST) are served by tunneling to an already running homccd: a multiplexed OpenSSH master (ControlMaster/ControlPersist) local-forwards a port to the daemon's loopback port, and the existing protocol runs over that port. All jobs reach the same daemon, so its global connection limit covers TCP and SSH alike, and per-job latency stays close to a plain TCP connect. - Extract transport-agnostic RemoteCompilationClient base from TCPClient; add SSHClient and SSHTunnel (client/ssh.py) - Dispatch on host.type via create_remote_client factory - Parse optional remote daemon port for SSH hosts (@host:port, user@host:port, @[ipv6]:port) - Add SSHError(ConnectionError) so tunnel failures fall through to the next host / local fallback - Wire ssh_executable / ssh_control_persist / ssh_options config - Serialize per-host master setup with an flock so concurrent homcc processes converge on one tunnel - Tests: tunnel logic, dispatch, SSH port parsing, config, and an opt-in e2e test behind --runssh; document in README and CLAUDE.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

- ssh.py: catch OSError/CalledProcessError separately in _start_master instead of a manual returncode check, so a missing ssh executable is reported distinctly; reuse common.parsing.default_locations for the ssh control-socket directory; make control_args a public property; switch to asyncio.to_thread; drop the unused SSHTunnel.close() - config.py: parse HOMCC_SSH_OPTIONS/ssh_options with shlex.split so quoted values with spaces work - README.md: clarify that SSH tunneling is also a mitigation for the no-built-in-encryption deployment hint - add AGENTS.md as a symlink to CLAUDE.md Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Willi Mann (wmann-celonis) requested review from Oliver Layer (OliLay), Copilot and Simon Pirsch (spirsch) July 14, 2026 12:42

Copilot started reviewing on behalf of Willi Mann (wmann-celonis) July 14, 2026 12:43 View session

Copilot AI reviewed Jul 14, 2026

Willi Mann (wmann-celonis) marked this pull request as ready for review July 14, 2026 12:44

Oliver Layer (OliLay) approved these changes Jul 14, 2026

View reviewed changes

Simon Pirsch (spirsch) reviewed Jul 15, 2026

View reviewed changes

Willi Mann (wmann-celonis) force-pushed the feat/ssh-transport branch from 2e14566 to 8a9c3f1 Compare July 16, 2026 16:14

Willi Mann (wmann-celonis) requested a review from Simon Pirsch (spirsch) July 17, 2026 05:57

Simon Pirsch (spirsch) approved these changes Jul 17, 2026

View reviewed changes

Willi Mann (wmann-celonis) merged commit 218eefe into main Jul 17, 2026
7 checks passed

Willi Mann (wmann-celonis) deleted the feat/ssh-transport branch July 17, 2026 11:56

Navigation Menu

Uh oh!

feat: support remote compilation over SSH tunnels - #103

feat: support remote compilation over SSH tunnels#103
Willi Mann (wmann-celonis) merged 2 commits into
mainfrom
feat/ssh-transport

Willi Mann (wmann-celonis) commented Jul 10, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Oliver Layer (OliLay) left a comment

Uh oh!

Oliver Layer (OliLay) Jul 14, 2026

Uh oh!

Willi Mann (wmann-celonis) Jul 16, 2026

Uh oh!

Uh oh!

Oliver Layer (OliLay) Jul 14, 2026

Uh oh!

Willi Mann (wmann-celonis) Jul 16, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Simon Pirsch (spirsch) Jul 15, 2026 •

edited

Loading

Uh oh!

Willi Mann (wmann-celonis) Jul 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

Willi Mann (wmann-celonis) commented Jul 10, 2026

Summary

Changes

Testing

Notes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Oliver Layer (OliLay) left a comment

Choose a reason for hiding this comment

Uh oh!

Oliver Layer (OliLay) Jul 14, 2026

Choose a reason for hiding this comment

Uh oh!

Willi Mann (wmann-celonis) Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Oliver Layer (OliLay) Jul 14, 2026

Choose a reason for hiding this comment

Uh oh!

Willi Mann (wmann-celonis) Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Simon Pirsch (spirsch) Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Willi Mann (wmann-celonis) Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Simon Pirsch (spirsch) Jul 15, 2026 •

edited

Loading