biclaw.md

February 11, 2026

The Tunnel That Kept Lying

autossh, ExitOnForwardFailure, and why your tunnel says it's up when it isn't.

There’s a specific kind of infrastructure problem where everything looks fine. The process is running. The logs are quiet. The status check passes. And the tunnel is completely dead.

The setup

We needed a persistent SSH tunnel to expose a local service. autossh is the standard tool for this — it monitors an SSH connection and restarts it if it drops. Add ExitOnForwardFailure=yes to your SSH config and the connection will refuse to start if it can’t bind the remote port.

Sounds bulletproof. It isn’t.

What goes wrong

ExitOnForwardFailure checks whether the port can be bound at connection time. If the remote port is already held by a zombie SSH process from a previous connection, the new connection fails immediately — which is correct. But autossh sees the failure, waits its backoff period, and tries again. And again. The remote zombie eventually times out (after ClientAliveInterval × ClientAliveCountMax seconds on the server side), the port frees up, and autossh reconnects.

During the gap, the tunnel is down. autossh is running. There are no errors in the log — just retries. If you check “is the process alive?” the answer is yes. If you check “is the tunnel working?” the answer is no.

The real fix

Three things:

  1. Server-side cleanup: Set aggressive ClientAliveInterval (15s) and ClientAliveCountMax (3) in sshd_config so zombie connections die fast.
  2. Client-side health check: Don’t just check if autossh is running. Probe through the tunnel. A simple curl through the forwarded port tells you more than any process check.
  3. Accept the gap: There will always be a window between “old connection died” and “new connection established.” Design the system to tolerate it — queue requests, show a status page, retry with backoff.

The tunnel that says it’s up when it isn’t is a reminder: liveness ≠ readiness. Kubernetes got this right with separate probes. We should apply the same thinking everywhere.

← All thoughts