# TailMux - Service Tunnels (GUI-managed SSH / DB endpoints)

Date: 2026-07-01

Status: Design proposal. Not yet implemented.

## Summary

Give TailMux a first-class concept of a **service tunnel**: a stable
loopback endpoint (`127.0.0.1:<port>`) that a plain client — a database GUI,
`psql`, an RDP/SMB client, `ssh` — points at once and forgets, while TailMux
relays the bytes through exactly one isolated profile. Tunnels are defined in
the GUI, kept alive automatically, and shown live in the menu bar. This is the
"Proton Bridge for internal services" experience.

Two tiers, because two kinds of target exist:

- **Tier A — native endpoint.** Pure TCP relay through the profile's `tsnet`
  dialer. Lab-proven to reach tailnet nodes, targets behind a subnet router,
  **and targets behind a Tailscale app connector** (see findings below).
- **Tier B — SSH-hop endpoint.** Fallback for targets the profile genuinely
  cannot dial (e.g. reachable only by jumping through an in-network host, or
  when tailnet ACLs block the direct path): TailMux runs `ssh -L` through a
  bastion, tunnelling the SSH transport itself through the profile. It reuses
  the user's system `ssh` (keys, agent, `known_hosts`); TailMux stores no
  credentials.

This document records why the split exists, the design, and the implementation
touchpoints. It complements [02-architecture](02-architecture.md) and
[04-security-isolation](04-security-isolation.md).

## Background: the problem to remove

Developers reach internal machines behind a bastion. The current documented
flows (`site/src/routes/docs/ssh.tsx`, `database-clients.tsx`) turn that into
manual CLI recipes:

```bash
# one-off SSH
ssh -o ProxyCommand="tailmux connect %h %p" user@host.work.ts.net

# fixed loopback for tools that cannot use ProxyCommand (DB GUIs, RDP, SMB)
tailmux tunnel --listen 127.0.0.1:15432 db.internal.example:5432
```

The goal is to make this near one-click from the GUI: define it once, keep it
alive, see it in the menu bar. The `tailmux tunnel` and `tailmux connect`
commands already implement the transport (`cmd/tailmux/tunnel.go`,
`cmd/tailmux/connect.go`); most of Tier A is productizing them.

## Investigation findings (evidence-based)

These findings drove the two-tier split. They were validated against the
running code and a live tailnet, not assumed.

1. **The transport already exists and is shared.** `tunnel` and `connect` both
   route by suffix and hand off to the single `proxy.Relay`
   ([proxy.go:1025](../internal/proxy/proxy.go)) via
   `proxy.DialUpstream` ([proxy.go:798](../internal/proxy/proxy.go)). A GUI
   tunnel manager is a lifecycle + UI layer over this, not new transport.

2. **The profile reaches the target with a plain `tsnet` dial.** The upstream
   dialer resolves via the profile's `tsnet` DNS and calls `server.Dial`
   ([internal/embedded/manager.go](../internal/embedded/manager.go),
   `serverDialer.DialContext`). There is **no SSH machinery anywhere in the Go
   code** — a full-repo grep finds only a comment. Reachability is therefore
   exactly "can this profile's `tsnet` node dial `host:port`".

3. **`tsnet` reaches normal tailnet nodes and subnet routes.** Confirmed live:
   `tailmux connect <node>.<tailnet>.ts.net 22` resolved MagicDNS to the node's
   `100.x` address and completed the TCP path (returned `connection refused`
   because that port had no listener — a *successful* network path).
   `accept_routes: true` maps to `RouteAll` for subnet routes
   ([manager.go](../internal/embedded/manager.go), `embeddedPrefs`).

4. **`tsnet` DOES traverse Tailscale app connectors** (lab-proven 2026-07-03,
   revising an earlier wrong assumption). With split-DNS configured and tailnet
   ACLs permitting, the warm service resolved an app-connector domain
   (`db.internal.example`) via the profile's DNS and completed TCP through the
   connector's dynamically advertised `/32` — verified in isolation with the
   subnet router taken down (HTTP 200, ~300ms, 3/3). The earlier real-world
   failure against a production app-connector DB was a **cold-start artifact**
   (the one-shot CLI's profile never reached `Running`, so the DNS query timed
   out), not a `tsnet`/app-connector incompatibility.

5. **Tailnet ACLs are a silent killer — and `tailscale ping` hides them.** A
   lab policy that dropped the default allow-all grant blocked every TCP dial
   while `tailscale ping` kept succeeding (disco pings bypass ACLs). Symptom
   through TailMux: hostname classifies, ping reports the peer reachable, yet
   every dial times out / SOCKS reply 1 / HTTP 503. The reachability
   diagnostic below must detect this signature and point at ACLs.

6. **One-shot cold start is the real UX hazard — and it caused the original
   bug report.** `connect`/`tunnel` cold-start fresh `tsnet` servers per
   invocation, sharing the state dir with the running service; a cold profile
   may not reach `Running` inside the dial deadline (`did not reach Running`,
   DNS `context deadline exceeded`), and repeated one-shot starts can
   destabilize the service's own node. Both real-world failures that motivated
   this design were cold-start artifacts. Tunnels must run **inside the
   persistent service**, never as one-shot CLI cold starts.

7. **Config applies on router restart — suffixes added for a new environment
   are not live until then.** Adding `.internal.example` to a profile while
   the router was running left the router classifying with the old suffix set
   (SOCKS reply 2, "no profile rule"). Expected per the design decision in
   AGENTS.md, but it must be surfaced in user docs as an environment-setup
   prerequisite ("save, then restart the router").

## Concept: Service Tunnel

A **service tunnel** is a named, persisted object owned by exactly one profile:

- a loopback listener `127.0.0.1:<localPort>` (auto-assigned, copyable);
- a target `host:port`;
- a **mode**: `native` (Tier A) or `ssh` (Tier B);
- lifecycle: started automatically when its profile is `Running`, restarted on
  failure, torn down on profile stop; startable/stoppable live from the UI.

The client is configured once against the loopback endpoint. Because the
endpoint is always present when the profile is up, there is no "did I start the
tunnel?" step — which is also why a DNS-interception feature is explicitly a
non-goal (see below).

### Naming

Do **not** call this a "bridge": it collides with the hard constraint *"Never
bridge traffic between profiles."* A tunnel is bound to one profile and never
falls back — it *reinforces* isolation. Use "tunnel" / "endpoint" / "service".

## Tier A — native endpoint

Wrap the existing `tunnel.go` relay inside the persistent service.

- Resolve + dial via the profile's `serverDialer` (unchanged transport).
- Per-connection dial at accept time (as `serveTCPTunnel` already does).
- Suffix classification decides the profile, or an explicit profile is set on
  the tunnel; the target host must be owned by that profile (no cross-profile).

Covers: tailnet nodes, subnet-router targets (the cupido case). Zero SSH, zero
credentials.

## Tier B — SSH-hop endpoint

Fallback for targets the profile cannot dial directly: hosts reachable only by
jumping through an in-network machine, or paths blocked by tailnet ACLs the
user cannot change. Lab-proven end-to-end (psql through the warm service's
SOCKS to a bastion, `-L` forward to the DB). TailMux supervises a system `ssh`
process equivalent to:

```bash
ssh -o ProxyCommand="tailmux connect %h %p" \
    -o ExitOnForwardFailure=yes \
    -N \
    -L 127.0.0.1:<localPort>:<target-host>:<target-port> \
    <user>@<bastion-host>
```

- The SSH transport to the bastion is tunnelled through the profile via
  `tailmux connect` (the same isolated path). The `-L` forward runs *on the
  bastion*, which sits inside the target network and resolves/reaches the
  app-connector DB with normal DNS/routing.
- **Reuse system `ssh`**: keys, `ssh-agent`, and `known_hosts` are the user's.
  TailMux stores no secrets and does not reimplement host-key verification.
- TailMux owns process lifecycle: spawn, health, auto-restart with backoff,
  surface the last error in the UI.

Bastion requirement: an in-VPC node that is a normal tailnet node (reachable by
`tsnet`, per finding 3), runs `sshd`, and can reach the target.

Infra-side alternative (out of TailMux's control, worth noting to users): if
the target's subnet is advertised by a **subnet router** instead of relying on
the app connector, Tier A reaches it directly and Tier B is unnecessary.

## Config model

Tunnels are persisted in the config file so they survive restarts, but
**starting/stopping is a live runtime action** (like `profile login/logout`),
not gated by the router-restart apply path documented in AGENTS.md. A top-level
map keyed by tunnel name, each referencing its profile:

```yaml
tunnels:
  prod-db:
    profile: work
    mode: ssh                       # native | ssh
    local_port: 15432               # optional; auto-assigned if omitted
    target: db.internal.example:5432
    ssh:                            # required when mode: ssh
      bastion: bastion.work.ts.net
      user: deploy
      # identity/agent come from the user's ssh config; nothing stored here
    autostart: true                 # start when the profile is Running

  desktop-rdp:
    profile: work
    mode: native
    local_port: 13389
    target: desktop.work.ts.net:3389
    autostart: true
```

Go sketch (mirrors `config.Profile` conventions in
[internal/config/config.go](../internal/config/config.go)):

```go
type Tunnel struct {
    Profile   string     `yaml:"profile"`
    Mode      string     `yaml:"mode"`       // "native" | "ssh"
    LocalPort int        `yaml:"local_port,omitempty"`
    Target    string     `yaml:"target"`     // host:port
    SSH       *TunnelSSH `yaml:"ssh,omitempty"`
    Autostart bool       `yaml:"autostart"`
}

type TunnelSSH struct {
    Bastion string `yaml:"bastion"`
    User    string `yaml:"user"`
}
```

Validation: loopback-only listener, `local_port` auto-assigned from the
existing allocator pattern (profiles use `43110+`; reserve a tunnel band),
`target` must be `host:port`, `ssh` block required iff `mode: ssh`, referenced
`profile` must exist. For `mode: native`, if `profile` is omitted the target's
suffix must classify to a single profile.

## Lifecycle and apply semantics

- Tunnel **definitions** persist through the normal `ConfigStore` save path.
- Tunnel **processes/listeners** are live: a new/edited tunnel can be started
  without a router restart, and start/stop is exposed as an explicit action.
  This intentionally differs from routing config (suffixes/ports), which still
  applies on restart per the AGENTS.md design decision.
- `autostart` tunnels come up when their profile reaches `Running` and are torn
  down when it stops. Restart on failure with capped backoff (reuse the accept
  backoff shape in `tunnel.go`).

## HTTP API

Extend the loopback control API (`cmd/tailmux/gui.go`, token-guarded like
`/api/config`):

```
GET    /api/tunnels                 list with live status
POST   /api/tunnels                 create (persist + optionally start)
DELETE /api/tunnels/{name}          remove (stop + persist)
POST   /api/tunnels/{name}/start    live start
POST   /api/tunnels/{name}/stop     live stop
```

Status payload per tunnel: `name`, `profile`, `mode`, `localAddr`, `target`,
`state` (`stopped` | `starting` | `listening` | `error`), `activeConns`,
`lastActivity`, `lastError`. Connection counts come from the relay layer.

## Reachability diagnostic ("why can't I reach this host?")

High support value and cheap given the findings above. A `POST
/api/tunnels/diagnose` (or a `tailmux doctor <host:port>` subcommand) that
classifies a failure into the exact buckets this investigation surfaced:

- profile not `Running` (cold start / needs login) — from `waitForRunning`;
- DNS resolution failed / hung — split-DNS missing for the domain, or profile
  not `Running`;
- classified but dial fails **while `tailscale ping` succeeds** — the
  **ACL-blocked** signature (disco pings bypass ACLs); point at the tailnet
  policy;
- classified but no route — subnet route/`accept_routes` missing, or the app
  connector has not advertised the route;
- connection refused / timeout — path reached, service down or wrong port;
- denied — unclassified `.ts.net`, raw IP literal, or no matching suffix (also
  raised when a just-added suffix is not live because the router has not been
  restarted).

This maps one-to-one onto existing error sites in `rules`, `embedded`, and
`proxy`, and turns the opaque failures we hit during design into actionable
messages.

## macOS menu-bar UI

Follow the existing Swift structure
(`Apps/TailMuxMenu/Sources/View/…`, `Store/TailMuxStore.swift`,
`Client/TailMuxAPIClient.swift`):

- **Settings — per profile.** A "Service tunnels" section in the profile
  editor: add/edit/remove; fields target, mode, (ssh) bastion + user; local
  port auto-filled with a copy button.
- **Menu-bar overview.** A live list of tunnels with the profile's color dot,
  state, active-connection count, copy-endpoint, and start/stop. This is the
  "see all open tunnels" the request asked for.
- New Swift model `TailMuxTunnel`; client methods `listTunnels`,
  `createTunnel`, `deleteTunnel`, `startTunnel`, `stopTunnel`.

## Security and constraints

- Listeners bind loopback only (enforce via the existing
  `requireLoopbackListen` / `TailMuxLoopback` checks).
- A tunnel is bound to one profile and never falls back — consistent with the
  hard constraints. No cross-profile relay.
- Tier B stores no credentials; it shells out to the user's `ssh` with a fixed,
  non-interactive argument set (`-N`, `-o ExitOnForwardFailure=yes`,
  `ProxyCommand=tailmux connect`). Host-key policy is the user's `ssh` default.
- `tailmux connect` already requires a valid license and loads validated config.

## Non-goals

- **DNS-shim interception** ("app tried to reach a managed host with no tunnel
  → prompt to create one"). Always-on autostart tunnels remove the pain this
  solves, and a system DNS shim conflicts with the "no system-wide route
  injection" stance. Revisit only if always-on proves insufficient.
- **Making `tsnet` natively traverse app connectors.** Deep upstream work; the
  SSH-hop is the pragmatic, shippable answer.

## Implementation touchpoints

| Area | File |
| --- | --- |
| Config model + validation | [internal/config/config.go](../internal/config/config.go) |
| Persist/save | [internal/manager/config_store.go](../internal/manager/config_store.go) |
| Tunnel supervisor (lifecycle, health) | new, alongside [cmd/tailmux/tunnel.go](../cmd/tailmux/tunnel.go) |
| Native relay | reuse `proxy.DialUpstream` + `proxy.Relay` ([internal/proxy/proxy.go](../internal/proxy/proxy.go)) |
| SSH-hop process mgmt | new; spawns system `ssh` with `ProxyCommand=tailmux connect` |
| Control API | [cmd/tailmux/gui.go](../cmd/tailmux/gui.go) |
| Swift models/UI/client | [Apps/TailMuxMenu/Sources](../Apps/TailMuxMenu/Sources) |
| Lab to validate | [10-orbstack-tailnet-lab.md](10-orbstack-tailnet-lab.md) |

## Open decisions

1. Confirm an SSH-able in-VPC bastion exists for the app-connector DBs (assumed
   yes — it is what the current guides rely on).
2. Confirm Tier B reuses system `ssh` + user keys/agent (recommended) rather
   than TailMux-managed credentials.
3. Phasing: ship Tier A (native, always-on, menu-bar) first — it already
   removes the cold-start pain and covers all `tsnet`-reachable targets — then
   Tier B for app connectors, then the diagnostic.

## Suggested phasing

1. **Tier A** — config model, supervisor, `/api/tunnels`, menu-bar overview,
   autostart. Covers tailnet + subnet-router targets; kills cold-start pain.
2. **Tier B** — SSH-hop mode over the same model/UI, using system `ssh`.
3. **Diagnostic** — reachability classifier + surfaced messages.
