ADR – 002 Named Connection Instances for Connectivity

Context

The v1 connectivity model allocated one exit per exit type with fixed, hardcoded fwmarks (wireguard_ext → 0x10, openvpn → 0x20, tor → 0x30, sshuttle → 0x40, proxy → 0x50) and fixed routing tables (110–150). This meant:

You could not run two WireGuard providers simultaneously (for example, a personal VPN and a work VPN).
You could not assign different peers to different exits of the same type.
Adding a new peer to a connection required touching the global exit config rather than a per-peer record.
UI navigation for connectivity was a single flat page with one configuration slot per type.

As the number of use-cases for connectivity grew (privacy routing for some peers, geographic routing for others, per-service egress), the one-per-type limit became a blocker.

Options Considered

Option A — Keep v1, add per-peer overrides

Extend the one-per-type model with a per-peer config field that allows overriding the type's single global config. For example, peer.wireguard_ext_config could point to an alternate server.

Rejected because: this still limits you to one WireGuard interface per type globally. Two peers needing different WireGuard upstreams would share one interface with multiplexed config, creating routing conflicts. The data model also becomes inconsistent — exit config partly global, partly per-peer.

Option B — Named connection instances (chosen)

Replace the fixed-per-type model with N named connection instances. Each instance is identified by a UUID, has a human label, and is allocated its own fwmark and routing table from a pool. Peer assignment references the instance UUID, not the type.

Decision

We adopted the named connection instance model (v2). The key decisions within that model:

fwmark pool: 0x1000–0x1FFF, stride 0x10 (allows up to 255 simultaneous instances). The old v1 marks (0x10–0x50) are in the pool but are not re-allocated to avoid conflicts on upgraded cells.
Routing table pool: starting from 1000, one table per instance.
Per-instance exit containers: each wireguard_ext or openvpn instance gets a dedicated interface name (wgext_<suffix> or ovpn_<suffix>). Redirect-type instances (tor, sshuttle, proxy) get a dedicated redirect port from 9100–9199.
Per-peer assignment: each peer record stores exit_connection_id (UUID of the assigned instance). The legacy route_via field is kept in sync for backward compatibility.
Fail-open/fail-closed: configurable per-instance with peer-level override. Default is fail-closed for all types except Tor (where fail-open is the default, because the typical Tor user prefers degraded-but-connected over completely blocked).
Per-connection health probes: each instance has its own health check mechanism appropriate to its type. Results are cached for 30 seconds.
cell_relay auto-derived from cell links: when a linked remote cell advertises an internet exit, a cell_relay connection instance appears automatically without admin configuration. These fail-closed by default.
UI moved to sub-pages: /connectivity became a connection list, with each connection's peers and config on a sub-page.

Migration v1 → v2: on the first call to get_connectivity() after upgrade, ConnectivityManager._migrate_connectivity_v1_to_v2() runs if the stored schema version is less than 2. It creates one named instance per previously-configured exit type, re-points vault secret references to the new conn_<id>_<field> naming scheme, deletes the old references, and writes the new schema version. The migration is idempotent.

Consequences

Operators must update client configs on major upgrade if they used connectivity in v1. The migration runs automatically but peer re-assignment to the new instance UUIDs happens in the background.
v1 fwmarks (0x10–0x50) are reserved and not re-used by the v2 allocator, to avoid collisions on upgraded cells during the migration window.
The Tor type remains limited to one instance per cell. The Tor container does not support multiple simultaneous SOCKS listeners with isolated routing in the current implementation. This constraint is enforced by ConnectivityManager.create_connection() which returns an error if a second tor instance is requested.
UI for connectivity became deeper. Admins navigate to a connection and then to its sub-pages (Peers, Config, Health). This is more clicks for simple use-cases but necessary for the multi-instance model.
Vault secret naming changed. Old keys (wireguard_ext_conf) became conn_<uuid>_conf. The migration handles existing cells; new cells only ever see the new naming.

Internals: see Dev – Architecture (Connectivity v2 data model section).

Personal Internet Cell

Home

ADR – 002 Named Connection Instances for Connectivity

Context

Options Considered

Option A — Keep v1, add per-peer overrides

Option B — Named connection instances (chosen)

Decision

Consequences

Personal Internet Cell

New here?

Users

Admins

Developers

Decisions (ADRs)

Meta

Archive