fix: unblock instanceable connectivity store-service install + clean up on delete
Live verification on pic1 of the connectivity v2 multi-instance feature
surfaced four integration bugs that prevented installing any published
connectivity store service (proxy/wireguard-ext/openvpn-client/sshuttle)
and left stale host routing state behind. All four are fixed here:
1. manifest_validator rejected the CI-published `name:tag@sha256:<digest>`
image form (it required digest-only), while service_store_manager already
accepted it — so every published store image failed validation. Allow an
optional tag before the digest, matching service_store_manager.
2. The cell-api image shipped the docker CLI but not the Compose v2 plugin,
so every `docker compose` ServiceComposer runs (pull/up/down for store
services) failed with "'compose' is not a docker command". Copy the
compose plugin binary from the docker-cli stage.
3. service_store_manager.install ran the base compose up for instanceable
services, whose template still contains ${INSTANCE_ID}/${REDIRECT_PORT}
(there is no base container — one runs per connection instance). It now
verifies the image signature but defers the container to connection
creation for instanceable manifests.
4. delete_connection freed the record/secrets/container but never removed the
connection's individually-managed `ip rule fwmark->table` or its FORWARD
kill-switch (apply_routes only flushes the PIC_CONNECTIVITY chains and
re-adds rules for surviving connections), leaking stale host routing state.
It now tears both down; added _remove_killswitch.
Verified end-to-end on pic1: two proxy instances allocate distinct
marks/tables/ports (skipping in-use resources), render distinct per-instance
containers, two peers route through distinct instances (per-peer MARK +
REDIRECT), delete is blocked while referenced (409) and cleans its ip rule.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -1419,6 +1419,25 @@ class ConnectivityManager(BaseServiceManager):
|
||||
logger.warning(f"delete_connection: container teardown failed "
|
||||
f"(non-fatal): {e}")
|
||||
|
||||
# Free this connection's host policy-routing rule and kill-switch.
|
||||
# apply_routes only re-adds rules for *existing* connections and only
|
||||
# flushes the PIC_CONNECTIVITY chains — it never removes the deleted
|
||||
# connection's individually-managed `ip rule fwmark→table` or its
|
||||
# FORWARD kill-switch, so they must be torn down here or they leak.
|
||||
mark, table = record.get('mark'), record.get('table')
|
||||
if (record.get('type') != self.CELL_RELAY_TYPE
|
||||
and isinstance(mark, int) and isinstance(table, int)):
|
||||
try:
|
||||
self._remove_ip_rule(mark, table)
|
||||
except Exception as e:
|
||||
logger.warning(f"delete_connection: ip rule cleanup failed "
|
||||
f"(non-fatal): {e}")
|
||||
try:
|
||||
self._remove_killswitch(mark, record.get('iface'))
|
||||
except Exception as e:
|
||||
logger.warning(f"delete_connection: killswitch cleanup failed "
|
||||
f"(non-fatal): {e}")
|
||||
|
||||
for secret_ref in record.get('secret_refs', []):
|
||||
if self.vault_manager is not None:
|
||||
try:
|
||||
@@ -2138,6 +2157,24 @@ class ConnectivityManager(BaseServiceManager):
|
||||
'-m', 'mark', '--mark', hex(mark),
|
||||
'!', '-o', iface, '-j', 'DROP'])
|
||||
|
||||
def _remove_killswitch(self, mark: int, iface: Optional[str]) -> None:
|
||||
"""Remove a connection's kill-switch FORWARD DROP (idempotent).
|
||||
|
||||
Unlike the per-peer MARK/REDIRECT rules (which live in the flushed
|
||||
PIC_CONNECTIVITY chains), the kill-switch is appended directly to
|
||||
FORWARD, so it is not cleared by apply_routes' chain flush — a deleted
|
||||
connection would otherwise leave a stale DROP that blocks a later
|
||||
connection reusing the same mark. Drain duplicates with a bounded loop.
|
||||
"""
|
||||
if not iface:
|
||||
return
|
||||
for _ in range(8):
|
||||
r = self._wg_iptables(['-D', 'FORWARD',
|
||||
'-m', 'mark', '--mark', hex(mark),
|
||||
'!', '-o', iface, '-j', 'DROP'])
|
||||
if r.returncode != 0:
|
||||
break
|
||||
|
||||
def _exit_status(self, exit_type: str) -> Dict[str, Any]:
|
||||
"""Return per-exit status (config presence + interface up/down).
|
||||
|
||||
|
||||
Reference in New Issue
Block a user