Commit Graph

297 Commits

Author SHA1 Message Date
roof 40f9d90fad feat: improve setup wizard and DDNS UX
Unit Tests / test (push) Successful in 7m29s
Setup wizard (Issue 1 — UI):
- pic.ngo subdomain input now uses the same split-field style as DuckDNS:
  input + static '.pic.ngo' suffix in a flex row, availability status below

Setup wizard (Issue 2 — Caddy not regenerating after completion):
- complete_setup route now fires IDENTITY_CHANGED after a successful wizard
  submission so CaddyManager regenerates the Caddyfile immediately; users
  no longer need to press 'Renew Certificate' to start ACME

Settings — DDNS status (Issue 2 — domain status missing):
- New GET /api/ddns/status endpoint: returns registered flag, domain_name,
  public_ip (ipify with 30s cache), last_ip from heartbeat
- Settings DDNS section for pic_ngo now shows a live status row with
  color-coded dot (green=registered+current, yellow=registered+stale,
  gray=not registered), current public IP, and a Check button
- Status auto-refreshes on mount and after each successful re-registration

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 00:36:47 -04:00
roof fb0326dae7 fix: remove auto-DDNS registration from installer; default to lan mode
Unit Tests / test (push) Successful in 7m27s
install.sh → make setup was registering 'mycell.pic.ngo' with DDNS at
install time (before the user ever opened the setup wizard). On a fresh
install the user would then open the wizard, choose 'pic1', and get a
401 OTP error because 'mycell' was already registered and the TOTP window
had moved on.

- Remove the register_with_ddns() call from setup_cell.py main(); DDNS
  registration now only happens through the setup wizard
- Change default DOMAIN_MODE from pic_ngo to lan so a bare 'make setup'
  no longer generates an ACME Caddyfile or pre-seeds a pic.ngo identity;
  the wizard collects the real cell name and domain mode from the user

make ddns-register still works for manual / scripted deployments.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 16:42:44 -04:00
roof e9077b2633 fix: Caddy health check must hit /config/ not /
Unit Tests / test (push) Successful in 7m35s
GET http://cell-caddy:2019/ returns 404 because Caddy's admin API has no
root handler.  The health monitor interpreted every response as a failure,
restarted Caddy every 3 minutes, and prevented ACME from ever completing.

/config/ returns 200 + the running config JSON whenever Caddy is up and
serving — that is the correct liveness indicator.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 15:57:32 -04:00
roof da302b5d54 fix: renew_cert regenerates Caddyfile before reload
Unit Tests / test (push) Successful in 7m32s
A stale or empty-token Caddyfile on disk caused Caddy to reject the
/load request, so the Renew button appeared to do nothing. Now
renew_cert() calls regenerate_with_installed([]) first, which writes a
fresh Caddyfile from current identity/config before reloading Caddy.
This ensures a broken on-disk file never blocks ACME renewal.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 14:38:30 -04:00
roof 6bd5f02b03 fix: surface DDNS registration failure during setup wizard
Unit Tests / test (push) Successful in 7m34s
Two problems on fresh install with pic_ngo mode:

1. Caddy crashed at startup because ddns.token was empty (registration
   hadn't completed yet), producing a bare `token` keyword in the
   Caddyfile that Caddy rejects with "wrong argument count".
   Fix: fall back to lan mode in _caddyfile_pic_ngo when the token is
   empty so Caddy always starts cleanly. The Caddyfile is regenerated
   once registration completes and the token is persisted.

2. DDNS registration failures were silently swallowed — the wizard
   showed "Setup complete!" with no indication that HTTPS wouldn't work.
   This made it look like everything was fine when the subdomain was
   never registered (e.g. name already taken from a previous install,
   or transient network error).
   Fix: capture the exception, classify it (name_taken vs transient),
   and return it as a `warnings` list in the setup response. The wizard
   done screen now shows amber warning cards with actionable text instead
   of auto-redirecting, giving the user a "Continue to login" button and
   a clear explanation of what went wrong.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 13:52:00 -04:00
roof 7ef294fd65 fix: fall back to lan mode in pic_ngo Caddyfile when token is empty
Unit Tests / test (push) Successful in 7m42s
On a fresh install before DDNS registration completes, ddns.token is
empty. Writing `token ` (bare keyword, no value) causes Caddy to reject
the Caddyfile at startup with "wrong argument count or unexpected line
ending after 'token'".

Guard added: if the token is empty, generate a LAN-mode Caddyfile so
Caddy starts cleanly. The Caddyfile is regenerated automatically once
registration completes and the token is persisted to cell_config.json.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 13:38:51 -04:00
roof 33d255f089 feat: TLS certificate management in Vault page
Unit Tests / test (push) Successful in 7m26s
Adds live cert status, one-click ACME renewal, and custom cert upload
directly to the Vault page so users never need to touch Caddy config.

Backend:
- CaddyManager.get_cert_status() now returns domain, domain_mode, and
  cert_type so the UI can render the right controls without a separate
  identity fetch
- CaddyManager.renew_cert() reloads Caddy and invalidates the status
  cache; the frontend polls until the cert turns valid
- CaddyManager.upload_custom_cert() validates PEM, writes cert+key to
  the shared config/caddy/certs/ volume, updates identity (cert_type=custom),
  and regenerates the Caddyfile so Caddy references the new paths
- LAN-mode Caddyfile switches from /etc/caddy/internal/ to the shared
  certs dir automatically when cert_type=custom is set
- ddns_api default no longer includes /api/v1 — the plugin appends it;
  legacy /api/v1 suffix is stripped at write time to keep the Caddyfile clean
- POST /api/caddy/cert-renew and POST /api/caddy/custom-cert routes added

Frontend:
- TLSPanel component at the top of Vault.jsx shows status badge
  (valid/expiring-soon/expired/pending/internal) with domain and expiry
- Renew button visible only for ACME modes; spins during the API call
  then polls GET /api/caddy/cert-status every 10 s until valid
- Upload Custom Cert opens a modal with PEM text areas; works for all modes
- caddyAPI.renewCert() and uploadCustomCert() added to api.js

Tests: 22 new tests across 5 classes covering enriched status,
renew_cert guards, upload_custom_cert validation/writes/persistence,
custom-cert Caddyfile path selection, and ddns_api suffix stripping.
All 2093 existing tests continue to pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 12:53:42 -04:00
roof 85d265187d fix: Caddy TLS cert acquisition — two DNS-01 blockers
Unit Tests / test (push) Successful in 7m32s
1. caddy_manager: embed ddns.token (registration bearer token) in
   Caddyfile, not DDNS_TOTP_SECRET. The pic_ngo plugin sends the token
   to POST /api/v1/dns-challenge; using the TOTP secret caused 401 on
   every attempt.

2. firewall_manager: add _acme-challenge.<zone> forwarding block before
   each split-horizon zone in the Corefile. Without this, CoreDNS was
   authoritative for the challenge name and returned NODATA for TXT
   queries (wildcard A record matches but wrong type), blocking Caddy's
   internal DNS pre-verification step.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 10:45:15 -04:00
roof 76bbc2b67a fix: EmailManager route calls get_email_users not get_users
Unit Tests / test (push) Successful in 7m27s
The method is named get_email_users in EmailManager; the route was
calling the non-existent get_users, causing an AttributeError on every
GET /api/email/users request.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 10:12:24 -04:00
roof bd71466a87 fix: split-horizon DNS zone uses WireGuard IP, not Docker bridge IP
Unit Tests / test (push) Successful in 7m31s
VPN peers can reach Caddy via the host's WireGuard interface (10.0.0.1),
not via the Docker bridge IP (172.20.0.2) which is unreachable outside
the container network. _bootstrap_dns now calls _get_wg_server_ip()
instead of ip_utils.get_service_ips() so the internal zone returns a
routable address for service subdomains.

Also log config save failures instead of silently swallowing them —
the silent PermissionError/OSError was masking write failures and
making it impossible to diagnose why installed services disappeared
after container restarts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 02:11:01 -04:00
roof e4c80149f4 fix: start-core missing cell-network creation breaks fresh install
Unit Tests / test (push) Successful in 7m34s
make start-core (called by install.sh step 6) used $(DCF) which includes
docker-compose.services.yml — that file declares cell-network as external:true.
On a fresh machine the network doesn't exist yet, so compose up failed with
"network cell-network declared as external, but could not be found".

Fix: add the same network-create idempotency guard that start and update
already have. Also add 26 regression tests (test_install_process.py) that
verify install.sh structure and that all start-* targets using DCF create
the network before running compose up.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-08 01:07:00 -04:00
roof 69862331e7 fix: DDNS update token in body, webdav gating, regression tests
Unit Tests / test (push) Successful in 7m25s
- PicNgoDDNS.update(): send token in request body instead of Authorization
  header; DDNS server validates it from body (was returning HTTP 422 on
  every heartbeat, leaving IP record stale after fresh install)
- peers.py / Peers.jsx: webdav service_access only valid when 'files' store
  service is installed; was always shown even with no services, confusing
  users into thinking WebDAV was pre-installed
- 10 new regression tests: DDNS update body contract, Caddy always
  regenerates on startup with no services, peer role allowed on
  /api/services/active, webdav gating by installed services

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 16:56:12 -04:00
roof 962d137093 fix: lockout countdown shows NaN minutes
Unit Tests / test (push) Successful in 7m31s
The API returns locked_until already ending in 'Z' (UTC ISO format).
Appending another 'Z' produces an invalid date string, so Date arithmetic
yielded NaN. Remove the redundant suffix.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 16:28:14 -04:00
roof 1607a2e86f fix: peer access to /api/services/active and unconditional Caddy startup regen
Unit Tests / test (push) Successful in 7m23s
- Add _PEER_READABLE_PATHS allowlist in enforce_auth so peer-role sessions
  can read /api/services/active; fixes My Services showing 'not installed'
  for cell members when services are installed
- Move Caddy regeneration before the early-return in reapply_on_startup so
  the Caddyfile is always rebuilt from current identity on startup, even when
  no store services are installed; fixes ERR_SSL_PROTOCOL_ERROR after a cell
  rename (Caddyfile retained old wildcard domain)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 15:58:27 -04:00
roof 9bdda6aaf8 fix: service credential provisioning and install reliability
Unit Tests / test (push) Successful in 7m21s
- calendar: create_calendar_user() now writes bcrypt htpasswd entry to
  data/services/calendar/config/users (the path Radicale reads at
  /etc/radicale/users); delete_calendar_user() removes the entry

- email: create_email_user() calls `docker exec cell-mail setup email add`
  to register the account in docker-mailserver's Dovecot/Postfix store;
  delete_email_user() calls the matching `setup email del` — both are
  non-fatal if the container isn't running

- service_composer.install(): pull image separately before up so slow
  registry pulls don't race with container startup; retry up once on
  failure so a transient registry hiccup on first install doesn't
  require the user to manually retry

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 13:41:41 -04:00
roof c696ca9ef6 fix: DNS split-horizon in DDNS mode, service access filter, health check, verbosity persistence
Unit Tests / test (push) Successful in 7m32s
- DNS (critical): add _configured_dns_params() that returns (primary_domain,
  split_horizon_zones) from config_manager so all apply_all_dns_rules() callers
  pass the correct primary zone (e.g. 'pic.ngo') and split-horizon list
  (e.g. ['pic1.pic.ngo']) instead of the FQDN as the primary — fixes
  DNS_PROBE_FINISHED_BAD_CONFIG for all external domains when on VPN

- firewall_manager: add split_horizon_zones param to apply_all_dns_rules()
  and forward it to generate_corefile()

- Peers: filter service_access list to installed services only; peers.py
  derives valid services from config_manager.get_installed_services() with
  the email→mail ID mapping; Peers.jsx fetches from /api/store/installed
  and filters the checkboxes and defaults accordingly

- Health check: fix file_manager→'files' ID mapping so files service health
  is checked when installed (was silently skipped due to 'file' vs 'files')

- Verbosity persistence: move log_levels.json from non-mounted
  /app/api/config/ to CONFIG_DIR (/app/config/) which maps to config/api/
  on the host; both load (managers.py) and save (routes/services.py) updated

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 13:05:58 -04:00
roof 4ebcb1d077 fix: don't overwrite split-horizon Corefile from _bootstrap_dns
Unit Tests / test (push) Successful in 7m29s
The apply_all_dns_rules() call at the end of _bootstrap_dns() was
added to force reload 30s into the Corefile on startup. Now that
reload 30s is removed (it broke CoreDNS zone serving), the call is
unnecessary in LAN mode and actively harmful in DDNS mode:
update_split_horizon_zone() already writes the correct Corefile
with the split-horizon block; the subsequent apply_all_dns_rules()
call would overwrite it without the split-horizon zones, causing
all service subdomain lookups to return NXDOMAIN.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 04:56:41 -04:00
roof 0507445d86 fix: remove file reload 30s from CoreDNS zone blocks
Unit Tests / test (push) Successful in 7m29s
CoreDNS 1.14.3 returns REFUSED for all zones that use
'file /data/zone reload 30s' — the reload timer defers the
initial zone load, causing the plugin to return REFUSED until
the timer fires. The timer never resolves this correctly.

Zone updates are already triggered by SIGUSR1 sent from
_reload_dns_service() after every zone file write, which
causes CoreDNS to reinitialise all plugins and re-read zone
files. No periodic zone polling is needed.

Also update config/dns/Corefile to remove the stale reload 30s.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 04:33:19 -04:00
roof 9b5c2e1994 fix: ensure DNS zone changes take effect immediately on startup
Unit Tests / test (push) Successful in 7m35s
Three related issues prevented CoreDNS from serving updated zone records:

1. The `file` plugin blocks in generate_corefile() lacked a `reload`
   option, so CoreDNS never re-read zone files after they were written.
   Added `reload 30s` so zone file changes are picked up within 30s.

2. _reload_dns_service() sent SIGHUP via `docker exec ... kill -HUP 1`,
   which doesn't trigger zone reloads. Changed to SIGUSR1 via
   `docker kill --signal=SIGUSR1` (same as firewall_manager.reload_coredns).

3. _bootstrap_dns() wrote the zone file but never regenerated the
   Corefile. CoreDNS's reload plugin only fires when the Corefile
   changes, so zone records from startup were invisible until the next
   peer modification triggered apply_all_dns_rules(). Now _bootstrap_dns()
   always calls apply_all_dns_rules() after the zone write.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 03:41:19 -04:00
roof 08f46332b0 fix: add built-in service subdomains to DNS zone on startup
Unit Tests / test (push) Successful in 7m45s
_build_dns_records() only hardcoded 'api' and 'webui', relying on the
optional service registry for the rest. Built-in services (calendar,
files, mail, webdav) were never registered, so they were absent from
the zone file and tests querying webdav.<domain> via CoreDNS got
NXDOMAIN.

Add _BUILTIN_SERVICE_SUBDOMAINS constant and include those names in
every zone build. Also update _stale and apply_cell_name exclusion
sets so DDNS mode correctly removes them from the parent zone.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-07 03:14:34 -04:00
roof e8b8e47aa4 fix: use sudo for nft list tables — /usr/sbin not in roof user PATH
Unit Tests / test (push) Successful in 7m26s
nft lives in /usr/sbin which is absent from the non-root PATH on Debian.
The delete call already used sudo; add it to the list call too so the
session-scoped cleanup fixture doesn't crash before any test runs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 15:46:09 -04:00
roof adce219a46 fix: clean up stale wg-quick nftables tables in e2e test teardown
Unit Tests / test (push) Successful in 7m29s
wg-quick creates an nftables 'preraw' table per interface that drops
decrypted ICMP replies arriving on any other interface. If a test run
crashes before bring_down(), the table persists and silently kills pings
on subsequent runs (handshake succeeds, replies are decrypted, but the
stale table drops them before the ping process sees them).

Extend cleanup_stale_e2e_interfaces() to also delete any orphaned
wg-quick-pic-e2e-* nftables tables found on the host.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 15:35:19 -04:00
roof 65d6d07c8d fix: get_status returns actual configured WG address instead of hardcoded default
Unit Tests / test (push) Successful in 7m41s
The address field in get_status() was hardcoded to SERVER_ADDRESS
('10.0.0.1/24') regardless of what wg0.conf contains, so instances
with a non-default subnet (e.g. pic1 at 10.0.1.1/24) always reported
the wrong server IP to callers such as the e2e WG conftest fixture.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 14:48:49 -04:00
roof ab6d6230dd Fix: read WG server IP and subnet from live API instead of hardcoding 10.0.0.x
Unit Tests / test (push) Successful in 7m30s
test_wg_connect_and_ping_server and the connected_peer fixture hardcoded
10.0.0.1 / 10.0.0.0/24 as the server VPN address. This breaks when the
server uses a different subnet (e.g. pic1 uses 10.0.1.1/24). Now both
read 'address' from /api/wireguard/status at session start and pass the
live server_ip / server_network through wg_server_info and connected_peer.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 14:09:48 -04:00
roof e2e9c50786 Test: skip peer-sync push test when WG tunnel between cells is not active
Unit Tests / test (push) Successful in 7m27s
The test_remote_permissions_pushed_to_cell2 test verifies that permission
changes on cell1 are pushed to cell2 via the WireGuard tunnel. When both
cells use a public endpoint (DDNS VPS) instead of LAN IPs, no tunnel is
established and the push silently fails. The test now probes cell2's API
at its WG DNS IP before asserting the push succeeded — skips gracefully
if the tunnel is down rather than failing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 12:52:03 -04:00
roof 568e4f9783 Fix: prevent wg0.conf truncation when remove_peer splits blocks
Unit Tests / test (push) Successful in 7m46s
_write_config() was stripping trailing newlines, causing the next
add_cell_peer() to create a single-newline separator between [Interface]
and [Peer] blocks instead of the required blank line. On the following
remove_peer() call, split('\n\n') treated both sections as one block,
matched the PublicKey filter, and wrote an empty string — destroying the
[Interface] section and reverting to the hardcoded SERVER_ADDRESS fallback.

Two-part fix:
1. _write_config() always ends content with a newline
2. remove_peer() normalises single-newline [Peer] headers to blank-line
   separators before splitting, and refuses to write if [Interface] would
   be lost

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 12:31:05 -04:00
roof 26576e1124 Fix: use domain_name (FQDN) in cell invite and conflict checks
Unit Tests / test (push) Successful in 7m39s
The GET /api/cells/invite endpoint was returning domain='pic.ngo' instead
of the full FQDN 'test5.pic.ngo' because it read _identity.domain rather
than _identity.domain_name.

Apply the same domain_name preference (domain_name || domain) to:
- routes/cells.py get_cell_invite() — the invite shown to connecting cells
- routes/cells.py update_cell_permissions() — Corefile DNS regeneration
- cell_link_manager.py _check_invite_conflicts() — incoming domain collision check
- cell_link_manager.py exchange_invites() — own invite construction

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 11:56:42 -04:00
roof 31f76c54fa Fix: use domain_name as service URL base and harden WG e2e tests
Unit Tests / test (push) Successful in 11m15s
API:
- _configured_domain() now prefers _identity.domain_name (full FQDN
  e.g. 'test5.pic.ngo') over domain ('pic.ngo'). Service URLs in
  /api/peer/services and /api/peer/dashboard now correctly return
  'calendar.test5.pic.ngo' instead of 'calendar.pic.ngo'.

WG e2e tests:
- test_api_domain_returns_json_not_webui: accept 3xx redirect as
  valid routing (Caddy redirects HTTP→HTTPS in pic_ngo mode).
- test_catchall_api_path_returns_json and test_catchall_root_serves_webui:
  skip when Caddy is in HTTPS-redirect mode — catch-all :80 block only
  exists in HTTP-mode cells (lan/local domain).
- test_http_api_domain_reaches_api: replace --dns-servers (requires
  c-ares) with dig + curl --host pattern.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 08:40:59 -04:00
roof b6af71acb5 Fix: accept both VIP and Caddy IP in DNS resolution test
Unit Tests / test (push) Successful in 11m9s
Cells with wildcard zone (e.g. * -> 172.20.0.2) and cells with per-service
VIP DNS records are both valid. Accept either in the assertion so the test
passes regardless of the zone file style.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 08:29:05 -04:00
roof 352bb6bb9e Fix: use api_base fixture instead of hardcoded pic0 IP in WG domain access tests
test_peer_services_* functions hardcoded 'http://192.168.31.51:3000' as the
fallback for PIC_API_BASE, causing failures when tests run on any other host
(including pic1 itself). Use the api_base fixture, which reads PIC_HOST and
PIC_API_PORT from the environment.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 08:06:29 -04:00
roof 463db029e1 Fix: expose listen_port in WG status API and add HTTPS DNAT to PostUp/PreDown
Unit Tests / test (push) Successful in 11m6s
Adds listen_port to /api/wireguard/status response so e2e test conftest
picks up the actual port (51821) instead of defaulting to 51820.

Extends PostUp/PreDown in generate_config to also DNAT and forward port
443 (HTTPS) through to cell-caddy — mirrors the ensure_service_dnat fix
so HTTPS works even after a WireGuard container restart without an API
restart. Updates _is_dnat_rule to recognize 443 rules.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 07:42:49 -04:00
roof 8da711e366 fix: DNAT and forward port 443 (HTTPS) to Caddy from WireGuard peers
Unit Tests / test (push) Successful in 11m9s
ensure_service_dnat() only wired port 80 → cell-caddy, so HTTPS was
silently dropped: no DNAT rule redirected 443 to the Caddy container,
and the FORWARD chain had no ACCEPT for dport 443. Refactored the
function to loop over both 80 and 443 so both are DNAT'd and forwarded.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 07:14:55 -04:00
roof 3e26186f85 fix: correct fake WireGuard key length and guard cell2_client teardown
Unit Tests / test (push) Successful in 11m14s
The synthetic cell fixture used a 46-char base64 key where the validator
expects exactly 43 chars before '='. The key failed format validation so
add_cell_peer returned False, making the cell connection store nothing and
all TestCellPermissionsApi tests hit 404.

The TestCellServiceAccessRestrictions and TestLiveCellConnection teardown
fixtures called _remove_connection(cell2_client, ...) without checking if
cell2_client is None (expected when no second cell is configured), causing
AttributeError on teardown.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 06:20:52 -04:00
roof f84f16fcd6 fix: add /api/network/dns/corefile endpoint and per-line iptables check
Unit Tests / test (push) Successful in 11m13s
The e2e tests were reading a stale Corefile at a hardcoded fallback path
(/home/roof/pic/config/dns/Corefile) instead of the live one written by
the API (/opt/pic/config/dns/Corefile on pic1). Adding a proper API
endpoint eliminates the path ambiguity.

The iptables test was checking whether peer_ip, DROP, and dpt:80 appeared
anywhere in the full multi-line output rather than on the same rule line,
producing false positives. Now checks per line.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 05:54:17 -04:00
roof eee0e800aa feat: add GET /api/peers/<peer_name> endpoint
Unit Tests / test (push) Successful in 11m19s
Allows fetching a single peer by name. E2E tests need this to verify
persisted peer state after PUT operations.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 05:19:10 -04:00
roof 2b29938a64 fix: set CSRF token in PicAPIClient after login
Unit Tests / test (push) Successful in 11m22s
POST requests from PicAPIClient were failing with 403 (CSRF token missing)
because the login response csrf_token was not being applied to subsequent
request headers.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 05:05:08 -04:00
roof 39c59fd3ef feat: WireGuard endpoint override + fix Docker network label issue
Unit Tests / test (push) Successful in 11m14s
Endpoint override:
- Add PUT /api/wireguard/endpoint to set endpoint_override in identity
  config; GET returns detected, override, and effective endpoints
- _effective_endpoint() helper applies override in peer config generation
  (wireguard.py and peer_dashboard.py); detected IP still shown in UI
- Add Endpoint Override input in WireGuard page — solves the common case
  where auto-detected IP is a gateway/VPS but peers connect via LAN IP

Docker cell-network fix:
- Declare cell-network external in docker-compose.yml; Docker Compose v5
  enforces label ownership and rejects networks created by older versions
- Makefile start/update pre-create cell-network idempotently
- reinstall/uninstall(full) explicitly delete and recreate the network
- Fix uninstall loop path: data/api/services/ (not data/services/)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 04:51:38 -04:00
roof 1b44a18062 fix: declare cell-network external; pre-create in Makefile start/update
Unit Tests / test (push) Successful in 11m16s
Docker Compose v5 enforces label ownership on networks it creates. On
systems where cell-network was created by an older compose version (no
labels), Caddy and other services fail to start with "incorrect label"
error.

Declaring the network external in docker-compose.yml skips label
validation. The Makefile start/update targets now create the network if
it doesn't exist (idempotent). The reinstall and uninstall (full) paths
explicitly delete the network so fresh recreations are clean.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 03:13:01 -04:00
roof f3737acfa4 fix: fall back to cell effective domain when email service domain not configured
Unit Tests / test (push) Successful in 11m10s
When the email store service is installed but no explicit domain has been
set in its config, _provision_email now falls back to
config_manager.get_effective_domain() so peer account creation works
immediately without requiring a separate config step.

Also threads config_manager into AccountManager.__init__ (optional kwarg,
no existing callers break) so the fallback is available without a global
import.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-05 17:06:51 -04:00
roof 64dd8b8488 fix: uninstall stops optional service containers before core teardown
Unit Tests / test (push) Successful in 11m11s
Iterates data/services/*/docker-compose.yml and runs `docker compose down`
for each before stopping core containers, so stale optional-service
containers (email, calendar, files, etc.) don't leave cell-network occupied
and block a subsequent fresh install.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-05 15:52:49 -04:00
roof 0267dce73d feat: HTTPS cert status, IDENTITY_CHANGED wiring, remove stale ip_utils Caddyfile writes
Unit Tests / test (push) Successful in 11m18s
- CaddyManager: add refresh_cert_status() and get_cert_status_fresh() that
  open a live TLS connection to cell-caddy:443 to read cert expiry; avoids
  needing a volume mount into the API container
- CaddyManager: periodic cert refresh in health_monitor_loop (every 60 cycles)
- config.py PUT /api/ddns: publish IDENTITY_CHANGED so CaddyManager regenerates
  the Caddyfile immediately after any domain/cell_name change — previously the
  event was never fired from this route
- config.py: remove all ip_utils.write_caddyfile() calls; CaddyManager is now
  the sole authority for Caddyfile generation
- app.py: add GET /api/caddy/cert-status route
- app.py: add GET /api/egress/status and PUT /api/egress/services/<id>/exit routes
- Settings.jsx: display cert status badge (valid/expired/internal/unknown) with
  expiry date and days-remaining in the domain section
- Tests: TestRefreshCertStatus (8 tests), TestDdnsConfigUpdatesFiresIdentityChanged,
  TestCaddyCertStatusRoute added; fix expired-cert helper to set not_valid_before
  relative to expiry so it's always earlier

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-05 11:39:36 -04:00
roof 41d09c598b wire: AccountManager HTTP dispatch + EgressManager startup + egress API routes
Unit Tests / test (push) Successful in 11m15s
- add_peer() now calls account_manager.provision() for any installed store
  service whose manifest declares accounts.manager == 'http', enabling
  per-peer credential provisioning to third-party HTTP services
- reapply_on_startup() calls egress_manager.apply_all() so fwmark rules
  survive container restarts without manual intervention
- add GET /api/egress/status and PUT /api/egress/services/<id>/exit routes
  so the UI can read and override per-service egress policy
- tests: HTTP provision wiring (happy path + non-fatal failure), egress
  apply_all at startup (wired/unwired/failure cases)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-05 10:30:41 -04:00
roof a906c26b5d fix: resolve Caddy env vars at write time to prevent parse errors
Unit Tests / test (push) Successful in 11m25s
acme_ca and the pic_ngo DNS credentials ({$PIC_NGO_DDNS_TOKEN},
{$PIC_NGO_DDNS_API}) were written as Caddy env-var placeholders, but the
Caddy container does not inherit the API container's environment, so the
substitutions always failed — Caddy saw bare directive names with no
arguments and rejected the Caddyfile.

- _global_acme_block: only emit the acme_ca directive when ACME_CA_URL is
  actually set; omitting it makes Caddy default to Let's Encrypt production.
- _caddyfile_pic_ngo: embed the DDNS_TOTP_SECRET and DDNS_URL values directly
  into the Caddyfile at write time rather than relying on Caddy env expansion.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-30 15:01:15 -04:00
roof e87022dc55 fix: cell-network name, install error surfacing, health history cleanup
Unit Tests / test (push) Successful in 11m22s
- docker-compose.services.yml: change external network name from
  pic_cell-network to cell-network so store-service compose files can find
  it.  The project-prefixed name was overriding the explicit name: cell-network
  fix in docker-compose.yml when both files were merged by make start.

- service_store.py: normalize docker compose stderr into the error key in
  the 400 response so the Store page shows the actual failure reason instead
  of the generic fallback message.

- app.py: skip health checks for email/calendar/files managers when those
  optional store services are not installed — prevents false Down alerts and
  unnecessary noise in health history.

- Logs.jsx: remove Email/Calendar/Files columns from the health history table;
  they are optional store services, not core builtins that should always appear.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-30 14:28:46 -04:00
roof 7d5c5421f1 Implement connectivity store services (wireguard-ext, openvpn-client, tor)
Unit Tests / test (push) Successful in 11m31s
- ConnectivityManager: move config dirs to data_dir/services/<id>/config so
  Docker can bind-mount them into store-service containers (Docker resolves
  bind-mount paths on the host, not inside the API container).  Add
  _migrate_legacy_configs to copy existing files from the old config_dir
  location on first boot.

- manifest_validator: add allow_host_network parameter to
  validate_rendered_compose.  When True, waives the external-network
  requirement, permits network_mode: host, and allows devices: — all needed
  by VPN/Tor containers that must share the host network namespace to create
  tun/wg interfaces.  Non-host services are unaffected.

- service_composer: read requires_host_network from the manifest and pass
  allow_host_network=True to validate_rendered_compose for connectivity
  services.

- Tests: update file-path assertions to new data_dir layout; add
  TestMigrateLegacyConfigs, TestValidateRenderedComposeHostNetwork, and
  two TestWriteCompose cases for the host-network path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-30 10:06:48 -04:00
roof 60601eb4af fix: give cell-network an explicit name to avoid compose project prefix
Unit Tests / test (push) Successful in 11m21s
Without name: cell-network, Docker Compose creates the network as
pic_cell-network (prefixed with the project name). Store service compose
templates declare cell-network as external: true and can't find it.
Adding name: cell-network makes the network name predictable regardless
of the Compose project name.

Existing installs need: make stop && make start to recreate the network.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-30 09:14:31 -04:00
roof 5ed75677c3 test: add e2e tests for service store install/uninstall flow
Unit Tests / test (push) Successful in 11m13s
Tests verify:
- /services page loads and lists all available services
- Admin can install calendar, files, email, and webmail via the store UI
- Install order respects dependencies (email before webmail)
- Uninstall flow shows confirmation dialog before removing
- Dashboard shows service links after install

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-30 04:51:10 -04:00
roof f7bb2cc962 fix: allow first-party store service subdomains and registry images
Unit Tests / test (push) Successful in 11m25s
Two manifest validation bugs blocked all store service installs:

1. service_store_manager.RESERVED_SUBDOMAINS included 'mail', which
   prevented the email service from using its required subdomain.
   Removed mail/calendar/files/webmail — they belong to official PIC
   store services and must be claimable by them.

2. manifest_validator required @sha256 digest pins on ALL images,
   including first-party git.pic.ngo/roof/* images that the PIC team
   builds and controls. service_store_manager._validate_manifest already
   only warned for first-party images; the secondary validator was
   stricter than intended, causing a hard reject on :latest tags.
   Aligned to warn-not-reject for first-party; malformed digests (when
   provided) are still a hard error.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-30 03:09:41 -04:00
roof c493630bb5 fix: Dashboard blank page — move state declarations before use
Unit Tests / test (push) Successful in 11m36s
SERVICES was computed on line 33 using activeServiceIds which was not
declared until line 36. In strict JS, const is not hoisted — this threw
a ReferenceError on mount, crashing the component and showing a blank page.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-30 02:44:41 -04:00
roof 0ed8669aec fix: dashboard only shows email/calendar/files if installed
Unit Tests / test (push) Successful in 11m25s
Fetches /api/services/active on load; service status cards and quick-
access links for email, calendar, files, and webmail are suppressed
until the service is installed via the Store. Core services (WireGuard,
Routing, Network) always show. Fixes #setup_complete gate on dev stack.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-30 01:38:16 -04:00