The pic1 commit (c65beb2) correctly removed rp_filter sysctl from
WireGuard PostUp/PostDown because writing /proc/sys fails in the
unprivileged (NET_ADMIN-only) container and crashed wg-quick. Two
tests that asserted rp_filter was present were left stale. Replace
them with a single test asserting rp_filter is NOT in the generated
config, restoring green main.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
sysctl writes to /proc/sys/net/ are blocked in unprivileged containers
(NET_ADMIN only, no SYS_ADMIN). The rp_filter=0 call at the end of
PostUp caused wg-quick to tear down wg0 immediately on every start,
putting cell-wireguard into a crash loop.
Remove the sysctl lines from both the seed (setup_cell.py) and the
API-regenerated (wireguard_manager.py) wg0.conf. Reverse-path filtering
is an optimisation, not required for VPN functionality; the iptables
FORWARD/MASQUERADE/DNAT rules all still work correctly without it.
Found during clean-install hardening verification on pic1 (f4b8d5c).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Security — WireGuard:
- Replace linuxserver/wireguard (privileged + SYS_MODULE + /lib/modules) with a
bespoke alpine image (wireguard/Dockerfile + entrypoint.sh): CAP_NET_ADMIN only,
119 MB → 14.7 MB. Modern kernels (≥5.6) have WireGuard built in; no module
loading required. Kernel-fallback comment left in compose for rare old kernels.
Security — supply-chain digest pins:
- CoreDNS image pinned by SHA-256 digest in docker-compose.yml.
- api/Dockerfile: python:3.11-slim and docker:27-cli pinned by digest.
- webui/Dockerfile: node:20-alpine and nginxinc/nginx-unprivileged:alpine pinned.
- ntp/Dockerfile: alpine:3.20 pinned by digest.
- wireguard/Dockerfile: alpine:3.20 pinned by digest.
Security — webui non-root:
- Switch from nginx:alpine (root, port 80) to nginxinc/nginx-unprivileged:alpine
(port 8080, runs as nginx uid 101). Compose port mapping and all Caddy upstream
references updated: cell-webui:80 → cell-webui:8080 everywhere.
API layer reduction (561 MB → 245 MB):
- Multi-stage api/Dockerfile: docker CLI copied from docker:27-cli stage instead
of being installed via apt from Docker's external repo (removes GPG key fetch,
lsb-release, gnupg, two apt-get update rounds). --no-install-recommends on
remaining apt install. mkdir folded into the same RUN layer.
Bug fix — WireGuard config path mismatch:
- setup_cell.py wrote wg0.conf to config/wireguard/wg0.conf but wireguard_manager
and the new entrypoint expect config/wireguard/wg_confs/wg0.conf (the standard
wg-quick sub-directory). Fixed by creating the wg_confs/ sub-dir and writing
there; REQUIRED_DIRS updated to pre-create it.
Bug fix — empty chrony.conf:
- config/ntp/chrony.conf was 0 bytes (pre-existing gap); added a real config
(pool.ntp.org + Cloudflare, allow 172.20/10.0, local stratum 10, driftfile,
makestep, rtcsync). NTP compose service now builds from ./ntp instead of
pulling alpine:latest and running apk at every container start.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds TestStartupCaddyRegen::test_startup_regenerates_caddyfile_first,
asserting that _apply_startup_enforcement() calls
caddy_manager.regenerate_with_installed([]) before any peer/iptables work.
This pins the fix that ensures a stale on-disk Caddyfile (e.g. missing
`admin 0.0.0.0:2019`) is overwritten at startup and cannot cause the health
monitor to restart Caddy every few minutes.
Also restores two displaced lines in test_health_history_maxlen_evicts_old_entries.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The installer dumped ~200 lines of docker layer spam, a leaked apt error,
and obsolete version warnings, alarming for non-technical users.
install.sh:
- Clean, progress-only default output; full log to /var/log/pic-install.log
- Admin password still surfaced on stdout at the end
- PIC_DEBUG=1 / --debug flag restores verbose output
- On error, prints the last 20 lines from the log file
Makefile:
- start / update / start-core compose invocations get @ prefix to suppress
command echo, plus --quiet-pull to kill layer-download spam
docker-compose.yml + docker-compose.services.yml:
- Removed obsolete `version: '3.3'` top-level key (triggers deprecation
warning with current Docker Compose)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
VPN clients got dns_probe_finished_bad_config / couldn't resolve any domain
after first setup because:
1. complete_setup() never wrote the split-horizon DNS zone for non-LAN modes;
SetupManager now accepts network_manager as an optional 3rd constructor
param, and complete_setup() calls
self.network_manager.update_split_horizon_zone(effective_domain, wg_ip,
primary_domain) for pic_ngo/cell_to_cell modes.
2. generate_corefile() used a tmp-file + os.replace pattern; the Corefile is
a Docker FILE bind-mount, so os.replace orphaned the inode and CoreDNS
never saw config updates. Fixed by truncating and rewriting in place
(open with 'w', seek(0), truncate()), preserving the inode CoreDNS holds.
api/managers.py passes network_manager into SetupManager.
Tests: new mock_network_manager fixture, 2 setup-zone tests, 1 inode
regression test in test_firewall_manager.py.
Verified live on pic1.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root-cause fix for ACME failures caused by clock drift breaking TOTP
during DDNS registration: install and start chrony (all supported package
managers) before the setup wizard runs, so the host clock is accurate from
day one.
Also enables and starts the pic systemd unit at the end of a cold install —
previously the unit file was written but never activated, so the stack would
not survive a reboot without a manual `systemctl enable --now pic`.
Makefile uninstall hardened: `disable --now` instead of bare `disable` so the
running unit is stopped before the unit file is removed; daemon-reload called
afterwards to flush the stale unit; and all lingering cell-* containers
(tor/sshuttle/redsocks/store services) are now force-removed so subsequent
reinstalls start from a clean Docker state.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Coverage was below acceptable levels and several newly-added code paths
(sshuttle egress, proxy egress, DDNS provider stubs, DNS overview route,
peer-registry provisioning) had zero test coverage.
~250 new unit tests are added across 16 new test files. Existing test files
are updated to match refactored interfaces (DHCP removed, constants
introduced, network_manager restructured). .coveragerc is added to pin the
source mapping and the 70% floor so regressions are caught at commit time.
tests/test_enhanced_api.py was previously living in api/ (wrong location)
and is moved to tests/ where it belongs.
Integration test files are updated to remove references to DHCP endpoints
and add coverage for the new DNS overview and DDNS sync endpoints.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Network Services page is rebuilt around real API data: GET /api/dns/overview
returns provider-aware records; per-service Cloudflare sync is exposed via
POST /api/ddns/sync; effective domain is displayed so operators can verify
what external name resolves to the cell; NTP status reflects the actual
systemd-timesyncd state rather than a hardcoded boolean.
DHCP is fully decommissioned: the cell-dhcp container is removed from
docker-compose.yml, DHCP methods are stripped from network_manager, the
setup_cell script no longer seeds DHCP config, and the Settings DHCP field
is gone. DHCP was never a PIC responsibility and the container was consuming
resources for no benefit.
Dead code removed: api/config.py (superseded by config_manager), the
standalone Email/Calendar/Files pages (these are now optional store services
and do not need dedicated pages). api/constants.py is introduced to hold
RESERVED_SUBDOMAINS in one place rather than scattered literals.
Docker resource limits (mem_limit, cpus, pids_limit) are added to all
compose services so a runaway process cannot starve the host.
Makefile gains a warning before the backup target so operators are not
surprised by the archive path. Settings same/accept state fix ensures
the Cell Identity section correctly shows the accept/discard banner and
does not flash a false-positive change indicator on first load.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The peer table was empty because it was not consulting the peer registry;
now peers are driven by PeerRegistry so the Connectivity page reflects actual
connected cells.
Exit-key handling is unified: all code paths now use the same key derivation
so a store-service exit bridge and a manual WireGuard peer both produce
consistent routing state.
Two new egress exit types are added (sshuttle via SSH tunnel and proxy via
redsocks SOCKS5), wiring through connectivity_manager, egress_manager, and
app.py routes. This lets a cell route its traffic through an SSH host or a
SOCKS5 proxy as an alternative to WireGuard exit nodes.
ServiceStoreManager and ServiceBus updated so the egress lifecycle (install /
uninstall) is cleanly signalled between components.
Connectivity.jsx gains the Service Egress section, letting operators assign
and reassign egress methods from the UI without touching config files.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CloudflareDDNS.update() was calling the wrong endpoint; fix to use the
correct zone-records API so DDNS updates actually land.
NoIP and FreeDNS providers now return explicit "not implemented" errors
instead of silently claiming success, preventing false-positive health state.
PicNgoDNS ACME dns-challenge now sends the token in the request body (was
missing), so cert issuance no longer silently fails.
add_peer gates builtin-service provisioning on the installed-services list
so a freshly-provisioned peer does not attempt to configure services that
aren't present, eliminating the startup error loop.
Startup Caddyfile regeneration added to routes/config.py so that a stale
on-disk Caddyfile no longer triggers the health-monitor restart loop after
a config change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Four bugs fixed:
1. Banner delay (up to 5 s): DraftConfigContext now exposes isDirty as
reactive useState so App.jsx re-renders immediately when any section
marks itself dirty, instead of waiting for the next checkPending() poll.
2. Banner re-triggers after Apply (race): For non-'*' container restarts
(e.g., cell_name → DNS restart) the background thread took ~300 ms to
clear _pending_restart. A concurrent checkPending() poll could see
needs_restart=True and overwrite the frontend's optimistic clear.
Fix: set needs_restart=False and applying=True synchronously before
spawning the thread.
3. Apply showed banner during applyPending() when hasDirty()==false:
setApplyStatus('saving') was skipped for the auto-save-then-apply
path, leaving applyStatus=null while applyPending() ran and the
banner stayed visible. Always set 'saving' before applyPending().
4. Cert status always 'unknown' in pic_ngo mode: _check_cert_via_ssl
connected to cell-caddy:443 but sent SNI='cell-caddy'. Caddy finds no
matching cert and returns nothing. Fix: pass the effective public
domain (e.g. pic1.pic.ngo) as SNI so Caddy returns the right cert.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
DraftConfig dirty state (set when any Cell Identity field changes) was
tracked in refs but never checked by the banner, which only looked at
backend pending state. Cell name changes in pic_ngo mode intentionally
block auto-save (to prevent premature DDNS re-registration), so the
backend never marked pending and the banner never appeared.
Fix: show the banner when hasDirty() is true in addition to backend
pending. Add clearAllDirty() to DraftConfigContext so Cancel immediately
clears frontend dirty state without waiting for the next 5-second poll.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The previous commit incorrectly added a standalone Save button to the
Cell Identity section. The Settings page already has a global
Accept/Discard flow (DraftConfig) where all section changes accumulate
in state and are only committed when the user presses Accept. The Save
button bypassed that pattern entirely.
Fix: remove the Save button. Cell Identity changes now follow the same
flow as every other section — edit → dirty state → Accept to commit,
Discard to revert. The pic_ngo cell-name auto-save block from the prior
commit is kept: the change accumulates until Accept, at which point the
DraftConfig flusher calls saveIdentity() and the DDNS re-registration
happens.
Update the regression tests to reflect the correct pattern: they now
verify that dirty state is set (triggering the Accept/Discard banner),
that auto-save is blocked for pic_ngo cell name changes, that auto-save
fires for ip_range changes, and that the flusher path (Accept) saves.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two changes:
1. Remove 'Internal zone name (advanced)' from Settings. The field
edited _identity.domain (the internal .cell TLD) which no user
should ever change post-install — changing it breaks all internal
service DNS. Removed the Advanced collapse section and the
showAdvancedZone state. The LAN-mode 'Local Domain' field is kept
since that mode genuinely needs a user-editable domain value.
2. Add an explicit Save button to the Cell Identity section. The
previous auto-save fix (no auto-save for pic_ngo cell name changes)
accidentally removed the only way to save those changes. The Save
button appears whenever the section is dirty and is disabled when:
- there are validation errors, or
- domainMode is pic_ngo, cell name changed, and the availability
check hasn't confirmed the name is free yet.
Adds 8 Vitest regression tests covering Save button visibility,
disabled states, that auto-save is blocked for pic_ngo cell name
changes, and that it still fires for ip_range-only changes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two bugs in the pic_ngo availability + auto-save flow:
1. Availability check fired on page load even when cell_name matched
the currently-registered name — sending unnecessary check requests
to the DDNS server and showing 'taken' for the user's own name.
Fix: skip the check when identity.cell_name === loadedCellName.
2. Auto-save triggered DDNS re-registration (release old subdomain +
register new one) as soon as picAvail became 'available' — without
the user pressing Accept. This happened because picAvail was in
the auto-save effect's dependency array, so it re-ran whenever the
availability check completed.
Fix: block auto-save entirely for pic_ngo cell name changes; the
user must press Accept explicitly since re-registration is
irreversible.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
checkDdnsStatus was declared via useCallback at line ~526 but referenced
in a useEffect dependency array at line 419 — before its declaration.
JavaScript const/let are not hoisted; accessing them before declaration
throws a ReferenceError (temporal dead zone). In the production build
this surfaced as:
ReferenceError: Cannot access 'Pn' before initialization
and caused the Settings page to crash blank on load.
Moved the checkDdnsStatus useCallback definition to immediately before
the useEffect that lists it as a dependency.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_bootstrap_dns runs at container start before the wizard, writing the
default cell name ('mycell') into cell.zone. When the wizard completed
it fired IDENTITY_CHANGED for Caddy but never updated the DNS zone, so
DNS records kept showing 'mycell.cell' even after naming the cell.
After successful wizard completion, call network_manager.apply_cell_name
to rename the hostname record in the primary zone file, then reload
CoreDNS. The empty old_name triggers auto-detection so it works even
when the zone was written with the env-var default.
Adds test_setup_route.py covering: apply_cell_name called on success,
not called on failure, 410 on repeat completion, and IDENTITY_CHANGED
publication.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix#2: Move DDNS bearer token from cell_config.json to data/api/ddns_token.
Token is now in the secrets store (data/) rather than the config store (config/).
Auto-migrates existing installs on first access. ConfigManager.get/set_ddns_token()
added. set_ddns_config() now strips 'token' key to prevent it leaking back.
- Fix#3: Set Caddyfile permissions to 0o600 after write so the token embedded
in the Caddyfile is not world-readable on the host filesystem.
- Fix#5: Heartbeat now fires IDENTITY_CHANGED after re-registration so Caddy
regenerates its config with the new token automatically — users no longer need
to click Re-register in Settings after a wizard registration failure.
Also: heartbeat skips the 401-cycle when no token exists and goes straight to
registration instead. DDNSManager now accepts service_bus= and is wired up.
- Fix#6: Settings page starts polling GET /api/caddy/cert-status every 15s
after a successful DDNS re-registration and shows "Acquiring certificate…"
feedback until Let's Encrypt issues the cert (up to 5 minutes).
- Fix#7: regenerate_with_installed() is debounced (5 s window) so two rapid
IDENTITY_CHANGED events (e.g. wizard + heartbeat) can't start simultaneous
ACME orders that interfere with each other.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Setup wizard (Issue 1 — UI):
- pic.ngo subdomain input now uses the same split-field style as DuckDNS:
input + static '.pic.ngo' suffix in a flex row, availability status below
Setup wizard (Issue 2 — Caddy not regenerating after completion):
- complete_setup route now fires IDENTITY_CHANGED after a successful wizard
submission so CaddyManager regenerates the Caddyfile immediately; users
no longer need to press 'Renew Certificate' to start ACME
Settings — DDNS status (Issue 2 — domain status missing):
- New GET /api/ddns/status endpoint: returns registered flag, domain_name,
public_ip (ipify with 30s cache), last_ip from heartbeat
- Settings DDNS section for pic_ngo now shows a live status row with
color-coded dot (green=registered+current, yellow=registered+stale,
gray=not registered), current public IP, and a Check button
- Status auto-refreshes on mount and after each successful re-registration
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
install.sh → make setup was registering 'mycell.pic.ngo' with DDNS at
install time (before the user ever opened the setup wizard). On a fresh
install the user would then open the wizard, choose 'pic1', and get a
401 OTP error because 'mycell' was already registered and the TOTP window
had moved on.
- Remove the register_with_ddns() call from setup_cell.py main(); DDNS
registration now only happens through the setup wizard
- Change default DOMAIN_MODE from pic_ngo to lan so a bare 'make setup'
no longer generates an ACME Caddyfile or pre-seeds a pic.ngo identity;
the wizard collects the real cell name and domain mode from the user
make ddns-register still works for manual / scripted deployments.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
GET http://cell-caddy:2019/ returns 404 because Caddy's admin API has no
root handler. The health monitor interpreted every response as a failure,
restarted Caddy every 3 minutes, and prevented ACME from ever completing.
/config/ returns 200 + the running config JSON whenever Caddy is up and
serving — that is the correct liveness indicator.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
A stale or empty-token Caddyfile on disk caused Caddy to reject the
/load request, so the Renew button appeared to do nothing. Now
renew_cert() calls regenerate_with_installed([]) first, which writes a
fresh Caddyfile from current identity/config before reloading Caddy.
This ensures a broken on-disk file never blocks ACME renewal.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two problems on fresh install with pic_ngo mode:
1. Caddy crashed at startup because ddns.token was empty (registration
hadn't completed yet), producing a bare `token` keyword in the
Caddyfile that Caddy rejects with "wrong argument count".
Fix: fall back to lan mode in _caddyfile_pic_ngo when the token is
empty so Caddy always starts cleanly. The Caddyfile is regenerated
once registration completes and the token is persisted.
2. DDNS registration failures were silently swallowed — the wizard
showed "Setup complete!" with no indication that HTTPS wouldn't work.
This made it look like everything was fine when the subdomain was
never registered (e.g. name already taken from a previous install,
or transient network error).
Fix: capture the exception, classify it (name_taken vs transient),
and return it as a `warnings` list in the setup response. The wizard
done screen now shows amber warning cards with actionable text instead
of auto-redirecting, giving the user a "Continue to login" button and
a clear explanation of what went wrong.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
On a fresh install before DDNS registration completes, ddns.token is
empty. Writing `token ` (bare keyword, no value) causes Caddy to reject
the Caddyfile at startup with "wrong argument count or unexpected line
ending after 'token'".
Guard added: if the token is empty, generate a LAN-mode Caddyfile so
Caddy starts cleanly. The Caddyfile is regenerated automatically once
registration completes and the token is persisted to cell_config.json.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds live cert status, one-click ACME renewal, and custom cert upload
directly to the Vault page so users never need to touch Caddy config.
Backend:
- CaddyManager.get_cert_status() now returns domain, domain_mode, and
cert_type so the UI can render the right controls without a separate
identity fetch
- CaddyManager.renew_cert() reloads Caddy and invalidates the status
cache; the frontend polls until the cert turns valid
- CaddyManager.upload_custom_cert() validates PEM, writes cert+key to
the shared config/caddy/certs/ volume, updates identity (cert_type=custom),
and regenerates the Caddyfile so Caddy references the new paths
- LAN-mode Caddyfile switches from /etc/caddy/internal/ to the shared
certs dir automatically when cert_type=custom is set
- ddns_api default no longer includes /api/v1 — the plugin appends it;
legacy /api/v1 suffix is stripped at write time to keep the Caddyfile clean
- POST /api/caddy/cert-renew and POST /api/caddy/custom-cert routes added
Frontend:
- TLSPanel component at the top of Vault.jsx shows status badge
(valid/expiring-soon/expired/pending/internal) with domain and expiry
- Renew button visible only for ACME modes; spins during the API call
then polls GET /api/caddy/cert-status every 10 s until valid
- Upload Custom Cert opens a modal with PEM text areas; works for all modes
- caddyAPI.renewCert() and uploadCustomCert() added to api.js
Tests: 22 new tests across 5 classes covering enriched status,
renew_cert guards, upload_custom_cert validation/writes/persistence,
custom-cert Caddyfile path selection, and ddns_api suffix stripping.
All 2093 existing tests continue to pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1. caddy_manager: embed ddns.token (registration bearer token) in
Caddyfile, not DDNS_TOTP_SECRET. The pic_ngo plugin sends the token
to POST /api/v1/dns-challenge; using the TOTP secret caused 401 on
every attempt.
2. firewall_manager: add _acme-challenge.<zone> forwarding block before
each split-horizon zone in the Corefile. Without this, CoreDNS was
authoritative for the challenge name and returned NODATA for TXT
queries (wildcard A record matches but wrong type), blocking Caddy's
internal DNS pre-verification step.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The method is named get_email_users in EmailManager; the route was
calling the non-existent get_users, causing an AttributeError on every
GET /api/email/users request.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
VPN peers can reach Caddy via the host's WireGuard interface (10.0.0.1),
not via the Docker bridge IP (172.20.0.2) which is unreachable outside
the container network. _bootstrap_dns now calls _get_wg_server_ip()
instead of ip_utils.get_service_ips() so the internal zone returns a
routable address for service subdomains.
Also log config save failures instead of silently swallowing them —
the silent PermissionError/OSError was masking write failures and
making it impossible to diagnose why installed services disappeared
after container restarts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
make start-core (called by install.sh step 6) used $(DCF) which includes
docker-compose.services.yml — that file declares cell-network as external:true.
On a fresh machine the network doesn't exist yet, so compose up failed with
"network cell-network declared as external, but could not be found".
Fix: add the same network-create idempotency guard that start and update
already have. Also add 26 regression tests (test_install_process.py) that
verify install.sh structure and that all start-* targets using DCF create
the network before running compose up.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- PicNgoDDNS.update(): send token in request body instead of Authorization
header; DDNS server validates it from body (was returning HTTP 422 on
every heartbeat, leaving IP record stale after fresh install)
- peers.py / Peers.jsx: webdav service_access only valid when 'files' store
service is installed; was always shown even with no services, confusing
users into thinking WebDAV was pre-installed
- 10 new regression tests: DDNS update body contract, Caddy always
regenerates on startup with no services, peer role allowed on
/api/services/active, webdav gating by installed services
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The API returns locked_until already ending in 'Z' (UTC ISO format).
Appending another 'Z' produces an invalid date string, so Date arithmetic
yielded NaN. Remove the redundant suffix.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add _PEER_READABLE_PATHS allowlist in enforce_auth so peer-role sessions
can read /api/services/active; fixes My Services showing 'not installed'
for cell members when services are installed
- Move Caddy regeneration before the early-return in reapply_on_startup so
the Caddyfile is always rebuilt from current identity on startup, even when
no store services are installed; fixes ERR_SSL_PROTOCOL_ERROR after a cell
rename (Caddyfile retained old wildcard domain)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- calendar: create_calendar_user() now writes bcrypt htpasswd entry to
data/services/calendar/config/users (the path Radicale reads at
/etc/radicale/users); delete_calendar_user() removes the entry
- email: create_email_user() calls `docker exec cell-mail setup email add`
to register the account in docker-mailserver's Dovecot/Postfix store;
delete_email_user() calls the matching `setup email del` — both are
non-fatal if the container isn't running
- service_composer.install(): pull image separately before up so slow
registry pulls don't race with container startup; retry up once on
failure so a transient registry hiccup on first install doesn't
require the user to manually retry
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- DNS (critical): add _configured_dns_params() that returns (primary_domain,
split_horizon_zones) from config_manager so all apply_all_dns_rules() callers
pass the correct primary zone (e.g. 'pic.ngo') and split-horizon list
(e.g. ['pic1.pic.ngo']) instead of the FQDN as the primary — fixes
DNS_PROBE_FINISHED_BAD_CONFIG for all external domains when on VPN
- firewall_manager: add split_horizon_zones param to apply_all_dns_rules()
and forward it to generate_corefile()
- Peers: filter service_access list to installed services only; peers.py
derives valid services from config_manager.get_installed_services() with
the email→mail ID mapping; Peers.jsx fetches from /api/store/installed
and filters the checkboxes and defaults accordingly
- Health check: fix file_manager→'files' ID mapping so files service health
is checked when installed (was silently skipped due to 'file' vs 'files')
- Verbosity persistence: move log_levels.json from non-mounted
/app/api/config/ to CONFIG_DIR (/app/config/) which maps to config/api/
on the host; both load (managers.py) and save (routes/services.py) updated
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The apply_all_dns_rules() call at the end of _bootstrap_dns() was
added to force reload 30s into the Corefile on startup. Now that
reload 30s is removed (it broke CoreDNS zone serving), the call is
unnecessary in LAN mode and actively harmful in DDNS mode:
update_split_horizon_zone() already writes the correct Corefile
with the split-horizon block; the subsequent apply_all_dns_rules()
call would overwrite it without the split-horizon zones, causing
all service subdomain lookups to return NXDOMAIN.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
CoreDNS 1.14.3 returns REFUSED for all zones that use
'file /data/zone reload 30s' — the reload timer defers the
initial zone load, causing the plugin to return REFUSED until
the timer fires. The timer never resolves this correctly.
Zone updates are already triggered by SIGUSR1 sent from
_reload_dns_service() after every zone file write, which
causes CoreDNS to reinitialise all plugins and re-read zone
files. No periodic zone polling is needed.
Also update config/dns/Corefile to remove the stale reload 30s.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three related issues prevented CoreDNS from serving updated zone records:
1. The `file` plugin blocks in generate_corefile() lacked a `reload`
option, so CoreDNS never re-read zone files after they were written.
Added `reload 30s` so zone file changes are picked up within 30s.
2. _reload_dns_service() sent SIGHUP via `docker exec ... kill -HUP 1`,
which doesn't trigger zone reloads. Changed to SIGUSR1 via
`docker kill --signal=SIGUSR1` (same as firewall_manager.reload_coredns).
3. _bootstrap_dns() wrote the zone file but never regenerated the
Corefile. CoreDNS's reload plugin only fires when the Corefile
changes, so zone records from startup were invisible until the next
peer modification triggered apply_all_dns_rules(). Now _bootstrap_dns()
always calls apply_all_dns_rules() after the zone write.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_build_dns_records() only hardcoded 'api' and 'webui', relying on the
optional service registry for the rest. Built-in services (calendar,
files, mail, webdav) were never registered, so they were absent from
the zone file and tests querying webdav.<domain> via CoreDNS got
NXDOMAIN.
Add _BUILTIN_SERVICE_SUBDOMAINS constant and include those names in
every zone build. Also update _stale and apply_cell_name exclusion
sets so DDNS mode correctly removes them from the parent zone.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
nft lives in /usr/sbin which is absent from the non-root PATH on Debian.
The delete call already used sudo; add it to the list call too so the
session-scoped cleanup fixture doesn't crash before any test runs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
wg-quick creates an nftables 'preraw' table per interface that drops
decrypted ICMP replies arriving on any other interface. If a test run
crashes before bring_down(), the table persists and silently kills pings
on subsequent runs (handshake succeeds, replies are decrypted, but the
stale table drops them before the ping process sees them).
Extend cleanup_stale_e2e_interfaces() to also delete any orphaned
wg-quick-pic-e2e-* nftables tables found on the host.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The address field in get_status() was hardcoded to SERVER_ADDRESS
('10.0.0.1/24') regardless of what wg0.conf contains, so instances
with a non-default subnet (e.g. pic1 at 10.0.1.1/24) always reported
the wrong server IP to callers such as the e2e WG conftest fixture.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
test_wg_connect_and_ping_server and the connected_peer fixture hardcoded
10.0.0.1 / 10.0.0.0/24 as the server VPN address. This breaks when the
server uses a different subnet (e.g. pic1 uses 10.0.1.1/24). Now both
read 'address' from /api/wireguard/status at session start and pass the
live server_ip / server_network through wg_server_info and connected_peer.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The test_remote_permissions_pushed_to_cell2 test verifies that permission
changes on cell1 are pushed to cell2 via the WireGuard tunnel. When both
cells use a public endpoint (DDNS VPS) instead of LAN IPs, no tunnel is
established and the push silently fails. The test now probes cell2's API
at its WG DNS IP before asserting the push succeeded — skips gracefully
if the tunnel is down rather than failing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_write_config() was stripping trailing newlines, causing the next
add_cell_peer() to create a single-newline separator between [Interface]
and [Peer] blocks instead of the required blank line. On the following
remove_peer() call, split('\n\n') treated both sections as one block,
matched the PublicKey filter, and wrote an empty string — destroying the
[Interface] section and reverting to the hardcoded SERVER_ADDRESS fallback.
Two-part fix:
1. _write_config() always ends content with a newline
2. remove_peer() normalises single-newline [Peer] headers to blank-line
separators before splitting, and refuses to write if [Interface] would
be lost
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The GET /api/cells/invite endpoint was returning domain='pic.ngo' instead
of the full FQDN 'test5.pic.ngo' because it read _identity.domain rather
than _identity.domain_name.
Apply the same domain_name preference (domain_name || domain) to:
- routes/cells.py get_cell_invite() — the invite shown to connecting cells
- routes/cells.py update_cell_permissions() — Corefile DNS regeneration
- cell_link_manager.py _check_invite_conflicts() — incoming domain collision check
- cell_link_manager.py exchange_invites() — own invite construction
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
API:
- _configured_domain() now prefers _identity.domain_name (full FQDN
e.g. 'test5.pic.ngo') over domain ('pic.ngo'). Service URLs in
/api/peer/services and /api/peer/dashboard now correctly return
'calendar.test5.pic.ngo' instead of 'calendar.pic.ngo'.
WG e2e tests:
- test_api_domain_returns_json_not_webui: accept 3xx redirect as
valid routing (Caddy redirects HTTP→HTTPS in pic_ngo mode).
- test_catchall_api_path_returns_json and test_catchall_root_serves_webui:
skip when Caddy is in HTTPS-redirect mode — catch-all :80 block only
exists in HTTP-mode cells (lan/local domain).
- test_http_api_domain_reaches_api: replace --dns-servers (requires
c-ares) with dig + curl --host pattern.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Cells with wildcard zone (e.g. * -> 172.20.0.2) and cells with per-service
VIP DNS records are both valid. Accept either in the assertion so the test
passes regardless of the zone file style.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
test_peer_services_* functions hardcoded 'http://192.168.31.51:3000' as the
fallback for PIC_API_BASE, causing failures when tests run on any other host
(including pic1 itself). Use the api_base fixture, which reads PIC_HOST and
PIC_API_PORT from the environment.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
roof
roof