Introduces api/manifest_validator.py as a single security chokepoint
imported by both ServiceComposer and ServiceStoreManager:
- validate_manifest(): rejects kind=builtin, reserved container names,
reserved subdomains, backend denylist (localhost, cell-api, etc.),
cap_add outside allowlist / in denylist, shell-string provision hooks,
and env values with shell-special characters
- validate_rendered_compose(): walks the rendered YAML and rejects
privileged:true, host network/pid/ipc/userns, absolute bind mounts,
denied capabilities, devices key, apparmor/seccomp unconfined, and
string-form command/entrypoint (shell-injection vector)
- validate_provision_hook(): requires argv list form, lowercase binary,
rejects NUL bytes
ServiceStoreManager changes:
- _validate_manifest() delegates to validate_manifest() after existing checks
- _fetch_manifest() and fetch_index() now stream with a 256 KB size cap
(prevents memory exhaustion from a malicious or compromised index)
- Digest-pin warning for images missing @sha256 (hard error for unknown
registries, warning for git.pic.ngo/roof/* and TRUSTED_IMAGES_NO_DIGEST)
ServiceComposer changes:
- write_compose() calls validate_rendered_compose() before any disk write
so no partial file is left if validation fails
- render_template() substitutes ${PIC_DATA_DIR} with the resolved data_dir path
102 new tests in tests/test_manifest_validator.py covering all five P0
security issues. Existing test mocks updated to use streaming response
pattern (stream=True + raw.read) and valid compose YAML templates.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Stray closing div was left in the ternary falsy branch after AdminConfigSection
was moved outside the ternary. esbuild interpreted it as an unterminated regex.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
These files were created during Steps 1-4 of the services architecture but were
never staged: AccountManager (per-service credential provisioning), ServiceComposer
(docker-compose lifecycle), built-in service manifests for email/calendar/files,
and their test suites (158 tests). Also un-tracks .coverage binaries that were
accidentally committed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Steps 3-6 were implemented since this doc was last written. Several
technical details had drifted from the actual code:
- Provision response shape was shown as echoing the password; corrected
to {provisioned: true} to match the security model (passwords are
never returned after creation)
- Restore command flag corrected from -C / to -C <path>; archives use
relative paths so the extraction target must be explicit
- Added ServiceRegistry validation chokepoint note: subdomain and
backend are validated at registration time, before Caddyfile
generation, not at request time
- Added Admin UI note: Accounts tab appears on service pages
- Added -- separator security note for backup command construction
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Admins previously had no UI path to provision per-peer accounts for
email, calendar, and files: they had to hit the AccountManager API
routes directly. This change wires those routes to a dedicated Accounts
tab on each service page so any peer can be granted or revoked service
access in two clicks.
- webui/src/services/api.js: add accountsAPI with list/provision/
deprovision/getCredentials, pointing to
/api/services/catalog/{serviceId}/accounts
- webui/src/components/ServiceAccountsPanel.jsx: new reusable panel;
handles credential reveal, removal confirmation, load-error state,
and humanized credential labels
- EmailPage, CalendarPage, FilesPage: Overview/Accounts tab nav (admin
only); Accounts tab renders ServiceAccountsPanel; AdminConfigSection
is hidden while on the Accounts tab
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously, CaddyManager and NetworkManager contained hardcoded lists of
service names (calendar, files, mail, webdav, etc.), meaning every new
service required a code change to appear in Caddy routes and DNS records.
Now both managers accept a service_registry parameter and derive their
service lists dynamically from the registry at runtime.
- CaddyManager: new _build_registry_service_routes() and
_http01_service_pairs() methods pull routes from the registry
- NetworkManager: new _get_service_subdomains() method returns registry
subdomains with a hardcoded fallback when no registry is wired in;
_build_dns_records, stale-record detection, and service name sets all
use the registry
- managers.py: service_registry constructed before network_manager so it
can be injected into both CaddyManager and NetworkManager
- service_registry.py: validation chokepoint in get_caddy_routes() rejects
invalid subdomain/backend values and reserved service names
- service_store_manager.py: _validate_manifest now validates top-level
subdomain, backend, extra_subdomains, and extra_backends fields
- tests: 24 new tests covering registry-driven routing and DNS subdomain
generation (test_caddy_registry_integration.py)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Rename Store → Services: ServicesIndex.jsx shows built-in core services
(Email, Calendar, Files) with Manage links, plus the existing add-on
store below.
New service sub-pages at /services/email|calendar|files serve both
admin and peer roles. Admins see connection info, service status, users
list, and an inline config form (port/data-dir). Peers see connection
info and their personal credentials fetched from peerAPI.
Navigation restructured: a Services parent item expands to show the
three sub-pages via a collapsible sidebar group (ChevronDown toggle).
Both admin and peer navigation include the Services group. Sidebar
extracted NavItem/NavList components to eliminate the duplicate mobile/
desktop rendering.
Settings.jsx drops EmailForm, CalendarForm, FilesForm and their
SERVICE_DEFS entries. Port conflict detection and per-service validation
logic extracted to utils/serviceConfig.js, shared by Settings and the
new service pages. Service form flushers are registered without cleanup
so the Apply banner saves dirty config even when the user navigates away
from a service page before clicking Apply.
Legacy routes /email, /calendar, /files, /store redirect to their new
canonical paths.
GET /api/config now includes installed_services so the nav can derive
which add-ons are installed without a separate store fetch.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
apply_cell_name() now skips multi-label zone files (split-horizon DDNS
zones like pic2.pic.ngo.zone) and excludes '*' and '@' from hostname
candidate detection, preventing the wildcard record from being renamed
to the old cell name during a cell rename.
update_split_horizon_zone() now deletes stale zone files from previous
cell names sharing the same TLD (e.g. pic3.pic.ngo.zone when renaming
to pic2.pic.ngo), eliminating orphaned DNS entries.
_bootstrap_dns() now detects non-LAN domain modes and calls
update_split_horizon_zone() instead of apply_ip_range(), preventing
service records (api, calendar, files…) from being re-injected into
the DDNS parent zone on every container restart.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Dashboard, Email, Calendar, and Files pages were building service URLs
with the internal LAN zone name (e.g. 'cell') instead of the public
effective domain (e.g. 'pic2.pic.ngo'), and always using http:// even
in DDNS mode where HTTPS is available.
Changes:
- Dashboard/Email/Calendar/Files: read effective_domain + domain_mode
from ConfigContext; use effective_domain in non-LAN mode and https://
for all DDNS domain modes.
- Calendar: show port 443 instead of 80 in DDNS mode.
- network_manager.update_split_horizon_zone: when the primary internal
zone name is a parent of the effective DDNS domain (e.g. pic.ngo is a
parent of pic2.pic.ngo), remove stale bootstrap service records (api,
calendar, files, mail, webmail, webdav) that pollute the DNS display
and would shadow public DNS responses.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
In DDNS modes (pic_ngo, cloudflare, duckdns, http01), all built-in
services are now reachable as subdomains of the cell domain, e.g.
calendar.pic1.pic.ngo instead of pic1.pic.ngo/calendar.
Key changes:
- CaddyManager._build_core_service_routes(): new helper generates
Caddy named-matcher host blocks for calendar, mail/webmail, files,
webdav, and api subdomains within the wildcard TLS server block.
- All ACME modes (pic_ngo, cloudflare, duckdns) use the new
subdomain matchers; http01 emits a dedicated server block per service.
- http01: installed store-plugin services whose name clashes with a
core service are skipped to prevent duplicate server blocks.
- routes/config.py: ip_utils.write_caddyfile() is skipped in non-LAN
modes so LAN Caddy config never overwrites the ACME config.
- firewall_manager.generate_corefile(): new split_horizon_zones param
adds local authoritative file zones so LAN clients resolve
*.pic1.pic.ngo to the internal Caddy IP without hairpin NAT.
- NetworkManager.update_split_horizon_zone(): writes the wildcard zone
file and regenerates the Corefile with the split-horizon block;
called automatically after every identity change in non-LAN mode.
- Added @ to allowed record-name chars in update_dns_zone validation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ConfigManager.get_effective_domain(): returns domain_name when DDNS
active (pic_ngo/cloudflare/duckdns), domain otherwise. Used by all
public-facing services so they use the real registered FQDN.
- ConfigManager.get_internal_domain(): always returns _identity.domain
(CoreDNS zone name, dnsmasq, cell-link invites — stays internal).
- Silent migration: if domain_mode != lan and domain is generic "cell",
auto-set to {cell_name}.local for unique CoreDNS zone naming.
- caddy_manager: fix custom_domain bug — cloudflare/http01 modes were
reading identity.get('custom_domain') which never exists; now reads
domain_name correctly.
- routes/config, app: expose effective_domain in GET /api/config and
/api/status responses.
- email_manager, routes/email: use get_effective_domain() for
OVERRIDE_HOSTNAME, POSTMASTER_ADDRESS, and new-user email defaults.
- ServiceBus.IDENTITY_CHANGED event: emitted from PUT /api/config and
POST /api/ddns/register after identity writes; caddy_manager and
email_manager subscribe to regenerate config automatically.
- Settings.jsx: hide Local Domain input in non-LAN modes; show
read-only effective_domain with "managed by DDNS" badge and an
Advanced toggle for the internal CoreDNS zone name.
- 11 new test classes covering all new helpers, event subscriptions,
caddy/email handlers, and the custom_domain fix.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
'unreachable' should not be a terminal state that triggers auto-save —
it was causing a 503 when the availability check failed and auto-save
fired the backend registration attempt. Only 'available' allows
auto-save when the cell name has changed from the loaded value.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add a short label explaining the button is for DDNS recovery (when the
DDNS server lost your record), not routine IP updates.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Auto-save was firing with picAvail === null (the moment the user typed a
new cell name, before the 900ms availability debounce even started), which
caused the backend to immediately register the subdomain on DDNS.
Track the last saved/loaded cell name in loadedCellName. When domainMode
is pic_ngo and the typed name differs from the loaded name, block
auto-save until picAvail reaches a terminal state (available or
unreachable). Also update loadedCellName on successful save so subsequent
edits to the same name are not blocked.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds DELETE /api/v1/registration to the DDNS server (token-authenticated,
owner-only) and PicNgoDDNS.release() on the client. DDNSManager.register()
now automatically releases the old subdomain before claiming the new one,
so stale names are freed for others to use. Release failures are logged as
warnings and do not block the new registration.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two bugs that prevented registration from working after wizard completion:
1. register(name, '') sent empty IP; server stored blank A record. Now calls
_get_public_ip() when ip is empty so the A record is always set correctly.
2. Token was saved to _identity.domain.ddns.token (TypeError when domain is a
string) instead of the top-level ddns config where update_ip() reads it.
Subdomain also now correctly written to _identity.domain_name.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- DDNSTokenExpired exception triggers auto re-register in update_ip()
so cells recover silently after a DDNS DB reset
- POST /api/ddns/register lets the user force re-registration from Settings
- Re-register button in Settings → External Domain & DDNS (pic_ngo only)
- 3 new tests covering register endpoint: wrong provider, missing name, success
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ddns_manager: DDNS_URL env var overrides stored api_base_url so
existing cells pick up the new HTTPS endpoint without re-registering
- docker-compose.yml: default DDNS_URL now points to https://ddns.pic.ngo
- setup_manager.py: add rstrip('/') before replacing /api/v1 to handle
URLs with or without trailing slash
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- GET /api/config now returns domain_mode, domain_name, ddns.{provider,subdomain,has_token}
- GET /api/ddns/check/<name> proxies availability check to DDNS service
- PUT /api/ddns validates and saves cloudflare/duckdns credentials post-setup
- When cell_name changes for pic_ngo provider, auto-registers the new subdomain
- Settings: Cell Name shows availability badge for pic_ngo; auto-save blocks on taken
- Settings: new External Domain & DDNS section — pic_ngo info, cloudflare/duckdns edit
- 11 new tests for the two new endpoints (all pass)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
docker-compose.yml DDNS_TOTP_SECRET defaulted to empty string —
containers on fresh installs had no OTP, so every /register call
was rejected with 401 and no domain was ever registered.
setup_cell.py still pointed to https://ddns.pic.ngo/api/v1 (no nginx
on VPS, so HTTPS fails). Both now default to the correct values; both
are still overridable via env var for custom DDNS deployments.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_check_pic_ngo_available was hardcoding https://ddns.pic.ngo, ignoring
DDNS_URL. Now imports DDNS_API_BASE from setup_manager so both the
availability check and DDNS registration use the same configured URL.
API container now receives DDNS_URL and DDNS_TOTP_SECRET from env.
Default DDNS_URL points to http://ddns.pic.ngo:8080/api/v1 (the
FastAPI service runs on port 8080 without TLS termination in front).
Also returns 503 (not 500) when the DDNS service is unreachable.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
For pic_ngo: name must be confirmed available (not just format-valid).
For cloudflare/duckdns: token is auto-verified on Next if not already
done — invalid or unreachable service blocks proceeding. Only lan and
http01 (no external dependency) allow Next without a live check.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The availability check was only triggered onBlur, so clicking Next
without blurring the field skipped the DDNS request entirely. Now
handleNext awaits the check and blocks with an error if the name is
taken. Unknown/unreachable DDNS is treated as available to avoid
blocking the wizard.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Domain name is now the cell identity (no separate cell name step).
All 5 providers (pic_ngo, cloudflare, duckdns, http01, lan) are
first-class options in a single Domain step. pic.ngo availability
is checked live via backend proxy to ddns.pic.ngo. Cloudflare and
DuckDNS tokens are verified via backend before proceeding.
cell_name is derived automatically from the chosen domain.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two bugs:
1. AttributeError: AuthManager.update_password does not exist — the
fallback when create_user fails should call set_password_admin().
This caused a 500 on every setup submit when an admin user already
existed (e.g. from a previous install attempt).
2. Wizard was jumping to step 2 and skipping domain steps 3-4 when
preconfigured data existed in cell_config.json. Since the installer
no longer sets that data, and the wizard must always show all steps,
the installerConfigured state and all step-skipping navigation is
removed. Values are still pre-filled if found in config.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
install.sh no longer prompts for anything. It installs packages (with sudo),
creates the system user, clones the repo, and runs 'make install' — all as
the invoking user. Only package installs and system-level ops use sudo.
All folder creation happens under the user's own account, no chown needed.
/setup wizard gains the missing validation that was previously in install.sh:
- Step 1: checks pic.ngo name availability via backend (non-blocking)
- Step 4: 'Verify token' button for Cloudflare and DuckDNS tokens,
validated server-side through new /api/setup/validate steps
API changes (routes/setup.py):
- validate step 'pic_ngo_available': proxy check to ddns.pic.ngo
- validate step 'cloudflare_token': verify via Cloudflare tokens API
- validate step 'duckdns_token': verify via DuckDNS update endpoint
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Reverts 8d1ef39. The installer must collect cell name, domain mode, and
provider tokens before 'make install' so that DDNS registration,
availability checks, and Caddy TLS can be configured at first boot.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When the bash installer collects cell name and domain mode, the first-run
wizard's /setup should only ask for a password, service selection, and
timezone. Previously the wizard pre-filled those fields but still showed
all 7 steps.
- useEffect fetches /api/setup/status on mount; if preconfigured.cell_name
and preconfigured.domain_mode are both set, sets installerConfigured=true
and jumps to step 2 (password)
- handleStep2Next → step 5 when installerConfigured (skips domain steps 3+4)
- handleStep2Back → step 1 when installerConfigured (review cell name)
- handleStep5Back returns to step 2 when installerConfigured
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
DOMAIN_MODE, CELL_DOMAIN_NAME, CLOUDFLARE_API_TOKEN, DUCKDNS_TOKEN,
DUCKDNS_SUBDOMAIN are now explicit in the setup target so they are
visible and documented, not silently inherited from the environment.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
DDNS registration (setup_cell.py):
- Replace pyotp dependency with stdlib TOTP (HMAC-SHA1, RFC 6238)
pyotp is only available inside the Docker container, not on the host
where setup_cell.py runs — registration was silently skipped every time
- OTP header still sent if generation succeeds; omitted gracefully if not
Wizard pre-fill (setup_manager + Setup.jsx):
- GET /api/setup/status now returns 'preconfigured' dict with cell_name,
domain_mode, domain_name, and provider tokens from installer-written config
- Setup.jsx fetches status on mount and pre-fills all form state so the
user only needs to set password, services, and timezone — not re-enter
the identity they already configured in the bash installer
- Fails silently so wizard still works on fresh installs with no config
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
install.sh now guides the user through the full identity setup before
running make install:
- Cell name prompt with format validation and pic.ngo availability check
- Domain mode selection: pic.ngo / Cloudflare / DuckDNS / HTTP-01 / LAN
- Cloudflare API token: collected and verified against CF tokens/verify API
- DuckDNS: subdomain + token verified against duckdns.org/update
- HTTP-01: domain name collected, port-80 warning shown
- All collected values passed as env vars to make install
- After two failed token attempts user can continue (re-verified at boot)
- Final banner shows configured cell name and domain
setup_cell.py: updated to handle all domain modes
- Reads DOMAIN_MODE / CELL_DOMAIN_NAME / CLOUDFLARE_API_TOKEN /
DUCKDNS_TOKEN / DUCKDNS_SUBDOMAIN from env
- write_cell_config() now writes domain_mode + domain_name to _identity
and builds the ddns section for each provider (not hardcoded to pic_ngo)
- register_with_ddns() only called when DOMAIN_MODE == 'pic_ngo'
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Password:
- Add lowercase to strength scoring; "Good" now requires all API criteria
(12 chars, upper, lower, digit) — no more submitting passwords the API rejects
- isReady gates the Next button on meeting API requirements, not just length
Domain steps 3 + 4:
- Step 3: choose pic_ngo / custom / lan (sends valid API domain_modes)
- Step 4 (pic.ngo): shows derived [cellName].pic.ngo domain preview
- Step 4 (custom): domain name field + TLS method selector
(Cloudflare DNS-01 + API token, DuckDNS + token, HTTP-01 + port-80 warning)
- Step 4 skipped entirely for LAN-only
- Review step shows actual domain string and TLS method instead of opaque codes
Cell name:
- Description and preview hint make clear it becomes the pic.ngo subdomain
- Step 1 shows live "name.pic.ngo" preview as you type
Backend:
- setup_manager now accepts and stores domain_name, cloudflare_api_token,
duckdns_token for Phase 3 DDNS registration use
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
make install runs as root so all generated files (config/, data/) land
as root:root. Added a chown pass in install.sh after make install
completes, re-applying REPO_OWNER ownership. Also fixed the make setup
chown to use SUDO_USER when invoked via sudo rather than always id -u
(which is 0 when running as root).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- setup_manager: fall back to update_password if admin already exists
(installer bootstrap creates admin; wizard now updates rather than fails)
- install.sh: chown repo to SUDO_USER instead of pic user so the
invoking operator can run make update without git safe.directory errors
- test: update mock to also stub update_password when testing total auth failure
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The installer runs as root and chowns /opt/pic to the pic user.
Any other user (roof, operator) running make update then hits
"detected dubious ownership". Fix: add /opt/pic to system-wide
git safe.directory after clone, and add same guard in make update.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The wizard was sending domain_type and services but the API expected
domain_mode and services_enabled, causing a validation error on submit.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The setup wizard runs before any account exists, but the installer's
setup_cell.py creates auth_users.json with an admin account first.
This meant enforce_auth was active by the time the browser hit /setup,
blocking all /api/setup/* calls with 401. The CSRF hook already exempted
/api/setup/* — auth enforcement now matches.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tests assumed write to /nonexistent/... fails, but CI runs as root where
Linux allows creating any path. Use unittest.mock.patch on builtins.open
with OSError side_effect so the test is environment-independent.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Option A is now the one-line curl installer (install.pic.ngo); Option B
is the manual git clone path. Wizard section covers all five domain modes
(pic_ngo, cloudflare, duckdns, http01, lan) and current password rules.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
README, QUICKSTART, and Wiki were pre-wizard, pre-auth, pre-DDNS, and
pre-service-store. Full rewrite covering:
- First-run wizard replaces manual make setup + .env identity config
- Session-based auth (admin/peer roles, CSRF protection)
- DDNS: pic.ngo registration with TOTP, provider abstraction
- Service store: install/remove optional services from manifest index
- Cell-to-cell networking and peer-sync protocol
- Extended connectivity: WG external, OpenVPN, Tor exit routing
- Caddy HTTPS: Let's Encrypt (DNS-01/HTTP-01) or internal CA
- Current container list, port bindings, and security model
- Accurate make targets (ddns-update, reset-admin-password, etc.)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
roof