Commit Graph

99 Commits

Author SHA1 Message Date
roof 8650704316 feat: add authentication and authorization system
Backend:
- AuthManager (api/auth_manager.py): server-side user store with bcrypt
  password hashing, account lockout after 5 failed attempts (15 min),
  and atomic file writes
- AuthRoutes (api/auth_routes.py): Blueprint at /api/auth/* — login,
  logout, me, change-password, admin reset-password, list-users
- app.py: register auth_bp blueprint; add enforce_auth before_request
  hook (401 for unauthenticated, 403 for wrong role; only active when
  auth store has users so pre-auth tests remain green); instantiate
  AuthManager; update POST /api/peers to require password >= 10 chars
  and auto-provision email + calendar + files + auth accounts with full
  rollback on any failure; extend DELETE /api/peers to tear down all
  four service accounts; add /api/peer/dashboard and /api/peer/services
  peer-scoped routes; fix is_local_request to also trust the last
  X-Forwarded-For entry appended by the reverse proxy (Caddy)
- Role-based access: admin for /api/* (except /api/auth/* which is
  public and /api/peer/* which is peer-only)
- setup_cell.py: generate and print initial admin password, store in
  .admin_initial_password with 0600 permissions; cleaned up on first
  admin login

Frontend:
- AuthContext.jsx: React context with login/logout/me state and Axios
  interceptor for automatic 401 redirect
- PrivateRoute.jsx: route guard component
- Login.jsx: login page with error handling and must-change-password
  redirect
- AccountSettings.jsx: change-password form for any authenticated user
- PeerDashboard.jsx: peer-role landing page (IP, service list)
- MyServices.jsx: peer service links page
- App.jsx, Sidebar.jsx: AuthContext integration, logout button,
  PrivateRoute wrappers, peer-role routing
- Peers.jsx, WireGuard.jsx, api.js: auth-aware API calls

Tests: 100 new auth tests all pass (test_auth_manager, test_auth_routes,
test_route_protection, test_peer_provisioning). Fix pre-existing test
failures: update WireGuard test keys to valid 44-char base64 format
(test_wireguard_manager, test_peer_wg_integration), add password field
and service manager mocks to test_api_endpoints peer tests, add auth
helpers to conftest.py. Full suite: 845 passed, 0 failures.

Fixed: .admin_initial_password security cleanup on bootstrap, username
minimum length (3 chars enforced by USERNAME_RE regex)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-25 15:00:06 -04:00
roof a338836bb8 add security fixes, port hardening, and expanded QA coverage
Security fixes:
- Replace debug=True with env-driven FLASK_DEBUG in app.py
- Add _safe_path helper and path-traversal protection to all 6 file routes
  in file_manager.py
- Add peer_name regex and input validation (public_key, name, endpoint_ip)
  in wireguard_manager.py
- Stop returning private key from GET /api/wireguard/keys; return only
  public_key + has_private_key boolean
- Fix is_local_request() XFF bypass by checking remote_addr only, ignoring
  X-Forwarded-For
- Remove duplicate get_all_configs / get_config_summary methods from
  config_manager.py

DevOps:
- Bind 6 internal service ports to 127.0.0.1 in docker-compose.yml
  (radicale, webdav, api, webui, rainloop, filegator)
- Move WebDAV credentials to env vars (WEBDAV_USER, WEBDAV_PASS)
- Pin flask, flask-cors, requests, cryptography, docker to secure minimum
  versions in requirements.txt

QA (560 tests, 0 failures):
- tests/test_wireguard_endpoints.py: 18 new endpoint tests
- tests/test_file_endpoints.py: 24 new endpoint tests incl. path traversal
- tests/test_container_manager.py: expanded from 2 to 30 tests
- tests/test_config_backup_restore_http.py: 25 new tests (new file)
- tests/test_config_apply.py: 9 new tests (new file)

Docs:
- Rewrite README.md with accurate architecture, ports, env vars, security notes
- Rewrite QUICKSTART.md with verified commands

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-25 13:08:24 -04:00
roof eb817ffdc5 fix: WireGuard sysctl || true, port check on page load, add peer status tests
Root cause: sysctl -q net.ipv4.conf.all.rp_filter=0 in PostUp exited non-zero
inside the linuxserver/wireguard container (no permission), causing wg-quick to
tear down the wg0 interface — breaking peer status, port check, and internet
access through full tunnel.

- wireguard_manager.py: add || true to both sysctl PostUp/PostDown lines
- docker-compose.yml: add net.ipv4.conf.all.rp_filter=0 to wireguard sysctls
- WireGuard.jsx: kick off port check asynchronously on page load (was refresh-only)
- tests: add TestWireGuardSysctlAndPortCheck — 14 new tests covering sysctl
  content, check_port_open (interface up / down / fallback-to-handshake),
  get_peer_status (online / offline / not-found / no-handshake), and
  get_all_peer_statuses (multi-peer / empty / skips interface line)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 10:31:57 -04:00
roof 4b994a5964 feat: domain validation for NTP servers and mail domain fields
- isValidDomain / isValidDomainOrIp helpers (RFC-compliant label regex)
- network.ntp_servers: each entry validated as hostname or IP; invalid entry shown in error message
- email.domain: validated as a proper domain name; blocks autosave until fixed

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 09:39:59 -04:00
roof 15e009bd94 feat: fix export/import, add backup download/upload, restore service checkboxes
- export_config: clean output (no internal _keys), identity exposed as 'identity'
- import_config: handle 'identity' key, merge into existing config (not replace)
- restore_config: accept optional services list for selective restore
- backup_config: include 'identity' in manifest services list
- new GET /api/config/backups/<id>/download → zip file download
- new POST /api/config/backup/upload → zip file upload
- webui: Download + Upload buttons, restore modal with per-service checkboxes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 08:51:40 -04:00
roof 2bd6545f0e fix: silent autosave, pending dedup, domain/cell_name pending, containers access
- Settings: remove Save buttons; autosave is silent (no toast on success, error only)
- Settings: loadAll() resets dirty flags to prevent stale autosave after discard
- app.py: fix domain/ip_range "actually changed" check — full identity is always
  sent on save so these were triggering pending on every keystroke regardless
- app.py: _dedup_changes handles port-change format "service field: old → new"
  (split on ':' not ' changed') so dns_port changed twice shows one entry
- app.py: domain + cell_name changes now go through pending restart banner;
  apply_domain/apply_cell_name write files immediately (reload=False) and set
  pending; Discard restores zone files + Caddyfile to pre-change state
- app.py: _set_pending_restart captures pre-change snapshot BEFORE config writes
  (was snapshotting after, making Discard a no-op)
- app.py: is_local_request reads /proc/net/route to allow the actual Docker
  bridge subnet (172.0.0.0/24) which is not RFC-1918; fixes Containers page 403
- container_manager: get_container_logs raises instead of swallowing exceptions
  so nonexistent container returns 500+error not 200+empty

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 07:16:13 -04:00
roof 4215e03ac6 fix: autosave, cell name overflow, length validation, apply-and-verify tests
Autosave on Apply (was broken):
- App.jsx called useDraftConfig() in the same component that rendered
  DraftConfigProvider — a component cannot consume context it provides.
  Fixed by splitting into AppCore (consumes context, all logic) and App
  (thin shell that wraps AppCore in DraftConfigProvider).  The hook now
  runs inside the provider and hasDirty()/flushAll() work correctly.

Cell name / domain length validation (255-char DNS standard):
- api/app.py: reject cell_name or domain > 255 chars or empty with 400
- api/app.py: reject ip_range without CIDR prefix (bare IPs shift all VIPs)
- webui/src/pages/Settings.jsx: cellNameError + domainError computed values
  block saveIdentity and show inline error; maxLength={255} on inputs
- tests/test_identity_validation.py: 8 unit tests for the new validation

Cell name overflow on all pages:
- Dashboard.jsx: add min-w-0 to flex child div + truncate + title on cell_name
- CellNetwork.jsx: min-w-0 + truncate + title on cell_name, domain, endpoint,
  vpn_subnet in invite cards and connected-cells list

Apply-and-verify integration tests:
- tests/integration/test_apply_propagation.py: TestPendingState (no restarts)
  and TestApplyAndVerify (triggers real container restart + health poll)
  covering the full save → apply → wait → verify propagation lifecycle

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 05:29:09 -04:00
roof 3ce45a8911 fix: get_live_service_vips uses config API, require CIDR prefix for ip_range
- tests/integration/conftest.py: get_live_service_vips() now reads from the
  config API's service_ips field instead of docker exec.  The docker exec approach
  spawns a fresh Python process that imports firewall_manager with its hardcoded
  initial SERVICE_IPS, ignoring any update_service_ips() calls made at runtime.
  The config API always computes VIPs from the current ip_range, so it matches what
  the running app actually uses when writing iptables rules.

- api/app.py: reject ip_range values without a CIDR prefix (e.g. '10.0.0.1')
  with a 400.  Bare IPs are parsed as /32 by ipaddress.ip_network(strict=False),
  which shifts all VIP offsets and produces unusable Docker subnet configs.

- tests/integration/test_config_api.py: update bare-ip test to expect 400 now
  that the API enforces the prefix requirement.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 04:54:47 -04:00
roof 768571f2b7 feat: port conflict validation, autosave on Apply, extended integration tests
Port conflict validation:
- api/port_registry.py: detect_conflicts() checks all service sections for shared port values
- api/app.py: returns HTTP 409 on port conflict after existing range validation
- webui/src/pages/Settings.jsx: JS-side detectPortConflicts() with useMemo shows inline
  conflict errors and blocks Save before the request is made; catch blocks surface server
  error messages (including 409) instead of generic fallbacks

Config autosave on Apply:
- webui/src/contexts/DraftConfigContext.jsx: new context; Settings registers flush callbacks
  per section; App calls flushAll() before applyPending() when any section is dirty
- webui/src/App.jsx: wraps tree with DraftConfigProvider, handleApply shows 'saving' banner
  state and awaits flushAll()
- webui/src/pages/Settings.jsx: registers identity + per-service flushers; propagates dirty
  state into context via setDirty; uses refs to avoid stale closures

Extended integration test coverage (114 new tests):
- tests/integration/test_config_api.py: GET/PUT config, export, import, backup lifecycle
- tests/integration/test_network_services.py: DNS records + DHCP reservations CRUD
- tests/integration/test_containers.py: list, restart, logs, stats; recovery polling
- tests/integration/test_negative_scenarios.py: error-path coverage for all endpoints
- tests/test_port_conflicts.py: 20 unit tests for port_registry.detect_conflicts()

Pre-commit hook updated to skip tests/integration/ (live-stack tests require a running
stack and must be run explicitly via `make test-integration`).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 04:45:47 -04:00
roof 596b06f171 fix: apply_domain now removes all stale zone files when renaming
Previously the loop broke after processing the first zone file it found.
If dev.zone already existed (created by apply_ip_range), it would be
processed and the loop would stop — leaving any other zone files (e.g.
cell.zone from an earlier domain) in place. get_dns_records() reads all
.zone files so the stale zone appeared doubled in the UI.

Fix: collect all non-local zone files first, write the target, then
delete every file that is not the current domain's zone.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 04:08:37 -04:00
roof dd5244279d Merge feature/fix-local-request-and-helper-image into main
Includes:
- Security audit fixes (XFF spoofing, is_local_request guards, service_access validation)
- Bug fixes (CoreDNS SIGHUP→SIGUSR1, dhcp/reservations args, atomic writes)
- State consistency (atomic config writes, backup/restore coverage)
- 66 new integration tests (live API + peer lifecycle + iptables + WebUI)
- 42 new unit tests (total 480)
- Pre-commit hook with pytest
2026-04-24 04:03:07 -04:00
roof d2ae18f047 feat: add integration test suite (66 tests covering live API + services + UI)
Tests cover health, config, all 12 containers, WireGuard, DNS/DHCP/NTP,
services status, peer CRUD with iptables rule verification, service_access
enforcement (full/restricted/no-access), and WebUI smoke tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 03:59:19 -04:00
roof d5018c2b34 fix: architecture audit — security, atomicity, broken endpoints, test coverage
Sprint 1 — Security & correctness:
- Restore all 10 commented-out is_local_request() checks (vault, containers, images, volumes)
- Fix XFF spoofing: only trust the LAST X-Forwarded-For entry (Caddy's append), not all
- Require prefix length in wireguard.address (was accepting bare IPs like 10.0.0.1)
- Validate service_access list in add_peer (valid: calendar/files/mail/webdav)
- Fix dhcp/reservations POST/DELETE: unpack mac/ip/hostname from body (was passing dict as positional arg)
- Fix network/test POST: remove spurious data arg (test_connectivity takes no args)
- Fix remove_peer: clear iptables rules and regenerate DNS ACLs on deletion (was leaving stale rules)
- Fix CoreDNS reload: SIGHUP → SIGUSR1 (SIGHUP kills the process; SIGUSR1 triggers reload plugin)
- Remove local.{domain} block from Corefile template (local.zone doesn't exist, caused log spam)
- Fix routing_manager._remove_nat_rule: targeted -D instead of flushing entire POSTROUTING chain

Sprint 2 — State consistency:
- Atomic config writes in config_manager, ip_utils, firewall_manager, network_manager
  (write to .tmp → fsync → os.replace, prevents truncated files on kill)
- backup_config: now also backs up Caddyfile, Corefile, .env, DNS zone files
- restore_config: restores all of the above so config stays consistent after restore

Sprint 3 — Dead code / documentation:
- Remove CellManager instantiation from app startup (was never called, double-instantiated all managers)
- Document routing_manager scope (targets host, not cell-wireguard; methods not called by any active route)

Sprint 4 — Test infrastructure:
- Add tests/conftest.py with shared tmp_dir, tmp_config_dir, tmp_data_dir, flask_client fixtures
- Add tests/test_config_validation.py: 400 paths for ip_range, port, wireguard.address validation
- Add tests/test_ip_utils_caddyfile.py: 14 tests for write_caddyfile (was completely untested)
- Expand test_app_misc.py: 7 new is_local_request tests covering XFF spoofing and cell-network IPs
- Add --cov-fail-under=70 to make test-coverage
- Add pre-commit hook that runs pytest before every commit

414 tests pass (was 372).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 03:27:52 -04:00
roof 55bec04603 Add port and IP validation across all service config forms
UI: validateServiceConfig() checks all port fields (1–65535) and
WireGuard address (IP/CIDR) on every keystroke; Save button is
disabled and saveService() guards against any field errors.
API: update_config() rejects out-of-range port values and invalid
WireGuard address before persisting, returning 400 with a clear
field path (e.g. email.smtp_port).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 00:48:20 -04:00
roof 323729e1ab feat: validate ip_range must be within RFC-1918 on save
API: rejects ip_range outside 10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16
with a 400 error before saving to config.

UI: isRFC1918Cidr() validates on every keystroke; error message shown inline
below the field; Save Identity button disabled while the value is invalid.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 00:33:30 -04:00
roof 60cf223293 fix: is_local_request rejects non-RFC1918 cell subnets; helper image hardcoded
Two bugs triggered when ip_range is set to a subnet outside 172.16.0.0/12
(e.g. 172.0.0.0/24):

1. is_local_request() used ip.is_private which returns False for 172.0.x.x,
   causing Caddy reverse-proxy requests to get 403 on the containers endpoint.
   Fix: also accept IPs in the configured cell-network subnet.

2. apply_pending_config() hardcoded 'pic_api:latest' as the helper container
   image. docker-compose v1 builds pic_api:latest (underscore) but compose v2+
   builds pic-api:latest (hyphen). On a v2 install the helper would fail to
   start silently, leaving the network unreconstructed after an ip_range change.
   Fix: read the actual image tag from cell-api's own container metadata.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 16:15:58 -04:00
Administrator 2c11db6cc1 Merge branch 'feature/fix-apply-network-recreate' into 'main'
fix: make uninstall keeps images and data when user answers no

See merge request root/pic!8
2026-04-22 19:45:13 +00:00
roof 6cc40f5755 fix: make uninstall keeps images and data when user answers no
Previously 'n' still ran --rmi all, removing Docker images. Now:
- n (default): docker-compose down only — containers gone, images and
  data/config untouched; 'make start' brings everything back immediately
- y: full wipe — containers, images, config/ and data/ all removed

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 15:43:14 -04:00
Administrator dc0525429f Merge branch 'feature/fix-apply-network-recreate' into 'main'
fix: spawn helper container for all-services restart so API survives

See merge request root/pic!7
2026-04-22 19:41:51 +00:00
roof 50671f71cb fix: use configured domain in CoreDNS Corefile generation
Two bugs caused DNS to fail when the domain name changes:
1. generate_corefile() hardcoded 'cell' as the zone name instead of
   using the configured domain — on startup it would silently reset any
   domain change back to 'cell'
2. apply_domain() regex replaced ALL non-dot zones (including local.cell)
   with the new domain → duplicate zone blocks → CoreDNS crash

Fix: add a domain parameter to generate_corefile/apply_all_dns_rules,
add _configured_domain() helper in app.py, and delegate Corefile updates
in apply_domain() to generate_corefile() so the logic is in one place.
Also parameterise SERVICE_HOSTS ACL entries via the domain argument.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 15:32:23 -04:00
roof e74d5e0504 fix: generate Caddyfile in setup and on identity changes
`make reinstall` wipes config/ then `make setup` creates an empty
Caddyfile (ensure_file just touches it). Add write_caddyfile() to
ip_utils.py that generates the full reverse-proxy config from ip_range,
cell_name, and domain. Call it from setup_cell.py so fresh installs
always get a valid Caddyfile. Also regenerate it in app.py whenever
ip_range, domain, or cell_name changes so Caddy stays in sync.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 15:18:37 -04:00
roof c9ed28f258 fix: spawn helper container for all-services restart so API survives
When containers=['*'] (ip_range change or full restart), the previous
code ran docker compose down/up in a background thread inside cell-api.
docker compose down killed cell-api, terminating the thread before
docker compose up could run — leaving all containers stopped.

Fix: spawn an independent docker run --rm container (pic_api:latest)
that has the docker socket and project dir mounted. This helper outlives
cell-api being stopped and completes the up -d independently.

For specific-container restarts (port changes), keep the direct approach
since the API container is not in the affected set.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 15:02:26 -04:00
Administrator cf3db09ea7 Delete ENHANCED_API_IMPROVEMENTS.md 2026-04-22 18:46:00 +00:00
Administrator 0c48fc25e7 Delete COVERAGE_REPORT.md 2026-04-22 18:45:46 +00:00
Administrator 55e8caecfb Delete COMPREHENSIVE_IMPROVEMENTS_SUMMARY.md 2026-04-22 18:45:30 +00:00
roof 5f4f635719 fix: make update runs setup automatically if config is missing
After make uninstall (which wipes config/ and data/), running make update
directly failed with "Couldn't find env file: config/mail/mailserver.env"
because docker-compose needs the generated config files to exist.

make update now checks for the sentinel config file and calls make setup
first if it's missing, so uninstall → update works as a valid flow.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 14:34:42 -04:00
roof 11c80124af fix: subprocess not imported in _do_apply background thread
The apply_pending_config endpoint spawns _do_apply in a background thread.
subprocess was used but not imported inside the closure, causing
NameError: name 'subprocess' is not defined on every Apply click —
silently swallowed, so containers never restarted and no error was shown.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 14:28:13 -04:00
Administrator 7e58300b6c Merge branch 'feature/service-ports' into 'main'
fix: propagate dynamic IPs/ports to service pages; add apply restart feedback

See merge request root/pic!6
2026-04-22 18:10:12 +00:00
roof 7c94d934e1 test: add .env write verification for port changes
TestEnvFileWrittenOnPortChange (7 tests) confirms that PUT /api/config with
a port change actually writes the new variable to the .env file consumed by
docker compose — the critical link between 'config saved' and 'docker binding
changes on next restart'. Tests cover calendar, webdav, filegator, wireguard,
email; also verifies changing one port does not reset unrelated ports, and
WG_PORT appears exactly once with the new value.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 14:06:20 -04:00
roof 255f9e2576 fix: port changes now correctly queue pending restart for all services
Two bugs fixed:

1. calendar_manager and wireguard_manager (port-only) called
   _restart_container immediately in apply_config, bypassing the pending
   restart banner and restarting the container before the docker port
   binding in .env was updated — leaving the service broken until the
   banner was applied manually. apply_config now only updates the config
   file (radicale.conf / wg0.conf); the docker compose restart happens
   via the banner as intended.

2. Port change detection in update_config used `if old_val is not None`
   to guard against triggering on unchanged values. When a service's port
   was never explicitly saved (first time), old_val was None, so the
   pending restart was never queued. Fix: fall back to PORT_DEFAULTS[key]
   so the comparison is always against the effective current value.

Add TestPortChangeDetection (5 tests) covering first-save and multi-service
accumulation cases.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 13:59:52 -04:00
roof 7a273ad43e fix: consolidate WireGuard port config and propagate port changes to UI
- docker-compose: fix WireGuard port mapping to ${WG_PORT}:${WG_PORT} so
  the daemon ListenPort matches the Docker host-to-container binding
- app.py: sync wireguard.port ↔ identity.wireguard_port in both directions
  so changing either keeps them consistent; identity path now also updates
  wg0.conf via wireguard_manager.update_config
- Settings.jsx: remove duplicate wireguard_port from Cell Identity section
  (port is configurable under WireGuard VPN service config); add
  refreshConfig() after saveService so other pages see new values immediately
- WireGuard.jsx: import useConfig() and use service_configs.wireguard.port
  as the reactive port source for endpoint display and port-open warnings

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 13:27:35 -04:00
roof f07df79f94 fix(apply): handle ip_range network recreation; propagate IPs+ports to service pages
When ip_range changes, Docker cannot modify a network subnet in-place.
_set_pending_restart now accepts network_recreate=True; apply endpoint
runs `docker compose down` before `up -d` in that case so the bridge
network is fully recreated with the new subnet.

Service page fixes:
- GET /api/config includes service_ips (dns, vip_mail, vip_calendar,
  vip_files, vip_webdav) computed via ip_utils
- Email/Calendar/Files pages read IPs and ports from useConfig() instead
  of hardcoded 172.20.0.x constants and default port literals
- Apply feedback: spinner → success/timeout/error banners via health polling

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 12:45:54 -04:00
roof 10878543a9 fix: propagate dynamic IPs/ports to service pages; add apply restart feedback
Service pages (Email, Calendar, Files) now read IPs and ports from the
config API instead of hardcoded 172.20.0.x constants:
- GET /api/config now includes service_ips (dns, vip_mail, vip_calendar,
  vip_files, vip_webdav) computed from ip_range via ip_utils
- Email.jsx: mailIp, dnsIp, imapPort, smtpPort, webmailPort from context
- Calendar.jsx: calendarIp, dnsIp, calendarPort from context
- Files.jsx: filesIp, webdavIp, webdavPort, filegatorPort from context

Apply button now shows restart progress:
- "Restarting containers — please wait…" spinner while polling /health
- "Containers restarted successfully" on success (clears after 4s)
- "Timed out" / error message if health doesn't come back in 45s

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 12:41:10 -04:00
Administrator 0e2f3c2961 Merge branch 'feature/service-ports' into 'main'
feat(service-ports): remove hardcoded ports from docker-compose, make all...

See merge request root/pic!5
2026-04-22 16:27:23 +00:00
roof b46d8d9b8f test(pending-restart): add 28 tests for pending restart system
Covers _set_pending_restart (accumulation, wildcard merge, no duplicates),
_clear_pending_restart, _collect_service_ports (all service port mappings),
GET /api/config/pending (containers field), and DELETE /api/config/pending
(cancel — clears state, idempotent, verified via follow-up GET).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 12:27:01 -04:00
roof 16609da529 feat(pending-banner): add Discard button to cancel pending restart without applying
- DELETE /api/config/pending endpoint calls _clear_pending_restart()
- cellAPI.cancelPending() calls the new endpoint
- PendingRestartBanner shows a "Discard" button alongside "Apply Now";
  clicking it drops the pending state without restarting any containers

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 12:07:39 -04:00
roof b5462f84e0 fix(setup): preserve existing ip_range when re-running make setup
setup_cell.py now reads ip_range from cell_config.json before falling
back to CELL_IP_RANGE env var, so re-running make setup on an existing
install doesn't reset the .env subnet to the default 172.20.0.0/16.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 11:59:07 -04:00
roof 673fe04164 feat(service-ports): remove hardcoded ports from docker-compose, make all service ports configurable
All host port bindings in docker-compose.yml now use \${VAR:-default} substitution,
driven by the .env file generated by ip_utils.write_env_file(). Changing a port in
Settings triggers a per-container pending-restart banner so only the affected container
is restarted on Apply.

- ip_utils: add PORT_DEFAULTS, PORT_ENV_VAR_NAMES, PORT_TO_CONTAINERS; extend
  write_env_file() to accept optional ports dict and write all port env vars
- docker-compose: convert all hardcoded port bindings to \${VAR:-default} form
- app.py: add _collect_service_ports helper; detect port changes in update_config,
  write updated .env and call _set_pending_restart with specific container list;
  update _set_pending_restart to merge/accumulate pending state with containers list;
  update apply_pending_config to use --no-deps <service> for targeted restarts
- config_manager: add submission_port, webmail_port to email schema; add manager_port
  to files schema
- Settings.jsx: make all email/files ports editable, add submission_port, webmail_port,
  manager_port fields; update stale identity note
- tests: 8 new tests for PORT_DEFAULTS, PORT_ENV_VAR_NAMES, and port override in write_env_file

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 11:51:10 -04:00
Administrator d34a23661c Merge branch 'feature/pending-restart' into 'main'
feat: replace hardcoded docker-compose IPs with .env-based substitution

See merge request root/pic!2
2026-04-22 15:35:14 +00:00
roof c3b2c8d8e5 feat: pending-restart banner + Apply button for config changes
When ip_range changes, a persistent amber banner appears at the top of
every page showing what changed and a "Apply Now" button.  Clicking it
shows a confirmation modal ("containers will restart briefly"), then
calls POST /api/config/apply which runs docker compose up -d from inside
the API container — no manual make start needed.

Backend:
- _set_pending_restart() / _clear_pending_restart() helpers track state
  in config_manager so it survives page refresh
- GET /api/config/pending returns { needs_restart, changed_at, changes }
- POST /api/config/apply runs docker compose up -d via the mounted
  docker.sock, using the project working_dir label to resolve host paths
- docker-compose.yml mounts docker-compose.yml itself read-only into
  the API container so docker compose can read it from inside

Frontend (App.jsx):
- Polls /api/config/pending every 5 s alongside the health check
- PendingRestartBanner component with confirmation modal
- Optimistically clears banner on Apply click; API and containers
  restart in the background

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 11:29:26 -04:00
roof 1c939249e4 feat: replace hardcoded docker-compose IPs with .env-based substitution
docker-compose.yml now uses ${VAR:-default} for every container IP and
the network subnet, so there are no hardcoded addresses in the YAML.

How it works:
- setup_cell.py generates .env at project root from ip_range (gitignored).
- docker-compose reads .env automatically at startup.
- When ip_range changes in Settings, the API writes a new .env via
  ip_utils.write_env_file(); DNS/firewall/vIPs update immediately.
- User runs `make start` to recreate containers with the new IPs.

api/ip_utils.py gains ENV_VAR_NAMES dict and write_env_file(ip_range, path).
The old update_docker_compose_ips() direct-patch approach is removed from app.py.
3 new tests added (TestWriteEnvFile); total 324 pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 10:43:33 -04:00
roof 615448b875 feat: dynamic ip_range propagation to DNS, firewall, and docker-compose
When ip_range changes in Settings, the new subnet is now applied to:
- DNS zone records (network_manager.apply_ip_range)
- Caddy virtual IPs (firewall_manager.ensure_caddy_virtual_ips)
- iptables per-service rules (firewall_manager.update_service_ips)
- docker-compose.yml static IPs if writable (ip_utils.update_docker_compose_ips)

New module ip_utils.py derives all container IPs from the subnet using
fixed offsets so the entire stack stays consistent from one setting.

321 tests pass (72 new tests added for ip_utils, apply_ip_range, update_service_ips).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 10:26:21 -04:00
roof 8e741b5729 feat: auto-generate DNS records on first API startup
- NetworkManager.bootstrap_dns_records(): creates A records for all
  cell services (api, webui, calendar, files, mail, webmail, webdav,
  <cell_name>) using their static container IPs — only runs when the
  zone file doesn't exist yet (idempotent)
- API startup: _bootstrap_dns() thread reads cell_name/domain from
  config_manager and calls bootstrap — runs alongside enforcement thread
- Fix: add_dns_record(data) and remove_dns_record(data) now correctly
  unpack dict kwargs instead of passing dict as positional arg
- Fix: remove duplicate cell{} block in config/dns/Corefile

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 10:00:56 -04:00
roof 16af657376 feat: make uninstall prompts before wiping data
Shows a warning then asks: y = full wipe, n/Enter = stop+remove images
only (keep config/data), anything else = cancel. Updates help + README.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 09:44:08 -04:00
roof 78310a70ba feat: add update, reinstall, uninstall, logs-<svc>, shell-<svc> targets
- update: git pull + rebuild + restart
- reinstall: full wipe (config/data) + setup + start
- uninstall: stop, remove images, wipe config/data
- logs-<svc>: follow logs for any single service
- shell-<svc>: exec into any container (bash with sh fallback)
- backup: use sudo tar to read container-owned files
- help: restructured with all commands documented
- README: updated Quick Start + added Management Commands reference

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 09:38:30 -04:00
roof a1a6b65e48 fix: support cryptography < 3.0 API for X25519 key serialization
private_bytes_raw() was added later; fall back to private_bytes(Raw)
for older system packages (e.g. Debian Bookworm python3-cryptography).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 09:33:23 -04:00
roof 42d27c85ef fix: use SUDO_USER for docker group to get the invoking user not root
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 09:32:43 -04:00
roof 30878fe539 fix: check-deps installs all required system packages via apt
scripts/check_deps.sh now checks and installs all prerequisites:
git, curl, openssl, python3, python3-cryptography, docker, docker-compose.
Runs apt-get update only once if anything needs installing.
Also adds current user to docker group if missing.
Makefile calls it with sudo so it has the rights to install packages.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 09:27:45 -04:00
roof 368457ecce fix: move dep checks into scripts/check_deps.sh for robustness
Replaces fragile one-liner Makefile chain with a proper shell script.
Tries: sudo -n apt → python3 -m pip --user → get-pip.py bootstrap.
Prints a clear manual-install message if all automated paths fail.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 09:23:01 -04:00
roof b2f12824ac fix: check-deps uses sudo -n and pip --user, no TTY required
sudo -n skips apt silently if no sudo rights (SSH non-interactive).
Falls back to python3 -m pip install --user which never needs root,
then ensurepip if pip module is missing. Friendly error if all fail.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 09:21:29 -04:00