After make uninstall (which wipes config/ and data/), running make update
directly failed with "Couldn't find env file: config/mail/mailserver.env"
because docker-compose needs the generated config files to exist.
make update now checks for the sentinel config file and calls make setup
first if it's missing, so uninstall → update works as a valid flow.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The apply_pending_config endpoint spawns _do_apply in a background thread.
subprocess was used but not imported inside the closure, causing
NameError: name 'subprocess' is not defined on every Apply click —
silently swallowed, so containers never restarted and no error was shown.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
TestEnvFileWrittenOnPortChange (7 tests) confirms that PUT /api/config with
a port change actually writes the new variable to the .env file consumed by
docker compose — the critical link between 'config saved' and 'docker binding
changes on next restart'. Tests cover calendar, webdav, filegator, wireguard,
email; also verifies changing one port does not reset unrelated ports, and
WG_PORT appears exactly once with the new value.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two bugs fixed:
1. calendar_manager and wireguard_manager (port-only) called
_restart_container immediately in apply_config, bypassing the pending
restart banner and restarting the container before the docker port
binding in .env was updated — leaving the service broken until the
banner was applied manually. apply_config now only updates the config
file (radicale.conf / wg0.conf); the docker compose restart happens
via the banner as intended.
2. Port change detection in update_config used `if old_val is not None`
to guard against triggering on unchanged values. When a service's port
was never explicitly saved (first time), old_val was None, so the
pending restart was never queued. Fix: fall back to PORT_DEFAULTS[key]
so the comparison is always against the effective current value.
Add TestPortChangeDetection (5 tests) covering first-save and multi-service
accumulation cases.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- docker-compose: fix WireGuard port mapping to ${WG_PORT}:${WG_PORT} so
the daemon ListenPort matches the Docker host-to-container binding
- app.py: sync wireguard.port ↔ identity.wireguard_port in both directions
so changing either keeps them consistent; identity path now also updates
wg0.conf via wireguard_manager.update_config
- Settings.jsx: remove duplicate wireguard_port from Cell Identity section
(port is configurable under WireGuard VPN service config); add
refreshConfig() after saveService so other pages see new values immediately
- WireGuard.jsx: import useConfig() and use service_configs.wireguard.port
as the reactive port source for endpoint display and port-open warnings
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When ip_range changes, Docker cannot modify a network subnet in-place.
_set_pending_restart now accepts network_recreate=True; apply endpoint
runs `docker compose down` before `up -d` in that case so the bridge
network is fully recreated with the new subnet.
Service page fixes:
- GET /api/config includes service_ips (dns, vip_mail, vip_calendar,
vip_files, vip_webdav) computed via ip_utils
- Email/Calendar/Files pages read IPs and ports from useConfig() instead
of hardcoded 172.20.0.x constants and default port literals
- Apply feedback: spinner → success/timeout/error banners via health polling
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Service pages (Email, Calendar, Files) now read IPs and ports from the
config API instead of hardcoded 172.20.0.x constants:
- GET /api/config now includes service_ips (dns, vip_mail, vip_calendar,
vip_files, vip_webdav) computed from ip_range via ip_utils
- Email.jsx: mailIp, dnsIp, imapPort, smtpPort, webmailPort from context
- Calendar.jsx: calendarIp, dnsIp, calendarPort from context
- Files.jsx: filesIp, webdavIp, webdavPort, filegatorPort from context
Apply button now shows restart progress:
- "Restarting containers — please wait…" spinner while polling /health
- "Containers restarted successfully" on success (clears after 4s)
- "Timed out" / error message if health doesn't come back in 45s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Covers _set_pending_restart (accumulation, wildcard merge, no duplicates),
_clear_pending_restart, _collect_service_ports (all service port mappings),
GET /api/config/pending (containers field), and DELETE /api/config/pending
(cancel — clears state, idempotent, verified via follow-up GET).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- DELETE /api/config/pending endpoint calls _clear_pending_restart()
- cellAPI.cancelPending() calls the new endpoint
- PendingRestartBanner shows a "Discard" button alongside "Apply Now";
clicking it drops the pending state without restarting any containers
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
setup_cell.py now reads ip_range from cell_config.json before falling
back to CELL_IP_RANGE env var, so re-running make setup on an existing
install doesn't reset the .env subnet to the default 172.20.0.0/16.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
All host port bindings in docker-compose.yml now use \${VAR:-default} substitution,
driven by the .env file generated by ip_utils.write_env_file(). Changing a port in
Settings triggers a per-container pending-restart banner so only the affected container
is restarted on Apply.
- ip_utils: add PORT_DEFAULTS, PORT_ENV_VAR_NAMES, PORT_TO_CONTAINERS; extend
write_env_file() to accept optional ports dict and write all port env vars
- docker-compose: convert all hardcoded port bindings to \${VAR:-default} form
- app.py: add _collect_service_ports helper; detect port changes in update_config,
write updated .env and call _set_pending_restart with specific container list;
update _set_pending_restart to merge/accumulate pending state with containers list;
update apply_pending_config to use --no-deps <service> for targeted restarts
- config_manager: add submission_port, webmail_port to email schema; add manager_port
to files schema
- Settings.jsx: make all email/files ports editable, add submission_port, webmail_port,
manager_port fields; update stale identity note
- tests: 8 new tests for PORT_DEFAULTS, PORT_ENV_VAR_NAMES, and port override in write_env_file
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When ip_range changes, a persistent amber banner appears at the top of
every page showing what changed and a "Apply Now" button. Clicking it
shows a confirmation modal ("containers will restart briefly"), then
calls POST /api/config/apply which runs docker compose up -d from inside
the API container — no manual make start needed.
Backend:
- _set_pending_restart() / _clear_pending_restart() helpers track state
in config_manager so it survives page refresh
- GET /api/config/pending returns { needs_restart, changed_at, changes }
- POST /api/config/apply runs docker compose up -d via the mounted
docker.sock, using the project working_dir label to resolve host paths
- docker-compose.yml mounts docker-compose.yml itself read-only into
the API container so docker compose can read it from inside
Frontend (App.jsx):
- Polls /api/config/pending every 5 s alongside the health check
- PendingRestartBanner component with confirmation modal
- Optimistically clears banner on Apply click; API and containers
restart in the background
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
docker-compose.yml now uses ${VAR:-default} for every container IP and
the network subnet, so there are no hardcoded addresses in the YAML.
How it works:
- setup_cell.py generates .env at project root from ip_range (gitignored).
- docker-compose reads .env automatically at startup.
- When ip_range changes in Settings, the API writes a new .env via
ip_utils.write_env_file(); DNS/firewall/vIPs update immediately.
- User runs `make start` to recreate containers with the new IPs.
api/ip_utils.py gains ENV_VAR_NAMES dict and write_env_file(ip_range, path).
The old update_docker_compose_ips() direct-patch approach is removed from app.py.
3 new tests added (TestWriteEnvFile); total 324 pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When ip_range changes in Settings, the new subnet is now applied to:
- DNS zone records (network_manager.apply_ip_range)
- Caddy virtual IPs (firewall_manager.ensure_caddy_virtual_ips)
- iptables per-service rules (firewall_manager.update_service_ips)
- docker-compose.yml static IPs if writable (ip_utils.update_docker_compose_ips)
New module ip_utils.py derives all container IPs from the subnet using
fixed offsets so the entire stack stays consistent from one setting.
321 tests pass (72 new tests added for ip_utils, apply_ip_range, update_service_ips).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- NetworkManager.bootstrap_dns_records(): creates A records for all
cell services (api, webui, calendar, files, mail, webmail, webdav,
<cell_name>) using their static container IPs — only runs when the
zone file doesn't exist yet (idempotent)
- API startup: _bootstrap_dns() thread reads cell_name/domain from
config_manager and calls bootstrap — runs alongside enforcement thread
- Fix: add_dns_record(data) and remove_dns_record(data) now correctly
unpack dict kwargs instead of passing dict as positional arg
- Fix: remove duplicate cell{} block in config/dns/Corefile
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Shows a warning then asks: y = full wipe, n/Enter = stop+remove images
only (keep config/data), anything else = cancel. Updates help + README.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- update: git pull + rebuild + restart
- reinstall: full wipe (config/data) + setup + start
- uninstall: stop, remove images, wipe config/data
- logs-<svc>: follow logs for any single service
- shell-<svc>: exec into any container (bash with sh fallback)
- backup: use sudo tar to read container-owned files
- help: restructured with all commands documented
- README: updated Quick Start + added Management Commands reference
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
private_bytes_raw() was added later; fall back to private_bytes(Raw)
for older system packages (e.g. Debian Bookworm python3-cryptography).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
scripts/check_deps.sh now checks and installs all prerequisites:
git, curl, openssl, python3, python3-cryptography, docker, docker-compose.
Runs apt-get update only once if anything needs installing.
Also adds current user to docker group if missing.
Makefile calls it with sudo so it has the rights to install packages.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
sudo -n skips apt silently if no sudo rights (SSH non-interactive).
Falls back to python3 -m pip install --user which never needs root,
then ensurepip if pip module is missing. Friendly error if all fail.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add check-deps target: auto-installs python3, pip3, cryptography via
apt (with pip fallback) before running setup_cell.py
- Add sudo chown in setup to reclaim config/data dirs if containers have
taken ownership (e.g. re-running setup after make start)
- Pass PUID/PGID=$(id -u/g) to docker-compose so linuxserver/wireguard
chowns its config dir to the host user instead of hardcoded UID 911
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
docker compose up --build ensures the API and webui images are always built
from the current source on first install or after code changes, so users
don't need a separate make build-api step.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Allows running make setup on hosts without wg binary or Python cryptography
library by passing pre-generated keys from another machine.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The implementation uses socket.getaddrinfo; the tests were patching
subprocess.run which had no effect, causing both tests to fail.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
DC variable auto-detects available command at build time.
Also fixes backup and dev targets that incorrectly used $(DC).yml.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- app.py: ConfigManager now uses CONFIG_DIR env var for config file path
instead of hardcoded './config/cell_config.json' — config was being read
from the image's working directory, making all settings writes ephemeral
(lost on container restart)
- wireguard_manager: generate_config uses configured address/port instead of
hardcoded 10.0.0.1 in DNAT rules and Address field
- scripts/setup_cell.py: full setup script — generates WireGuard keys (wg
binary or Python cryptography fallback), writes wg0.conf and cell_config.json
with correct _identity key; CELL_NAME / VPN_ADDRESS / WG_PORT env vars
- Makefile: setup target passes env vars through; build-api / build-webui targets
- README: replace install.sh references with make setup && make start
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Site-to-site WireGuard tunnels between PIC cells with automatic DNS forwarding.
Each cell generates an invite JSON (public key, endpoint, VPN subnet, DNS IP,
domain); the remote cell imports it to establish a bidirectional tunnel and
CoreDNS forwarding block so each cell's domain resolves across the mesh.
Backend:
- CellLinkManager: invite generation, add/remove connections, live WireGuard
handshake status; stores links in data/cell_links.json
- WireGuardManager: add_cell_peer() accepts subnet CIDRs (not /32) and an
optional endpoint for site-to-site peers; _read_iface_field() reads port,
address, and network directly from wg0.conf at runtime instead of constants
- NetworkManager: add/remove CoreDNS forwarding blocks per remote cell domain
- app.py: /api/cells/* routes; _next_peer_ip() derives VPN range from
configured address so peer allocation follows any address change
Frontend:
- CellNetwork page: invite panel (JSON + QR), connect form (paste JSON),
connected cells list (green/red status, disconnect button)
- App.jsx: Cell Network nav entry and route
Tests: 25 new tests across test_wireguard_manager, test_network_manager,
test_cell_link_manager (263 total)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Backend:
- wireguard_manager: _get_configured_port/address/network() read from wg0.conf
instead of module-level constants; get_split_tunnel_ips() derives VPN network
from configured Address; get_server_config() returns configured port, dns_ip,
split_tunnel_ips, vpn_network
- add_peer() and get_peer_config() use configured port (not hardcoded 51820)
- _next_peer_ip() derives subnet from wireguard_manager._get_configured_address()
so new peers are allocated IPs from the correct VPN range after address change
- refresh-ip and check-port API endpoints return configured port, not 51820
- PUT /api/config: when wireguard port/address changes, all peers are marked
config_needs_reinstall so users know to re-download tunnel configs
- get_peer_config endpoint: uses configured split tunnel IPs (not hardcoded)
Frontend:
- Peers.jsx: SERVICES domains use live domain from ConfigContext; generateConfig()
uses serverConf.dns_ip and serverConf.split_tunnel_ips; vpn_network shown in
peer-access description; DNS hint uses live domain; server config loaded at
mount time so it is available without re-fetching on every peer action;
handleUpdatePeer uses /32 for server-side AllowedIPs (was incorrectly using
full/split tunnel CIDRs which the backend rejects)
- WireGuard.jsx: generateWireGuardConfig() uses serverConfig.dns_ip,
split_tunnel_ips from server-config API; split-tunnel description shows
live IPs
Tests: 9 new tests in TestWireGuardConfigReads verify all config reads
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Dashboard: SERVICES array (Cell Home, Calendar, Files, Webmail) now builds
URLs from useConfig() domain + cell_name instead of hardcoded 'mycell.cell'
- NetworkServices: imports useConfig() and cellAPI; shows current DNS zone and
DHCP range in page header; fetches service_configs.network to display
configured DHCP range and NTP servers alongside live operational data
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Changes:
- ConfigContext.jsx: React context that loads /api/config once; exposes domain,
cell_name, refresh() — wraps entire app in App.jsx
- Email/Calendar/Files pages: replace hardcoded 'mail.cell', 'calendar.cell',
'files.cell', 'webdav.cell' with domain from ConfigContext; hostname updates
immediately after Settings save (refreshConfig() called on save)
- /api/status: cell_name and domain now read from stored _identity in config_manager,
not hardcoded 'personal-internet-cell' / 'cell.local'
- network_manager.apply_cell_name(old, new): updates hostname A-record in primary
zone file and reloads CoreDNS; called from PUT /api/config when cell_name changes
- Old identity captured before save so apply_cell_name gets the correct old value
- Settings EmailForm: smtp/imap ports are read-only with note (docker-compose.yml level)
- Settings FilesForm: port is read-only with note (Caddy proxies on 80 externally)
- Settings CalendarForm: port labeled "Internal port; clients use 80 via Caddy"
Tests added:
- test_apply_cell_name_renames_host_record
- test_apply_cell_name_noop_when_same
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
config_manager restore_config and import_config previously injected zero-filled
entries (port=0, domain='') for every service schema regardless of whether that
service was in the backup/import data. Removed this logic — only restore what's
actually in the backup.
network_manager.apply_domain now:
- updates dnsmasq.conf domain= line (reload cell-dhcp)
- rewrites Corefile zone blocks to the new domain name
- renames and rewrites the primary zone file $ORIGIN + SOA records
- reloads CoreDNS
Tests added first (TDD):
- test_restore_does_not_zero_unconfigured_services
- test_restore_does_not_zero_import
- test_apply_domain_updates_corefile (zone file + Corefile)
- test_apply_domain_updates_dnsmasq
- test_apply_config_writes_dhcp_range / ntp_servers
- test_apply_config_updates_mailserver_env / no_domain_no_restart
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each service manager now has apply_config() that writes to the actual config:
- network: dhcp_range → dnsmasq.conf (reload cell-dhcp), ntp_servers → chrony.conf
(restart cell-ntp), domain → dnsmasq.conf domain= line
- email: domain → mailserver.env OVERRIDE_HOSTNAME + POSTMASTER_ADDRESS,
restart cell-mail
- wireguard: port/address/private_key → wg0.conf ListenPort/Address/PrivateKey,
restart cell-wireguard
- calendar: port → radicale config hosts=, restart cell-radicale
PUT /api/config now calls apply_config() after persisting JSON, and returns
{restarted: [...], warnings: [...]} so Settings UI can show which containers
were restarted. _restart_container() helper added to BaseServiceManager.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- PUT /api/config now calls service_manager.update_config() for each service
so changes write to the service's own config file, not just cell_config.json
- email_manager.get_status() now reads smtp_port/imap_port/domain from its
config file (defaults: 587/993/cell.local) and includes them in the response
- calendar_manager.get_status() includes configured port (default 5232)
- file_manager.get_status() uses configured port from service config
- Email.jsx reads imap_port/smtp_port from API status instead of hardcoding
- Settings service sections show "port changes require container restart" note
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
app.py:
- Alert logic now checks status.running (container up/down) instead of healthy
(which requires connectivity tests) — services are only alerted when actually down
- Add POST /api/health/history/clear endpoint to reset history + alert counters
log_manager.py:
- get_all_log_file_infos: include rotated backup files (*.log.1, *.log.2 ...) in listing,
marked with backup=true so UI can dim them and hide rotate button
api.js: add monitoringAPI.clearHealthHistory
Logs page:
- Health History: add Clear button with confirmation
- File panel: show full filename (including .log.1 backups), explain host path and naming,
hide rotate button for backup files
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Service manager fixes (connectivity tests):
- email_manager: replace telnet with socket.create_connection for SMTP/IMAP;
replace nslookup with socket.getaddrinfo for DNS; exclude unconfigured domain
from success (email healthy=False now correctly means ports refused, not missing domain)
- calendar_manager: replace localhost:5232 with cell-radicale:5232;
fix database check to test dir writability instead of file existence (files created on demand)
- file_manager: replace localhost:8080 with cell-webdav:80; add top-level success key
- network_manager: replace nslookup with socket.getaddrinfo;
add success key to dhcp_test and ntp_test return values
- routing_manager: exclude iptables_access from success
(iptables runs in cell-wireguard, not API container)
- wireguard_manager: add success key to no-arg test_connectivity result
Health history UI:
- SvcCol reads data?.status?.running || data?.status?.status — handles nested health check shape
Result: network/wireguard/calendar/files/routing/vault all healthy=True.
Email healthy=False is correct — mail server needs ≥1 account before Dovecot starts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
docker-compose.yml:
- Add json-file logging driver (max-size: 10m, max-file: 5) to all 13 containers
- Docker now owns container stdout/stderr rotation automatically
- Add ./data/logs:/app/api/data/logs volume to API — service logs now persist across restarts
log_manager.py:
- Remove container log collection hack (Docker handles it natively)
- Add set_service_level(service, level) — change log level at runtime without restart
- Add get_service_levels() — return current per-service levels
- Simplify get_all_log_file_infos to return only service log files
app.py:
- Add GET /api/logs/verbosity — return current per-service log levels
- Add PUT /api/logs/verbosity — update levels at runtime, persist to config/log_levels.json
- Load persisted log level overrides at startup from log_levels.json
- Simplify rotate endpoint (service logs only, container logs owned by Docker)
wireguard_manager.py:
- get_keys(): return empty strings if key files don't exist (prevents get_status crash
when wg0.conf is missing at startup and falls through to generate_config)
Logs page (4 tabs):
- API Service Logs: structured JSON logs from Python managers, with search/filter/rotate panel
- Container Logs: live docker logs (read via existing /api/containers/<name>/logs endpoint)
- Verbosity Config: per-service level dropdowns, apply immediately + persist
- Health History: existing health poll table
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- log_manager: add collect_container_logs (appends docker logs to container_<name>.log),
get_container_log_lines, rotate_container_log, get_all_log_file_infos
- app.py: new endpoints /api/logs/files (all log file sizes), /api/logs/containers/<name>
(collect+return stored container logs); rotate endpoint now handles both service and container logs
- Logs page: split into API Service Logs tab (python manager logs) and Container Logs tab
(persistent docker stdout/stderr); Statistics tab shows both kinds with per-row rotate;
each tab has a description explaining what it shows and where files live
- wireguard_manager: test_connectivity peer_ip=None guard (already in previous commit, now rebuilt)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- WireGuardManager.test_connectivity: make peer_ip optional so health_check
can call it without args (was logging ERROR on every health poll)
- Logs page: add ALL option to service selector (uses search across all services)
- Logs page: show service tag on each log line when in ALL/search mode
- Logs page: require window.confirm before rotating logs to prevent accidental data loss
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Added info banners on Firewall, Peer Routes, and Live iptables tabs
explaining that stored rules (NAT/Firewall/Peer Routes forms) and live
rules (pic-peer-* from Peers page, PostUp from wg0.conf) are separate
by design — Live iptables shows everything, each form tab shows only
what it manages.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Administrator
roof