Commit Graph

30 Commits

Author SHA1 Message Date
roof 16609da529 feat(pending-banner): add Discard button to cancel pending restart without applying
- DELETE /api/config/pending endpoint calls _clear_pending_restart()
- cellAPI.cancelPending() calls the new endpoint
- PendingRestartBanner shows a "Discard" button alongside "Apply Now";
  clicking it drops the pending state without restarting any containers

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 12:07:39 -04:00
roof 673fe04164 feat(service-ports): remove hardcoded ports from docker-compose, make all service ports configurable
All host port bindings in docker-compose.yml now use \${VAR:-default} substitution,
driven by the .env file generated by ip_utils.write_env_file(). Changing a port in
Settings triggers a per-container pending-restart banner so only the affected container
is restarted on Apply.

- ip_utils: add PORT_DEFAULTS, PORT_ENV_VAR_NAMES, PORT_TO_CONTAINERS; extend
  write_env_file() to accept optional ports dict and write all port env vars
- docker-compose: convert all hardcoded port bindings to \${VAR:-default} form
- app.py: add _collect_service_ports helper; detect port changes in update_config,
  write updated .env and call _set_pending_restart with specific container list;
  update _set_pending_restart to merge/accumulate pending state with containers list;
  update apply_pending_config to use --no-deps <service> for targeted restarts
- config_manager: add submission_port, webmail_port to email schema; add manager_port
  to files schema
- Settings.jsx: make all email/files ports editable, add submission_port, webmail_port,
  manager_port fields; update stale identity note
- tests: 8 new tests for PORT_DEFAULTS, PORT_ENV_VAR_NAMES, and port override in write_env_file

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 11:51:10 -04:00
roof c3b2c8d8e5 feat: pending-restart banner + Apply button for config changes
When ip_range changes, a persistent amber banner appears at the top of
every page showing what changed and a "Apply Now" button.  Clicking it
shows a confirmation modal ("containers will restart briefly"), then
calls POST /api/config/apply which runs docker compose up -d from inside
the API container — no manual make start needed.

Backend:
- _set_pending_restart() / _clear_pending_restart() helpers track state
  in config_manager so it survives page refresh
- GET /api/config/pending returns { needs_restart, changed_at, changes }
- POST /api/config/apply runs docker compose up -d via the mounted
  docker.sock, using the project working_dir label to resolve host paths
- docker-compose.yml mounts docker-compose.yml itself read-only into
  the API container so docker compose can read it from inside

Frontend (App.jsx):
- Polls /api/config/pending every 5 s alongside the health check
- PendingRestartBanner component with confirmation modal
- Optimistically clears banner on Apply click; API and containers
  restart in the background

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 11:29:26 -04:00
roof 1c939249e4 feat: replace hardcoded docker-compose IPs with .env-based substitution
docker-compose.yml now uses ${VAR:-default} for every container IP and
the network subnet, so there are no hardcoded addresses in the YAML.

How it works:
- setup_cell.py generates .env at project root from ip_range (gitignored).
- docker-compose reads .env automatically at startup.
- When ip_range changes in Settings, the API writes a new .env via
  ip_utils.write_env_file(); DNS/firewall/vIPs update immediately.
- User runs `make start` to recreate containers with the new IPs.

api/ip_utils.py gains ENV_VAR_NAMES dict and write_env_file(ip_range, path).
The old update_docker_compose_ips() direct-patch approach is removed from app.py.
3 new tests added (TestWriteEnvFile); total 324 pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 10:43:33 -04:00
roof 615448b875 feat: dynamic ip_range propagation to DNS, firewall, and docker-compose
When ip_range changes in Settings, the new subnet is now applied to:
- DNS zone records (network_manager.apply_ip_range)
- Caddy virtual IPs (firewall_manager.ensure_caddy_virtual_ips)
- iptables per-service rules (firewall_manager.update_service_ips)
- docker-compose.yml static IPs if writable (ip_utils.update_docker_compose_ips)

New module ip_utils.py derives all container IPs from the subnet using
fixed offsets so the entire stack stays consistent from one setting.

321 tests pass (72 new tests added for ip_utils, apply_ip_range, update_service_ips).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 10:26:21 -04:00
roof 8e741b5729 feat: auto-generate DNS records on first API startup
- NetworkManager.bootstrap_dns_records(): creates A records for all
  cell services (api, webui, calendar, files, mail, webmail, webdav,
  <cell_name>) using their static container IPs — only runs when the
  zone file doesn't exist yet (idempotent)
- API startup: _bootstrap_dns() thread reads cell_name/domain from
  config_manager and calls bootstrap — runs alongside enforcement thread
- Fix: add_dns_record(data) and remove_dns_record(data) now correctly
  unpack dict kwargs instead of passing dict as positional arg
- Fix: remove duplicate cell{} block in config/dns/Corefile

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 10:00:56 -04:00
roof 4ed2a6cbae fix: config persistence, setup script, and install docs
- app.py: ConfigManager now uses CONFIG_DIR env var for config file path
  instead of hardcoded './config/cell_config.json' — config was being read
  from the image's working directory, making all settings writes ephemeral
  (lost on container restart)
- wireguard_manager: generate_config uses configured address/port instead of
  hardcoded 10.0.0.1 in DNAT rules and Address field
- scripts/setup_cell.py: full setup script — generates WireGuard keys (wg
  binary or Python cryptography fallback), writes wg0.conf and cell_config.json
  with correct _identity key; CELL_NAME / VPN_ADDRESS / WG_PORT env vars
- Makefile: setup target passes env vars through; build-api / build-webui targets
- README: replace install.sh references with make setup && make start

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 07:37:11 -04:00
roof 848f8cfc7c feat: cell-to-cell (PIC mesh) connection feature
Site-to-site WireGuard tunnels between PIC cells with automatic DNS forwarding.
Each cell generates an invite JSON (public key, endpoint, VPN subnet, DNS IP,
domain); the remote cell imports it to establish a bidirectional tunnel and
CoreDNS forwarding block so each cell's domain resolves across the mesh.

Backend:
- CellLinkManager: invite generation, add/remove connections, live WireGuard
  handshake status; stores links in data/cell_links.json
- WireGuardManager: add_cell_peer() accepts subnet CIDRs (not /32) and an
  optional endpoint for site-to-site peers; _read_iface_field() reads port,
  address, and network directly from wg0.conf at runtime instead of constants
- NetworkManager: add/remove CoreDNS forwarding blocks per remote cell domain
- app.py: /api/cells/* routes; _next_peer_ip() derives VPN range from
  configured address so peer allocation follows any address change

Frontend:
- CellNetwork page: invite panel (JSON + QR), connect form (paste JSON),
  connected cells list (green/red status, disconnect button)
- App.jsx: Cell Network nav entry and route

Tests: 25 new tests across test_wireguard_manager, test_network_manager,
test_cell_link_manager (263 total)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-21 08:34:21 -04:00
roof 3912452fd6 fix: wireguard port/subnet/domain propagate to peer configs and new peer IPs
Backend:
- wireguard_manager: _get_configured_port/address/network() read from wg0.conf
  instead of module-level constants; get_split_tunnel_ips() derives VPN network
  from configured Address; get_server_config() returns configured port, dns_ip,
  split_tunnel_ips, vpn_network
- add_peer() and get_peer_config() use configured port (not hardcoded 51820)
- _next_peer_ip() derives subnet from wireguard_manager._get_configured_address()
  so new peers are allocated IPs from the correct VPN range after address change
- refresh-ip and check-port API endpoints return configured port, not 51820
- PUT /api/config: when wireguard port/address changes, all peers are marked
  config_needs_reinstall so users know to re-download tunnel configs
- get_peer_config endpoint: uses configured split tunnel IPs (not hardcoded)

Frontend:
- Peers.jsx: SERVICES domains use live domain from ConfigContext; generateConfig()
  uses serverConf.dns_ip and serverConf.split_tunnel_ips; vpn_network shown in
  peer-access description; DNS hint uses live domain; server config loaded at
  mount time so it is available without re-fetching on every peer action;
  handleUpdatePeer uses /32 for server-side AllowedIPs (was incorrectly using
  full/split tunnel CIDRs which the backend rejects)
- WireGuard.jsx: generateWireGuardConfig() uses serverConfig.dns_ip,
  split_tunnel_ips from server-config API; split-tunnel description shows
  live IPs

Tests: 9 new tests in TestWireGuardConfigReads verify all config reads

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-21 07:47:19 -04:00
roof 1f3386d43b fix: all service pages use live domain; cell_name/domain propagate to DNS; /api/status reads stored identity
Changes:
- ConfigContext.jsx: React context that loads /api/config once; exposes domain,
  cell_name, refresh() — wraps entire app in App.jsx
- Email/Calendar/Files pages: replace hardcoded 'mail.cell', 'calendar.cell',
  'files.cell', 'webdav.cell' with domain from ConfigContext; hostname updates
  immediately after Settings save (refreshConfig() called on save)
- /api/status: cell_name and domain now read from stored _identity in config_manager,
  not hardcoded 'personal-internet-cell' / 'cell.local'
- network_manager.apply_cell_name(old, new): updates hostname A-record in primary
  zone file and reloads CoreDNS; called from PUT /api/config when cell_name changes
- Old identity captured before save so apply_cell_name gets the correct old value
- Settings EmailForm: smtp/imap ports are read-only with note (docker-compose.yml level)
- Settings FilesForm: port is read-only with note (Caddy proxies on 80 externally)
- Settings CalendarForm: port labeled "Internal port; clients use 80 via Caddy"

Tests added:
- test_apply_cell_name_renames_host_record
- test_apply_cell_name_noop_when_same

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-21 05:05:51 -04:00
roof 87ff50c378 feat: Settings changes now apply to real service config files and restart containers
Each service manager now has apply_config() that writes to the actual config:
- network: dhcp_range → dnsmasq.conf (reload cell-dhcp), ntp_servers → chrony.conf
  (restart cell-ntp), domain → dnsmasq.conf domain= line
- email: domain → mailserver.env OVERRIDE_HOSTNAME + POSTMASTER_ADDRESS,
  restart cell-mail
- wireguard: port/address/private_key → wg0.conf ListenPort/Address/PrivateKey,
  restart cell-wireguard
- calendar: port → radicale config hosts=, restart cell-radicale

PUT /api/config now calls apply_config() after persisting JSON, and returns
{restarted: [...], warnings: [...]} so Settings UI can show which containers
were restarted. _restart_container() helper added to BaseServiceManager.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-21 04:27:22 -04:00
roof ae73246878 fix: propagate Settings config changes to service managers and live pages
- PUT /api/config now calls service_manager.update_config() for each service
  so changes write to the service's own config file, not just cell_config.json
- email_manager.get_status() now reads smtp_port/imap_port/domain from its
  config file (defaults: 587/993/cell.local) and includes them in the response
- calendar_manager.get_status() includes configured port (default 5232)
- file_manager.get_status() uses configured port from service config
- Email.jsx reads imap_port/smtp_port from API status instead of hardcoding
- Settings service sections show "port changes require container restart" note

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-21 03:46:31 -04:00
roof c778ee8eb8 feat: fully editable Settings page with service configs, backup/restore, export/import
- Rewrote Settings.jsx: Cell Identity editor, per-service config sections
  (network, wireguard, email, calendar, files, routing, vault) with
  collapsible cards, appropriate input types, and per-section Save buttons
- Added Backup & Restore panel with create/restore/delete actions
- Added Export (download JSON) and Import (upload JSON) panel
- Added PUT /api/config identity field persistence (_identity key in cell_config.json)
  so cell_name/domain/ip_range/wireguard_port survive restarts
- GET /api/config now returns service_configs separately and prefers stored identity
- Added DELETE /api/config/backups/<id> endpoint
- Extended cellAPI in api.js with createBackup, listBackups, restoreBackup,
  deleteBackup, exportConfig, importConfig

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-21 03:19:46 -04:00
roof 8e1814c7d2 fix: spurious health alerts, show rotated logs, clear history button
app.py:
- Alert logic now checks status.running (container up/down) instead of healthy
  (which requires connectivity tests) — services are only alerted when actually down
- Add POST /api/health/history/clear endpoint to reset history + alert counters

log_manager.py:
- get_all_log_file_infos: include rotated backup files (*.log.1, *.log.2 ...) in listing,
  marked with backup=true so UI can dim them and hide rotate button

api.js: add monitoringAPI.clearHealthHistory

Logs page:
- Health History: add Clear button with confirmation
- File panel: show full filename (including .log.1 backups), explain host path and naming,
  hide rotate button for backup files

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-21 03:05:04 -04:00
roof f848a1d056 feat: proper logging architecture — Docker rotation, persisted service logs, verbosity config
docker-compose.yml:
- Add json-file logging driver (max-size: 10m, max-file: 5) to all 13 containers
- Docker now owns container stdout/stderr rotation automatically
- Add ./data/logs:/app/api/data/logs volume to API — service logs now persist across restarts

log_manager.py:
- Remove container log collection hack (Docker handles it natively)
- Add set_service_level(service, level) — change log level at runtime without restart
- Add get_service_levels() — return current per-service levels
- Simplify get_all_log_file_infos to return only service log files

app.py:
- Add GET /api/logs/verbosity — return current per-service log levels
- Add PUT /api/logs/verbosity — update levels at runtime, persist to config/log_levels.json
- Load persisted log level overrides at startup from log_levels.json
- Simplify rotate endpoint (service logs only, container logs owned by Docker)

wireguard_manager.py:
- get_keys(): return empty strings if key files don't exist (prevents get_status crash
  when wg0.conf is missing at startup and falls through to generate_config)

Logs page (4 tabs):
- API Service Logs: structured JSON logs from Python managers, with search/filter/rotate panel
- Container Logs: live docker logs (read via existing /api/containers/<name>/logs endpoint)
- Verbosity Config: per-service level dropdowns, apply immediately + persist
- Health History: existing health poll table

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-21 02:07:57 -04:00
roof 7b39331417 feat: persistent container log collection, unified rotation, logs page redesign
- log_manager: add collect_container_logs (appends docker logs to container_<name>.log),
  get_container_log_lines, rotate_container_log, get_all_log_file_infos
- app.py: new endpoints /api/logs/files (all log file sizes), /api/logs/containers/<name>
  (collect+return stored container logs); rotate endpoint now handles both service and container logs
- Logs page: split into API Service Logs tab (python manager logs) and Container Logs tab
  (persistent docker stdout/stderr); Statistics tab shows both kinds with per-row rotate;
  each tab has a description explaining what it shows and where files live
- wireguard_manager: test_connectivity peer_ip=None guard (already in previous commit, now rebuilt)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-21 01:54:33 -04:00
roof 4bf583c071 fix: diagnostics tab — run ping/traceroute in cell-wireguard, fix wrong method call
The connectivity endpoint was calling routing_manager.test_connectivity()
(no args, internal health check) instead of test_routing_connectivity(target_ip).
Also ping/traceroute aren't installed in the API container; run them via
docker exec cell-wireguard instead.

Updated test_api_endpoints to mock test_routing_connectivity and cover
the new DELETE /firewall/<id> and GET /live-iptables endpoints.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-21 01:26:40 -04:00
roof 901094f60a feat: routing page — port forwarding tab, live iptables, diagnostics, firewall delete
Backend:
- routing_manager.remove_firewall_rule(): remove stored rule + iptables -D
- routing_manager.get_live_iptables(): dump filter/nat tables from cell-wireguard
- DELETE /api/routing/firewall/<rule_id> endpoint (was missing)
- GET /api/routing/live-iptables endpoint

Frontend Routing.jsx — 7 tabs:
- Overview: proper routing table with destination/gateway/interface columns
- Port Forwarding: clean DNAT form (protocol, ext port → internal IP:port)
- NAT Rules: MASQUERADE/SNAT only, cleaner layout
- Peer Routes: IP route entries through VPN peers
- Firewall: custom rules with working delete button
- Live iptables: read-only terminal view of actual running rules in cell-wireguard
- Diagnostics: ping + traceroute test from server with output display

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-21 01:14:49 -04:00
roof 53c7661812 feat: per-peer access enforcement, live peer status, auto IP assignment
Server-side access control:
- firewall_manager.py: per-peer iptables FORWARD rules in WireGuard container;
  virtual IPs on Caddy (172.20.0.21-24) for per-service DROP/ACCEPT targeting
- CoreDNS Corefile regenerated with ACL blocks for blocked services per peer
- POST /api/wireguard/apply-enforcement re-applies rules after WireGuard restart;
  wg0.conf PostUp calls it via curl so rules restore automatically on container start

WireGuard fixes:
- _syncconf uses `wg set peer` instead of `wg syncconf` to avoid resetting ListenPort
- add_peer validates AllowedIPs must be /32 — rejects full/split tunnel CIDRs that
  would route internet or LAN traffic to that peer
- _config_file() checks for linuxserver wg_confs/ subdirectory first

UI:
- Peers page fetches /api/wireguard/peers/statuses for live handshake data;
  status badge now shows real Online/Offline + seconds since last handshake
- IP field removed from Add Peer form (auto-assigned from 10.0.0.0/24)

Tests (246 pass):
- test_firewall_manager.py: 22 tests for ACL generation, iptables rule correctness,
  comment tagging, clear_peer_rules filter logic
- test_peer_wg_integration.py: 10 tests for /32 enforcement, IP auto-assignment,
  syncconf called on add/remove
- test_wireguard_manager.py: updated to reflect correct IPs and /32 requirement

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-21 01:01:07 -04:00
roof 8e41568964 feat: peer access config, DNS fix, real routing table, reinstall notifications
Peer creation/edit form now configures:
- Tunnel mode: full (0.0.0.0/0) or split (PIC only)
- Per-service access toggles (calendar, files, mail, webdav)
- Peer-to-peer communication toggle
- Optional calendar account creation
- Access capability badges in peer list

Bug fixes:
- DNS in client configs was 8.8.8.8 / 172.20.0.2 — now 172.20.0.3 (CoreDNS)
  This was why .cell domains didn't resolve on connected VPN peers
- get_peer_config API uses stored internet_access to set AllowedIPs
- New PUT /api/peers/<name> endpoint with config_changed detection
- POST /api/peers/<name>/clear-reinstall clears reinstall flag after download
- Routing page reads real host routes via /proc/1/net/route (pid: host)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-20 15:40:19 -04:00
roof 9d7d74f3f4 fix: full-tunnel default, real host routing table, peer config tunnel mode
- WireGuard default changed to full tunnel (0.0.0.0/0) — all peer traffic
  routes through PIC server so internet latency matches server's clean 41ms
- UI tunnel toggle now defaults to Full tunnel
- API /peers/config accepts allowed_ips param so UI toggle wires through
- Routing page reads real host routes via /proc/1/net/route (pid: host)
  instead of mock data; shows ens18/192.168.31.1 correctly
- Add iproute2 + util-linux to API Dockerfile

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-20 15:20:55 -04:00
roof e79ee08c63 fix: WireGuard routing, DNS, service access, and UI improvements
- Fix CoreDNS not loading .cell zones (wrong Corefile path, now uses -conf flag)
- Fix WireGuard server address conflict (172.20.0.1/16 overlapped with Docker
  network; changed to 10.0.0.1/24 to eliminate duplicate routes)
- Add SERVERMODE=true and sysctls to WireGuard docker-compose for server mode
- Fix DNS zone file parser to handle 4-field records (name IN type value)
- Add get_dns_records() to NetworkManager; mount data/dns into API container
- Fix peer config endpoint: look up IP/key from registry, use real endpoint
- Add bulk peer statuses endpoint keyed by public_key
- Normalize snake_case API fields to camelCase in WireGuard UI
- Add port check endpoint (checks via live handshake, not unreliable TCP probe)
- Add Caddy virtual hosts for ui/calendar/files/mail .cell domains (HTTP only)
- Fix cell config domain default from cell.local to cell
- Fix Routing Network Config tab (was calling hardcoded localhost:3000)
- Fix DNS records display (record.value not record.ip)
- Move service access guide to top of Dashboard with login hints
- Add /api/routing/setup endpoint

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-20 12:43:23 -04:00
roof bd67764bf4 feat: external IP detection, port status, fix peer config generation
- WireGuardManager: get_external_ip() (cached 1h), check_port_open(),
  get_server_config() returning public_key + detected endpoint
- API: /api/wireguard/server-config returns real external IP;
  /api/wireguard/refresh-ip forces re-detection;
  /api/wireguard/peers/config now looks up peer IP + private key from
  registry and uses real server endpoint automatically
- Fix doubled port in Endpoint (178.x:51820:51820 → 178.x:51820)
- Fix Address=/32 when peer_ip already has mask
- WebUI nginx: proxy /api/ and /health to cell-api (fixes localhost:3000
  hardcode — UI now works from any machine)
- api.js: baseURL='' so all calls go through nginx proxy
- WireGuard page: show Server Endpoint card with external IP, endpoint,
  public key, and Refresh IP button

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-20 02:41:50 -04:00
roof 5239751a71 fix: all 214 tests passing (from 36 failures)
Key fixes:
- safe_makedirs() in all managers so tests run outside Docker (/app paths)
- WireGuardManager: rewrote with X25519 key gen, corrected method names
- VaultManager: init ca_cert=None, guard generate_certificate when CA missing
- ConfigManager: _save_all_configs wraps mkdir+write in try/except
- app.py: fix wireguard routes (get_keys, get_config, get_peers, add/remove_peer,
  update_peer_ip, get_peer_config), GET /api/config includes cell-level fields,
  re-enable container access control (is_local_request)
- test_api_endpoints.py: patch paths api.app.X -> app.X
- test_app_misc.py: patch paths api.app.X -> app.X, relax status assertions
- test_vault_api.py: replace patch('api.vault_manager') with patch.object(app, ...)
  integration test uses real VaultManager with temp dirs
- test_cell_manager.py: pass config_path to both managers in persistence test

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-19 16:43:07 -04:00
Cloud bb6ccfe023 wip: wireguard 2025-09-14 03:31:14 -05:00
Constantin 4f65f95ac9 wip: peers 2025-09-13 18:56:00 +03:00
Constantin 36776353b9 wip: Fix ContainerDashboard 2025-09-13 15:49:32 +03:00
Constantin b40e4f277e fix uptime on dashboard 2025-09-13 14:42:44 +03:00
Constantin f0b6d1cff1 wip: make work Services Status 2025-09-13 14:23:31 +03:00
Constantin 2277b11563 init 2025-09-12 23:04:52 +03:00