roof/pic
1
0
Fork 0
Code Issues Pull Requests Actions Wiki Activity
Files
238db6070288f31590bd096f33a113a7463d241a
pic/Personal Internet Cell – Project Wiki.md
T
roof 8a9f4f50c6
Unit Tests / test (push) Successful in 12m12s
Details
docs: bring all docs current with this session's changes 
Update README, QUICKSTART, wiki, service-developer-guide, and CLAUDE.md for:
optional store services (email/calendar/files), sshuttle+proxy egress exits,
provider-aware Network Services/DNS overview, DHCP/dnsmasq removal, split-horizon
VPN DNS, container hardening (slim images, unprivileged WireGuard, webui port 8080,
pinned ntp/coredns), installer changes (host NTP, PIC_DEBUG, clean output, systemd),
and the backup overhaul (full secrets coverage + optional passphrase encryption).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 15:56:03 -04:00

522 lines
24 KiB
Markdown
Raw Blame History

# Personal Internet Cell – Project Wiki
## Overview
Personal Internet Cell (PIC) is a self-hosted digital infrastructure platform. It runs DNS, NTP, WireGuard VPN, HTTPS reverse proxy, a certificate authority, and optional services (email, calendar/contacts, file storage, connectivity exits, and more) — all managed from a single REST API and React web UI.
The goal is to give a person full ownership of their core internet services on their own hardware, without relying on cloud providers.
---
## Table of Contents
1. [Architecture](#architecture)
2. [Service Managers](#service-managers)
3. [First-Run Wizard](#first-run-wizard)
4. [Authentication](#authentication)
5. [API Reference](#api-reference)
6. [DDNS](#ddns)
7. [Network Services and DNS](#network-services-and-dns)
8. [Services UI](#services-ui)
9. [Service Store (Add-ons)](#service-store-add-ons)
10. [Backup and Restore](#backup-and-restore)
11. [Cell-to-Cell Networking](#cell-to-cell-networking)
12. [Extended Connectivity](#extended-connectivity)
13. [Security Model](#security-model)
14. [Testing](#testing)
15. [Development](#development)
---
## Architecture
```
Browser / WireGuard peer
└── Caddy (:80/:443) reverse proxy, TLS termination
└── React SPA (:8081→8080) Vite + Tailwind (Nginx in container)
└── Flask API (:3000) REST API, bound to 127.0.0.1
├── NetworkManager CoreDNS, chrony (split-horizon DNS)
├── WireGuardManager WireGuard VPN peer lifecycle
├── PeerRegistry peer registration and trust
├── EmailManager Postfix + Dovecot (optional service)
├── CalendarManager Radicale CalDAV/CardDAV (optional service)
├── FileManager WebDAV + Filegator (optional service)
├── RoutingManager iptables NAT and routing
├── FirewallManager iptables firewall rules
├── VaultManager internal CA, cert lifecycle, Fernet encryption
├── ContainerManager Docker SDK
├── CellLinkManager cell-to-cell WireGuard links
├── ConnectivityManager exit routing (WG ext, OpenVPN, Tor, sshuttle, proxy)
├── DDNSManager dynamic DNS heartbeat
├── ServiceStoreManager optional service install/remove
├── CaddyManager Caddyfile generation and reload
├── AuthManager session auth, RBAC
└── SetupManager first-run wizard state
```
Six core containers run on a Docker bridge network (`cell-network`, `172.20.0.0/16` default): `cell-caddy`, `cell-dns`, `cell-ntp`, `cell-wireguard`, `cell-api`, `cell-webui`. Static IPs per container are defined in `docker-compose.yml`. Installed optional services join the same network via their own compose projects, managed by `ServiceComposer`.
The WireGuard container runs unprivileged — it uses `NET_ADMIN` only and requires the WireGuard module on the host kernel (Linux 5.6+ or a loadable module). The `cell-api` and `cell-webui` images are slim builds. The CoreDNS image is pinned to a specific digest.
Runtime configuration lives in `config/api/cell_config.json`, managed by `ConfigManager`. All service managers read and write through `ConfigManager`, which validates and backs up automatically.
---
## Service Managers
All managers inherit `BaseServiceManager` (`api/base_service_manager.py`), which provides:
- `get_status()` — current running state
- `get_config()` / `update_config()` — config read/write
- `test_connectivity()` — reachability check
- `get_logs()` — last N lines from the service log
- `restart_service()` — container restart via Docker SDK
The `ServiceBus` (`api/service_bus.py`) handles pub/sub events between managers (e.g., `CONFIG_CHANGED`, `SERVICE_STARTED`). Dependencies are declared in the bus (wireguard depends on network; email depends on network and vault).
### Manager summary
| Manager | Responsibilities |
|---|---|
| `NetworkManager` | CoreDNS zone files and split-horizon DNS, chrony NTP |
| `WireGuardManager` | Key generation, `wg0.conf` generation, peer add/remove, route sync |
| `PeerRegistry` | Peer registration, trust tracking, peer statistics |
| `EmailManager` | docker-mailserver accounts, mailbox config, alias management |
| `CalendarManager` | Radicale user/calendar/contacts lifecycle |
| `FileManager` | WebDAV user directories, Filegator access |
| `RoutingManager` | NAT rules, per-peer routing policy, fwmark-based exit routing |
| `FirewallManager` | iptables INPUT/FORWARD/OUTPUT rule management |
| `VaultManager` | Internal CA (self-signed root), TLS cert issue/revoke, Age public key |
| `ContainerManager` | Docker container/image/volume management via SDK |
| `CellLinkManager` | Site-to-site WireGuard links to other PIC cells, peer-sync protocol |
| `ConnectivityManager` | Per-peer exit routing via WireGuard external, OpenVPN, Tor, sshuttle, or proxy (redsocks) |
| `DDNSManager` | Public IP heartbeat, provider abstraction (pic_ngo, cloudflare, duckdns, noip, freedns) |
| `ServiceStoreManager` | Fetch manifest index, install/remove optional services |
| `CaddyManager` | Caddyfile generation, reload-on-change |
| `AuthManager` | bcrypt password store, session management, admin/peer RBAC |
| `SetupManager` | First-run wizard state, setup-complete flag |
---
## First-Run Wizard
On first start, `SetupManager.is_setup_complete()` returns `False`. The `enforce_setup` before-request hook returns HTTP 428 for all `/api/*` requests except `/api/setup/*` and `/health`, redirecting clients to `/setup`.
The wizard collects:
- **Cell name** — used for hostnames and DDNS subdomain (e.g. `myhome` → `myhome.pic.ngo`)
- **Domain mode** — determines TLS certificate source: `lan` (internal CA), `pic_ngo`, `cloudflare`, `duckdns`, `http01`
- **Timezone**
- **Services to install** — optional services (email, calendar, files) to install after setup; each starts a background install via `ServiceStoreManager`
- **Admin password** — minimum 12 characters
Domain modes and their TLS behavior:
| Mode | Certificate |
|---|---|
| `pic_ngo` | Wildcard Let's Encrypt cert via DNS-01 (requires accurate host clock for ACME + DDNS token) |
| `cloudflare` | Wildcard Let's Encrypt cert via Cloudflare DNS-01 |
| `duckdns` | Let's Encrypt via DuckDNS DNS-01 |
| `http01` | Let's Encrypt per-subdomain cert via HTTP-01 (no wildcard; port 80 must be reachable) |
| `lan` | Internal CA only; no internet required |
On completion:
1. Admin account is created in `data/auth_users.json`
2. Cell identity is written to `config/api/cell_config.json`
3. Caddy config is generated
4. If domain mode is `pic_ngo`, the cell registers `<name>.pic.ngo` with the DDNS service
5. Each selected service is installed in a background thread
Wizard endpoints: `GET/POST /api/setup/step`, `GET /api/setup/status`, `POST /api/setup/complete`.
---
## Authentication
`AuthManager` stores bcrypt-hashed credentials in `data/auth_users.json`. Two roles:
| Role | Access |
|---|---|
| `admin` | All `/api/*` endpoints except `/api/peer/*` |
| `peer` | `/api/peer/*` only (peer dashboard, key exchange) |
Session auth flow:
- `POST /api/auth/login` — creates a Flask session
- `GET /api/auth/me` — current session info
- `POST /api/auth/logout` — clears session
- `POST /api/auth/change-password` — change own password
- `POST /api/auth/admin/reset-password` — admin resets another user's password
CSRF protection: all `POST`, `PUT`, `DELETE`, `PATCH` on `/api/*` (except `/api/auth/*` and `/api/setup/*`) require the `X-CSRF-Token` header matching the session token, obtained via `GET /api/auth/csrf-token`.
Cell-to-cell peer-sync endpoints (`/api/cells/peer-sync/*`) use source-IP + WireGuard public key auth, not session cookies.
Auth enforcement is active once any user exists in the store. If the store is empty (fresh install before wizard), all requests bypass auth — `enforce_setup` already blocks them with 428.
---
## API Reference
**Base URL:** `http://localhost:3000`
**Auth:** session cookie (`X-CSRF-Token` header required for mutations)
### Core
| Method | Path | Description |
|---|---|---|
| GET | `/health` | Health check (always public) |
| GET | `/api/status` | All-service status summary |
| GET | `/api/config` | Full cell config |
| PUT | `/api/config` | Update cell config |
| GET | `/api/health/history` | Recent health check history |
### Auth (`/api/auth/`)
| Method | Path | Description |
|---|---|---|
| POST | `/api/auth/login` | Create session |
| POST | `/api/auth/logout` | Destroy session |
| GET | `/api/auth/me` | Current user info |
| GET | `/api/auth/csrf-token` | Get CSRF token |
| POST | `/api/auth/change-password` | Change own password |
| POST | `/api/auth/admin/reset-password` | Admin: reset another user's password |
| GET | `/api/auth/users` | Admin: list users |
### Setup (`/api/setup/`)
| Method | Path | Description |
|---|---|---|
| GET | `/api/setup/status` | Setup complete flag + current step |
| GET | `/api/setup/step` | Current wizard step data |
| POST | `/api/setup/step` | Submit current step |
| POST | `/api/setup/complete` | Finalize setup |
### Network Services (`/api/dns/`, `/api/ntp/`, `/api/network/`)
DNS overview (effective domain, public records, internal records, per-mode actions), NTP status, network connectivity test.
### WireGuard (`/api/wireguard/`, `/api/peers/`)
Peer add/remove, key generation, QR code export, per-peer routing policy, WireGuard status.
### Email (`/api/email/`) _(available when email service is installed)_
User account management, mailbox config, alias management, connectivity test. Returns HTTP 404 when the email service is not installed (except `/api/email/status`).
### Calendar (`/api/calendar/`) _(available when calendar service is installed)_
User, calendar, and contacts (CardDAV) management. Returns HTTP 404 when the calendar service is not installed (except `/api/calendar/status`).
### Files (`/api/files/`) _(available when files service is installed)_
WebDAV user management, file upload/download/delete, folder management. Returns HTTP 404 when the files service is not installed (except `/api/files/status`).
### Routing (`/api/routing/`)
NAT rules, peer routes, exit node configuration.
### Vault (`/api/vault/`)
Certificate issue/revoke, CA certificate, trust key management, Age public key.
### Containers (`/api/containers/`)
List, start, stop, inspect containers; manage images and volumes.
### Cell Network (`/api/cells/`)
List connected cells, add/remove cell links, peer-sync.
### Connectivity (`/api/connectivity/`)
List exit nodes, configure WireGuard external / OpenVPN / Tor / sshuttle / proxy exits, assign per-peer exit policy, assign per-service egress.
### Service Store (`/api/store/`)
List available services, install, remove.
### Logs (`/api/logs/`)
Per-service log retrieval, log search, log statistics.
---
## DDNS
`DDNSManager` maintains a `<cell-name>.pic.ngo` DNS A record pointing at the cell's public IP. A background thread runs every 5 minutes and calls `provider.update(token, ip)` only when the IP changes.
Registration happens during the setup wizard (if domain mode is `pic_ngo`) via `provider.register(name, ip)`, which returns a bearer token stored in `data/api/.ddns_token`.
DDNS config lives in `cell_config.json` under the top-level `ddns` key:
```json
{
"ddns": {
"provider": "pic_ngo",
"api_base_url": "https://ddns.pic.ngo",
"totp_secret": "<base32 secret>"
}
}
```
Registration requires a time-based OTP (`X-Register-OTP` header) derived from the shared `REGISTER_TOTP_SECRET` on the DDNS server. This prevents unauthorized subdomain registration.
Supported providers: `pic_ngo`, `cloudflare`, `duckdns`, `noip`, `freedns`.
---
## Network Services and DNS
### Split-horizon DNS
PIC operates a split-horizon DNS configuration for the cell domain.
- **Outside the VPN** — the cell domain (e.g. `myhome.pic.ngo`) resolves to the public IP via the DDNS provider.
- **Inside the VPN** — CoreDNS answers the same cell domain with the WireGuard internal IP of `cell-caddy`. Traffic therefore flows through the WireGuard tunnel and Caddy serves it on both the public and WireGuard interface.
`NetworkManager.update_split_horizon_zone()` writes a zone file for the effective domain and reloads CoreDNS via SIGUSR1 whenever the cell name or domain mode changes.
### Network Services page
The **Network Services** page (`/network`) shows a provider-aware DNS overview:
- Current domain mode label and effective domain
- Public DNS records (A records registered with the DDNS provider)
- Service subdomains (shown for `pic_ngo` and `duckdns` modes)
- Internal records served by CoreDNS on the WireGuard network
- Per-mode action buttons (e.g. force-refresh DDNS, reload CoreDNS)
- NTP status (running/stopped, current time source)
DHCP has been removed from PIC. There is no `cell-dhcp` container and no DHCP configuration in the UI. The `cell-dns` container runs CoreDNS only.
### NTP
The `cell-ntp` container runs chrony in a pinned Alpine image. It provides NTP to WireGuard peers that configure the cell as their NTP server. The installer also enables host-level chrony to keep the host clock accurate for ACME certificate issuance and DDNS TOTP tokens.
---
## Backup and Restore
### What `make backup` captures
`make backup` creates `backups/cell-backup-<timestamp>.tar.gz` containing:
- `config/` — all service configuration including `cell_config.json`
- `data/` excluding logs (`data/logs/`) and internal config-backup snapshots (`data/api/config_backups/`)
- `docker-compose.yml` and `Makefile`
The archive is written mode `0600`. It contains key material (WireGuard keys, internal CA, vault fernet.key, admin credentials, DDNS token, cell links, Caddy ACME certs). Store it securely.
Data volumes of installed store services (email mailboxes, calendar collections, file trees) are **not** included in `make backup`. They are captured by the API-driven backup described below.
### API-driven backup (`POST /api/config/backup`)
The API backup captures everything `make backup` does, plus:
- `.env`
- The Caddyfile and Corefile (runtime-generated)
- DNS zone files
- WireGuard key material and live peer configs
- Vault directory (CA, certificates, fernet.key, trust store)
- Per-service connectivity configs (sshuttle keys, redsocks config, OpenVPN configs, WireGuard external config)
- Auth users, Flask secret key, DDNS token, cell links, peer service credentials
- Caddy issued ACME certificates and ACME state
- Live service data volumes (streamed via `docker exec tar`)
**Passphrase encryption**: pass `{"passphrase": "..."}` in the request body to encrypt the archive. The encrypted file is named `<backup_id>.tar.gz.age` and uses Fernet with an scrypt-derived key. The plaintext staging directory is removed immediately after encryption. Supply the same passphrase when calling `POST /api/config/restore/<backup_id>`.
Both backup and encrypted archive files are written mode `0600`.
### Restore
**From `make backup`:**
```bash
tar -xzf backups/cell-backup-YYYYMMDD-HHMMSS.tar.gz
make restart
```
**From API backup:**
Call `POST /api/config/restore/<backup_id>` (with `{"passphrase": "..."}` for encrypted archives). The restore process:
1. Restores the vault first (fernet.key must be present before any encrypted secrets are read)
2. Restores identity, `.env`, WireGuard key material, cell links
3. Restores Caddy ACME certs, Caddyfile, Corefile, DNS zones
4. Restores connectivity configs, auth users, DDNS token
5. Restores service user account files
6. Reloads `cell_config.json` into memory
7. Restores live service data volumes (if service containers are running)
8. Calls `_reapply_runtime_state()` — regenerates Caddyfile and Corefile from the restored config and re-applies routing rules
After an API restore, run `make restart` to ensure all containers pick up the restored configuration.
---
## Services UI
### Navigation
The left-hand navigation contains a **Services** group. Both admin and peer users see it. Sub-items for installed services (Email, Calendar, Files, etc.) are added dynamically: the UI fetches `GET /api/services/active` on load and after each install/uninstall. Services not yet installed do not appear in the nav.
Legacy paths redirect to their new canonical locations:
| Old path | New path |
|---|---|
| `/email` | `/services/email` |
| `/calendar` | `/services/calendar` |
| `/files` | `/services/files` |
| `/store` | `/services` |
### Services page (`/services`)
A single unified catalog of all available services from the store index. Each card shows:
- Service name, description, version
- **Install** button (not installed) or **Uninstall** button (installed)
- **Open** link for installed services (navigates to the service sub-page)
- Running/stopped status dot for installed services
The `pic-services-changed` custom DOM event is dispatched after install/uninstall, causing the nav to re-fetch active services immediately.
### Service sub-pages — admin view
Each sub-page at `/services/email`, `/services/calendar`, and `/services/files` shows:
1. **Connection info** — hostnames, ports, and protocol details (e.g. IMAP/SMTP/Webmail, CalDAV/CardDAV, WebDAV/Filegator).
2. **Service status** — current running state fetched from the API.
3. **Users list** — accounts registered with that service.
4. **Inline config form** — editable fields for that service's settings.
If the service is not installed, the page shows a `ServiceNotInstalledBanner` with a link to the catalog for admins, or a "contact your admin" message for peer users. All non-status API routes for uninstalled services return HTTP 404.
Config forms save automatically with an 800 ms debounce after the last change.
### Service sub-pages — peer view
Peers access the same URLs. The peer view shows only:
- Connection info (hostnames, ports, copy buttons).
- Personal credentials for that service, fetched from `/api/peer/*`.
The config form and users list are not shown to peers.
### Settings page
The Email, Calendar, and Files configuration forms have been removed from the Settings page. Settings now covers: Identity, DDNS, Network (DNS, NTP), WireGuard, Routing & Firewall, Vault & Trust, and Backup & Restore.
### Relevant API endpoints
| Method | Path | Description |
|---|---|---|
| GET | `/api/services/active` | List installed services with id, name, subdomain, capabilities |
| GET | `/api/config` | Full cell config, includes `installed_services` dict |
---
## Service Store (Add-ons)
Email, calendar, and file storage are store services — not part of the core stack. All optional functionality ships through this mechanism.
`ServiceStoreManager` fetches a manifest index from `https://git.pic.ngo/roof/pic-services/raw/branch/main/index.json`. Each manifest (`schema_version: 3`) declares:
- Container image and compose template
- Caddy subdomain routes
- Capabilities: `has_subdomain`, `has_accounts`, `has_admin_config`, `has_storage`, `has_egress`
- Account provisioning interface (`accounts.manager`)
- Backup declarations (`backup.volumes`, `backup.config_paths`)
- Egress routing policy (`egress.allowed`)
- Per-peer connection info template (`peer_config_template`)
`POST /api/store/install` fetches the manifest and compose template, validates them, renders the template with PIC-specific variables (`${PIC_DOMAIN}`, `${PIC_DATA_DIR}`, etc.), writes a per-service compose file, and brings the containers up via `ServiceComposer`. Caddy routes and DNS entries are applied automatically.
`POST /api/store/remove` checks for dependent services, stops and removes containers, and regenerates Caddy.
**`ServiceComposer`** (`api/service_composer.py`) manages the per-service compose lifecycle independently of the main stack. Each service gets its own compose project at `data/services/<id>/docker-compose.yml`. On startup, `reapply_active_services()` brings up containers for all recorded installs.
See `docs/service-developer-guide.md` for the full manifest schema reference and submission process.
---
## Cell-to-Cell Networking
`CellLinkManager` manages WireGuard site-to-site tunnels between PIC cells. Each link is a WireGuard peer configured with a dedicated `/32` address and allowed-IPs covering the remote cell's subnet.
The peer-sync protocol (`/api/cells/peer-sync/`) exchanges public keys and allowed networks between cells using source-IP + WireGuard public key authentication (no session required).
Access control is per-service (calendar, files, mail, WebDAV) and enforced at the iptables level.
---
## Extended Connectivity
`ConnectivityManager` provides per-peer and per-service exit routing: traffic from a specific WireGuard peer (or a specific installed service) can be routed through an alternate exit instead of the cell's default gateway.
All exit types are optional store services installed from the Services catalog. Each exit type corresponds to a store service ID:
| Exit type | Store service | Mechanism |
|---|---|---|
| `wireguard_ext` | `wireguard-ext` | WireGuard client tunnel to an external server; iface `wg_ext0`; fwmark `0x10`, table 110 |
| `openvpn` | `openvpn-client` | OpenVPN client tunnel; iface `tun0`; fwmark `0x20`, table 120 |
| `tor` | `tor` | Transparent proxy → Tor SOCKS on port 9040; fwmark `0x30`, table 130 |
| `sshuttle` | `sshuttle` | SSH tunnel via sshuttle to any SSH server; transparent proxy on port 12300; fwmark `0x40`, table 140 |
| `proxy` | `proxy` | HTTP or SOCKS5 upstream proxy via redsocks transparent redirection on port 12345; fwmark `0x50`, table 150 |
Routing uses fwmark and `ip rule` / `ip route` in separate routing tables inside the `cell-wireguard` container. A kill-switch FORWARD rule drops traffic for configured exits if the exit container is not active.
Configuration is via `PUT /api/connectivity/peers/<peer_name>/exit`. Service-level egress is declared in the service manifest's `egress` block and configured via the Connectivity page in the UI.
---
## Security Model
- **No open ports for the API** — Flask API binds to `127.0.0.1:3000` only; Caddy proxies HTTPS requests to it.
- **Session auth** — bcrypt passwords, Flask server-side sessions, CSRF double-submit.
- **Setup wizard gate** — all `/api/*` requests return 428 until setup is complete.
- **Role separation** — admin cannot access peer endpoints; peer cannot access admin endpoints.
- **HTTPS everywhere** — Caddy handles TLS; internal services are reached via reverse proxy paths.
- **Internal CA** — VaultManager issues certificates for services that don't use Let's Encrypt.
- **Docker socket isolation** — the Docker socket is mounted only into `cell-api`; other containers have no Docker access.
- **iptables firewall** — FirewallManager manages INPUT/FORWARD rules; WireGuard peer isolation is enforced at the packet level.
---
## Testing
```bash
make test # unit tests (pytest, ~1900+ functions)
make test-coverage # coverage report in htmlcov/
```
Test layout:
- `tests/` — unit and endpoint tests; no running services required
- `tests/integration/` — require a running PIC stack
- `tests/e2e/` — Playwright UI tests and WireGuard integration tests
CI: Gitea Actions runs `pytest tests/ --ignore=tests/e2e --ignore=tests/integration` on every push.
---
## Development
```bash
# Full stack in Docker
make start
make stop
make logs
# Flask API without Docker (port 3000)
pip install -r api/requirements.txt
python api/app.py
# React UI dev server (port 5173, proxies /api → :3000)
cd webui && npm install && npm run dev
# Rebuild containers after code change
make build-api
make build-webui
```
Key files:
- `api/app.py` — Flask app, blueprint registration, before-request hooks, health monitor thread
- `api/managers.py` — singleton instantiation of all service managers
- `api/base_service_manager.py` — abstract base class all managers implement
- `api/config_manager.py` — `cell_config.json` read/write/validate/backup
- `api/service_bus.py` — pub/sub event system
- `webui/src/services/api.js` — Axios API client used by all UI pages
- `docker-compose.yml` — container definitions and network topology
- `Makefile` — all operational commands
Reference in New Issue View Git Blame Copy Permalink