roof/pic
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update all documentation to reflect current architecture
Unit Tests / test (push) Failing after 8m47s
Details

Browse Source 
README, QUICKSTART, and Wiki were pre-wizard, pre-auth, pre-DDNS, and
pre-service-store.  Full rewrite covering:
- First-run wizard replaces manual make setup + .env identity config
- Session-based auth (admin/peer roles, CSRF protection)
- DDNS: pic.ngo registration with TOTP, provider abstraction
- Service store: install/remove optional services from manifest index
- Cell-to-cell networking and peer-sync protocol
- Extended connectivity: WG external, OpenVPN, Tor exit routing
- Caddy HTTPS: Let's Encrypt (DNS-01/HTTP-01) or internal CA
- Current container list, port bindings, and security model
- Accurate make targets (ddns-update, reset-admin-password, etc.)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Dmitrii Iurco
2026-05-10 04:35:37 -04:00
parent f1b48208fc
commit 35993bc79d
3 changed files with 540 additions and 690 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+95 -58
View File
@@ -1,6 +1,6 @@
# Personal Internet Cell (PIC)
PIC is a self-hosted digital infrastructure platform. It manages DNS, DHCP, NTP, WireGuard VPN, email, calendar/contacts (CalDAV), file storage (WebDAV), a reverse proxy, and a certificate authority — all controlled from a single REST API and React web UI. No manual config file editing is required for normal operations.
PIC is a self-hosted digital infrastructure platform. It packages DNS, DHCP, NTP, WireGuard VPN, email, calendar/contacts (CalDAV), file storage (WebDAV), a reverse proxy, a certificate authority, and optional third-party services — all managed through a single REST API and a React web UI. No manual config file editing is required for normal operations.
---
@@ -10,33 +10,56 @@ PIC is a self-hosted digital infrastructure platform. It manages DNS, DHCP, NTP,
Browser
└── React SPA (cell-webui :8081)
└── Flask REST API (cell-api :3000, bound to 127.0.0.1)
└── Docker SDK / config files
├── cell-caddy :80/:443 reverse proxy
├── cell-dns :53 CoreDNS
├── cell-dhcp :67/udp dnsmasq
├── cell-ntp :123/udp chrony
├── cell-wireguard :51820/udp WireGuard VPN
├── cell-mail :25/:587/:993 Postfix + Dovecot
├── cell-radicale 127.0.0.1:5232 CalDAV/CardDAV
├── cell-webdav 127.0.0.1:8080 WebDAV
├── cell-rainloop :8888 webmail (RainLoop)
├── cell-filegator :8082 file manager UI
└── cell-webui :8081 React UI (Nginx)
└── Service managers + Docker SDK
├── cell-caddy :80/:443 Caddy reverse proxy (HTTPS/TLS)
├── cell-dns :53 CoreDNS
├── cell-dhcp :67/udp dnsmasq
├── cell-ntp :123/udp chrony
├── cell-wireguard :51820/udp WireGuard VPN
├── cell-mail :25/:587/:993 Postfix + Dovecot
├── cell-radicale 127.0.0.1:5232 CalDAV/CardDAV
├── cell-webdav 127.0.0.1:8080 WebDAV
├── cell-rainloop 127.0.0.1:8888 Webmail (RainLoop)
├── cell-filegator 127.0.0.1:8082 File manager (Filegator)
└── cell-webui :8081 React UI (Nginx)
```
All containers run on a custom Docker bridge network (`cell-network`, default `172.20.0.0/16`). Static IPs per container are set in `docker-compose.yml` and overridden via `.env`.
All containers run on a custom Docker bridge network (`cell-network`, default subnet `172.20.0.0/16`). Static IPs per container are set in `docker-compose.yml` and can be overridden via `.env`.
The Flask API (`api/app.py`, ~2800 lines) contains all REST endpoints, runs a background health-monitoring thread, and manages the entire lifecycle of generated config artefacts: `Caddyfile`, `Corefile`, `wg0.conf`, and `cell_config.json` (the single source of truth at `config/api/cell_config.json`).
The Flask API (`api/app.py`) contains REST endpoints and a background health-monitoring thread. Service managers are instantiated as singletons in `api/managers.py`. The single source of truth for runtime configuration is `config/api/cell_config.json`, managed by `ConfigManager`.
The React frontend (`webui/`) is built with Vite + Tailwind CSS. All API calls go through `src/services/api.js` (Axios). Pages: Dashboard, Peers, Network Services, WireGuard, Email, Calendar, Files, Routing, Vault, Containers, Cell Network, Logs, Settings.
The React frontend (`webui/`) is built with Vite + Tailwind CSS. All API calls go through `src/services/api.js` (Axios).
**Web UI pages:** Dashboard, Peers, Network Services, WireGuard, Email, Calendar, Files, Routing, Vault, Containers, Cell Network, Connectivity, Service Store, Logs, Settings.
---
## Features
- **First-run wizard** — browser-based setup at `/setup`. On first start, all API requests redirect to `/setup` (HTTP 428) until the wizard is completed. Sets cell name, domain mode, timezone, admin password, and initial services. No manual `.env` editing required for identity.
- **Session-based auth** — admin and peer roles. All `/api/*` endpoints require an authenticated session after setup. CSRF protection on all state-changing requests.
- **WireGuard VPN** — peer lifecycle management, automatic key generation, QR code config export, per-peer routing policy.
- **Caddy HTTPS** — automatic TLS via Let's Encrypt (DNS-01 or HTTP-01) or an internal CA, depending on domain mode.
- **DDNS (pic.ngo)** — registers a `<cell-name>.pic.ngo` subdomain. Supported providers: `pic_ngo`, `cloudflare`, `duckdns`, `noip`, `freedns`. A background thread re-publishes the public IP every 5 minutes.
- **Service store** — install/remove optional third-party services from the `pic-services` index at `git.pic.ngo`. Manifests declare container images, Caddy routes, and iptables rules.
- **Extended connectivity** — per-peer egress routing through alternate exits: WireGuard external, OpenVPN, or Tor. Configured via policy routing (fwmark + ip rule) in the WireGuard container.
- **Cell-to-cell networking** — WireGuard-based site-to-site links between PIC cells with service-level access control (calendar, files, mail, WebDAV) and a peer-sync protocol.
- **Certificate authority** — `vault_manager` issues and revokes TLS certificates for internal services.
- **Network services** — CoreDNS (`.cell` TLD), dnsmasq DHCP, chrony NTP.
- **Email** — Postfix + Dovecot via `docker-mailserver`.
- **Calendar/contacts** — Radicale CalDAV/CardDAV.
- **File storage** — WebDAV with per-user accounts; Filegator for browser-based file management.
- **Container manager** — start/stop/inspect containers, pull images, manage volumes via the Docker SDK.
- **Firewall manager** — iptables rule management (`firewall_manager.py`).
- **Structured logging** — JSON logs with rotation (5 MB / 5 backups per service), log search, and per-service verbosity control.
---
## Requirements
- Linux host with the WireGuard kernel module loaded
- Linux host with the WireGuard kernel module loaded (`modprobe wireguard` to verify)
- Docker Engine and Docker Compose (v2 plugin or v1 standalone)
- Python 3.10+ (for `make setup` and local dev only; not needed at runtime)
- Python 3.10+ (for `make setup` and local development; not needed at runtime)
- 2 GB+ RAM, 10 GB+ disk
- Ports available: 53, 67/udp, 80, 443, 51820/udp, 25, 587, 993
@@ -44,62 +67,77 @@ The React frontend (`webui/`) is built with Vite + Tailwind CSS. All API calls g
## Quick Start
See [QUICKSTART.md](QUICKSTART.md) for step-by-step setup.
See [QUICKSTART.md](QUICKSTART.md) for step-by-step instructions.
The short version:
```bash
git clone gitea@192.168.31.50:roof/pic.git pic
cd pic
make start
# open http://<host-ip>:8081 — the setup wizard appears automatically
```
---
## Configuration
Runtime configuration is controlled by `.env` in the project root. Copy `.env.example` to `.env` before first run.
Port assignments and container IPs are configured in `.env` in the project root. A `.env` file is not required for first start — all variables have defaults. Create one only if you need to change ports or container IPs.
| Variable | Default | Description |
|---|---|---|
| `CELL_NETWORK` | `172.20.0.0/16` | Docker bridge subnet for all containers |
| `CADDY_IP` through `FILEGATOR_IP` | `172.20.0.2`–`.13` | Static IP for each container |
| `DNS_PORT` | `53` | DNS (UDP+TCP) |
| `CELL_NETWORK` | `172.20.0.0/16` | Docker bridge subnet |
| `CADDY_IP` through `FILEGATOR_IP` | `172.20.0.2`–`.13` | Static IP per container |
| `DNS_PORT` | `53` | DNS (UDP + TCP) |
| `DHCP_PORT` | `67` | DHCP (UDP) |
| `NTP_PORT` | `123` | NTP (UDP) |
| `WG_PORT` | `51820` | WireGuard listen port (UDP) |
| `API_PORT` | `3000` | Flask API (bound to `127.0.0.1`) |
| `API_PORT` | `3000` | Flask API (127.0.0.1 only) |
| `WEBUI_PORT` | `8081` | React UI |
| `MAIL_SMTP_PORT` | `25` | SMTP |
| `MAIL_SUBMISSION_PORT` | `587` | SMTP submission |
| `MAIL_IMAP_PORT` | `993` | IMAP |
| `RADICALE_PORT` | `5232` | CalDAV (bound to `127.0.0.1`) |
| `WEBDAV_PORT` | `8080` | WebDAV (bound to `127.0.0.1`) |
| `RADICALE_PORT` | `5232` | CalDAV (127.0.0.1 only) |
| `WEBDAV_PORT` | `8080` | WebDAV (127.0.0.1 only) |
| `RAINLOOP_PORT` | `8888` | Webmail |
| `FILEGATOR_PORT` | `8082` | File manager UI |
| `WEBDAV_USER` | `admin` | WebDAV basic-auth username |
| `WEBDAV_PASS` | _(required)_ | WebDAV basic-auth password — must be set before `make start` |
| `FLASK_DEBUG` | _(unset)_ | Set to `1` to enable Flask debug mode; do not use in production |
| `WEBDAV_PASS` | _(unset)_ | WebDAV basic-auth password |
| `FLASK_DEBUG` | _(unset)_ | Set to `1` for Flask debug mode; do not use in production |
| `PUID` / `PGID` | current user | UID/GID passed to the WireGuard container |
Cell identity (cell name, domain, VPN IP range) is configured via `make setup` or the Settings → Identity page in the UI after startup. The VPN IP range must be an RFC-1918 CIDR (`10.0.0.0/8`, `172.16.0.0/12`, or `192.168.0.0/16`); the API and UI both enforce this.
Cell identity (cell name, domain mode, timezone) is set through the first-run wizard on first start, or later through the Settings page in the UI.
---
## Security Notes
## Security
**Ports exposed to the network:**
**Ports exposed on all interfaces by default:**
- `80` / `443` — Caddy (HTTP/HTTPS reverse proxy)
- `51820/udp` — WireGuard
- `25` / `587` / `993` — Mail (SMTP, submission, IMAP)
- `53` — DNS (UDP + TCP)
- `25` / `587` / `993` — mail
- `53` — DNS
- `67/udp` — DHCP
- `8081` — Web UI
- `8888` — Webmail (RainLoop)
- `8082` — File manager (Filegator)
**Ports bound to `127.0.0.1` only** (not directly reachable from the network):
**Ports bound to `127.0.0.1` only:**
- `3000` — Flask API
- `5232` — Radicale (CalDAV)
- `8080` — WebDAV
- `8888` — Webmail
- `8082` — Filegator
The API has no authentication layer. It relies on `is_local_request()` to restrict sensitive endpoints (containers, vault) to requests originating from loopback or the cell's Docker network. The Docker socket is mounted into `cell-api`; treat access to port 3000 as equivalent to root access on the host.
The API uses session-based authentication (admin and peer roles). The Docker socket is mounted into `cell-api`; treat access to port 3000 as equivalent to root access on the host.
For internet-facing deployments, place the host behind a firewall or VPN and restrict access to the API and UI ports.
Before setup is complete, all `/api/*` requests except `/api/setup/*` and `/health` return HTTP 428 and a redirect to `/setup`.
CSRF protection (double-submit token in `X-CSRF-Token` header) applies to all `POST`, `PUT`, `DELETE`, and `PATCH` requests on `/api/*` once a user session exists, except `/api/auth/*` and `/api/setup/*`.
Cell-to-cell peer-sync endpoints (`/api/cells/peer-sync/*`) authenticate via source IP and WireGuard public key, not session cookies.
For internet-facing deployments, place the host behind a firewall and restrict access to the API and UI ports.
---
@@ -123,7 +161,7 @@ cd webui && npm install && npm run dev
# Follow all container logs
make logs
# Follow logs for one service (e.g. api, dns, caddy, wireguard, mail)
# Follow logs for one service
make logs-api
# Open a shell inside a container
@@ -135,41 +173,38 @@ make shell-api
## Testing
```bash
make test # run the full pytest suite
make test # run all unit tests (pytest, excludes e2e and integration)
make test-coverage # run with coverage; HTML report in htmlcov/
make test-api # run API endpoint tests only
```
Tests live in `tests/` (34 files, 642 test functions). Coverage includes:
- All service managers (network, WireGuard, email, calendar, file, routing, vault, container)
- API endpoint tests for each service area
- Config manager (CRUD, validation, backup/restore)
- IP utilities and Caddyfile generation
- Peer registry and WireGuard peer lifecycle
- Service bus pub/sub
- Firewall manager
- Pending-restart logic
Integration tests (`tests/integration/`) require a running PIC stack:
Tests live in `tests/`. Integration tests require a running stack:
```bash
make test-integration # full suite (creates peers)
make test-integration # full suite (creates peers, modifies state)
make test-integration-readonly # read-only checks, safe to run anytime
```
End-to-end tests use Playwright:
```bash
make test-e2e-deps # install Playwright and dependencies (run once)
make test-e2e-api # API-level e2e tests
make test-e2e-ui # UI-level e2e tests
```
---
## Management Commands
```bash
make setup # generate WireGuard keys, write configs, create data dirs
make start # docker compose up -d --build
make start # docker compose up -d --build (full profile)
make stop # docker compose down
make restart # docker compose restart
make status # container status + API health check
make logs # follow all service logs
make logs-<svc> # follow logs for one service
make shell-<svc> # shell inside a container
make logs-<svc> # follow logs for one service (e.g. make logs-api)
make shell-<svc> # shell inside a container (e.g. make shell-api)
make update # git pull + rebuild + restart
make reinstall # full wipe of config/ and data/, then setup + start
@@ -180,7 +215,9 @@ make restore # list available backups
make list-peers # show WireGuard peers via API
make show-routes # wg show inside the wireguard container
make add-peer PEER_NAME=foo PEER_IP=10.0.0.5 PEER_KEY=<pubkey>
make show-admin-password # print current admin password
make reset-admin-password # generate and set a new random admin password
```
---