roof
6bc1d625bf
Live verification on pic1 of the connectivity v2 multi-instance feature
surfaced four integration bugs that prevented installing any published
connectivity store service (proxy/wireguard-ext/openvpn-client/sshuttle)
and left stale host routing state behind. All four are fixed here:
1. manifest_validator rejected the CI-published `name:tag@sha256:<digest>`
image form (it required digest-only), while service_store_manager already
accepted it — so every published store image failed validation. Allow an
optional tag before the digest, matching service_store_manager.
2. The cell-api image shipped the docker CLI but not the Compose v2 plugin,
so every `docker compose` ServiceComposer runs (pull/up/down for store
services) failed with "'compose' is not a docker command". Copy the
compose plugin binary from the docker-cli stage.
3. service_store_manager.install ran the base compose up for instanceable
services, whose template still contains ${INSTANCE_ID}/${REDIRECT_PORT}
(there is no base container — one runs per connection instance). It now
verifies the image signature but defers the container to connection
creation for instanceable manifests.
4. delete_connection freed the record/secrets/container but never removed the
connection's individually-managed `ip rule fwmark->table` or its FORWARD
kill-switch (apply_routes only flushes the PIC_CONNECTIVITY chains and
re-adds rules for surviving connections), leaking stale host routing state.
It now tears both down; added _remove_killswitch.
Verified end-to-end on pic1: two proxy instances allocate distinct
marks/tables/ports (skipping in-use resources), render distinct per-instance
containers, two peers route through distinct instances (per-peer MARK +
REDIRECT), delete is blocked while referenced (409) and cleans its ip rule.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Personal Internet Cell (PIC)
PIC is a self-hosted digital infrastructure platform. It packages DNS, NTP, WireGuard VPN, a reverse proxy, a certificate authority, and optional third-party services (email, calendar/contacts, file storage, and more) — all managed through a single REST API and a React web UI. No manual config file editing is required for normal operations.
Architecture
Browser
└── React SPA (cell-webui :8081, container port 8080)
└── Flask REST API (cell-api :3000, bound to 127.0.0.1)
└── Service managers + Docker SDK
├── cell-caddy :80/:443 Caddy reverse proxy (HTTPS/TLS)
├── cell-dns :53 CoreDNS
├── cell-ntp :123/udp chrony
├── cell-wireguard :51820/udp WireGuard VPN (NET_ADMIN only, not privileged)
└── cell-webui :8081→8080 React UI (Nginx)
(+ per-service containers, started when a service is installed)
Six core containers run on a Docker bridge network (cell-network, default subnet 172.20.0.0/16). Static IPs per container are set in docker-compose.yml and can be overridden via .env. Installed service containers join the same network with their own compose projects managed by ServiceComposer.
The Flask API (api/app.py) contains REST endpoints and a background health-monitoring thread. Service managers are instantiated as singletons in api/managers.py. The single source of truth for runtime configuration is config/api/cell_config.json, managed by ConfigManager.
The React frontend (webui/) is built with Vite + Tailwind CSS. All API calls go through src/services/api.js (Axios).
Web UI pages: Dashboard, Peers, Network Services, WireGuard, Connectivity (tunnels, proxies, SSH, Tor, cells, assignments), Services (store catalog + per-service pages), Routing, Vault, Containers, Activity, Logs, Settings — plus peer-facing My Services and Account pages.
Features
- First-run wizard — browser-based setup at
/setup. On first start, all API requests redirect to/setup(HTTP 428) until the wizard is completed. Sets cell name, domain mode, timezone, admin password, and initial services. No manual.envediting required for identity. - Session-based auth — admin and peer roles. All
/api/*endpoints require an authenticated session after setup. CSRF protection on all state-changing requests. - WireGuard VPN — peer lifecycle management, automatic key generation, QR code config export, per-peer routing policy.
- Caddy HTTPS — automatic TLS via Let's Encrypt (DNS-01 or HTTP-01) or an internal CA, depending on domain mode.
- DDNS (pic.ngo) — registers a
<cell-name>.pic.ngosubdomain. Supported providers:pic_ngo,cloudflare,duckdns. A background thread re-publishes the public IP every 5 minutes. - Service store — install/remove optional third-party services from the
pic-servicesindex atgit.pic.ngo. Manifests declare container images, Caddy routes, and iptables rules. Store images are digest-pinned and cosign-signed by the build pipeline; the cell verifies signatures before starting a container (enforced by default). - Extended connectivity — named connection instances per exit type: WireGuard external, OpenVPN, Tor, sshuttle (SSH tunnel), or proxy (HTTP/SOCKS5 via redsocks), plus cell-relay through another cell. Peers are assigned per-peer to a connection with configurable fail-open/fail-closed; per-connection health is tracked. Per-service egress policy is also supported. Routing uses per-instance fwmarks and
ip rulein the WireGuard container. - Cell-to-cell networking — WireGuard-based site-to-site links between PIC cells with service-level access control (calendar, files, mail, WebDAV) and a peer-sync protocol.
- Certificate authority —
vault_managerissues and revokes TLS certificates for internal services. - Network services — CoreDNS (
.cellTLD and split-horizon DNS for the cell domain), chrony NTP. - Split-horizon DNS — from outside the VPN, the cell domain resolves to the public IP. Inside the VPN, CoreDNS resolves it to the WireGuard IP so traffic stays in the tunnel. Caddy serves on both interfaces.
- Email (optional, install via Service Store) — Postfix + Dovecot via
docker-mailserver. - Calendar/contacts (optional, install via Service Store) — Radicale CalDAV/CardDAV.
- File storage (optional, install via Service Store) — WebDAV with per-user accounts; Filegator for browser-based file management.
- Container manager — start/stop/inspect containers, pull images, manage volumes via the Docker SDK.
- Firewall manager — iptables rule management (
firewall_manager.py). - Structured logging — JSON logs with rotation (5 MB / 5 backups per service), log search, and per-service verbosity control.
- Audit log — append-only, hash-chained change log of all admin actions, with CSV export and an Activity page in the UI.
- Backup / restore — full backup of config, secrets, key material, and live service data volumes, with optional passphrase encryption; ordered restore with automatic runtime reapply.
Requirements
- Linux host with the WireGuard kernel module loaded (
modprobe wireguardto verify; required — userspace WireGuard is not supported) - Docker Engine and Docker Compose (v2 plugin or v1 standalone)
- Python 3.10+ (for
make setupand local development; not needed at runtime) - 2 GB+ RAM, 10 GB+ disk
- Ports available: 53, 80, 443, 51820/udp (plus 25, 587, 993 only when the email service is installed)
Documentation
Full documentation lives in the project wiki — installation walkthrough, admin guide (setup, domains/TLS, services, connectivity, peers, backup, logging/audit, troubleshooting), user guide, and developer documentation (architecture, API reference, building store services, testing).
Quick Start
See the wiki's Setup and First Run for step-by-step instructions.
The short version — one-line installer (recommended):
curl -fsSL https://install.pic.ngo | sudo bash
# open http://<host-ip>:8081/setup — the setup wizard appears automatically
Or clone manually for development:
git clone https://git.pic.ngo/roof/pic.git pic
cd pic
make start
# open http://<host-ip>:8081 — the setup wizard appears automatically
Configuration
Port assignments and container IPs are configured in .env in the project root. A .env file is not required for first start — all variables have defaults. Create one only if you need to change ports or container IPs.
| Variable | Default | Description |
|---|---|---|
CELL_NETWORK |
172.20.0.0/16 |
Docker bridge subnet |
CADDY_IP through WG_IP |
172.20.0.2–.11 |
Static IP per core container |
DNS_PORT |
53 |
DNS (UDP + TCP) |
NTP_PORT |
123 |
NTP (UDP) |
WG_PORT |
51820 |
WireGuard listen port (UDP) |
API_PORT |
3000 |
Flask API (127.0.0.1 only) |
WEBUI_PORT |
8081 |
Host port mapped to container port 8080 |
FLASK_DEBUG |
(unset) | Set to 1 for Flask debug mode; do not use in production |
PUID / PGID |
current user | UID/GID passed to the WireGuard container |
Cell identity (cell name, domain mode, timezone) is set through the first-run wizard on first start, or later through the Settings page in the UI.
Security
Ports exposed on all interfaces by default:
80/443— Caddy (HTTP/HTTPS reverse proxy)51820/udp— WireGuard53— DNS8081— Web UI25/587/993— mail (only when the email service is installed)
Ports bound to 127.0.0.1 only:
3000— Flask API
The API uses session-based authentication (admin and peer roles). The Docker socket is mounted into cell-api; treat access to port 3000 as equivalent to root access on the host.
Before setup is complete, all /api/* requests except /api/setup/* and /health return HTTP 428 and a redirect to /setup.
CSRF protection (double-submit token in X-CSRF-Token header) applies to all POST, PUT, DELETE, and PATCH requests on /api/* once a user session exists, except /api/auth/* and /api/setup/*.
Cell-to-cell peer-sync endpoints (/api/cells/peer-sync/*) authenticate via source IP and WireGuard public key, not session cookies.
For internet-facing deployments, place the host behind a firewall and restrict access to the API and UI ports.
Development
# Start the full stack (builds api and webui images)
make start
# Rebuild a single image after code changes
make build-api
make build-webui
# Run Flask API locally without Docker (port 3000)
pip install -r api/requirements.txt
python api/app.py
# Run React UI dev server locally (port 5173, proxies /api to :3000)
cd webui && npm install && npm run dev
# Follow all container logs
make logs
# Follow logs for one service
make logs-api
# Open a shell inside a container
make shell-api
Testing
make test # run all unit tests (pytest, excludes e2e and integration)
make test-coverage # run with coverage; HTML report in htmlcov/
make test-api # run API endpoint tests only
Tests live in tests/. Integration tests require a running stack:
make test-integration # full suite (creates peers, modifies state)
make test-integration-readonly # read-only checks, safe to run anytime
End-to-end tests use Playwright:
make test-e2e-deps # install Playwright and dependencies (run once)
make test-e2e-api # API-level e2e tests
make test-e2e-ui # UI-level e2e tests
Management Commands
make start # docker compose up -d --build (full profile)
make stop # docker compose down
make restart # docker compose restart
make status # container status + API health check
make logs # follow all service logs
make logs-<svc> # follow logs for one service (e.g. make logs-api)
make shell-<svc> # shell inside a container (e.g. make shell-api)
make update # git pull + rebuild + restart
make reinstall # full wipe of config/ and data/, then setup + start
make uninstall # stop containers; prompts whether to also delete config/ and data/
make backup # tar config/ + data/ into backups/
make restore # list available backups
make list-peers # show WireGuard peers via API
make show-routes # wg show inside the wireguard container
make show-admin-password # print current admin password
make reset-admin-password # generate and set a new random admin password
License
MIT.