roof/pic
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update all documentation to reflect current architecture
Unit Tests / test (push) Failing after 8m47s
Details

Browse Source 
README, QUICKSTART, and Wiki were pre-wizard, pre-auth, pre-DDNS, and
pre-service-store.  Full rewrite covering:
- First-run wizard replaces manual make setup + .env identity config
- Session-based auth (admin/peer roles, CSRF protection)
- DDNS: pic.ngo registration with TOTP, provider abstraction
- Service store: install/remove optional services from manifest index
- Cell-to-cell networking and peer-sync protocol
- Extended connectivity: WG external, OpenVPN, Tor exit routing
- Caddy HTTPS: Let's Encrypt (DNS-01/HTTP-01) or internal CA
- Current container list, port bindings, and security model
- Accurate make targets (ddns-update, reset-admin-password, etc.)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Dmitrii Iurco
2026-05-10 04:35:37 -04:00
parent f1b48208fc
commit 35993bc79d
3 changed files with 540 additions and 690 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Personal Internet Cell – Project Wiki.md
+343 -535
View File
@@ -1,535 +1,343 @@
# Personal Internet Cell – Project Wiki # Personal Internet Cell – Project Wiki
## 🌟 Overview ## Overview
Personal Internet Cell is a **production-grade, self-hosted, decentralized digital infrastructure** solution designed to provide individuals with full control over their digital services and data. The project has evolved from a phase-based implementation to a **unified, enterprise-ready system** with modern architecture, comprehensive testing, and production-grade features. Personal Internet Cell (PIC) is a self-hosted digital infrastructure platform. It runs DNS, DHCP, NTP, WireGuard VPN, email, calendar/contacts, file storage, HTTPS reverse proxy, a certificate authority, and optional services — all managed from a single REST API and React web UI.
## 📋 Table of Contents The goal is to give a person full ownership of their core internet services on their own hardware, without relying on cloud providers.
1. [Project Goals](#project-goals) ---
2. [Architecture & Components](#architecture--components)
3. [Service Manager Architecture](#service-manager-architecture) ## Table of Contents
4. [Core Services](#core-services)
5. [API Reference](#api-reference) 1. [Architecture](#architecture)
6. [Enhanced CLI](#enhanced-cli) 2. [Service Managers](#service-managers)
7. [Security Model](#security-model) 3. [First-Run Wizard](#first-run-wizard)
8. [Testing & Quality Assurance](#testing--quality-assurance) 4. [Authentication](#authentication)
9. [Usage Examples](#usage-examples) 5. [API Reference](#api-reference)
10. [Development & Deployment](#development--deployment) 6. [DDNS](#ddns)
11. [Future Enhancements](#future-enhancements) 7. [Service Store](#service-store)
12. [Project Status](#project-status) 8. [Cell-to-Cell Networking](#cell-to-cell-networking)
9. [Extended Connectivity](#extended-connectivity)
## 🎯 Project Goals 10. [Security Model](#security-model)
11. [Testing](#testing)
- **Self-Hosted**: Run your own digital services (email, calendar, files, VPN, etc.) on your hardware 12. [Development](#development)
- **Decentralized**: Peer-to-peer networking and trust, no central authority
- **Production-Grade**: Enterprise-ready architecture with comprehensive monitoring ---
- **Secure**: Modern cryptography, certificate management, and encrypted storage
- **User-Friendly**: Professional CLI and API for easy management ## Architecture
- **Extensible**: Modular architecture for future services and integrations
- **Event-Driven**: Real-time service communication and orchestration ```
Browser / WireGuard peer
## 🏗️ Architecture & Components └── Caddy (:80/:443) reverse proxy, TLS termination
└── React SPA (:8081) Vite + Tailwind (Nginx in container)
### **Modern Architecture Stack** └── Flask API (:3000) REST API, bound to 127.0.0.1
├── NetworkManager CoreDNS, dnsmasq, chrony
- **Backend**: Python (Flask) with production-grade service managers ├── WireGuardManager WireGuard VPN peer lifecycle
- **Service Architecture**: BaseServiceManager pattern with unified interfaces ├── PeerRegistry peer registration and trust
- **Event System**: Service bus for real-time communication and orchestration ├── EmailManager Postfix + Dovecot
- **Configuration**: Centralized configuration management with validation ├── CalendarManager Radicale CalDAV/CardDAV
- **Logging**: Structured JSON logging with rotation and search ├── FileManager WebDAV + Filegator
- **Containerization**: Docker-based deployment and service isolation ├── RoutingManager iptables NAT and routing
- **API**: RESTful endpoints with comprehensive documentation ├── FirewallManager iptables firewall rules
├── VaultManager internal CA, cert lifecycle, Age encryption
### **Core Architecture Components** ├── ContainerManager Docker SDK
├── CellLinkManager cell-to-cell WireGuard links
``` ├── ConnectivityManager exit routing (WG ext, OpenVPN, Tor)
┌─────────────────────────────────────────────────────────────┐ ├── DDNSManager dynamic DNS heartbeat
│ Personal Internet Cell │ ├── ServiceStoreManager optional service install/remove
├─────────────────────────────────────────────────────────────┤ ├── CaddyManager Caddyfile generation and reload
│ Enhanced CLI │ Web UI │ REST API │ Service Bus │ Logging │ ├── AuthManager session auth, RBAC
├─────────────────────────────────────────────────────────────┤ └── SetupManager first-run wizard state
│ Service Managers │ ```
│ Network │ WireGuard │ Email │ Calendar │ Files │ Routing │
│ Vault │ Container │ Cell │ Peer │ │ │ All 12 service containers run on a Docker bridge network (`cell-network`, `172.20.0.0/16` default). Static IPs per container are defined in `docker-compose.yml`.
├─────────────────────────────────────────────────────────────┤
│ Core Infrastructure │ Runtime configuration lives in `config/api/cell_config.json`, managed by `ConfigManager`. All service managers read and write through `ConfigManager`, which validates and backs up automatically.
│ DNS │ DHCP │ NTP │ VPN │ CA │ Encryption │ Trust │ Storage │
└─────────────────────────────────────────────────────────────┘ ---
```
## Service Managers
## 🔧 Service Manager Architecture
All managers inherit `BaseServiceManager` (`api/base_service_manager.py`), which provides:
### **BaseServiceManager Pattern** - `get_status()` — current running state
- `get_config()` / `update_config()` — config read/write
All services inherit from `BaseServiceManager`, providing: - `test_connectivity()` — reachability check
- `get_logs()` — last N lines from the service log
```python - `restart_service()` — container restart via Docker SDK
class BaseServiceManager(ABC):
def __init__(self, service_name: str, data_dir: str, config_dir: str) The `ServiceBus` (`api/service_bus.py`) handles pub/sub events between managers (e.g., `CONFIG_CHANGED`, `SERVICE_STARTED`). Dependencies are declared in the bus (wireguard depends on network; email depends on network and vault).
@abstractmethod ### Manager summary
def get_status(self) -> Dict[str, Any]
| Manager | Responsibilities |
@abstractmethod |---|---|
def test_connectivity(self) -> Dict[str, Any] | `NetworkManager` | CoreDNS zone files, dnsmasq DHCP config and lease monitoring, chrony NTP |
| `WireGuardManager` | Key generation, `wg0.conf` generation, peer add/remove, route sync |
# Common methods | `PeerRegistry` | Peer registration, trust tracking, peer statistics |
def get_logs(self, lines: int = 50) -> List[str] | `EmailManager` | docker-mailserver accounts, mailbox config, alias management |
def restart_service(self) -> bool | `CalendarManager` | Radicale user/calendar/contacts lifecycle |
def get_config(self) -> Dict[str, Any] | `FileManager` | WebDAV user directories, Filegator access |
def update_config(self, config: Dict[str, Any]) -> bool | `RoutingManager` | NAT rules, per-peer routing policy, fwmark-based exit routing |
def health_check(self) -> Dict[str, Any] | `FirewallManager` | iptables INPUT/FORWARD/OUTPUT rule management |
def handle_error(self, error: Exception, context: str) -> Dict[str, Any] | `VaultManager` | Internal CA (self-signed root), TLS cert issue/revoke, Age public key |
``` | `ContainerManager` | Docker container/image/volume management via SDK |
| `CellLinkManager` | Site-to-site WireGuard links to other PIC cells, peer-sync protocol |
### **Service Bus Integration** | `ConnectivityManager` | Per-peer exit routing via WireGuard external, OpenVPN, or Tor |
| `DDNSManager` | Public IP heartbeat, provider abstraction (pic_ngo, cloudflare, duckdns, noip, freedns) |
```python | `ServiceStoreManager` | Fetch manifest index, install/remove optional services |
# Event-driven service communication | `CaddyManager` | Caddyfile generation, reload-on-change |
service_bus.register_service('network', network_manager) | `AuthManager` | bcrypt password store, session management, admin/peer RBAC |
service_bus.register_service('wireguard', wireguard_manager) | `SetupManager` | First-run wizard state, setup-complete flag |
service_bus.publish_event(EventType.SERVICE_STARTED, 'network', data)
---
# Service dependencies
service_dependencies = { ## First-Run Wizard
'wireguard': ['network'],
'email': ['network', 'vault'], On first start, `SetupManager.is_setup_complete()` returns `False`. The `enforce_setup` before-request hook returns HTTP 428 for all `/api/*` requests except `/api/setup/*` and `/health`, redirecting clients to `/setup`.
'calendar': ['network', 'vault'],
'files': ['network', 'vault'], The wizard collects:
'routing': ['network', 'wireguard'], - **Cell name** — used for hostnames and DDNS subdomain (e.g. `myhome` → `myhome.pic.ngo`)
'vault': ['network'] - **Domain mode** — determines TLS certificate source: `lan` (internal CA), `pic_ngo`, `cloudflare`, `duckdns`, `http01`
} - **Timezone**
``` - **Initial services to enable**
- **Admin password** — minimum 12 characters
## 🔧 Core Services
On completion:
### **Network Services** 1. Admin account is created in `data/auth_users.json`
- **NetworkManager**: DNS, DHCP, NTP with dynamic management 2. Cell identity is written to `config/api/cell_config.json`
- Dynamic zone file generation 3. Caddy config is generated
- DHCP lease monitoring 4. If domain mode is `pic_ngo`, the cell registers `<name>.pic.ngo` with the DDNS service
- Network connectivity testing
- Service health monitoring Wizard endpoints: `GET/POST /api/setup/step`, `GET /api/setup/status`, `POST /api/setup/complete`.
### **VPN & Mesh Networking** ---
- **WireGuardManager**: WireGuard VPN configuration and peer management
- Key generation and management ## Authentication
- Peer configuration
- Connectivity testing `AuthManager` stores bcrypt-hashed credentials in `data/auth_users.json`. Two roles:
- Dynamic IP updates
| Role | Access |
- **PeerRegistry**: Peer registration and trust management |---|---|
- Peer lifecycle management | `admin` | All `/api/*` endpoints except `/api/peer/*` |
- Trust relationship tracking | `peer` | `/api/peer/*` only (peer dashboard, key exchange) |
- Data integrity validation
- Peer statistics Session auth flow:
- `POST /api/auth/login` — creates a Flask session
### **Digital Services** - `GET /api/auth/me` — current session info
- **EmailManager**: SMTP/IMAP email services - `POST /api/auth/logout` — clears session
- User account management - `POST /api/auth/change-password` — change own password
- Mailbox configuration - `POST /api/auth/admin/reset-password` — admin resets another user's password
- Service connectivity testing
- Email delivery monitoring CSRF protection: all `POST`, `PUT`, `DELETE`, `PATCH` on `/api/*` (except `/api/auth/*` and `/api/setup/*`) require the `X-CSRF-Token` header matching the session token, obtained via `GET /api/auth/csrf-token`.
- **CalendarManager**: CalDAV/CardDAV calendar and contacts Cell-to-cell peer-sync endpoints (`/api/cells/peer-sync/*`) use source-IP + WireGuard public key auth, not session cookies.
- User and calendar management
- Event synchronization Auth enforcement is active once any user exists in the store. If the store is empty (fresh install before wizard), all requests bypass auth — `enforce_setup` already blocks them with 428.
- Service health monitoring
- Connectivity testing ---
- **FileManager**: WebDAV file storage ## API Reference
- User directory management
- Storage quota monitoring **Base URL:** `http://localhost:3000`
- File system access testing **Auth:** session cookie (`X-CSRF-Token` header required for mutations)
- Backup and restore capabilities
### Core
### **Infrastructure Services**
- **RoutingManager**: Advanced routing and NAT | Method | Path | Description |
- NAT rule management |---|---|---|
- Firewall configuration | GET | `/health` | Health check (always public) |
- Exit node routing | GET | `/api/status` | All-service status summary |
- Bridge and split routing | GET | `/api/config` | Full cell config |
- Connectivity testing | PUT | `/api/config` | Update cell config |
| GET | `/api/health/history` | Recent health check history |
- **VaultManager**: Security and trust management
- Self-hosted Certificate Authority ### Auth (`/api/auth/`)
- Certificate lifecycle management
- Age/Fernet encryption | Method | Path | Description |
- Trust relationship management |---|---|---|
- Cryptographic verification | POST | `/api/auth/login` | Create session |
| POST | `/api/auth/logout` | Destroy session |
- **ContainerManager**: Docker orchestration | GET | `/api/auth/me` | Current user info |
- Container lifecycle management | GET | `/api/auth/csrf-token` | Get CSRF token |
- Image and volume management | POST | `/api/auth/change-password` | Change own password |
- Docker daemon connectivity | POST | `/api/auth/admin/reset-password` | Admin: reset another user's password |
- Service isolation | GET | `/api/auth/users` | Admin: list users |
- **CellManager**: Overall cell orchestration ### Setup (`/api/setup/`)
- Service coordination
- Health monitoring | Method | Path | Description |
- Configuration management |---|---|---|
- Peer management | GET | `/api/setup/status` | Setup complete flag + current step |
| GET | `/api/setup/step` | Current wizard step data |
## 📡 API Reference | POST | `/api/setup/step` | Submit current step |
| POST | `/api/setup/complete` | Finalize setup |
### **Core API Endpoints**
### Network Services (`/api/dns/`, `/api/dhcp/`, `/api/ntp/`, `/api/network/`)
```bash
# Service Status and Health DNS records, DHCP leases and reservations, NTP status, network connectivity test.
GET /api/services/status # All services status
GET /api/services/connectivity # Service connectivity tests ### WireGuard (`/api/wireguard/`, `/api/peers/`)
GET /health # API health check
Peer add/remove, key generation, QR code export, per-peer routing policy, WireGuard status.
# Configuration Management
GET /api/config # Get configuration ### Email (`/api/email/`)
PUT /api/config # Update configuration
POST /api/config/backup # Create backup User account management, mailbox config, alias management, connectivity test.
GET /api/config/backups # List backups
POST /api/config/restore/<id> # Restore backup ### Calendar (`/api/calendar/`)
GET /api/config/export # Export configuration
POST /api/config/import # Import configuration User, calendar, and contacts (CardDAV) management.
# Service Bus ### Files (`/api/files/`)
GET /api/services/bus/status # Service bus status
GET /api/services/bus/events # Event history WebDAV user management, file upload/download/delete, folder management.
POST /api/services/bus/services/<service>/start
POST /api/services/bus/services/<service>/stop ### Routing (`/api/routing/`)
POST /api/services/bus/services/<service>/restart
NAT rules, peer routes, exit node configuration.
# Logging
GET /api/logs/services/<service> # Service logs ### Vault (`/api/vault/`)
POST /api/logs/search # Log search
POST /api/logs/export # Log export Certificate issue/revoke, CA certificate, trust key management, Age public key.
GET /api/logs/statistics # Log statistics
POST /api/logs/rotate # Log rotation ### Containers (`/api/containers/`)
```
List, start, stop, inspect containers; manage images and volumes.
### **Service-Specific Endpoints**
### Cell Network (`/api/cells/`)
```bash
# Network Services List connected cells, add/remove cell links, peer-sync.
GET /api/dns/records # DNS records
POST /api/dns/records # Add DNS record ### Connectivity (`/api/connectivity/`)
DELETE /api/dns/records # Remove DNS record
GET /api/dhcp/leases # DHCP leases List exit nodes, configure WireGuard external / OpenVPN / Tor exits, assign per-peer exit policy.
POST /api/dhcp/reservations # Add DHCP reservation
GET /api/ntp/status # NTP status ### Service Store (`/api/store/`)
GET /api/network/info # Network information
POST /api/network/test # Network connectivity test List available services, install, remove.
# WireGuard & Peers ### Logs (`/api/logs/`)
GET /api/wireguard/keys # WireGuard keys
POST /api/wireguard/keys/peer # Generate peer keys Per-service log retrieval, log search, log statistics.
GET /api/wireguard/config # WireGuard configuration
GET /api/wireguard/peers # List peers ---
POST /api/wireguard/peers # Add peer
DELETE /api/wireguard/peers # Remove peer ## DDNS
GET /api/wireguard/status # WireGuard status
POST /api/wireguard/connectivity # Connectivity test `DDNSManager` maintains a `<cell-name>.pic.ngo` DNS A record pointing at the cell's public IP. A background thread runs every 5 minutes and calls `provider.update(token, ip)` only when the IP changes.
PUT /api/wireguard/peers/ip # Update peer IP
Registration happens during the setup wizard (if domain mode is `pic_ngo`) via `provider.register(name, ip)`, which returns a bearer token stored in `data/api/.ddns_token`.
# Digital Services
GET /api/email/users # Email users DDNS config lives in `cell_config.json` under the top-level `ddns` key:
POST /api/email/users # Add email user
DELETE /api/email/users/<user> # Remove email user ```json
GET /api/email/status # Email service status {
GET /api/email/connectivity # Email connectivity "ddns": {
POST /api/email/send # Send email "provider": "pic_ngo",
GET /api/email/mailbox/<user> # User mailbox "api_base_url": "https://ddns.pic.ngo",
"totp_secret": "<base32 secret>"
GET /api/calendar/users # Calendar users }
POST /api/calendar/users # Add calendar user }
DELETE /api/calendar/users/<user> # Remove calendar user ```
POST /api/calendar/calendars # Create calendar
POST /api/calendar/events # Add event Registration requires a time-based OTP (`X-Register-OTP` header) derived from the shared `REGISTER_TOTP_SECRET` on the DDNS server. This prevents unauthorized subdomain registration.
GET /api/calendar/events/<user>/<calendar> # List events
GET /api/calendar/status # Calendar service status Supported providers: `pic_ngo`, `cloudflare`, `duckdns`, `noip`, `freedns`.
GET /api/calendar/connectivity # Calendar connectivity
---
GET /api/files/users # File users
POST /api/files/users # Add file user ## Service Store
DELETE /api/files/users/<user> # Remove file user
POST /api/files/folders # Create folder `ServiceStoreManager` fetches a manifest index from `http://git.pic.ngo/roof/pic-services/raw/branch/main/index.json`. Each manifest declares:
DELETE /api/files/folders/<user>/<path> # Remove folder - Container image
POST /api/files/upload/<user> # Upload file - Caddy routes (added to the Caddyfile)
GET /api/files/download/<user>/<path> # Download file - iptables rules
DELETE /api/files/delete/<user>/<path> # Delete file - Environment variables
GET /api/files/list/<user> # List files - Health check endpoint
GET /api/files/status # File service status
GET /api/files/connectivity # File connectivity `POST /api/store/install` pulls the image, writes the Caddy route, applies iptables rules, and starts the container. `POST /api/store/remove` reverses this.
# Routing & Security ---
GET /api/routing/status # Routing status
POST /api/routing/nat # Add NAT rule ## Cell-to-Cell Networking
DELETE /api/routing/nat/<id> # Remove NAT rule
POST /api/routing/peers # Add peer route `CellLinkManager` manages WireGuard site-to-site tunnels between PIC cells. Each link is a WireGuard peer configured with a dedicated `/32` address and allowed-IPs covering the remote cell's subnet.
DELETE /api/routing/peers/<peer> # Remove peer route
POST /api/routing/exit-nodes # Add exit node The peer-sync protocol (`/api/cells/peer-sync/`) exchanges public keys and allowed networks between cells using source-IP + WireGuard public key authentication (no session required).
POST /api/routing/bridge # Add bridge route
POST /api/routing/split # Add split route Access control is per-service (calendar, files, mail, WebDAV) and enforced at the iptables level.
POST /api/routing/firewall # Add firewall rule
POST /api/routing/connectivity # Routing connectivity test ---
GET /api/routing/logs # Routing logs
GET /api/routing/nat # List NAT rules ## Extended Connectivity
GET /api/routing/peers # List peer routes
GET /api/routing/firewall # List firewall rules `ConnectivityManager` provides per-peer exit routing: traffic from a specific WireGuard peer can be routed through an alternate exit instead of the cell's default gateway.
GET /api/vault/status # Vault status Supported exits:
GET /api/vault/certificates # List certificates - **WireGuard external** — another WireGuard endpoint (e.g. a VPS)
POST /api/vault/certificates # Generate certificate - **OpenVPN** — OpenVPN client running in a container
DELETE /api/vault/certificates/<name> # Revoke certificate - **Tor** — Tor SOCKS proxy with transparent redirection
GET /api/vault/ca/certificate # CA certificate
GET /api/vault/age/public-key # Age public key Routing uses fwmark and `ip rule` / `ip route` in separate routing tables. Configuration is via `PUT /api/connectivity/peers/<peer_name>/exit`.
GET /api/vault/trust/keys # Trusted keys
POST /api/vault/trust/keys # Add trusted key ---
DELETE /api/vault/trust/keys/<name> # Remove trusted key
POST /api/vault/trust/verify # Verify trust ## Security Model
GET /api/vault/trust/chains # Trust chains
``` - **No open ports for the API** — Flask API binds to `127.0.0.1:3000` only; Caddy proxies HTTPS requests to it.
- **Session auth** — bcrypt passwords, Flask server-side sessions, CSRF double-submit.
## 💻 Enhanced CLI - **Setup wizard gate** — all `/api/*` requests return 428 until setup is complete.
- **Role separation** — admin cannot access peer endpoints; peer cannot access admin endpoints.
### **CLI Features** - **HTTPS everywhere** — Caddy handles TLS; internal services are reached via reverse proxy paths.
- **Internal CA** — VaultManager issues certificates for services that don't use Let's Encrypt.
```bash - **Docker socket isolation** — the Docker socket is mounted only into `cell-api`; other containers have no Docker access.
# Interactive mode with tab completion - **iptables firewall** — FirewallManager manages INPUT/FORWARD rules; WireGuard peer isolation is enforced at the packet level.
python api/enhanced_cli.py --interactive
---
# Batch operations
python api/enhanced_cli.py --batch "status" "services" "health" ## Testing
# Configuration management ```bash
python api/enhanced_cli.py --export-config json make test # unit tests (pytest, ~1500 functions)
python api/enhanced_cli.py --import-config config.json make test-coverage # coverage report in htmlcov/
```
# Service wizards
python api/enhanced_cli.py --wizard network Test layout:
python api/enhanced_cli.py --wizard email - `tests/` — unit and endpoint tests; no running services required
- `tests/integration/` — require a running PIC stack
# Health monitoring - `tests/e2e/` — Playwright UI tests and WireGuard integration tests
python api/enhanced_cli.py --health
python api/enhanced_cli.py --logs network CI: Gitea Actions runs `pytest tests/ --ignore=tests/e2e --ignore=tests/integration` on every push.
# Service status ---
python api/enhanced_cli.py --status
python api/enhanced_cli.py --services ## Development
python api/enhanced_cli.py --peers
``` ```bash
# Full stack in Docker
### **CLI Capabilities** make start
- **Interactive Mode**: Tab completion, command history, help system make stop
- **Batch Operations**: Execute multiple commands in sequence make logs
- **Configuration Wizards**: Guided setup for complex services
- **Real-time Monitoring**: Live status updates and health checks # Flask API without Docker (port 3000)
- **Log Management**: View, search, and export service logs pip install -r api/requirements.txt
- **Service Management**: Start, stop, restart, and configure services python api/app.py
## 🔒 Security Model # React UI dev server (port 5173, proxies /api → :3000)
cd webui && npm install && npm run dev
### **Certificate Management**
- **Self-hosted CA**: Issue and manage TLS certificates for all services # Rebuild containers after code change
- **Certificate Lifecycle**: Generate, renew, revoke, and monitor certificates make build-api
- **Trust Management**: Direct, indirect, and verified trust relationships make build-webui
- **Age Encryption**: Modern encryption for sensitive data and keys ```
### **Network Security** Key files:
- **WireGuard VPN**: Secure peer-to-peer communication with key rotation - `api/app.py` — Flask app, blueprint registration, before-request hooks, health monitor thread
- **Firewall & NAT**: Granular control over network access and routing - `api/managers.py` — singleton instantiation of all service managers
- **Service Isolation**: Docker containers for each service - `api/base_service_manager.py` — abstract base class all managers implement
- **Input Validation**: All API endpoints validate and sanitize input - `api/config_manager.py` — `cell_config.json` read/write/validate/backup
- `api/service_bus.py` — pub/sub event system
### **Data Protection** - `webui/src/services/api.js` — Axios API client used by all UI pages
- **Encrypted Storage**: Sensitive data encrypted at rest using Age/Fernet - `docker-compose.yml` — container definitions and network topology
- **Secure Communication**: TLS for all API endpoints and service communication - `Makefile` — all operational commands
- **Access Control**: Role-based access for services and API endpoints
- **Audit Logging**: Comprehensive security event logging and monitoring
## 🧪 Testing & Quality Assurance
### **Test Coverage**
- **BaseServiceManager**: 100% coverage
- **ConfigManager**: 95%+ coverage
- **ServiceBus**: 95%+ coverage
- **LogManager**: 95%+ coverage
- **All Service Managers**: 77%+ overall coverage
- **API Endpoints**: 100% endpoint coverage
### **Test Types**
- **Unit Tests**: Individual component testing
- **Integration Tests**: Service interaction testing
- **API Tests**: Endpoint functionality testing
- **Error Handling**: Exception and edge case testing
- **Performance Tests**: Load and stress testing
### **Testing Commands**
```bash
# Run all tests
python api/test_enhanced_api.py
# Run specific test suites
python -m pytest api/tests/test_network_manager.py
python -m pytest api/tests/test_service_bus.py
# Generate coverage report
coverage run -m pytest api/tests/
coverage html
```
## 📝 Usage Examples
### **Add DNS Record**
```bash
curl -X POST http://localhost:3000/api/dns/records \
-H "Content-Type: application/json" \
-d '{
"name": "www",
"type": "A",
"value": "192.168.1.100",
"ttl": 300
}'
```
### **Register Peer**
```bash
curl -X POST http://localhost:3000/api/wireguard/peers \
-H "Content-Type: application/json" \
-d '{
"name": "bob",
"ip": "203.0.113.22",
"public_key": "peer_public_key_here",
"allowed_networks": ["10.0.0.0/24"]
}'
```
### **Generate Certificate**
```bash
curl -X POST http://localhost:3000/api/vault/certificates \
-H "Content-Type: application/json" \
-d '{
"common_name": "myapp.example.com",
"domains": ["myapp.example.com", "www.myapp.example.com"],
"days": 365
}'
```
### **Configure NAT Rule**
```bash
curl -X POST http://localhost:3000/api/routing/nat \
-H "Content-Type: application/json" \
-d '{
"source_network": "10.0.0.0/24",
"target_interface": "eth0",
"nat_type": "MASQUERADE",
"protocol": "ALL"
}'
```
## 🛠️ Development & Deployment
### **Development Setup**
```bash
# Install dependencies
pip install -r api/requirements.txt
# Start development server
python api/app.py
# Run tests
python api/test_enhanced_api.py
# Start frontend (if available)
cd webui && bun install && npm run dev
```
### **Production Deployment**
```bash
# Docker deployment
docker-compose up --build -d
# Health check
curl http://localhost:3000/health
# Service status
curl http://localhost:3000/api/services/status
```
### **Service Development**
```python
from base_service_manager import BaseServiceManager
class MyServiceManager(BaseServiceManager):
def __init__(self, data_dir='/app/data', config_dir='/app/config'):
super().__init__('myservice', data_dir, config_dir)
def get_status(self) -> Dict[str, Any]:
# Implement service status
return {
'running': True,
'status': 'online',
'timestamp': datetime.utcnow().isoformat()
}
def test_connectivity(self) -> Dict[str, Any]:
# Implement connectivity test
return {
'success': True,
'message': 'Service connectivity working',
'timestamp': datetime.utcnow().isoformat()
}
```
## 🚀 Future Enhancements
### **Planned Features**
- **Certificate Auto-renewal**: Automatic certificate renewal and monitoring
- **Web of Trust Models**: Advanced trust relationship management
- **Certificate Transparency**: CT log integration and monitoring
- **Hardware Security Module (HSM)**: HSM integration for key management
- **WebSocket Updates**: Real-time service status updates
- **Advanced Monitoring**: Metrics collection and alerting systems
- **Mobile App**: Mobile application for remote management
- **Plugin System**: Extensible architecture for custom services
### **Architecture Improvements**
- **Service Discovery**: Dynamic service registration and discovery
- **Load Balancing**: Multi-instance service deployment
- **Advanced Caching**: Redis-based caching for performance
- **Message Queues**: RabbitMQ/Kafka for reliable messaging
- **Distributed Tracing**: OpenTelemetry integration
- **Configuration Management**: GitOps-style configuration management
## 📊 Project Status
### **✅ Completed Features**
- **Production-Grade Architecture**: BaseServiceManager pattern implemented
- **Event-Driven Communication**: Service bus with real-time events
- **Centralized Configuration**: Type-safe configuration with validation
- **Comprehensive Logging**: Structured logging with search and export
- **Enhanced CLI**: Interactive CLI with batch operations
- **Health Monitoring**: Real-time health checks across all services
- **Security Framework**: Self-hosted CA, encryption, and trust management
- **Complete API**: RESTful API with comprehensive documentation
- **Testing Framework**: Comprehensive test suite with high coverage
### **🎯 Current Status**
- **All Services**: 10 service managers fully implemented and integrated
- **API Server**: Running on port 3000 with all endpoints functional
- **CLI Tool**: Enhanced CLI with all features working
- **Test Coverage**: 77%+ overall coverage with comprehensive testing
- **Documentation**: Complete documentation for all components
- **Production Ready**: Suitable for personal and small business deployment
### **🌟 Key Achievements**
- **Unified Architecture**: All services follow the same patterns and interfaces
- **Event-Driven Design**: Services communicate and orchestrate automatically
- **Configuration Management**: Centralized, validated configuration system
- **Comprehensive Logging**: Production-grade logging with advanced features
- **Enhanced CLI**: Professional command-line interface for management
- **Health Monitoring**: Real-time monitoring and alerting capabilities
- **Security Framework**: Enterprise-grade security with modern cryptography
- **Complete Testing**: Comprehensive test suite ensuring reliability
---
**The Personal Internet Cell empowers users with full control over their digital infrastructure, combining privacy, security, and usability in a single, production-ready, self-hosted platform.** 🌟
QUICKSTART.md
+102 -97
View File
@@ -1,85 +1,53 @@
# Quick Start # Quick Start
This guide walks through a first-time PIC installation from a clean Linux host. This guide walks through a first-time PIC installation on a clean Linux host.
--- ---
## Prerequisites ## Prerequisites
- Linux host with the WireGuard kernel module (`modprobe wireguard` to verify) - Linux host with the WireGuard kernel module loaded
- Docker Engine and Docker Compose installed - Docker Engine and Docker Compose installed (v2 plugin or v1 standalone)
- Python 3.10+ (needed for `make setup` only) - Python 3.10+ (for `make setup` only; not needed at runtime)
- 2 GB+ RAM, 10 GB+ disk - 2 GB+ RAM, 10 GB+ disk
- Ports 53, 80, 443, 51820/udp, 25, 587, 993 available on the host
Verify the WireGuard module is available:
```bash
sudo modprobe wireguard
```
--- ---
## 1. Clone the repository ## 1. Clone the repository
```bash ```bash
git clone <repo-url> pic git clone gitea@192.168.31.50:roof/pic.git pic
cd pic cd pic
``` ```
--- ---
## 2. Configure the environment ## 2. Start the stack
Copy the example environment file and edit it:
```bash
cp .env.example .env
```
Open `.env` and set at minimum:
```
WEBDAV_PASS=changeme
```
`WEBDAV_PASS` must be set before starting — the WebDAV container will fail to start without it.
All other variables have working defaults. See the Configuration section in [README.md](README.md) for the full list.
---
## 3. Run setup
`make setup` installs system dependencies, generates WireGuard keys, and writes all required config files under `config/`:
```bash
make check-deps # installs docker, python3-cryptography, etc. via apt
make setup # generates keys and writes configs
```
To customise the cell identity at setup time, pass overrides on the command line:
```bash
CELL_NAME=myhome CELL_DOMAIN=cell VPN_ADDRESS=10.0.0.1/24 WG_PORT=51820 make setup
```
`VPN_ADDRESS` must be an RFC-1918 address (e.g. `10.0.0.1/24`).
---
## 4. Start the stack
```bash ```bash
make start make start
``` ```
This builds the `cell-api` and `cell-webui` images and starts all 13 containers. The first run takes a few minutes while images are pulled and built. This builds the `cell-api` and `cell-webui` Docker images and starts all containers. The first run pulls images and compiles the frontend, which takes a few minutes.
Check that everything came up: Check that containers came up:
```bash ```bash
make status make status
``` ```
You should see all containers in the `Up` state and the API responding at `http://localhost:3000/health`. All containers should show as `Up`.
--- ---
## 5. Open the web UI ## 3. Complete the setup wizard
Open a browser and go to: Open a browser and go to:
@@ -87,53 +55,56 @@ Open a browser and go to:
http://<host-ip>:8081 http://<host-ip>:8081
``` ```
If you are running locally: The setup wizard appears automatically on first start. Until the wizard is complete, all API requests redirect to `/setup`.
``` The wizard asks for:
http://localhost:8081
```
The sidebar contains: Dashboard, Peers, Network Services, WireGuard, Email, Calendar, Files, Routing, Vault, Containers, Cell Network, Logs, Settings. - **Cell name** — a short identifier used in hostnames and DDNS. Must start with a lowercase letter and contain only lowercase letters, digits, and hyphens (2–31 characters total). Example: `myhome`.
- **Domain mode** — how HTTPS certificates are issued:
- `lan` — internal CA, no internet required
- `pic_ngo` — automatic `<cell-name>.pic.ngo` subdomain with Let's Encrypt via DNS-01
- `cloudflare` — Let's Encrypt via Cloudflare DNS-01
- `duckdns` — Let's Encrypt via DuckDNS DNS-01
- `http01` — Let's Encrypt via HTTP-01 (no wildcard certificates)
- **Timezone** — used for NTP and log timestamps.
- **Services to enable** — choose from email, calendar, files, WireGuard.
- **Admin password** — at least 12 characters with uppercase, lowercase, and a digit.
Click **Complete Setup**. The wizard creates the admin account, writes the cell identity to `config/api/cell_config.json`, and redirects to the login page.
--- ---
## 6. Set cell identity ## 4. Log in
Go to **Settings** in the sidebar. After the wizard, you are redirected to the login page at `/login`.
Set your: Log in with:
- **Cell name** — a short identifier, e.g. `myhome`
- **Domain** — the TLD your cell will use internally, e.g. `cell`
- **VPN IP range** — the CIDR for WireGuard peers, e.g. `10.0.0.0/24`
After saving, the UI will show a banner asking you to apply the changes. Click **Apply Now**. The containers will restart briefly to pick up the new configuration. - **Username:** `admin`
- **Password:** the password you set in the wizard
--- ---
## 7. Add a WireGuard peer ## 5. Add a WireGuard peer
Go to **WireGuard** in the sidebar. Go to **Peers** in the sidebar.
1. Click **Add Peer**. 1. Click **Add Peer**.
2. Enter a name for the peer (e.g. `laptop`). 2. Enter a peer name (e.g. `laptop`).
3. The API generates a key pair and assigns the next available VPN IP automatically. 3. The API generates a key pair and assigns the next available VPN IP automatically.
4. Click the QR code icon to display the peer config as a QR code. 4. Click the QR code icon to display the peer configuration as a QR code.
5. Scan the QR code with a WireGuard client (Android, iOS, or the WireGuard desktop app). 5. Scan the QR code with a WireGuard client (Android, iOS, or the WireGuard desktop app).
The peer config sets your cell as the DNS server. Once connected, `*.cell` names resolve through the cell's CoreDNS. Once connected, `*.cell` names resolve through the cell's CoreDNS and all traffic can be routed through the cell.
To manage peers from the command line:
```bash
make list-peers
make add-peer PEER_NAME=phone PEER_IP=10.0.0.3 PEER_KEY=<base64-pubkey>
```
--- ---
## 8. Day-to-day operations ## 6. Day-to-day operations
```bash ```bash
# Check container status and API health
make status
# Follow logs from all services # Follow logs from all services
make logs make logs
@@ -142,9 +113,6 @@ make logs-api
make logs-wireguard make logs-wireguard
make logs-caddy make logs-caddy
# Check container status and API health
make status
# Open a shell inside a container # Open a shell inside a container
make shell-api make shell-api
make shell-dns make shell-dns
@@ -152,7 +120,7 @@ make shell-dns
--- ---
## 9. Backup ## 7. Backup
Before making significant changes, create a backup: Before making significant changes, create a backup:
@@ -175,34 +143,47 @@ tar -xzf backups/cell-backup-YYYYMMDD-HHMMSS.tar.gz
make start make start
``` ```
Backup and restore is also available in the UI under **Settings**.
--- ---
## 10. Updating PIC ## 8. Updating PIC
```bash ```bash
make update make update
``` ```
This runs `git pull`, then rebuilds and restarts all containers. If `config/` is missing (e.g. after a fresh clone), it runs `make setup` automatically. This runs `git pull`, then rebuilds and restarts all containers. If `config/` is missing (e.g. after a fresh clone on a new machine with no data), it runs `make setup` first to generate directory structure and base config files.
---
## 9. Uninstalling
```bash
make uninstall
```
This stops and removes all containers, then prompts:
- **y** — also deletes `config/` and `data/` (full wipe; cannot be undone)
- **n** (default) — stops containers but leaves images, `config/`, and `data/` intact; `make start` will bring everything back
The systemd unit (`pic.service`) is disabled and removed either way.
--- ---
## Troubleshooting ## Troubleshooting
**Containers not starting** ### Containers not starting
```bash ```bash
make logs make logs
make logs-api make logs-api
``` ```
Look for errors related to missing config files or port conflicts. Look for errors about missing config files or port conflicts.
**Port 53 already in use** ### Port 53 already in use
On Ubuntu/Debian, `systemd-resolved` listens on port 53. Disable it: On Ubuntu and Debian, `systemd-resolved` listens on port 53. Disable it:
```bash ```bash
sudo systemctl disable --now systemd-resolved sudo systemctl disable --now systemd-resolved
@@ -212,21 +193,33 @@ echo "nameserver 8.8.8.8" | sudo tee /etc/resolv.conf
Then run `make start` again. Then run `make start` again.
**WebDAV container exits immediately** ### WireGuard container fails to load the kernel module
`WEBDAV_PASS` is not set in `.env`. Set it and run `make start` again.
**WireGuard container fails to load kernel module**
Ensure the WireGuard kernel module is available:
```bash ```bash
sudo modprobe wireguard sudo modprobe wireguard
``` ```
On some minimal installs you may need to install `wireguard-tools` and the kernel headers for your running kernel. On minimal installs you may need to install `wireguard-tools` and the kernel headers for the running kernel before `modprobe` succeeds.
**API returns 503 or UI shows "Backend Unavailable"** ### API returns 428 and redirects to /setup
The first-run wizard has not been completed. Open `http://<host-ip>:8081` and finish the wizard.
### API returns 401 or the UI shows "Not authenticated"
Your session has expired or you have not logged in yet. Go to `http://<host-ip>:8081/login`.
### API returns 503 with "Authentication not configured"
The `users` file exists but contains no accounts. This should not happen after a normal wizard completion. To recover:
```bash
make reset-admin-password
```
This generates a new admin password and prints it to the terminal.
### API returns 503 or the UI shows "Backend Unavailable"
The Flask API may still be starting. Wait 10–15 seconds after `make start` and refresh. If it persists: The Flask API may still be starting. Wait 10–15 seconds after `make start` and refresh. If it persists:
@@ -234,6 +227,18 @@ The Flask API may still be starting. Wait 10–15 seconds after `make start` and
make logs-api make logs-api
``` ```
**Config changes not taking effect** ### Config changes not taking effect
After changing identity or service settings in the UI, a yellow banner appears at the top of the page. Click **Apply Now** to restart the affected containers. After changing identity or service settings in the UI, a yellow banner appears. Click **Apply Now** to restart the affected containers with the new configuration.
### Forgot the admin password
```bash
make show-admin-password
```
If that doesn't help (e.g. the setup data file is missing):
```bash
make reset-admin-password
```
README.md
+95 -58
View File
@@ -1,6 +1,6 @@
# Personal Internet Cell (PIC) # Personal Internet Cell (PIC)
PIC is a self-hosted digital infrastructure platform. It manages DNS, DHCP, NTP, WireGuard VPN, email, calendar/contacts (CalDAV), file storage (WebDAV), a reverse proxy, and a certificate authority — all controlled from a single REST API and React web UI. No manual config file editing is required for normal operations. PIC is a self-hosted digital infrastructure platform. It packages DNS, DHCP, NTP, WireGuard VPN, email, calendar/contacts (CalDAV), file storage (WebDAV), a reverse proxy, a certificate authority, and optional third-party services — all managed through a single REST API and a React web UI. No manual config file editing is required for normal operations.
--- ---
@@ -10,33 +10,56 @@ PIC is a self-hosted digital infrastructure platform. It manages DNS, DHCP, NTP,
Browser Browser
└── React SPA (cell-webui :8081) └── React SPA (cell-webui :8081)
└── Flask REST API (cell-api :3000, bound to 127.0.0.1) └── Flask REST API (cell-api :3000, bound to 127.0.0.1)
└── Docker SDK / config files └── Service managers + Docker SDK
├── cell-caddy :80/:443 reverse proxy ├── cell-caddy :80/:443 Caddy reverse proxy (HTTPS/TLS)
├── cell-dns :53 CoreDNS ├── cell-dns :53 CoreDNS
├── cell-dhcp :67/udp dnsmasq ├── cell-dhcp :67/udp dnsmasq
├── cell-ntp :123/udp chrony ├── cell-ntp :123/udp chrony
├── cell-wireguard :51820/udp WireGuard VPN ├── cell-wireguard :51820/udp WireGuard VPN
├── cell-mail :25/:587/:993 Postfix + Dovecot ├── cell-mail :25/:587/:993 Postfix + Dovecot
├── cell-radicale 127.0.0.1:5232 CalDAV/CardDAV ├── cell-radicale 127.0.0.1:5232 CalDAV/CardDAV
├── cell-webdav 127.0.0.1:8080 WebDAV ├── cell-webdav 127.0.0.1:8080 WebDAV
├── cell-rainloop :8888 webmail (RainLoop) ├── cell-rainloop 127.0.0.1:8888 Webmail (RainLoop)
├── cell-filegator :8082 file manager UI ├── cell-filegator 127.0.0.1:8082 File manager (Filegator)
└── cell-webui :8081 React UI (Nginx) └── cell-webui :8081 React UI (Nginx)
``` ```
All containers run on a custom Docker bridge network (`cell-network`, default `172.20.0.0/16`). Static IPs per container are set in `docker-compose.yml` and overridden via `.env`. All containers run on a custom Docker bridge network (`cell-network`, default subnet `172.20.0.0/16`). Static IPs per container are set in `docker-compose.yml` and can be overridden via `.env`.
The Flask API (`api/app.py`, ~2800 lines) contains all REST endpoints, runs a background health-monitoring thread, and manages the entire lifecycle of generated config artefacts: `Caddyfile`, `Corefile`, `wg0.conf`, and `cell_config.json` (the single source of truth at `config/api/cell_config.json`). The Flask API (`api/app.py`) contains REST endpoints and a background health-monitoring thread. Service managers are instantiated as singletons in `api/managers.py`. The single source of truth for runtime configuration is `config/api/cell_config.json`, managed by `ConfigManager`.
The React frontend (`webui/`) is built with Vite + Tailwind CSS. All API calls go through `src/services/api.js` (Axios). Pages: Dashboard, Peers, Network Services, WireGuard, Email, Calendar, Files, Routing, Vault, Containers, Cell Network, Logs, Settings. The React frontend (`webui/`) is built with Vite + Tailwind CSS. All API calls go through `src/services/api.js` (Axios).
**Web UI pages:** Dashboard, Peers, Network Services, WireGuard, Email, Calendar, Files, Routing, Vault, Containers, Cell Network, Connectivity, Service Store, Logs, Settings.
---
## Features
- **First-run wizard** — browser-based setup at `/setup`. On first start, all API requests redirect to `/setup` (HTTP 428) until the wizard is completed. Sets cell name, domain mode, timezone, admin password, and initial services. No manual `.env` editing required for identity.
- **Session-based auth** — admin and peer roles. All `/api/*` endpoints require an authenticated session after setup. CSRF protection on all state-changing requests.
- **WireGuard VPN** — peer lifecycle management, automatic key generation, QR code config export, per-peer routing policy.
- **Caddy HTTPS** — automatic TLS via Let's Encrypt (DNS-01 or HTTP-01) or an internal CA, depending on domain mode.
- **DDNS (pic.ngo)** — registers a `<cell-name>.pic.ngo` subdomain. Supported providers: `pic_ngo`, `cloudflare`, `duckdns`, `noip`, `freedns`. A background thread re-publishes the public IP every 5 minutes.
- **Service store** — install/remove optional third-party services from the `pic-services` index at `git.pic.ngo`. Manifests declare container images, Caddy routes, and iptables rules.
- **Extended connectivity** — per-peer egress routing through alternate exits: WireGuard external, OpenVPN, or Tor. Configured via policy routing (fwmark + ip rule) in the WireGuard container.
- **Cell-to-cell networking** — WireGuard-based site-to-site links between PIC cells with service-level access control (calendar, files, mail, WebDAV) and a peer-sync protocol.
- **Certificate authority** — `vault_manager` issues and revokes TLS certificates for internal services.
- **Network services** — CoreDNS (`.cell` TLD), dnsmasq DHCP, chrony NTP.
- **Email** — Postfix + Dovecot via `docker-mailserver`.
- **Calendar/contacts** — Radicale CalDAV/CardDAV.
- **File storage** — WebDAV with per-user accounts; Filegator for browser-based file management.
- **Container manager** — start/stop/inspect containers, pull images, manage volumes via the Docker SDK.
- **Firewall manager** — iptables rule management (`firewall_manager.py`).
- **Structured logging** — JSON logs with rotation (5 MB / 5 backups per service), log search, and per-service verbosity control.
--- ---
## Requirements ## Requirements
- Linux host with the WireGuard kernel module loaded - Linux host with the WireGuard kernel module loaded (`modprobe wireguard` to verify)
- Docker Engine and Docker Compose (v2 plugin or v1 standalone) - Docker Engine and Docker Compose (v2 plugin or v1 standalone)
- Python 3.10+ (for `make setup` and local dev only; not needed at runtime) - Python 3.10+ (for `make setup` and local development; not needed at runtime)
- 2 GB+ RAM, 10 GB+ disk - 2 GB+ RAM, 10 GB+ disk
- Ports available: 53, 67/udp, 80, 443, 51820/udp, 25, 587, 993 - Ports available: 53, 67/udp, 80, 443, 51820/udp, 25, 587, 993
@@ -44,62 +67,77 @@ The React frontend (`webui/`) is built with Vite + Tailwind CSS. All API calls g
## Quick Start ## Quick Start
See [QUICKSTART.md](QUICKSTART.md) for step-by-step setup. See [QUICKSTART.md](QUICKSTART.md) for step-by-step instructions.
The short version:
```bash
git clone gitea@192.168.31.50:roof/pic.git pic
cd pic
make start
# open http://<host-ip>:8081 — the setup wizard appears automatically
```
--- ---
## Configuration ## Configuration
Runtime configuration is controlled by `.env` in the project root. Copy `.env.example` to `.env` before first run. Port assignments and container IPs are configured in `.env` in the project root. A `.env` file is not required for first start — all variables have defaults. Create one only if you need to change ports or container IPs.
| Variable | Default | Description | | Variable | Default | Description |
|---|---|---| |---|---|---|
| `CELL_NETWORK` | `172.20.0.0/16` | Docker bridge subnet for all containers | | `CELL_NETWORK` | `172.20.0.0/16` | Docker bridge subnet |
| `CADDY_IP` through `FILEGATOR_IP` | `172.20.0.2`–`.13` | Static IP for each container | | `CADDY_IP` through `FILEGATOR_IP` | `172.20.0.2`–`.13` | Static IP per container |
| `DNS_PORT` | `53` | DNS (UDP+TCP) | | `DNS_PORT` | `53` | DNS (UDP + TCP) |
| `DHCP_PORT` | `67` | DHCP (UDP) | | `DHCP_PORT` | `67` | DHCP (UDP) |
| `NTP_PORT` | `123` | NTP (UDP) | | `NTP_PORT` | `123` | NTP (UDP) |
| `WG_PORT` | `51820` | WireGuard listen port (UDP) | | `WG_PORT` | `51820` | WireGuard listen port (UDP) |
| `API_PORT` | `3000` | Flask API (bound to `127.0.0.1`) | | `API_PORT` | `3000` | Flask API (127.0.0.1 only) |
| `WEBUI_PORT` | `8081` | React UI | | `WEBUI_PORT` | `8081` | React UI |
| `MAIL_SMTP_PORT` | `25` | SMTP | | `MAIL_SMTP_PORT` | `25` | SMTP |
| `MAIL_SUBMISSION_PORT` | `587` | SMTP submission | | `MAIL_SUBMISSION_PORT` | `587` | SMTP submission |
| `MAIL_IMAP_PORT` | `993` | IMAP | | `MAIL_IMAP_PORT` | `993` | IMAP |
| `RADICALE_PORT` | `5232` | CalDAV (bound to `127.0.0.1`) | | `RADICALE_PORT` | `5232` | CalDAV (127.0.0.1 only) |
| `WEBDAV_PORT` | `8080` | WebDAV (bound to `127.0.0.1`) | | `WEBDAV_PORT` | `8080` | WebDAV (127.0.0.1 only) |
| `RAINLOOP_PORT` | `8888` | Webmail | | `RAINLOOP_PORT` | `8888` | Webmail |
| `FILEGATOR_PORT` | `8082` | File manager UI | | `FILEGATOR_PORT` | `8082` | File manager UI |
| `WEBDAV_USER` | `admin` | WebDAV basic-auth username | | `WEBDAV_USER` | `admin` | WebDAV basic-auth username |
| `WEBDAV_PASS` | _(required)_ | WebDAV basic-auth password — must be set before `make start` | | `WEBDAV_PASS` | _(unset)_ | WebDAV basic-auth password |
| `FLASK_DEBUG` | _(unset)_ | Set to `1` to enable Flask debug mode; do not use in production | | `FLASK_DEBUG` | _(unset)_ | Set to `1` for Flask debug mode; do not use in production |
| `PUID` / `PGID` | current user | UID/GID passed to the WireGuard container | | `PUID` / `PGID` | current user | UID/GID passed to the WireGuard container |
Cell identity (cell name, domain, VPN IP range) is configured via `make setup` or the Settings → Identity page in the UI after startup. The VPN IP range must be an RFC-1918 CIDR (`10.0.0.0/8`, `172.16.0.0/12`, or `192.168.0.0/16`); the API and UI both enforce this. Cell identity (cell name, domain mode, timezone) is set through the first-run wizard on first start, or later through the Settings page in the UI.
--- ---
## Security Notes ## Security
**Ports exposed to the network:** **Ports exposed on all interfaces by default:**
- `80` / `443` — Caddy (HTTP/HTTPS reverse proxy) - `80` / `443` — Caddy (HTTP/HTTPS reverse proxy)
- `51820/udp` — WireGuard - `51820/udp` — WireGuard
- `25` / `587` / `993` — Mail (SMTP, submission, IMAP) - `25` / `587` / `993` — mail
- `53` — DNS (UDP + TCP) - `53` — DNS
- `67/udp` — DHCP - `67/udp` — DHCP
- `8081` — Web UI - `8081` — Web UI
- `8888` — Webmail (RainLoop)
- `8082` — File manager (Filegator)
**Ports bound to `127.0.0.1` only** (not directly reachable from the network): **Ports bound to `127.0.0.1` only:**
- `3000` — Flask API - `3000` — Flask API
- `5232` — Radicale (CalDAV) - `5232` — Radicale (CalDAV)
- `8080` — WebDAV - `8080` — WebDAV
- `8888` — Webmail
- `8082` — Filegator
The API has no authentication layer. It relies on `is_local_request()` to restrict sensitive endpoints (containers, vault) to requests originating from loopback or the cell's Docker network. The Docker socket is mounted into `cell-api`; treat access to port 3000 as equivalent to root access on the host. The API uses session-based authentication (admin and peer roles). The Docker socket is mounted into `cell-api`; treat access to port 3000 as equivalent to root access on the host.
For internet-facing deployments, place the host behind a firewall or VPN and restrict access to the API and UI ports. Before setup is complete, all `/api/*` requests except `/api/setup/*` and `/health` return HTTP 428 and a redirect to `/setup`.
CSRF protection (double-submit token in `X-CSRF-Token` header) applies to all `POST`, `PUT`, `DELETE`, and `PATCH` requests on `/api/*` once a user session exists, except `/api/auth/*` and `/api/setup/*`.
Cell-to-cell peer-sync endpoints (`/api/cells/peer-sync/*`) authenticate via source IP and WireGuard public key, not session cookies.
For internet-facing deployments, place the host behind a firewall and restrict access to the API and UI ports.
--- ---
@@ -123,7 +161,7 @@ cd webui && npm install && npm run dev
# Follow all container logs # Follow all container logs
make logs make logs
# Follow logs for one service (e.g. api, dns, caddy, wireguard, mail) # Follow logs for one service
make logs-api make logs-api
# Open a shell inside a container # Open a shell inside a container
@@ -135,41 +173,38 @@ make shell-api
## Testing ## Testing
```bash ```bash
make test # run the full pytest suite make test # run all unit tests (pytest, excludes e2e and integration)
make test-coverage # run with coverage; HTML report in htmlcov/ make test-coverage # run with coverage; HTML report in htmlcov/
make test-api # run API endpoint tests only
``` ```
Tests live in `tests/` (34 files, 642 test functions). Coverage includes: Tests live in `tests/`. Integration tests require a running stack:
- All service managers (network, WireGuard, email, calendar, file, routing, vault, container)
- API endpoint tests for each service area
- Config manager (CRUD, validation, backup/restore)
- IP utilities and Caddyfile generation
- Peer registry and WireGuard peer lifecycle
- Service bus pub/sub
- Firewall manager
- Pending-restart logic
Integration tests (`tests/integration/`) require a running PIC stack:
```bash ```bash
make test-integration # full suite (creates peers) make test-integration # full suite (creates peers, modifies state)
make test-integration-readonly # read-only checks, safe to run anytime make test-integration-readonly # read-only checks, safe to run anytime
``` ```
End-to-end tests use Playwright:
```bash
make test-e2e-deps # install Playwright and dependencies (run once)
make test-e2e-api # API-level e2e tests
make test-e2e-ui # UI-level e2e tests
```
--- ---
## Management Commands ## Management Commands
```bash ```bash
make setup # generate WireGuard keys, write configs, create data dirs make start # docker compose up -d --build (full profile)
make start # docker compose up -d --build
make stop # docker compose down make stop # docker compose down
make restart # docker compose restart make restart # docker compose restart
make status # container status + API health check make status # container status + API health check
make logs # follow all service logs make logs # follow all service logs
make logs-<svc> # follow logs for one service make logs-<svc> # follow logs for one service (e.g. make logs-api)
make shell-<svc> # shell inside a container make shell-<svc> # shell inside a container (e.g. make shell-api)
make update # git pull + rebuild + restart make update # git pull + rebuild + restart
make reinstall # full wipe of config/ and data/, then setup + start make reinstall # full wipe of config/ and data/, then setup + start
@@ -180,7 +215,9 @@ make restore # list available backups
make list-peers # show WireGuard peers via API make list-peers # show WireGuard peers via API
make show-routes # wg show inside the wireguard container make show-routes # wg show inside the wireguard container
make add-peer PEER_NAME=foo PEER_IP=10.0.0.5 PEER_KEY=<pubkey>
make show-admin-password # print current admin password
make reset-admin-password # generate and set a new random admin password
``` ```
--- ---