Compare commits
19 Commits
8efe8c1225
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
 roof
|
925ab1f696 | ||
|
roof
|
439886624e | ||
|
roof
|
24877df976 | ||
|
roof
|
bfa0d99dd1 | ||
|
roof
|
1e2cf5580f | ||
|
roof
|
1989dfa0a3 | ||
|
roof
|
5dab6377bc | ||
|
roof
|
0a24d20bbc | ||
|
roof
|
46599bd37e | ||
|
roof
|
dde4d9a53f | ||
|
roof
|
674a66f7a0 | ||
|
roof
|
9df3bf6a17 | ||
|
roof
|
0773179962 | ||
|
roof
|
3a35cf72d3 | ||
|
roof
|
515f3d5075 | ||
|
roof
|
35993bc79d | ||
|
roof
|
f1b48208fc | ||
|
roof
|
ffe1dbeed6 | ||
|
roof
|
15376b67c7 |
+7
-1
@@ -21,8 +21,10 @@ config/api/caddy/Caddyfile
|
||||
config/api/calendar.json
|
||||
config/api/cell_config.json
|
||||
config/api/wireguard.json
|
||||
config/api/webdav/webdav.conf
|
||||
config/api/webdav/
|
||||
config/api/dhcp/
|
||||
config/api/dns/
|
||||
config/api/network.json
|
||||
config/caddy/Caddyfile
|
||||
config/dhcp/dnsmasq.conf
|
||||
config/dns/Corefile
|
||||
@@ -85,3 +87,7 @@ backups/
|
||||
# Temporary files
|
||||
*.tmp
|
||||
*.temp
|
||||
|
||||
# Coverage data
|
||||
.coverage
|
||||
htmlcov/
|
||||
@@ -1,87 +1,282 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
This file is the primary context source for Claude Code in this repository. Read it fully before touching any code.
|
||||
|
||||
## What This Project Is
|
||||
---
|
||||
|
||||
**Personal Internet Cell (PIC)** — a self-hosted digital infrastructure platform. It manages DNS, DHCP, NTP, WireGuard VPN, email, calendar/contacts (CalDAV), file storage (WebDAV), reverse proxy (Caddy), a certificate authority, and container orchestration, all from a single API + React UI.
|
||||
## Project Overview
|
||||
|
||||
## Common Commands
|
||||
**Personal Internet Cell (PIC)** is a self-hosted digital infrastructure platform for individuals who want full ownership of their core internet services without relying on cloud providers.
|
||||
|
||||
```bash
|
||||
# Full stack
|
||||
make start # docker-compose up -d
|
||||
make stop # docker-compose down
|
||||
make restart # docker-compose restart
|
||||
make status # docker status + API health
|
||||
make logs # docker-compose logs -f
|
||||
make build # rebuild api image
|
||||
A PIC instance runs DNS, DHCP, NTP, WireGuard VPN, email (SMTP/IMAP), calendar/contacts (CalDAV/CardDAV), file storage (WebDAV), HTTPS reverse proxy (Caddy), an internal certificate authority, and optional third-party services — all managed from a single REST API and a React web UI. No manual config-file editing is required for normal operations.
|
||||
|
||||
# Tests
|
||||
make test # pytest tests/ api/tests/
|
||||
make test-coverage # pytest with coverage HTML report
|
||||
make test-api # pytest tests/test_api_endpoints.py
|
||||
pytest tests/test_<module>.py # single test file
|
||||
**Primary users:** technically capable individuals, homelab operators, small families or teams.
|
||||
|
||||
# Local dev (no Docker)
|
||||
pip install -r api/requirements.txt
|
||||
python api/app.py # Flask API on :3000
|
||||
**What the product optimizes for:**
|
||||
- One-command install, browser-based first-run wizard, no manual `.env` editing for identity
|
||||
- Everything managed through the API and UI — the user should never need to `ssh` for day-to-day operations
|
||||
- Security by default: session auth, CSRF protection, WireGuard isolation, internal CA, no open API port
|
||||
- Reliability and observability: structured logs, health monitoring, automated config backups
|
||||
|
||||
cd webui && npm install && npm run dev # React UI on :5173 (proxies API to :3000)
|
||||
**Key constraints:**
|
||||
- Runs on a single Linux host with Docker; no Kubernetes, no swarm
|
||||
- Must work on Debian, Ubuntu, Fedora, RHEL, and Alpine
|
||||
- The Flask API must never be exposed directly; Caddy always proxies it
|
||||
- All secrets live in `data/` (git-ignored), never in the repo
|
||||
|
||||
# WireGuard
|
||||
make show-routes
|
||||
make add-peer PEER_NAME=foo PEER_IP=10.0.0.5 PEER_KEY=<pubkey>
|
||||
make list-peers
|
||||
```
|
||||
---
|
||||
|
||||
## Tech Stack
|
||||
|
||||
### Backend
|
||||
- **Python 3.11** — Flask REST API (`api/app.py`)
|
||||
- **Flask** — routing, sessions, before-request hooks (enforce_setup, enforce_auth, check_csrf)
|
||||
- **bcrypt** — password hashing in `AuthManager`
|
||||
- **Docker SDK for Python** — container lifecycle in `ContainerManager`
|
||||
- **PyNaCl / Age** — encryption in `VaultManager`
|
||||
- **pyotp** — TOTP for DDNS registration
|
||||
|
||||
### Frontend
|
||||
- **React 18** — SPA
|
||||
- **Vite** — dev server and build (proxies `/api` → `:3000`)
|
||||
- **Tailwind CSS** — all styling; no custom CSS files
|
||||
- **Axios** — all API calls go through `src/services/api.js`
|
||||
|
||||
### Infrastructure
|
||||
- **Docker Compose** — all 12+ service containers
|
||||
- **Caddy** — reverse proxy, TLS termination (Let's Encrypt DNS-01 or HTTP-01 or internal CA)
|
||||
- **CoreDNS** — `.cell` TLD authoritative DNS
|
||||
- **dnsmasq** — DHCP
|
||||
- **chrony** — NTP
|
||||
- **WireGuard** — VPN (kernel module, not userspace)
|
||||
- **Postfix + Dovecot** — email via `docker-mailserver`
|
||||
- **Radicale** — CalDAV/CardDAV
|
||||
- **PowerDNS** — authoritative DNS on the DDNS VPS (separate repo: `pic-ddns`)
|
||||
|
||||
### CI/CD
|
||||
- **Gitea Actions** — unit tests on every push, image builds on tag
|
||||
- **act_runner** — self-hosted runner on pic0 (192.168.31.51)
|
||||
- **Gitea Container Registry** — images pushed to `git.pic.ngo`
|
||||
|
||||
Do not introduce: Redux, styled-components, SQLAlchemy, Celery, or any async framework (asyncio/FastAPI) into the main API unless explicitly requested.
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
### Backend (`api/`)
|
||||
```
|
||||
Browser / WireGuard peer
|
||||
└── Caddy (:80/:443) TLS termination, reverse proxy
|
||||
└── React SPA (:8081) Vite + Tailwind (Nginx in container)
|
||||
└── Flask API (:3000) REST API, bound to 127.0.0.1 only
|
||||
├── NetworkManager CoreDNS, dnsmasq, chrony
|
||||
├── WireGuardManager WireGuard peer lifecycle
|
||||
├── PeerRegistry peer registration and trust
|
||||
├── EmailManager Postfix + Dovecot
|
||||
├── CalendarManager Radicale CalDAV/CardDAV
|
||||
├── FileManager WebDAV + Filegator
|
||||
├── RoutingManager iptables NAT and routing
|
||||
├── FirewallManager iptables INPUT/FORWARD rules
|
||||
├── VaultManager internal CA, TLS certs, Age encryption
|
||||
├── ContainerManager Docker SDK
|
||||
├── CellLinkManager site-to-site WireGuard links
|
||||
├── ConnectivityManager per-peer exit routing (WG ext, OpenVPN, Tor)
|
||||
├── DDNSManager dynamic DNS heartbeat
|
||||
├── ServiceStoreManager optional service install/remove
|
||||
├── CaddyManager Caddyfile generation and reload
|
||||
├── AuthManager bcrypt passwords, session auth, RBAC
|
||||
└── SetupManager first-run wizard state
|
||||
```
|
||||
|
||||
All service managers inherit `BaseServiceManager` (`api/base_service_manager.py`). This enforces a consistent interface: `get_status()`, `get_config()`, `update_config()`, `validate_config()`, `test_connectivity()`, `get_logs()`, `restart_service()`. When adding or modifying a service manager, follow this pattern.
|
||||
### Key files
|
||||
|
||||
The `ServiceBus` (`api/service_bus.py`) is a pub/sub event system used for inter-service communication. Services publish events (e.g., `SERVICE_STARTED`, `CONFIG_CHANGED`, `PEER_CONNECTED`) and subscribe to events from dependencies. Dependency graph is declared in the bus — e.g., `wireguard` depends on `network`; `email` depends on `network` and `vault`.
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| `api/app.py` | Flask app, all REST endpoints, before-request hooks, health monitor thread |
|
||||
| `api/managers.py` | Singleton instantiation of all service managers |
|
||||
| `api/base_service_manager.py` | Abstract base class: `get_status`, `get_config`, `update_config`, `validate_config`, `test_connectivity`, `get_logs`, `restart_service` |
|
||||
| `api/config_manager.py` | Single source of truth for `cell_config.json` — all read/write goes through here |
|
||||
| `api/service_bus.py` | Pub/sub event system between managers |
|
||||
| `webui/src/services/api.js` | Axios API client — all UI→API calls |
|
||||
| `docker-compose.yml` | Container definitions and network topology |
|
||||
| `Makefile` | All operational commands |
|
||||
| `install.sh` | Bash installer served via `https://install.pic.ngo` |
|
||||
|
||||
`ConfigManager` (`api/config_manager.py`) is the single source of truth. Config lives in `/app/config/cell_config.json` (mapped from `config/api/`). All managers read/write through ConfigManager, which validates against per-service schemas and maintains automatic backups.
|
||||
### Directory layout
|
||||
|
||||
`LogManager` (`api/log_manager.py`) provides structured JSON logging with rotation (5 MB / 5 backups per service). Use it instead of `print()` or raw `logging`.
|
||||
```
|
||||
api/ Flask API and all service managers
|
||||
webui/ React SPA (Vite + Tailwind)
|
||||
tests/ pytest unit tests (no running services required)
|
||||
tests/integration/ require a running PIC stack
|
||||
tests/e2e/ Playwright UI and WireGuard e2e tests
|
||||
config/ Runtime config per service (mostly git-ignored)
|
||||
data/ Runtime secrets and state (fully git-ignored)
|
||||
scripts/ Setup and maintenance scripts
|
||||
install.sh One-line installer entry point
|
||||
Makefile All make targets
|
||||
docker-compose.yml
|
||||
```
|
||||
|
||||
`app.py` (2000+ lines) contains all Flask REST endpoints, organized by service. It runs a background health-monitoring thread.
|
||||
### Config and secrets
|
||||
|
||||
Service managers:
|
||||
- `network_manager.py` — DNS (CoreDNS), DHCP (dnsmasq), NTP (chrony)
|
||||
- `wireguard_manager.py` — VPN peer lifecycle, QR codes
|
||||
- `peer_registry.py` — peer registration/lookup
|
||||
- `routing_manager.py` — NAT, firewall rules, VPN gateway
|
||||
- `vault_manager.py` — internal certificate authority
|
||||
- `email_manager.py` — Postfix + Dovecot
|
||||
- `calendar_manager.py` — Radicale CalDAV/CardDAV
|
||||
- `file_manager.py` — WebDAV storage
|
||||
- `container_manager.py` — Docker SDK wrappers
|
||||
- `cell_manager.py` — top-level orchestration
|
||||
- Runtime config: `config/api/cell_config.json` — managed by `ConfigManager`, never edit directly
|
||||
- Secrets and user data: `data/` — git-ignored, contains `auth_users.json`, WireGuard keys, DDNS token, CA key
|
||||
- DDNS config lives under the top-level `ddns` key in `cell_config.json`, accessed via `config_manager.configs.get('ddns', {})`
|
||||
- Do not read `_identity.domain` expecting a dict — it is a plain string (the domain mode, e.g. `"pic_ngo"`)
|
||||
|
||||
### Frontend (`webui/`)
|
||||
### Before-request hooks (app.py)
|
||||
|
||||
React 18 + Vite + Tailwind CSS. All API calls go through `src/services/api.js` (Axios). Vite dev server proxies `/api` to `localhost:3000`. Pages in `src/pages/`, shared components in `src/components/`.
|
||||
Three hooks run on every request in this order:
|
||||
1. `enforce_setup` — returns 428 for all `/api/*` except `/api/setup/*` and `/health` until setup is complete. Skipped when `app.config['TESTING']` is True.
|
||||
2. `enforce_auth` — returns 401 if no session; returns 503 if users file exists but is empty (misconfiguration). Skipped when `app.config['TESTING']` is True.
|
||||
3. `check_csrf` — requires `X-CSRF-Token` header on all mutating requests except `/api/auth/*` and `/api/setup/*`.
|
||||
|
||||
### Infrastructure
|
||||
---
|
||||
|
||||
`docker-compose.yml` defines 13 services on a custom bridge network `cell-network` (172.20.0.0/16). Cell IPs default to 10.0.0.0/24. Key ports: 53 (DNS), 80/443 (Caddy), 3000 (API), 5173/8081 (WebUI), 51820/udp (WireGuard), 25/587/993 (mail), 5232 (CalDAV), 8080 (WebDAV).
|
||||
## Coding Conventions
|
||||
|
||||
Config files for each service live under `config/<service>/`. Persistent data is under `data/` (git-ignored). WireGuard configs are also git-ignored.
|
||||
### Python (API)
|
||||
|
||||
## Testing
|
||||
- All managers inherit `BaseServiceManager` — always implement all abstract methods
|
||||
- Use `self.logger` (from `BaseServiceManager`) — never `print()` or raw `logging`
|
||||
- Config reads go through `self.config_manager` — never open `cell_config.json` directly
|
||||
- Use `threading.RLock` for shared state; managers run in a multi-threaded Flask app
|
||||
- Do not use `any` typing; be explicit
|
||||
- Keep Flask route handlers thin — business logic belongs in the manager, not in `app.py`
|
||||
- Error responses must be JSON: `jsonify({'error': '...'}), <status_code>`
|
||||
- Do not catch bare `Exception` and silently swallow it — log at minimum
|
||||
|
||||
Tests live in `tests/` (28 files). Use mocking (`pytest-mock`) for external system calls. Integration tests in `test_integration.py` require Docker services running.
|
||||
### JavaScript (webui)
|
||||
|
||||
## AI Collaboration Rules (Claude Code)
|
||||
- All API calls go through `src/services/api.js` — never use `fetch` or a new Axios instance directly
|
||||
- Use functional components; no class components
|
||||
- Tailwind utilities only — no inline styles, no custom CSS files
|
||||
- Keep page components in `src/pages/`, reusable UI in `src/components/`
|
||||
- State: local `useState`/`useEffect` is fine; no Redux or global state library
|
||||
|
||||
### General
|
||||
|
||||
- No comments that describe *what* the code does — only *why* if non-obvious
|
||||
- No dead code, no commented-out blocks
|
||||
- No backwards-compat shims for things being removed
|
||||
- Prefer editing existing files over creating new ones
|
||||
- Tests that write to disk: mock `builtins.open` with `OSError` rather than relying on `/nonexistent/path` (CI runs as root and can create any path)
|
||||
|
||||
---
|
||||
|
||||
## Testing and Quality
|
||||
|
||||
Before considering any task complete:
|
||||
1. Run `make test` — all 1500+ unit tests must pass
|
||||
2. Fix failures before committing — the pre-commit hook will block the commit anyway
|
||||
|
||||
### Rules
|
||||
|
||||
- Use `unittest.mock` / `pytest-mock` for all Docker, filesystem, and subprocess calls
|
||||
- Tests must pass in CI (rootless environment where filesystem assumptions don't hold)
|
||||
- When testing write-failure paths, mock `builtins.open` with `side_effect=OSError` — do not rely on unwritable paths
|
||||
- Integration tests (`tests/integration/`) require a running stack — exclude from CI with `--ignore=tests/integration`
|
||||
- E2e tests (`tests/e2e/`) require Playwright — exclude from CI with `--ignore=tests/e2e`
|
||||
- Add tests for any new API endpoint, manager method, or utility function
|
||||
- Do not add tests for Flask routing boilerplate or trivial getters — test behaviour, not structure
|
||||
|
||||
---
|
||||
|
||||
## File Placement Rules
|
||||
|
||||
| New thing | Where it goes |
|
||||
|---|---|
|
||||
| New service manager | `api/<name>_manager.py`, registered in `api/managers.py` and wired into `app.py` |
|
||||
| New API endpoints | `app.py` — grouped with the relevant manager's existing endpoints |
|
||||
| New React page | `webui/src/pages/` |
|
||||
| Reusable UI component | `webui/src/components/` |
|
||||
| New pytest test file | `tests/test_<module>.py` |
|
||||
| Operational script | `scripts/` |
|
||||
| Documentation | Update `README.md`, `QUICKSTART.md`, or `Personal Internet Cell – Project Wiki.md` as appropriate |
|
||||
|
||||
Do not create a new abstraction for a single use case. Do not create near-duplicate files — edit the existing one.
|
||||
|
||||
---
|
||||
|
||||
## Safety Rules
|
||||
|
||||
- **Never expose the Flask API port (3000) directly** — it must always be behind Caddy
|
||||
- **Never commit secrets** — `data/`, `.env`, `*.key`, `*.pem` are all git-ignored; keep it that way
|
||||
- **Do not modify `enforce_setup` or `enforce_auth` hooks** without understanding the full auth flow — these are the security boundary
|
||||
- **Do not change the `cell_config.json` schema** without updating `ConfigManager` validation and all manager reads
|
||||
- **Do not rename API route paths** without checking the webui `api.js` client and any external callers
|
||||
- **Do not modify WireGuard key generation** — losing the server private key means all peers must be re-provisioned
|
||||
- Flag any change to auth flow, CSRF logic, or session management as security-sensitive before implementing
|
||||
|
||||
---
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
# Stack lifecycle (always use make — never call docker/docker-compose directly)
|
||||
make start # build and start all containers
|
||||
make stop # stop all containers
|
||||
make restart # restart containers
|
||||
make status # container status + API health check
|
||||
make logs # follow all container logs
|
||||
make logs-api # follow API logs only
|
||||
make logs-caddy # follow Caddy logs
|
||||
make shell-api # shell inside the API container
|
||||
make build-api # rebuild API image after code change
|
||||
make build-webui # rebuild webui image after code change
|
||||
|
||||
# Tests
|
||||
make test # pytest tests/ --ignore=tests/e2e --ignore=tests/integration
|
||||
make test-coverage # coverage report in htmlcov/
|
||||
pytest tests/test_<module>.py -v # single test file
|
||||
|
||||
# Local dev (no Docker)
|
||||
pip install -r api/requirements.txt
|
||||
python3 api/app.py # Flask API on :3000
|
||||
|
||||
cd webui && npm install && npm run dev # React UI on :5173 (proxies /api → :3000)
|
||||
|
||||
# Peer / WireGuard
|
||||
make list-peers
|
||||
make show-routes
|
||||
|
||||
# Admin password
|
||||
make show-admin-password
|
||||
make reset-admin-password
|
||||
|
||||
# Backup / restore
|
||||
make backup
|
||||
make restore
|
||||
|
||||
# Maintenance
|
||||
make update # git pull + rebuild + restart
|
||||
make uninstall # stop containers; prompt to delete config/ and data/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Infrastructure Topology
|
||||
|
||||
| Machine | IP | Role |
|
||||
|---|---|---|
|
||||
| pic0 | 192.168.31.51 | Dev machine — you are here. Run all commands directly. |
|
||||
| pic1 | 192.168.31.52 | Test/staging PIC instance |
|
||||
| Gitea | 192.168.31.50 | Self-hosted git server (`gitea@192.168.31.50:roof/pic.git`) |
|
||||
| DDNS VPS | 192.168.31.101 (LAN) / 178.168.15.65 (public) | PowerDNS + FastAPI for `*.pic.ngo` DDNS |
|
||||
|
||||
The `roof` user on pic0 has passwordless sudo and is in the `docker` group — use both freely.
|
||||
|
||||
---
|
||||
|
||||
## AI Collaboration Rules
|
||||
|
||||
These rules apply to every Claude Code session in this repo:
|
||||
|
||||
- **Read memory first** — load `/home/roof/.claude/projects/-home-roof/memory/MEMORY.md` and referenced files at session start.
|
||||
- **Dev machine context** — you are already on pic0 (192.168.31.51), the dev machine. Execute commands here directly; do not ask the user to run them.
|
||||
- **Use all available agents** — spawn specialized sub-agents (pic-remote, pic-qa, pic-architect, etc.) for tasks that match their description.
|
||||
- **make is the only interface** — never call docker/docker-compose directly. All container lifecycle operations go through `make start`, `make stop`, `make build`, `make logs`, etc.
|
||||
- **Test every new feature** — after implementing any change, run `make test` before considering the task done.
|
||||
- **Test before commit** — the pre-commit hook enforces this, but run `make test` manually first and fix all failures before staging files.
|
||||
- **Read memory first** — load `/home/roof/.claude/projects/-home-roof/memory/MEMORY.md` at session start; follow referenced memory files for relevant context.
|
||||
- **You are on pic0** — execute commands directly here; do not ask the user to run them.
|
||||
- **`make` is the only container interface** — never call `docker` or `docker-compose` directly. All container lifecycle goes through `make start`, `make stop`, `make build`, `make logs`, etc.
|
||||
- **Use specialized agents** — spawn `pic-remote` for VPS/pic1 SSH tasks, `pic-qa` for test writing, `pic-architect` for design decisions, `pic-designer` for UI review, `pic-devops` for docker-compose/Makefile changes, `pic-writer` for documentation.
|
||||
- **Test before commit** — run `make test` and fix all failures before staging. The pre-commit hook enforces this, but run it manually first.
|
||||
- **No skipping hooks** — never use `--no-verify` unless the only change is documentation or a workflow file with no Python/JS.
|
||||
- **Commits need context** — write commit messages that explain *why*, not just *what*. Always add the Co-Authored-By trailer.
|
||||
|
||||
@@ -12,7 +12,8 @@
|
||||
test-e2e-deps test-e2e-api test-e2e-ui test-e2e-wg test-e2e \
|
||||
reset-test-admin-pass \
|
||||
show-admin-password reset-admin-password \
|
||||
show-routes add-peer list-peers
|
||||
show-routes add-peer list-peers \
|
||||
ddns-update ddns-register
|
||||
|
||||
# Detect docker compose command (v2 plugin preferred, fallback to v1 standalone)
|
||||
DC := $(shell docker compose version >/dev/null 2>&1 && echo "docker compose" || echo "docker-compose")
|
||||
@@ -78,7 +79,7 @@ check-deps:
|
||||
|
||||
setup: check-deps
|
||||
@echo "Setting up Personal Internet Cell..."
|
||||
@sudo chown -R $$(id -u):$$(id -g) config/ data/ 2>/dev/null || true
|
||||
@sudo chown -R $${SUDO_USER:-$$(id -un)}:$${SUDO_USER:-$$(id -un)} config/ data/ 2>/dev/null || true
|
||||
CELL_NAME=$(or $(CELL_NAME),mycell) \
|
||||
CELL_DOMAIN=$(or $(CELL_DOMAIN),cell) \
|
||||
VPN_ADDRESS=$(or $(VPN_ADDRESS),10.0.0.1/24) \
|
||||
@@ -130,6 +131,7 @@ shell-%:
|
||||
|
||||
update:
|
||||
@echo "Pulling latest code..."
|
||||
@git config --global --add safe.directory $$(pwd) 2>/dev/null || true
|
||||
@git stash --include-untracked --quiet 2>/dev/null || true
|
||||
git pull
|
||||
@git stash pop --quiet 2>/dev/null || true
|
||||
@@ -335,6 +337,21 @@ add-peer:
|
||||
echo "Usage: make add-peer PEER_NAME=name PEER_IP=10.0.0.x PEER_KEY=<pubkey>"; \
|
||||
fi
|
||||
|
||||
# ── DDNS ─────────────────────────────────────────────────────────────────────
|
||||
|
||||
ddns-update:
|
||||
@python3 scripts/ddns_update.py
|
||||
|
||||
ddns-register:
|
||||
@DDNS_TOTP_SECRET="$(DDNS_TOTP_SECRET)" python3 -c "\
|
||||
import os, sys; sys.path.insert(0, 'scripts'); \
|
||||
from setup_cell import register_with_ddns, _read_existing_ip_range; \
|
||||
import json; \
|
||||
cfg = json.load(open('config/api/cell_config.json')) if os.path.exists('config/api/cell_config.json') else {}; \
|
||||
name = cfg.get('_identity', {}).get('cell_name', os.environ.get('CELL_NAME', 'mycell')); \
|
||||
import os; os.remove('data/api/.ddns_token') if os.path.exists('data/api/.ddns_token') else None; \
|
||||
register_with_ddns(name)"
|
||||
|
||||
# ── Dev ───────────────────────────────────────────────────────────────────────
|
||||
|
||||
dev:
|
||||
|
||||
@@ -1,535 +1,343 @@
|
||||
# Personal Internet Cell – Project Wiki
|
||||
|
||||
## 🌟 Overview
|
||||
## Overview
|
||||
|
||||
Personal Internet Cell is a **production-grade, self-hosted, decentralized digital infrastructure** solution designed to provide individuals with full control over their digital services and data. The project has evolved from a phase-based implementation to a **unified, enterprise-ready system** with modern architecture, comprehensive testing, and production-grade features.
|
||||
Personal Internet Cell (PIC) is a self-hosted digital infrastructure platform. It runs DNS, DHCP, NTP, WireGuard VPN, email, calendar/contacts, file storage, HTTPS reverse proxy, a certificate authority, and optional services — all managed from a single REST API and React web UI.
|
||||
|
||||
## 📋 Table of Contents
|
||||
|
||||
1. [Project Goals](#project-goals)
|
||||
2. [Architecture & Components](#architecture--components)
|
||||
3. [Service Manager Architecture](#service-manager-architecture)
|
||||
4. [Core Services](#core-services)
|
||||
5. [API Reference](#api-reference)
|
||||
6. [Enhanced CLI](#enhanced-cli)
|
||||
7. [Security Model](#security-model)
|
||||
8. [Testing & Quality Assurance](#testing--quality-assurance)
|
||||
9. [Usage Examples](#usage-examples)
|
||||
10. [Development & Deployment](#development--deployment)
|
||||
11. [Future Enhancements](#future-enhancements)
|
||||
12. [Project Status](#project-status)
|
||||
|
||||
## 🎯 Project Goals
|
||||
|
||||
- **Self-Hosted**: Run your own digital services (email, calendar, files, VPN, etc.) on your hardware
|
||||
- **Decentralized**: Peer-to-peer networking and trust, no central authority
|
||||
- **Production-Grade**: Enterprise-ready architecture with comprehensive monitoring
|
||||
- **Secure**: Modern cryptography, certificate management, and encrypted storage
|
||||
- **User-Friendly**: Professional CLI and API for easy management
|
||||
- **Extensible**: Modular architecture for future services and integrations
|
||||
- **Event-Driven**: Real-time service communication and orchestration
|
||||
|
||||
## 🏗️ Architecture & Components
|
||||
|
||||
### **Modern Architecture Stack**
|
||||
|
||||
- **Backend**: Python (Flask) with production-grade service managers
|
||||
- **Service Architecture**: BaseServiceManager pattern with unified interfaces
|
||||
- **Event System**: Service bus for real-time communication and orchestration
|
||||
- **Configuration**: Centralized configuration management with validation
|
||||
- **Logging**: Structured JSON logging with rotation and search
|
||||
- **Containerization**: Docker-based deployment and service isolation
|
||||
- **API**: RESTful endpoints with comprehensive documentation
|
||||
|
||||
### **Core Architecture Components**
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Personal Internet Cell │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ Enhanced CLI │ Web UI │ REST API │ Service Bus │ Logging │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ Service Managers │
|
||||
│ Network │ WireGuard │ Email │ Calendar │ Files │ Routing │
|
||||
│ Vault │ Container │ Cell │ Peer │ │ │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ Core Infrastructure │
|
||||
│ DNS │ DHCP │ NTP │ VPN │ CA │ Encryption │ Trust │ Storage │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## 🔧 Service Manager Architecture
|
||||
|
||||
### **BaseServiceManager Pattern**
|
||||
|
||||
All services inherit from `BaseServiceManager`, providing:
|
||||
|
||||
```python
|
||||
class BaseServiceManager(ABC):
|
||||
def __init__(self, service_name: str, data_dir: str, config_dir: str)
|
||||
|
||||
@abstractmethod
|
||||
def get_status(self) -> Dict[str, Any]
|
||||
|
||||
@abstractmethod
|
||||
def test_connectivity(self) -> Dict[str, Any]
|
||||
|
||||
# Common methods
|
||||
def get_logs(self, lines: int = 50) -> List[str]
|
||||
def restart_service(self) -> bool
|
||||
def get_config(self) -> Dict[str, Any]
|
||||
def update_config(self, config: Dict[str, Any]) -> bool
|
||||
def health_check(self) -> Dict[str, Any]
|
||||
def handle_error(self, error: Exception, context: str) -> Dict[str, Any]
|
||||
```
|
||||
|
||||
### **Service Bus Integration**
|
||||
|
||||
```python
|
||||
# Event-driven service communication
|
||||
service_bus.register_service('network', network_manager)
|
||||
service_bus.register_service('wireguard', wireguard_manager)
|
||||
service_bus.publish_event(EventType.SERVICE_STARTED, 'network', data)
|
||||
|
||||
# Service dependencies
|
||||
service_dependencies = {
|
||||
'wireguard': ['network'],
|
||||
'email': ['network', 'vault'],
|
||||
'calendar': ['network', 'vault'],
|
||||
'files': ['network', 'vault'],
|
||||
'routing': ['network', 'wireguard'],
|
||||
'vault': ['network']
|
||||
}
|
||||
```
|
||||
|
||||
## 🔧 Core Services
|
||||
|
||||
### **Network Services**
|
||||
- **NetworkManager**: DNS, DHCP, NTP with dynamic management
|
||||
- Dynamic zone file generation
|
||||
- DHCP lease monitoring
|
||||
- Network connectivity testing
|
||||
- Service health monitoring
|
||||
|
||||
### **VPN & Mesh Networking**
|
||||
- **WireGuardManager**: WireGuard VPN configuration and peer management
|
||||
- Key generation and management
|
||||
- Peer configuration
|
||||
- Connectivity testing
|
||||
- Dynamic IP updates
|
||||
|
||||
- **PeerRegistry**: Peer registration and trust management
|
||||
- Peer lifecycle management
|
||||
- Trust relationship tracking
|
||||
- Data integrity validation
|
||||
- Peer statistics
|
||||
|
||||
### **Digital Services**
|
||||
- **EmailManager**: SMTP/IMAP email services
|
||||
- User account management
|
||||
- Mailbox configuration
|
||||
- Service connectivity testing
|
||||
- Email delivery monitoring
|
||||
|
||||
- **CalendarManager**: CalDAV/CardDAV calendar and contacts
|
||||
- User and calendar management
|
||||
- Event synchronization
|
||||
- Service health monitoring
|
||||
- Connectivity testing
|
||||
|
||||
- **FileManager**: WebDAV file storage
|
||||
- User directory management
|
||||
- Storage quota monitoring
|
||||
- File system access testing
|
||||
- Backup and restore capabilities
|
||||
|
||||
### **Infrastructure Services**
|
||||
- **RoutingManager**: Advanced routing and NAT
|
||||
- NAT rule management
|
||||
- Firewall configuration
|
||||
- Exit node routing
|
||||
- Bridge and split routing
|
||||
- Connectivity testing
|
||||
|
||||
- **VaultManager**: Security and trust management
|
||||
- Self-hosted Certificate Authority
|
||||
- Certificate lifecycle management
|
||||
- Age/Fernet encryption
|
||||
- Trust relationship management
|
||||
- Cryptographic verification
|
||||
|
||||
- **ContainerManager**: Docker orchestration
|
||||
- Container lifecycle management
|
||||
- Image and volume management
|
||||
- Docker daemon connectivity
|
||||
- Service isolation
|
||||
|
||||
- **CellManager**: Overall cell orchestration
|
||||
- Service coordination
|
||||
- Health monitoring
|
||||
- Configuration management
|
||||
- Peer management
|
||||
|
||||
## 📡 API Reference
|
||||
|
||||
### **Core API Endpoints**
|
||||
|
||||
```bash
|
||||
# Service Status and Health
|
||||
GET /api/services/status # All services status
|
||||
GET /api/services/connectivity # Service connectivity tests
|
||||
GET /health # API health check
|
||||
|
||||
# Configuration Management
|
||||
GET /api/config # Get configuration
|
||||
PUT /api/config # Update configuration
|
||||
POST /api/config/backup # Create backup
|
||||
GET /api/config/backups # List backups
|
||||
POST /api/config/restore/<id> # Restore backup
|
||||
GET /api/config/export # Export configuration
|
||||
POST /api/config/import # Import configuration
|
||||
|
||||
# Service Bus
|
||||
GET /api/services/bus/status # Service bus status
|
||||
GET /api/services/bus/events # Event history
|
||||
POST /api/services/bus/services/<service>/start
|
||||
POST /api/services/bus/services/<service>/stop
|
||||
POST /api/services/bus/services/<service>/restart
|
||||
|
||||
# Logging
|
||||
GET /api/logs/services/<service> # Service logs
|
||||
POST /api/logs/search # Log search
|
||||
POST /api/logs/export # Log export
|
||||
GET /api/logs/statistics # Log statistics
|
||||
POST /api/logs/rotate # Log rotation
|
||||
```
|
||||
|
||||
### **Service-Specific Endpoints**
|
||||
|
||||
```bash
|
||||
# Network Services
|
||||
GET /api/dns/records # DNS records
|
||||
POST /api/dns/records # Add DNS record
|
||||
DELETE /api/dns/records # Remove DNS record
|
||||
GET /api/dhcp/leases # DHCP leases
|
||||
POST /api/dhcp/reservations # Add DHCP reservation
|
||||
GET /api/ntp/status # NTP status
|
||||
GET /api/network/info # Network information
|
||||
POST /api/network/test # Network connectivity test
|
||||
|
||||
# WireGuard & Peers
|
||||
GET /api/wireguard/keys # WireGuard keys
|
||||
POST /api/wireguard/keys/peer # Generate peer keys
|
||||
GET /api/wireguard/config # WireGuard configuration
|
||||
GET /api/wireguard/peers # List peers
|
||||
POST /api/wireguard/peers # Add peer
|
||||
DELETE /api/wireguard/peers # Remove peer
|
||||
GET /api/wireguard/status # WireGuard status
|
||||
POST /api/wireguard/connectivity # Connectivity test
|
||||
PUT /api/wireguard/peers/ip # Update peer IP
|
||||
|
||||
# Digital Services
|
||||
GET /api/email/users # Email users
|
||||
POST /api/email/users # Add email user
|
||||
DELETE /api/email/users/<user> # Remove email user
|
||||
GET /api/email/status # Email service status
|
||||
GET /api/email/connectivity # Email connectivity
|
||||
POST /api/email/send # Send email
|
||||
GET /api/email/mailbox/<user> # User mailbox
|
||||
|
||||
GET /api/calendar/users # Calendar users
|
||||
POST /api/calendar/users # Add calendar user
|
||||
DELETE /api/calendar/users/<user> # Remove calendar user
|
||||
POST /api/calendar/calendars # Create calendar
|
||||
POST /api/calendar/events # Add event
|
||||
GET /api/calendar/events/<user>/<calendar> # List events
|
||||
GET /api/calendar/status # Calendar service status
|
||||
GET /api/calendar/connectivity # Calendar connectivity
|
||||
|
||||
GET /api/files/users # File users
|
||||
POST /api/files/users # Add file user
|
||||
DELETE /api/files/users/<user> # Remove file user
|
||||
POST /api/files/folders # Create folder
|
||||
DELETE /api/files/folders/<user>/<path> # Remove folder
|
||||
POST /api/files/upload/<user> # Upload file
|
||||
GET /api/files/download/<user>/<path> # Download file
|
||||
DELETE /api/files/delete/<user>/<path> # Delete file
|
||||
GET /api/files/list/<user> # List files
|
||||
GET /api/files/status # File service status
|
||||
GET /api/files/connectivity # File connectivity
|
||||
|
||||
# Routing & Security
|
||||
GET /api/routing/status # Routing status
|
||||
POST /api/routing/nat # Add NAT rule
|
||||
DELETE /api/routing/nat/<id> # Remove NAT rule
|
||||
POST /api/routing/peers # Add peer route
|
||||
DELETE /api/routing/peers/<peer> # Remove peer route
|
||||
POST /api/routing/exit-nodes # Add exit node
|
||||
POST /api/routing/bridge # Add bridge route
|
||||
POST /api/routing/split # Add split route
|
||||
POST /api/routing/firewall # Add firewall rule
|
||||
POST /api/routing/connectivity # Routing connectivity test
|
||||
GET /api/routing/logs # Routing logs
|
||||
GET /api/routing/nat # List NAT rules
|
||||
GET /api/routing/peers # List peer routes
|
||||
GET /api/routing/firewall # List firewall rules
|
||||
|
||||
GET /api/vault/status # Vault status
|
||||
GET /api/vault/certificates # List certificates
|
||||
POST /api/vault/certificates # Generate certificate
|
||||
DELETE /api/vault/certificates/<name> # Revoke certificate
|
||||
GET /api/vault/ca/certificate # CA certificate
|
||||
GET /api/vault/age/public-key # Age public key
|
||||
GET /api/vault/trust/keys # Trusted keys
|
||||
POST /api/vault/trust/keys # Add trusted key
|
||||
DELETE /api/vault/trust/keys/<name> # Remove trusted key
|
||||
POST /api/vault/trust/verify # Verify trust
|
||||
GET /api/vault/trust/chains # Trust chains
|
||||
```
|
||||
|
||||
## 💻 Enhanced CLI
|
||||
|
||||
### **CLI Features**
|
||||
|
||||
```bash
|
||||
# Interactive mode with tab completion
|
||||
python api/enhanced_cli.py --interactive
|
||||
|
||||
# Batch operations
|
||||
python api/enhanced_cli.py --batch "status" "services" "health"
|
||||
|
||||
# Configuration management
|
||||
python api/enhanced_cli.py --export-config json
|
||||
python api/enhanced_cli.py --import-config config.json
|
||||
|
||||
# Service wizards
|
||||
python api/enhanced_cli.py --wizard network
|
||||
python api/enhanced_cli.py --wizard email
|
||||
|
||||
# Health monitoring
|
||||
python api/enhanced_cli.py --health
|
||||
python api/enhanced_cli.py --logs network
|
||||
|
||||
# Service status
|
||||
python api/enhanced_cli.py --status
|
||||
python api/enhanced_cli.py --services
|
||||
python api/enhanced_cli.py --peers
|
||||
```
|
||||
|
||||
### **CLI Capabilities**
|
||||
- **Interactive Mode**: Tab completion, command history, help system
|
||||
- **Batch Operations**: Execute multiple commands in sequence
|
||||
- **Configuration Wizards**: Guided setup for complex services
|
||||
- **Real-time Monitoring**: Live status updates and health checks
|
||||
- **Log Management**: View, search, and export service logs
|
||||
- **Service Management**: Start, stop, restart, and configure services
|
||||
|
||||
## 🔒 Security Model
|
||||
|
||||
### **Certificate Management**
|
||||
- **Self-hosted CA**: Issue and manage TLS certificates for all services
|
||||
- **Certificate Lifecycle**: Generate, renew, revoke, and monitor certificates
|
||||
- **Trust Management**: Direct, indirect, and verified trust relationships
|
||||
- **Age Encryption**: Modern encryption for sensitive data and keys
|
||||
|
||||
### **Network Security**
|
||||
- **WireGuard VPN**: Secure peer-to-peer communication with key rotation
|
||||
- **Firewall & NAT**: Granular control over network access and routing
|
||||
- **Service Isolation**: Docker containers for each service
|
||||
- **Input Validation**: All API endpoints validate and sanitize input
|
||||
|
||||
### **Data Protection**
|
||||
- **Encrypted Storage**: Sensitive data encrypted at rest using Age/Fernet
|
||||
- **Secure Communication**: TLS for all API endpoints and service communication
|
||||
- **Access Control**: Role-based access for services and API endpoints
|
||||
- **Audit Logging**: Comprehensive security event logging and monitoring
|
||||
|
||||
## 🧪 Testing & Quality Assurance
|
||||
|
||||
### **Test Coverage**
|
||||
- **BaseServiceManager**: 100% coverage
|
||||
- **ConfigManager**: 95%+ coverage
|
||||
- **ServiceBus**: 95%+ coverage
|
||||
- **LogManager**: 95%+ coverage
|
||||
- **All Service Managers**: 77%+ overall coverage
|
||||
- **API Endpoints**: 100% endpoint coverage
|
||||
|
||||
### **Test Types**
|
||||
- **Unit Tests**: Individual component testing
|
||||
- **Integration Tests**: Service interaction testing
|
||||
- **API Tests**: Endpoint functionality testing
|
||||
- **Error Handling**: Exception and edge case testing
|
||||
- **Performance Tests**: Load and stress testing
|
||||
|
||||
### **Testing Commands**
|
||||
```bash
|
||||
# Run all tests
|
||||
python api/test_enhanced_api.py
|
||||
|
||||
# Run specific test suites
|
||||
python -m pytest api/tests/test_network_manager.py
|
||||
python -m pytest api/tests/test_service_bus.py
|
||||
|
||||
# Generate coverage report
|
||||
coverage run -m pytest api/tests/
|
||||
coverage html
|
||||
```
|
||||
|
||||
## 📝 Usage Examples
|
||||
|
||||
### **Add DNS Record**
|
||||
```bash
|
||||
curl -X POST http://localhost:3000/api/dns/records \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"name": "www",
|
||||
"type": "A",
|
||||
"value": "192.168.1.100",
|
||||
"ttl": 300
|
||||
}'
|
||||
```
|
||||
|
||||
### **Register Peer**
|
||||
```bash
|
||||
curl -X POST http://localhost:3000/api/wireguard/peers \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"name": "bob",
|
||||
"ip": "203.0.113.22",
|
||||
"public_key": "peer_public_key_here",
|
||||
"allowed_networks": ["10.0.0.0/24"]
|
||||
}'
|
||||
```
|
||||
|
||||
### **Generate Certificate**
|
||||
```bash
|
||||
curl -X POST http://localhost:3000/api/vault/certificates \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"common_name": "myapp.example.com",
|
||||
"domains": ["myapp.example.com", "www.myapp.example.com"],
|
||||
"days": 365
|
||||
}'
|
||||
```
|
||||
|
||||
### **Configure NAT Rule**
|
||||
```bash
|
||||
curl -X POST http://localhost:3000/api/routing/nat \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"source_network": "10.0.0.0/24",
|
||||
"target_interface": "eth0",
|
||||
"nat_type": "MASQUERADE",
|
||||
"protocol": "ALL"
|
||||
}'
|
||||
```
|
||||
|
||||
## 🛠️ Development & Deployment
|
||||
|
||||
### **Development Setup**
|
||||
```bash
|
||||
# Install dependencies
|
||||
pip install -r api/requirements.txt
|
||||
|
||||
# Start development server
|
||||
python api/app.py
|
||||
|
||||
# Run tests
|
||||
python api/test_enhanced_api.py
|
||||
|
||||
# Start frontend (if available)
|
||||
cd webui && bun install && npm run dev
|
||||
```
|
||||
|
||||
### **Production Deployment**
|
||||
```bash
|
||||
# Docker deployment
|
||||
docker-compose up --build -d
|
||||
|
||||
# Health check
|
||||
curl http://localhost:3000/health
|
||||
|
||||
# Service status
|
||||
curl http://localhost:3000/api/services/status
|
||||
```
|
||||
|
||||
### **Service Development**
|
||||
```python
|
||||
from base_service_manager import BaseServiceManager
|
||||
|
||||
class MyServiceManager(BaseServiceManager):
|
||||
def __init__(self, data_dir='/app/data', config_dir='/app/config'):
|
||||
super().__init__('myservice', data_dir, config_dir)
|
||||
|
||||
def get_status(self) -> Dict[str, Any]:
|
||||
# Implement service status
|
||||
return {
|
||||
'running': True,
|
||||
'status': 'online',
|
||||
'timestamp': datetime.utcnow().isoformat()
|
||||
}
|
||||
|
||||
def test_connectivity(self) -> Dict[str, Any]:
|
||||
# Implement connectivity test
|
||||
return {
|
||||
'success': True,
|
||||
'message': 'Service connectivity working',
|
||||
'timestamp': datetime.utcnow().isoformat()
|
||||
}
|
||||
```
|
||||
|
||||
## 🚀 Future Enhancements
|
||||
|
||||
### **Planned Features**
|
||||
- **Certificate Auto-renewal**: Automatic certificate renewal and monitoring
|
||||
- **Web of Trust Models**: Advanced trust relationship management
|
||||
- **Certificate Transparency**: CT log integration and monitoring
|
||||
- **Hardware Security Module (HSM)**: HSM integration for key management
|
||||
- **WebSocket Updates**: Real-time service status updates
|
||||
- **Advanced Monitoring**: Metrics collection and alerting systems
|
||||
- **Mobile App**: Mobile application for remote management
|
||||
- **Plugin System**: Extensible architecture for custom services
|
||||
|
||||
### **Architecture Improvements**
|
||||
- **Service Discovery**: Dynamic service registration and discovery
|
||||
- **Load Balancing**: Multi-instance service deployment
|
||||
- **Advanced Caching**: Redis-based caching for performance
|
||||
- **Message Queues**: RabbitMQ/Kafka for reliable messaging
|
||||
- **Distributed Tracing**: OpenTelemetry integration
|
||||
- **Configuration Management**: GitOps-style configuration management
|
||||
|
||||
## 📊 Project Status
|
||||
|
||||
### **✅ Completed Features**
|
||||
- **Production-Grade Architecture**: BaseServiceManager pattern implemented
|
||||
- **Event-Driven Communication**: Service bus with real-time events
|
||||
- **Centralized Configuration**: Type-safe configuration with validation
|
||||
- **Comprehensive Logging**: Structured logging with search and export
|
||||
- **Enhanced CLI**: Interactive CLI with batch operations
|
||||
- **Health Monitoring**: Real-time health checks across all services
|
||||
- **Security Framework**: Self-hosted CA, encryption, and trust management
|
||||
- **Complete API**: RESTful API with comprehensive documentation
|
||||
- **Testing Framework**: Comprehensive test suite with high coverage
|
||||
|
||||
### **🎯 Current Status**
|
||||
- **All Services**: 10 service managers fully implemented and integrated
|
||||
- **API Server**: Running on port 3000 with all endpoints functional
|
||||
- **CLI Tool**: Enhanced CLI with all features working
|
||||
- **Test Coverage**: 77%+ overall coverage with comprehensive testing
|
||||
- **Documentation**: Complete documentation for all components
|
||||
- **Production Ready**: Suitable for personal and small business deployment
|
||||
|
||||
### **🌟 Key Achievements**
|
||||
- **Unified Architecture**: All services follow the same patterns and interfaces
|
||||
- **Event-Driven Design**: Services communicate and orchestrate automatically
|
||||
- **Configuration Management**: Centralized, validated configuration system
|
||||
- **Comprehensive Logging**: Production-grade logging with advanced features
|
||||
- **Enhanced CLI**: Professional command-line interface for management
|
||||
- **Health Monitoring**: Real-time monitoring and alerting capabilities
|
||||
- **Security Framework**: Enterprise-grade security with modern cryptography
|
||||
- **Complete Testing**: Comprehensive test suite ensuring reliability
|
||||
The goal is to give a person full ownership of their core internet services on their own hardware, without relying on cloud providers.
|
||||
|
||||
---
|
||||
|
||||
**The Personal Internet Cell empowers users with full control over their digital infrastructure, combining privacy, security, and usability in a single, production-ready, self-hosted platform.** 🌟
|
||||
## Table of Contents
|
||||
|
||||
1. [Architecture](#architecture)
|
||||
2. [Service Managers](#service-managers)
|
||||
3. [First-Run Wizard](#first-run-wizard)
|
||||
4. [Authentication](#authentication)
|
||||
5. [API Reference](#api-reference)
|
||||
6. [DDNS](#ddns)
|
||||
7. [Service Store](#service-store)
|
||||
8. [Cell-to-Cell Networking](#cell-to-cell-networking)
|
||||
9. [Extended Connectivity](#extended-connectivity)
|
||||
10. [Security Model](#security-model)
|
||||
11. [Testing](#testing)
|
||||
12. [Development](#development)
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Browser / WireGuard peer
|
||||
└── Caddy (:80/:443) reverse proxy, TLS termination
|
||||
└── React SPA (:8081) Vite + Tailwind (Nginx in container)
|
||||
└── Flask API (:3000) REST API, bound to 127.0.0.1
|
||||
├── NetworkManager CoreDNS, dnsmasq, chrony
|
||||
├── WireGuardManager WireGuard VPN peer lifecycle
|
||||
├── PeerRegistry peer registration and trust
|
||||
├── EmailManager Postfix + Dovecot
|
||||
├── CalendarManager Radicale CalDAV/CardDAV
|
||||
├── FileManager WebDAV + Filegator
|
||||
├── RoutingManager iptables NAT and routing
|
||||
├── FirewallManager iptables firewall rules
|
||||
├── VaultManager internal CA, cert lifecycle, Age encryption
|
||||
├── ContainerManager Docker SDK
|
||||
├── CellLinkManager cell-to-cell WireGuard links
|
||||
├── ConnectivityManager exit routing (WG ext, OpenVPN, Tor)
|
||||
├── DDNSManager dynamic DNS heartbeat
|
||||
├── ServiceStoreManager optional service install/remove
|
||||
├── CaddyManager Caddyfile generation and reload
|
||||
├── AuthManager session auth, RBAC
|
||||
└── SetupManager first-run wizard state
|
||||
```
|
||||
|
||||
All 12 service containers run on a Docker bridge network (`cell-network`, `172.20.0.0/16` default). Static IPs per container are defined in `docker-compose.yml`.
|
||||
|
||||
Runtime configuration lives in `config/api/cell_config.json`, managed by `ConfigManager`. All service managers read and write through `ConfigManager`, which validates and backs up automatically.
|
||||
|
||||
---
|
||||
|
||||
## Service Managers
|
||||
|
||||
All managers inherit `BaseServiceManager` (`api/base_service_manager.py`), which provides:
|
||||
- `get_status()` — current running state
|
||||
- `get_config()` / `update_config()` — config read/write
|
||||
- `test_connectivity()` — reachability check
|
||||
- `get_logs()` — last N lines from the service log
|
||||
- `restart_service()` — container restart via Docker SDK
|
||||
|
||||
The `ServiceBus` (`api/service_bus.py`) handles pub/sub events between managers (e.g., `CONFIG_CHANGED`, `SERVICE_STARTED`). Dependencies are declared in the bus (wireguard depends on network; email depends on network and vault).
|
||||
|
||||
### Manager summary
|
||||
|
||||
| Manager | Responsibilities |
|
||||
|---|---|
|
||||
| `NetworkManager` | CoreDNS zone files, dnsmasq DHCP config and lease monitoring, chrony NTP |
|
||||
| `WireGuardManager` | Key generation, `wg0.conf` generation, peer add/remove, route sync |
|
||||
| `PeerRegistry` | Peer registration, trust tracking, peer statistics |
|
||||
| `EmailManager` | docker-mailserver accounts, mailbox config, alias management |
|
||||
| `CalendarManager` | Radicale user/calendar/contacts lifecycle |
|
||||
| `FileManager` | WebDAV user directories, Filegator access |
|
||||
| `RoutingManager` | NAT rules, per-peer routing policy, fwmark-based exit routing |
|
||||
| `FirewallManager` | iptables INPUT/FORWARD/OUTPUT rule management |
|
||||
| `VaultManager` | Internal CA (self-signed root), TLS cert issue/revoke, Age public key |
|
||||
| `ContainerManager` | Docker container/image/volume management via SDK |
|
||||
| `CellLinkManager` | Site-to-site WireGuard links to other PIC cells, peer-sync protocol |
|
||||
| `ConnectivityManager` | Per-peer exit routing via WireGuard external, OpenVPN, or Tor |
|
||||
| `DDNSManager` | Public IP heartbeat, provider abstraction (pic_ngo, cloudflare, duckdns, noip, freedns) |
|
||||
| `ServiceStoreManager` | Fetch manifest index, install/remove optional services |
|
||||
| `CaddyManager` | Caddyfile generation, reload-on-change |
|
||||
| `AuthManager` | bcrypt password store, session management, admin/peer RBAC |
|
||||
| `SetupManager` | First-run wizard state, setup-complete flag |
|
||||
|
||||
---
|
||||
|
||||
## First-Run Wizard
|
||||
|
||||
On first start, `SetupManager.is_setup_complete()` returns `False`. The `enforce_setup` before-request hook returns HTTP 428 for all `/api/*` requests except `/api/setup/*` and `/health`, redirecting clients to `/setup`.
|
||||
|
||||
The wizard collects:
|
||||
- **Cell name** — used for hostnames and DDNS subdomain (e.g. `myhome` → `myhome.pic.ngo`)
|
||||
- **Domain mode** — determines TLS certificate source: `lan` (internal CA), `pic_ngo`, `cloudflare`, `duckdns`, `http01`
|
||||
- **Timezone**
|
||||
- **Initial services to enable**
|
||||
- **Admin password** — minimum 12 characters
|
||||
|
||||
On completion:
|
||||
1. Admin account is created in `data/auth_users.json`
|
||||
2. Cell identity is written to `config/api/cell_config.json`
|
||||
3. Caddy config is generated
|
||||
4. If domain mode is `pic_ngo`, the cell registers `<name>.pic.ngo` with the DDNS service
|
||||
|
||||
Wizard endpoints: `GET/POST /api/setup/step`, `GET /api/setup/status`, `POST /api/setup/complete`.
|
||||
|
||||
---
|
||||
|
||||
## Authentication
|
||||
|
||||
`AuthManager` stores bcrypt-hashed credentials in `data/auth_users.json`. Two roles:
|
||||
|
||||
| Role | Access |
|
||||
|---|---|
|
||||
| `admin` | All `/api/*` endpoints except `/api/peer/*` |
|
||||
| `peer` | `/api/peer/*` only (peer dashboard, key exchange) |
|
||||
|
||||
Session auth flow:
|
||||
- `POST /api/auth/login` — creates a Flask session
|
||||
- `GET /api/auth/me` — current session info
|
||||
- `POST /api/auth/logout` — clears session
|
||||
- `POST /api/auth/change-password` — change own password
|
||||
- `POST /api/auth/admin/reset-password` — admin resets another user's password
|
||||
|
||||
CSRF protection: all `POST`, `PUT`, `DELETE`, `PATCH` on `/api/*` (except `/api/auth/*` and `/api/setup/*`) require the `X-CSRF-Token` header matching the session token, obtained via `GET /api/auth/csrf-token`.
|
||||
|
||||
Cell-to-cell peer-sync endpoints (`/api/cells/peer-sync/*`) use source-IP + WireGuard public key auth, not session cookies.
|
||||
|
||||
Auth enforcement is active once any user exists in the store. If the store is empty (fresh install before wizard), all requests bypass auth — `enforce_setup` already blocks them with 428.
|
||||
|
||||
---
|
||||
|
||||
## API Reference
|
||||
|
||||
**Base URL:** `http://localhost:3000`
|
||||
**Auth:** session cookie (`X-CSRF-Token` header required for mutations)
|
||||
|
||||
### Core
|
||||
|
||||
| Method | Path | Description |
|
||||
|---|---|---|
|
||||
| GET | `/health` | Health check (always public) |
|
||||
| GET | `/api/status` | All-service status summary |
|
||||
| GET | `/api/config` | Full cell config |
|
||||
| PUT | `/api/config` | Update cell config |
|
||||
| GET | `/api/health/history` | Recent health check history |
|
||||
|
||||
### Auth (`/api/auth/`)
|
||||
|
||||
| Method | Path | Description |
|
||||
|---|---|---|
|
||||
| POST | `/api/auth/login` | Create session |
|
||||
| POST | `/api/auth/logout` | Destroy session |
|
||||
| GET | `/api/auth/me` | Current user info |
|
||||
| GET | `/api/auth/csrf-token` | Get CSRF token |
|
||||
| POST | `/api/auth/change-password` | Change own password |
|
||||
| POST | `/api/auth/admin/reset-password` | Admin: reset another user's password |
|
||||
| GET | `/api/auth/users` | Admin: list users |
|
||||
|
||||
### Setup (`/api/setup/`)
|
||||
|
||||
| Method | Path | Description |
|
||||
|---|---|---|
|
||||
| GET | `/api/setup/status` | Setup complete flag + current step |
|
||||
| GET | `/api/setup/step` | Current wizard step data |
|
||||
| POST | `/api/setup/step` | Submit current step |
|
||||
| POST | `/api/setup/complete` | Finalize setup |
|
||||
|
||||
### Network Services (`/api/dns/`, `/api/dhcp/`, `/api/ntp/`, `/api/network/`)
|
||||
|
||||
DNS records, DHCP leases and reservations, NTP status, network connectivity test.
|
||||
|
||||
### WireGuard (`/api/wireguard/`, `/api/peers/`)
|
||||
|
||||
Peer add/remove, key generation, QR code export, per-peer routing policy, WireGuard status.
|
||||
|
||||
### Email (`/api/email/`)
|
||||
|
||||
User account management, mailbox config, alias management, connectivity test.
|
||||
|
||||
### Calendar (`/api/calendar/`)
|
||||
|
||||
User, calendar, and contacts (CardDAV) management.
|
||||
|
||||
### Files (`/api/files/`)
|
||||
|
||||
WebDAV user management, file upload/download/delete, folder management.
|
||||
|
||||
### Routing (`/api/routing/`)
|
||||
|
||||
NAT rules, peer routes, exit node configuration.
|
||||
|
||||
### Vault (`/api/vault/`)
|
||||
|
||||
Certificate issue/revoke, CA certificate, trust key management, Age public key.
|
||||
|
||||
### Containers (`/api/containers/`)
|
||||
|
||||
List, start, stop, inspect containers; manage images and volumes.
|
||||
|
||||
### Cell Network (`/api/cells/`)
|
||||
|
||||
List connected cells, add/remove cell links, peer-sync.
|
||||
|
||||
### Connectivity (`/api/connectivity/`)
|
||||
|
||||
List exit nodes, configure WireGuard external / OpenVPN / Tor exits, assign per-peer exit policy.
|
||||
|
||||
### Service Store (`/api/store/`)
|
||||
|
||||
List available services, install, remove.
|
||||
|
||||
### Logs (`/api/logs/`)
|
||||
|
||||
Per-service log retrieval, log search, log statistics.
|
||||
|
||||
---
|
||||
|
||||
## DDNS
|
||||
|
||||
`DDNSManager` maintains a `<cell-name>.pic.ngo` DNS A record pointing at the cell's public IP. A background thread runs every 5 minutes and calls `provider.update(token, ip)` only when the IP changes.
|
||||
|
||||
Registration happens during the setup wizard (if domain mode is `pic_ngo`) via `provider.register(name, ip)`, which returns a bearer token stored in `data/api/.ddns_token`.
|
||||
|
||||
DDNS config lives in `cell_config.json` under the top-level `ddns` key:
|
||||
|
||||
```json
|
||||
{
|
||||
"ddns": {
|
||||
"provider": "pic_ngo",
|
||||
"api_base_url": "https://ddns.pic.ngo",
|
||||
"totp_secret": "<base32 secret>"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Registration requires a time-based OTP (`X-Register-OTP` header) derived from the shared `REGISTER_TOTP_SECRET` on the DDNS server. This prevents unauthorized subdomain registration.
|
||||
|
||||
Supported providers: `pic_ngo`, `cloudflare`, `duckdns`, `noip`, `freedns`.
|
||||
|
||||
---
|
||||
|
||||
## Service Store
|
||||
|
||||
`ServiceStoreManager` fetches a manifest index from `http://git.pic.ngo/roof/pic-services/raw/branch/main/index.json`. Each manifest declares:
|
||||
- Container image
|
||||
- Caddy routes (added to the Caddyfile)
|
||||
- iptables rules
|
||||
- Environment variables
|
||||
- Health check endpoint
|
||||
|
||||
`POST /api/store/install` pulls the image, writes the Caddy route, applies iptables rules, and starts the container. `POST /api/store/remove` reverses this.
|
||||
|
||||
---
|
||||
|
||||
## Cell-to-Cell Networking
|
||||
|
||||
`CellLinkManager` manages WireGuard site-to-site tunnels between PIC cells. Each link is a WireGuard peer configured with a dedicated `/32` address and allowed-IPs covering the remote cell's subnet.
|
||||
|
||||
The peer-sync protocol (`/api/cells/peer-sync/`) exchanges public keys and allowed networks between cells using source-IP + WireGuard public key authentication (no session required).
|
||||
|
||||
Access control is per-service (calendar, files, mail, WebDAV) and enforced at the iptables level.
|
||||
|
||||
---
|
||||
|
||||
## Extended Connectivity
|
||||
|
||||
`ConnectivityManager` provides per-peer exit routing: traffic from a specific WireGuard peer can be routed through an alternate exit instead of the cell's default gateway.
|
||||
|
||||
Supported exits:
|
||||
- **WireGuard external** — another WireGuard endpoint (e.g. a VPS)
|
||||
- **OpenVPN** — OpenVPN client running in a container
|
||||
- **Tor** — Tor SOCKS proxy with transparent redirection
|
||||
|
||||
Routing uses fwmark and `ip rule` / `ip route` in separate routing tables. Configuration is via `PUT /api/connectivity/peers/<peer_name>/exit`.
|
||||
|
||||
---
|
||||
|
||||
## Security Model
|
||||
|
||||
- **No open ports for the API** — Flask API binds to `127.0.0.1:3000` only; Caddy proxies HTTPS requests to it.
|
||||
- **Session auth** — bcrypt passwords, Flask server-side sessions, CSRF double-submit.
|
||||
- **Setup wizard gate** — all `/api/*` requests return 428 until setup is complete.
|
||||
- **Role separation** — admin cannot access peer endpoints; peer cannot access admin endpoints.
|
||||
- **HTTPS everywhere** — Caddy handles TLS; internal services are reached via reverse proxy paths.
|
||||
- **Internal CA** — VaultManager issues certificates for services that don't use Let's Encrypt.
|
||||
- **Docker socket isolation** — the Docker socket is mounted only into `cell-api`; other containers have no Docker access.
|
||||
- **iptables firewall** — FirewallManager manages INPUT/FORWARD rules; WireGuard peer isolation is enforced at the packet level.
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
make test # unit tests (pytest, ~1500 functions)
|
||||
make test-coverage # coverage report in htmlcov/
|
||||
```
|
||||
|
||||
Test layout:
|
||||
- `tests/` — unit and endpoint tests; no running services required
|
||||
- `tests/integration/` — require a running PIC stack
|
||||
- `tests/e2e/` — Playwright UI tests and WireGuard integration tests
|
||||
|
||||
CI: Gitea Actions runs `pytest tests/ --ignore=tests/e2e --ignore=tests/integration` on every push.
|
||||
|
||||
---
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
# Full stack in Docker
|
||||
make start
|
||||
make stop
|
||||
make logs
|
||||
|
||||
# Flask API without Docker (port 3000)
|
||||
pip install -r api/requirements.txt
|
||||
python api/app.py
|
||||
|
||||
# React UI dev server (port 5173, proxies /api → :3000)
|
||||
cd webui && npm install && npm run dev
|
||||
|
||||
# Rebuild containers after code change
|
||||
make build-api
|
||||
make build-webui
|
||||
```
|
||||
|
||||
Key files:
|
||||
- `api/app.py` — Flask app, blueprint registration, before-request hooks, health monitor thread
|
||||
- `api/managers.py` — singleton instantiation of all service managers
|
||||
- `api/base_service_manager.py` — abstract base class all managers implement
|
||||
- `api/config_manager.py` — `cell_config.json` read/write/validate/backup
|
||||
- `api/service_bus.py` — pub/sub event system
|
||||
- `webui/src/services/api.js` — Axios API client used by all UI pages
|
||||
- `docker-compose.yml` — container definitions and network topology
|
||||
- `Makefile` — all operational commands
|
||||
|
||||
+104
-132
@@ -1,139 +1,112 @@
|
||||
# Quick Start
|
||||
|
||||
This guide walks through a first-time PIC installation from a clean Linux host.
|
||||
This guide walks through a first-time PIC installation on a clean Linux host.
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Linux host with the WireGuard kernel module (`modprobe wireguard` to verify)
|
||||
- Docker Engine and Docker Compose installed
|
||||
- Python 3.10+ (needed for `make setup` only)
|
||||
- Linux x86-64 host — Debian, Ubuntu, Fedora, RHEL, or Alpine
|
||||
- 2 GB+ RAM, 10 GB+ disk
|
||||
- Ports 53, 80, 443, 51820/udp, 25, 587, 993 available
|
||||
|
||||
The installer handles all software dependencies (git, docker, make, etc.) automatically.
|
||||
|
||||
---
|
||||
|
||||
## 1. Clone the repository
|
||||
## Option A — One-line installer (recommended)
|
||||
|
||||
```bash
|
||||
git clone <repo-url> pic
|
||||
curl -fsSL https://install.pic.ngo | sudo bash
|
||||
```
|
||||
|
||||
Always review the script before running it:
|
||||
|
||||
```bash
|
||||
curl -fsSL https://install.pic.ngo | less
|
||||
```
|
||||
|
||||
The installer:
|
||||
1. Detects your OS and installs Docker, git, make via the system package manager
|
||||
2. Creates a `pic` system user and adds it to the `docker` group
|
||||
3. Clones the repository to `/opt/pic`
|
||||
4. Runs `make install` (generates keys and config, writes a systemd unit)
|
||||
5. Runs `make start-core` to bring up the core containers
|
||||
6. Waits for the API to respond, then prints the wizard URL
|
||||
|
||||
When it finishes, open the URL it prints:
|
||||
|
||||
```
|
||||
http://<host-ip>:8081/setup
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Option B — Manual install
|
||||
|
||||
Use this if you want to control where PIC is installed, or if you are installing on a machine that already has Docker.
|
||||
|
||||
```bash
|
||||
git clone https://git.pic.ngo/roof/pic.git pic
|
||||
cd pic
|
||||
sudo make install
|
||||
make start-core
|
||||
```
|
||||
|
||||
Then open `http://<host-ip>:8081` in a browser.
|
||||
|
||||
---
|
||||
|
||||
## 2. Configure the environment
|
||||
## Complete the setup wizard
|
||||
|
||||
Copy the example environment file and edit it:
|
||||
The setup wizard appears automatically on first start. All API requests redirect to `/setup` until it is finished.
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
```
|
||||
The wizard asks for:
|
||||
|
||||
Open `.env` and set at minimum:
|
||||
- **Cell name** — used for hostnames and DDNS subdomain. Lowercase letters, digits, hyphens, 2–31 characters. Example: `myhome`.
|
||||
- **Domain mode** — how HTTPS certificates are issued:
|
||||
- `pic_ngo` — automatic `<cell-name>.pic.ngo` subdomain with Let's Encrypt via DNS-01 (recommended for internet-facing cells)
|
||||
- `cloudflare` — Let's Encrypt via Cloudflare DNS-01 (bring your own domain)
|
||||
- `duckdns` — Let's Encrypt via DuckDNS DNS-01
|
||||
- `http01` — Let's Encrypt via HTTP-01 (no wildcard; cell must be reachable on port 80)
|
||||
- `lan` — internal CA, no internet required (for LAN-only installs)
|
||||
- **Timezone**
|
||||
- **Services to enable** — email, calendar, files, WireGuard
|
||||
- **Admin password** — minimum 12 characters, must contain uppercase, lowercase, and a digit
|
||||
|
||||
```
|
||||
WEBDAV_PASS=changeme
|
||||
```
|
||||
|
||||
`WEBDAV_PASS` must be set before starting — the WebDAV container will fail to start without it.
|
||||
|
||||
All other variables have working defaults. See the Configuration section in [README.md](README.md) for the full list.
|
||||
Click **Complete Setup**. The wizard creates the admin account, writes cell identity to `config/api/cell_config.json`, and redirects to the login page.
|
||||
|
||||
---
|
||||
|
||||
## 3. Run setup
|
||||
## Log in
|
||||
|
||||
`make setup` installs system dependencies, generates WireGuard keys, and writes all required config files under `config/`:
|
||||
After the wizard you are redirected to `/login`.
|
||||
|
||||
```bash
|
||||
make check-deps # installs docker, python3-cryptography, etc. via apt
|
||||
make setup # generates keys and writes configs
|
||||
```
|
||||
|
||||
To customise the cell identity at setup time, pass overrides on the command line:
|
||||
|
||||
```bash
|
||||
CELL_NAME=myhome CELL_DOMAIN=cell VPN_ADDRESS=10.0.0.1/24 WG_PORT=51820 make setup
|
||||
```
|
||||
|
||||
`VPN_ADDRESS` must be an RFC-1918 address (e.g. `10.0.0.1/24`).
|
||||
- **Username:** `admin`
|
||||
- **Password:** the password you set in the wizard
|
||||
|
||||
---
|
||||
|
||||
## 4. Start the stack
|
||||
## Add a WireGuard peer
|
||||
|
||||
```bash
|
||||
make start
|
||||
```
|
||||
|
||||
This builds the `cell-api` and `cell-webui` images and starts all 13 containers. The first run takes a few minutes while images are pulled and built.
|
||||
|
||||
Check that everything came up:
|
||||
|
||||
```bash
|
||||
make status
|
||||
```
|
||||
|
||||
You should see all containers in the `Up` state and the API responding at `http://localhost:3000/health`.
|
||||
|
||||
---
|
||||
|
||||
## 5. Open the web UI
|
||||
|
||||
Open a browser and go to:
|
||||
|
||||
```
|
||||
http://<host-ip>:8081
|
||||
```
|
||||
|
||||
If you are running locally:
|
||||
|
||||
```
|
||||
http://localhost:8081
|
||||
```
|
||||
|
||||
The sidebar contains: Dashboard, Peers, Network Services, WireGuard, Email, Calendar, Files, Routing, Vault, Containers, Cell Network, Logs, Settings.
|
||||
|
||||
---
|
||||
|
||||
## 6. Set cell identity
|
||||
|
||||
Go to **Settings** in the sidebar.
|
||||
|
||||
Set your:
|
||||
- **Cell name** — a short identifier, e.g. `myhome`
|
||||
- **Domain** — the TLD your cell will use internally, e.g. `cell`
|
||||
- **VPN IP range** — the CIDR for WireGuard peers, e.g. `10.0.0.0/24`
|
||||
|
||||
After saving, the UI will show a banner asking you to apply the changes. Click **Apply Now**. The containers will restart briefly to pick up the new configuration.
|
||||
|
||||
---
|
||||
|
||||
## 7. Add a WireGuard peer
|
||||
|
||||
Go to **WireGuard** in the sidebar.
|
||||
Go to **Peers** in the sidebar.
|
||||
|
||||
1. Click **Add Peer**.
|
||||
2. Enter a name for the peer (e.g. `laptop`).
|
||||
2. Enter a peer name (e.g. `laptop`).
|
||||
3. The API generates a key pair and assigns the next available VPN IP automatically.
|
||||
4. Click the QR code icon to display the peer config as a QR code.
|
||||
4. Click the QR code icon to display the peer configuration as a QR code.
|
||||
5. Scan the QR code with a WireGuard client (Android, iOS, or the WireGuard desktop app).
|
||||
|
||||
The peer config sets your cell as the DNS server. Once connected, `*.cell` names resolve through the cell's CoreDNS.
|
||||
|
||||
To manage peers from the command line:
|
||||
|
||||
```bash
|
||||
make list-peers
|
||||
make add-peer PEER_NAME=phone PEER_IP=10.0.0.3 PEER_KEY=<base64-pubkey>
|
||||
```
|
||||
Once connected, `*.cell` names resolve through the cell's CoreDNS and traffic can be routed through the cell.
|
||||
|
||||
---
|
||||
|
||||
## 8. Day-to-day operations
|
||||
## Day-to-day operations
|
||||
|
||||
```bash
|
||||
# Check container status and API health
|
||||
make status
|
||||
|
||||
# Follow logs from all services
|
||||
make logs
|
||||
|
||||
@@ -142,9 +115,6 @@ make logs-api
|
||||
make logs-wireguard
|
||||
make logs-caddy
|
||||
|
||||
# Check container status and API health
|
||||
make status
|
||||
|
||||
# Open a shell inside a container
|
||||
make shell-api
|
||||
make shell-dns
|
||||
@@ -152,20 +122,11 @@ make shell-dns
|
||||
|
||||
---
|
||||
|
||||
## 9. Backup
|
||||
|
||||
Before making significant changes, create a backup:
|
||||
## Backup and restore
|
||||
|
||||
```bash
|
||||
make backup
|
||||
```
|
||||
|
||||
This archives `config/` and `data/` into `backups/cell-backup-<timestamp>.tar.gz`.
|
||||
|
||||
To list available backups:
|
||||
|
||||
```bash
|
||||
make restore
|
||||
make backup # archives config/ and data/ into backups/cell-backup-<timestamp>.tar.gz
|
||||
make restore # list available backups
|
||||
```
|
||||
|
||||
To restore manually:
|
||||
@@ -175,34 +136,38 @@ tar -xzf backups/cell-backup-YYYYMMDD-HHMMSS.tar.gz
|
||||
make start
|
||||
```
|
||||
|
||||
Backup and restore is also available in the UI under **Settings**.
|
||||
---
|
||||
|
||||
## Updating PIC
|
||||
|
||||
```bash
|
||||
make update # git pull + rebuild + restart
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10. Updating PIC
|
||||
## Uninstalling
|
||||
|
||||
```bash
|
||||
make update
|
||||
make uninstall # stops containers; prompts to also delete config/ and data/
|
||||
```
|
||||
|
||||
This runs `git pull`, then rebuilds and restarts all containers. If `config/` is missing (e.g. after a fresh clone), it runs `make setup` automatically.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**Containers not starting**
|
||||
### Containers not starting
|
||||
|
||||
```bash
|
||||
make logs
|
||||
make logs-api
|
||||
```
|
||||
|
||||
Look for errors related to missing config files or port conflicts.
|
||||
Look for errors about missing config files or port conflicts.
|
||||
|
||||
**Port 53 already in use**
|
||||
### Port 53 already in use
|
||||
|
||||
On Ubuntu/Debian, `systemd-resolved` listens on port 53. Disable it:
|
||||
On Ubuntu and Debian, `systemd-resolved` listens on port 53. Disable it:
|
||||
|
||||
```bash
|
||||
sudo systemctl disable --now systemd-resolved
|
||||
@@ -212,28 +177,35 @@ echo "nameserver 8.8.8.8" | sudo tee /etc/resolv.conf
|
||||
|
||||
Then run `make start` again.
|
||||
|
||||
**WebDAV container exits immediately**
|
||||
|
||||
`WEBDAV_PASS` is not set in `.env`. Set it and run `make start` again.
|
||||
|
||||
**WireGuard container fails to load kernel module**
|
||||
|
||||
Ensure the WireGuard kernel module is available:
|
||||
### WireGuard container fails to load the kernel module
|
||||
|
||||
```bash
|
||||
sudo modprobe wireguard
|
||||
```
|
||||
|
||||
On some minimal installs you may need to install `wireguard-tools` and the kernel headers for your running kernel.
|
||||
On minimal installs you may need `wireguard-tools` and the kernel headers for the running kernel.
|
||||
|
||||
**API returns 503 or UI shows "Backend Unavailable"**
|
||||
### API returns 428 and redirects to /setup
|
||||
|
||||
The Flask API may still be starting. Wait 10–15 seconds after `make start` and refresh. If it persists:
|
||||
The first-run wizard has not been completed. Open `http://<host-ip>:8081` and finish the wizard.
|
||||
|
||||
### API returns 401 / UI shows "Not authenticated"
|
||||
|
||||
Your session expired or you have not logged in. Go to `http://<host-ip>:8081/login`.
|
||||
|
||||
### API returns 503 "Authentication not configured"
|
||||
|
||||
The auth file exists but contains no accounts. To recover:
|
||||
|
||||
```bash
|
||||
make logs-api
|
||||
make reset-admin-password
|
||||
```
|
||||
|
||||
**Config changes not taking effect**
|
||||
This generates a new admin password and prints it.
|
||||
|
||||
After changing identity or service settings in the UI, a yellow banner appears at the top of the page. Click **Apply Now** to restart the affected containers.
|
||||
### Forgot the admin password
|
||||
|
||||
```bash
|
||||
make show-admin-password # print current password
|
||||
make reset-admin-password # generate a new random password
|
||||
```
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Personal Internet Cell (PIC)
|
||||
|
||||
PIC is a self-hosted digital infrastructure platform. It manages DNS, DHCP, NTP, WireGuard VPN, email, calendar/contacts (CalDAV), file storage (WebDAV), a reverse proxy, and a certificate authority — all controlled from a single REST API and React web UI. No manual config file editing is required for normal operations.
|
||||
PIC is a self-hosted digital infrastructure platform. It packages DNS, DHCP, NTP, WireGuard VPN, email, calendar/contacts (CalDAV), file storage (WebDAV), a reverse proxy, a certificate authority, and optional third-party services — all managed through a single REST API and a React web UI. No manual config file editing is required for normal operations.
|
||||
|
||||
---
|
||||
|
||||
@@ -10,8 +10,8 @@ PIC is a self-hosted digital infrastructure platform. It manages DNS, DHCP, NTP,
|
||||
Browser
|
||||
└── React SPA (cell-webui :8081)
|
||||
└── Flask REST API (cell-api :3000, bound to 127.0.0.1)
|
||||
└── Docker SDK / config files
|
||||
├── cell-caddy :80/:443 reverse proxy
|
||||
└── Service managers + Docker SDK
|
||||
├── cell-caddy :80/:443 Caddy reverse proxy (HTTPS/TLS)
|
||||
├── cell-dns :53 CoreDNS
|
||||
├── cell-dhcp :67/udp dnsmasq
|
||||
├── cell-ntp :123/udp chrony
|
||||
@@ -19,24 +19,47 @@ Browser
|
||||
├── cell-mail :25/:587/:993 Postfix + Dovecot
|
||||
├── cell-radicale 127.0.0.1:5232 CalDAV/CardDAV
|
||||
├── cell-webdav 127.0.0.1:8080 WebDAV
|
||||
├── cell-rainloop :8888 webmail (RainLoop)
|
||||
├── cell-filegator :8082 file manager UI
|
||||
├── cell-rainloop 127.0.0.1:8888 Webmail (RainLoop)
|
||||
├── cell-filegator 127.0.0.1:8082 File manager (Filegator)
|
||||
└── cell-webui :8081 React UI (Nginx)
|
||||
```
|
||||
|
||||
All containers run on a custom Docker bridge network (`cell-network`, default `172.20.0.0/16`). Static IPs per container are set in `docker-compose.yml` and overridden via `.env`.
|
||||
All containers run on a custom Docker bridge network (`cell-network`, default subnet `172.20.0.0/16`). Static IPs per container are set in `docker-compose.yml` and can be overridden via `.env`.
|
||||
|
||||
The Flask API (`api/app.py`, ~2800 lines) contains all REST endpoints, runs a background health-monitoring thread, and manages the entire lifecycle of generated config artefacts: `Caddyfile`, `Corefile`, `wg0.conf`, and `cell_config.json` (the single source of truth at `config/api/cell_config.json`).
|
||||
The Flask API (`api/app.py`) contains REST endpoints and a background health-monitoring thread. Service managers are instantiated as singletons in `api/managers.py`. The single source of truth for runtime configuration is `config/api/cell_config.json`, managed by `ConfigManager`.
|
||||
|
||||
The React frontend (`webui/`) is built with Vite + Tailwind CSS. All API calls go through `src/services/api.js` (Axios). Pages: Dashboard, Peers, Network Services, WireGuard, Email, Calendar, Files, Routing, Vault, Containers, Cell Network, Logs, Settings.
|
||||
The React frontend (`webui/`) is built with Vite + Tailwind CSS. All API calls go through `src/services/api.js` (Axios).
|
||||
|
||||
**Web UI pages:** Dashboard, Peers, Network Services, WireGuard, Email, Calendar, Files, Routing, Vault, Containers, Cell Network, Connectivity, Service Store, Logs, Settings.
|
||||
|
||||
---
|
||||
|
||||
## Features
|
||||
|
||||
- **First-run wizard** — browser-based setup at `/setup`. On first start, all API requests redirect to `/setup` (HTTP 428) until the wizard is completed. Sets cell name, domain mode, timezone, admin password, and initial services. No manual `.env` editing required for identity.
|
||||
- **Session-based auth** — admin and peer roles. All `/api/*` endpoints require an authenticated session after setup. CSRF protection on all state-changing requests.
|
||||
- **WireGuard VPN** — peer lifecycle management, automatic key generation, QR code config export, per-peer routing policy.
|
||||
- **Caddy HTTPS** — automatic TLS via Let's Encrypt (DNS-01 or HTTP-01) or an internal CA, depending on domain mode.
|
||||
- **DDNS (pic.ngo)** — registers a `<cell-name>.pic.ngo` subdomain. Supported providers: `pic_ngo`, `cloudflare`, `duckdns`, `noip`, `freedns`. A background thread re-publishes the public IP every 5 minutes.
|
||||
- **Service store** — install/remove optional third-party services from the `pic-services` index at `git.pic.ngo`. Manifests declare container images, Caddy routes, and iptables rules.
|
||||
- **Extended connectivity** — per-peer egress routing through alternate exits: WireGuard external, OpenVPN, or Tor. Configured via policy routing (fwmark + ip rule) in the WireGuard container.
|
||||
- **Cell-to-cell networking** — WireGuard-based site-to-site links between PIC cells with service-level access control (calendar, files, mail, WebDAV) and a peer-sync protocol.
|
||||
- **Certificate authority** — `vault_manager` issues and revokes TLS certificates for internal services.
|
||||
- **Network services** — CoreDNS (`.cell` TLD), dnsmasq DHCP, chrony NTP.
|
||||
- **Email** — Postfix + Dovecot via `docker-mailserver`.
|
||||
- **Calendar/contacts** — Radicale CalDAV/CardDAV.
|
||||
- **File storage** — WebDAV with per-user accounts; Filegator for browser-based file management.
|
||||
- **Container manager** — start/stop/inspect containers, pull images, manage volumes via the Docker SDK.
|
||||
- **Firewall manager** — iptables rule management (`firewall_manager.py`).
|
||||
- **Structured logging** — JSON logs with rotation (5 MB / 5 backups per service), log search, and per-service verbosity control.
|
||||
|
||||
---
|
||||
|
||||
## Requirements
|
||||
|
||||
- Linux host with the WireGuard kernel module loaded
|
||||
- Linux host with the WireGuard kernel module loaded (`modprobe wireguard` to verify)
|
||||
- Docker Engine and Docker Compose (v2 plugin or v1 standalone)
|
||||
- Python 3.10+ (for `make setup` and local dev only; not needed at runtime)
|
||||
- Python 3.10+ (for `make setup` and local development; not needed at runtime)
|
||||
- 2 GB+ RAM, 10 GB+ disk
|
||||
- Ports available: 53, 67/udp, 80, 443, 51820/udp, 25, 587, 993
|
||||
|
||||
@@ -44,62 +67,77 @@ The React frontend (`webui/`) is built with Vite + Tailwind CSS. All API calls g
|
||||
|
||||
## Quick Start
|
||||
|
||||
See [QUICKSTART.md](QUICKSTART.md) for step-by-step setup.
|
||||
See [QUICKSTART.md](QUICKSTART.md) for step-by-step instructions.
|
||||
|
||||
The short version:
|
||||
|
||||
```bash
|
||||
git clone gitea@192.168.31.50:roof/pic.git pic
|
||||
cd pic
|
||||
make start
|
||||
# open http://<host-ip>:8081 — the setup wizard appears automatically
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
Runtime configuration is controlled by `.env` in the project root. Copy `.env.example` to `.env` before first run.
|
||||
Port assignments and container IPs are configured in `.env` in the project root. A `.env` file is not required for first start — all variables have defaults. Create one only if you need to change ports or container IPs.
|
||||
|
||||
| Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `CELL_NETWORK` | `172.20.0.0/16` | Docker bridge subnet for all containers |
|
||||
| `CADDY_IP` through `FILEGATOR_IP` | `172.20.0.2`–`.13` | Static IP for each container |
|
||||
| `DNS_PORT` | `53` | DNS (UDP+TCP) |
|
||||
| `CELL_NETWORK` | `172.20.0.0/16` | Docker bridge subnet |
|
||||
| `CADDY_IP` through `FILEGATOR_IP` | `172.20.0.2`–`.13` | Static IP per container |
|
||||
| `DNS_PORT` | `53` | DNS (UDP + TCP) |
|
||||
| `DHCP_PORT` | `67` | DHCP (UDP) |
|
||||
| `NTP_PORT` | `123` | NTP (UDP) |
|
||||
| `WG_PORT` | `51820` | WireGuard listen port (UDP) |
|
||||
| `API_PORT` | `3000` | Flask API (bound to `127.0.0.1`) |
|
||||
| `API_PORT` | `3000` | Flask API (127.0.0.1 only) |
|
||||
| `WEBUI_PORT` | `8081` | React UI |
|
||||
| `MAIL_SMTP_PORT` | `25` | SMTP |
|
||||
| `MAIL_SUBMISSION_PORT` | `587` | SMTP submission |
|
||||
| `MAIL_IMAP_PORT` | `993` | IMAP |
|
||||
| `RADICALE_PORT` | `5232` | CalDAV (bound to `127.0.0.1`) |
|
||||
| `WEBDAV_PORT` | `8080` | WebDAV (bound to `127.0.0.1`) |
|
||||
| `RADICALE_PORT` | `5232` | CalDAV (127.0.0.1 only) |
|
||||
| `WEBDAV_PORT` | `8080` | WebDAV (127.0.0.1 only) |
|
||||
| `RAINLOOP_PORT` | `8888` | Webmail |
|
||||
| `FILEGATOR_PORT` | `8082` | File manager UI |
|
||||
| `WEBDAV_USER` | `admin` | WebDAV basic-auth username |
|
||||
| `WEBDAV_PASS` | _(required)_ | WebDAV basic-auth password — must be set before `make start` |
|
||||
| `FLASK_DEBUG` | _(unset)_ | Set to `1` to enable Flask debug mode; do not use in production |
|
||||
| `WEBDAV_PASS` | _(unset)_ | WebDAV basic-auth password |
|
||||
| `FLASK_DEBUG` | _(unset)_ | Set to `1` for Flask debug mode; do not use in production |
|
||||
| `PUID` / `PGID` | current user | UID/GID passed to the WireGuard container |
|
||||
|
||||
Cell identity (cell name, domain, VPN IP range) is configured via `make setup` or the Settings → Identity page in the UI after startup. The VPN IP range must be an RFC-1918 CIDR (`10.0.0.0/8`, `172.16.0.0/12`, or `192.168.0.0/16`); the API and UI both enforce this.
|
||||
Cell identity (cell name, domain mode, timezone) is set through the first-run wizard on first start, or later through the Settings page in the UI.
|
||||
|
||||
---
|
||||
|
||||
## Security Notes
|
||||
## Security
|
||||
|
||||
**Ports exposed to the network:**
|
||||
**Ports exposed on all interfaces by default:**
|
||||
|
||||
- `80` / `443` — Caddy (HTTP/HTTPS reverse proxy)
|
||||
- `51820/udp` — WireGuard
|
||||
- `25` / `587` / `993` — Mail (SMTP, submission, IMAP)
|
||||
- `53` — DNS (UDP + TCP)
|
||||
- `25` / `587` / `993` — mail
|
||||
- `53` — DNS
|
||||
- `67/udp` — DHCP
|
||||
- `8081` — Web UI
|
||||
- `8888` — Webmail (RainLoop)
|
||||
- `8082` — File manager (Filegator)
|
||||
|
||||
**Ports bound to `127.0.0.1` only** (not directly reachable from the network):
|
||||
**Ports bound to `127.0.0.1` only:**
|
||||
|
||||
- `3000` — Flask API
|
||||
- `5232` — Radicale (CalDAV)
|
||||
- `8080` — WebDAV
|
||||
- `8888` — Webmail
|
||||
- `8082` — Filegator
|
||||
|
||||
The API has no authentication layer. It relies on `is_local_request()` to restrict sensitive endpoints (containers, vault) to requests originating from loopback or the cell's Docker network. The Docker socket is mounted into `cell-api`; treat access to port 3000 as equivalent to root access on the host.
|
||||
The API uses session-based authentication (admin and peer roles). The Docker socket is mounted into `cell-api`; treat access to port 3000 as equivalent to root access on the host.
|
||||
|
||||
For internet-facing deployments, place the host behind a firewall or VPN and restrict access to the API and UI ports.
|
||||
Before setup is complete, all `/api/*` requests except `/api/setup/*` and `/health` return HTTP 428 and a redirect to `/setup`.
|
||||
|
||||
CSRF protection (double-submit token in `X-CSRF-Token` header) applies to all `POST`, `PUT`, `DELETE`, and `PATCH` requests on `/api/*` once a user session exists, except `/api/auth/*` and `/api/setup/*`.
|
||||
|
||||
Cell-to-cell peer-sync endpoints (`/api/cells/peer-sync/*`) authenticate via source IP and WireGuard public key, not session cookies.
|
||||
|
||||
For internet-facing deployments, place the host behind a firewall and restrict access to the API and UI ports.
|
||||
|
||||
---
|
||||
|
||||
@@ -123,7 +161,7 @@ cd webui && npm install && npm run dev
|
||||
# Follow all container logs
|
||||
make logs
|
||||
|
||||
# Follow logs for one service (e.g. api, dns, caddy, wireguard, mail)
|
||||
# Follow logs for one service
|
||||
make logs-api
|
||||
|
||||
# Open a shell inside a container
|
||||
@@ -135,41 +173,38 @@ make shell-api
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
make test # run the full pytest suite
|
||||
make test # run all unit tests (pytest, excludes e2e and integration)
|
||||
make test-coverage # run with coverage; HTML report in htmlcov/
|
||||
make test-api # run API endpoint tests only
|
||||
```
|
||||
|
||||
Tests live in `tests/` (34 files, 642 test functions). Coverage includes:
|
||||
|
||||
- All service managers (network, WireGuard, email, calendar, file, routing, vault, container)
|
||||
- API endpoint tests for each service area
|
||||
- Config manager (CRUD, validation, backup/restore)
|
||||
- IP utilities and Caddyfile generation
|
||||
- Peer registry and WireGuard peer lifecycle
|
||||
- Service bus pub/sub
|
||||
- Firewall manager
|
||||
- Pending-restart logic
|
||||
|
||||
Integration tests (`tests/integration/`) require a running PIC stack:
|
||||
Tests live in `tests/`. Integration tests require a running stack:
|
||||
|
||||
```bash
|
||||
make test-integration # full suite (creates peers)
|
||||
make test-integration # full suite (creates peers, modifies state)
|
||||
make test-integration-readonly # read-only checks, safe to run anytime
|
||||
```
|
||||
|
||||
End-to-end tests use Playwright:
|
||||
|
||||
```bash
|
||||
make test-e2e-deps # install Playwright and dependencies (run once)
|
||||
make test-e2e-api # API-level e2e tests
|
||||
make test-e2e-ui # UI-level e2e tests
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Management Commands
|
||||
|
||||
```bash
|
||||
make setup # generate WireGuard keys, write configs, create data dirs
|
||||
make start # docker compose up -d --build
|
||||
make start # docker compose up -d --build (full profile)
|
||||
make stop # docker compose down
|
||||
make restart # docker compose restart
|
||||
make status # container status + API health check
|
||||
make logs # follow all service logs
|
||||
make logs-<svc> # follow logs for one service
|
||||
make shell-<svc> # shell inside a container
|
||||
make logs-<svc> # follow logs for one service (e.g. make logs-api)
|
||||
make shell-<svc> # shell inside a container (e.g. make shell-api)
|
||||
|
||||
make update # git pull + rebuild + restart
|
||||
make reinstall # full wipe of config/ and data/, then setup + start
|
||||
@@ -180,7 +215,9 @@ make restore # list available backups
|
||||
|
||||
make list-peers # show WireGuard peers via API
|
||||
make show-routes # wg show inside the wireguard container
|
||||
make add-peer PEER_NAME=foo PEER_IP=10.0.0.5 PEER_KEY=<pubkey>
|
||||
|
||||
make show-admin-password # print current admin password
|
||||
make reset-admin-password # generate and set a new random admin password
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
+2
-6
@@ -199,8 +199,8 @@ def enforce_auth():
|
||||
backward-compatibility with pre-auth test suites.
|
||||
"""
|
||||
path = request.path
|
||||
# Always allow non-API paths and auth namespace
|
||||
if not path.startswith('/api/') or path.startswith('/api/auth/'):
|
||||
# Always allow non-API paths, auth namespace, and setup namespace
|
||||
if not path.startswith('/api/') or path.startswith('/api/auth/') or path.startswith('/api/setup/'):
|
||||
return None
|
||||
# Cell peer-sync endpoints authenticate via source IP + WG pubkey — not session
|
||||
if path.startswith('/api/cells/peer-sync/'):
|
||||
@@ -216,10 +216,6 @@ def enforce_auth():
|
||||
return None
|
||||
users = auth_manager.list_users()
|
||||
if not users:
|
||||
# Only fail closed when the auth file is readable but empty —
|
||||
# that's an explicit misconfiguration. If the file is missing or
|
||||
# unreadable (test env, wrong host path, permission denied), bypass
|
||||
# so pre-auth test suites continue to work.
|
||||
users_file = getattr(auth_manager, '_users_file', None)
|
||||
if users_file:
|
||||
try:
|
||||
|
||||
@@ -47,16 +47,6 @@ class AuthManager(BaseServiceManager):
|
||||
os.makedirs(os.path.dirname(self._users_file), exist_ok=True)
|
||||
except Exception:
|
||||
pass
|
||||
if not os.path.exists(self._users_file):
|
||||
try:
|
||||
with open(self._users_file, 'w') as f:
|
||||
f.write('[]')
|
||||
try:
|
||||
os.chmod(self._users_file, 0o600)
|
||||
except Exception:
|
||||
pass
|
||||
except Exception as e:
|
||||
self.logger.error(f'Could not create users file: {e}')
|
||||
|
||||
def _load_users(self) -> List[Dict[str, Any]]:
|
||||
with self._lock:
|
||||
|
||||
+29
-22
@@ -17,6 +17,7 @@ every 5 minutes, skipping the call when the IP has not changed.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import os
|
||||
import threading
|
||||
import time
|
||||
from typing import Any, Dict, Optional
|
||||
@@ -68,13 +69,25 @@ class PicNgoDDNS(DDNSProvider):
|
||||
DEFAULT_API_BASE = 'https://ddns.pic.ngo'
|
||||
TIMEOUT = 10
|
||||
|
||||
def __init__(self, api_base_url: Optional[str] = None):
|
||||
def __init__(self, api_base_url: Optional[str] = None, totp_secret: Optional[str] = None):
|
||||
self.api_base_url = (api_base_url or self.DEFAULT_API_BASE).rstrip('/')
|
||||
self._totp_secret = totp_secret or ''
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Internal helpers
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _otp_header(self) -> Dict[str, str]:
|
||||
"""Generate a fresh TOTP header for /register calls."""
|
||||
if not self._totp_secret:
|
||||
return {}
|
||||
try:
|
||||
import pyotp
|
||||
return {'X-Register-OTP': pyotp.TOTP(self._totp_secret).now()}
|
||||
except ImportError:
|
||||
logger.warning("pyotp not installed — X-Register-OTP header omitted")
|
||||
return {}
|
||||
|
||||
def _headers(self, token: Optional[str] = None) -> Dict[str, str]:
|
||||
h: Dict[str, str] = {'Content-Type': 'application/json'}
|
||||
if token:
|
||||
@@ -95,7 +108,8 @@ class PicNgoDDNS(DDNSProvider):
|
||||
"""POST /api/v1/register — register subdomain, returns token + subdomain."""
|
||||
url = f'{self.api_base_url}/api/v1/register'
|
||||
payload = {'name': name, 'ip': ip}
|
||||
resp = requests.post(url, json=payload, headers=self._headers(), timeout=self.TIMEOUT)
|
||||
headers = {**self._headers(), **self._otp_header()}
|
||||
resp = requests.post(url, json=payload, headers=headers, timeout=self.TIMEOUT)
|
||||
self._raise_for_status(resp, 'register')
|
||||
return resp.json()
|
||||
|
||||
@@ -280,11 +294,9 @@ class DDNSManager(BaseServiceManager):
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_status(self) -> Dict[str, Any]:
|
||||
identity = self._identity()
|
||||
domain_cfg = identity.get('domain', {})
|
||||
return {
|
||||
'service': 'ddns',
|
||||
'provider': domain_cfg.get('ddns', {}).get('provider') if domain_cfg else None,
|
||||
'provider': self._ddns_cfg().get('provider'),
|
||||
'last_ip': self._last_ip,
|
||||
'heartbeat_running': (
|
||||
self._heartbeat_thread is not None and
|
||||
@@ -310,17 +322,20 @@ class DDNSManager(BaseServiceManager):
|
||||
return {}
|
||||
return self.config_manager.get_identity() or {}
|
||||
|
||||
def _ddns_cfg(self) -> Dict[str, Any]:
|
||||
if self.config_manager is None:
|
||||
return {}
|
||||
return self.config_manager.configs.get('ddns', {}) or {}
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Provider factory
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_provider(self) -> Optional[DDNSProvider]:
|
||||
"""Instantiate and return the configured DDNS provider, or None."""
|
||||
identity = self._identity()
|
||||
domain_cfg = identity.get('domain', {})
|
||||
if not domain_cfg:
|
||||
if self.config_manager is None:
|
||||
return None
|
||||
ddns_cfg = domain_cfg.get('ddns', {})
|
||||
ddns_cfg = self.config_manager.configs.get('ddns', {})
|
||||
if not ddns_cfg:
|
||||
return None
|
||||
|
||||
@@ -330,7 +345,8 @@ class DDNSManager(BaseServiceManager):
|
||||
|
||||
if provider_name == 'pic_ngo':
|
||||
api_base = ddns_cfg.get('api_base_url')
|
||||
return PicNgoDDNS(api_base_url=api_base)
|
||||
totp_secret = ddns_cfg.get('totp_secret') or os.environ.get('DDNS_TOTP_SECRET', '')
|
||||
return PicNgoDDNS(api_base_url=api_base, totp_secret=totp_secret)
|
||||
|
||||
if provider_name == 'cloudflare':
|
||||
return CloudflareDDNS(
|
||||
@@ -405,10 +421,7 @@ class DDNSManager(BaseServiceManager):
|
||||
logger.debug("DDNS update_ip: IP unchanged (%s), skipping", current_ip)
|
||||
return
|
||||
|
||||
identity = self._identity()
|
||||
domain_cfg = identity.get('domain', {})
|
||||
ddns_cfg = domain_cfg.get('ddns', {}) if domain_cfg else {}
|
||||
token = ddns_cfg.get('token', '')
|
||||
token = self._ddns_cfg().get('token', '')
|
||||
|
||||
try:
|
||||
success = provider.update(token, current_ip)
|
||||
@@ -468,10 +481,7 @@ class DDNSManager(BaseServiceManager):
|
||||
provider = self.get_provider()
|
||||
if provider is None:
|
||||
raise DDNSError("No DDNS provider configured")
|
||||
identity = self._identity()
|
||||
domain_cfg = identity.get('domain', {})
|
||||
ddns_cfg = domain_cfg.get('ddns', {}) if domain_cfg else {}
|
||||
token = ddns_cfg.get('token', '')
|
||||
token = self._ddns_cfg().get('token', '')
|
||||
return provider.dns_challenge_create(token, fqdn, value)
|
||||
|
||||
def dns_challenge_delete(self, fqdn: str) -> bool:
|
||||
@@ -479,8 +489,5 @@ class DDNSManager(BaseServiceManager):
|
||||
provider = self.get_provider()
|
||||
if provider is None:
|
||||
raise DDNSError("No DDNS provider configured")
|
||||
identity = self._identity()
|
||||
domain_cfg = identity.get('domain', {})
|
||||
ddns_cfg = domain_cfg.get('ddns', {}) if domain_cfg else {}
|
||||
token = ddns_cfg.get('token', '')
|
||||
token = self._ddns_cfg().get('token', '')
|
||||
return provider.dns_challenge_delete(token, fqdn)
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
flask>=3.0.3
|
||||
flask-cors>=4.0.1
|
||||
requests>=2.32.3
|
||||
pyotp>=2.9.0
|
||||
cryptography>=42.0.5
|
||||
pyyaml==6.0.1
|
||||
icalendar==5.0.7
|
||||
|
||||
+18
-7
@@ -128,9 +128,12 @@ class SetupManager:
|
||||
cell_name = payload.get('cell_name', '')
|
||||
password = payload.get('password', '')
|
||||
domain_mode = payload.get('domain_mode', '')
|
||||
domain_name = payload.get('domain_name', '')
|
||||
timezone = payload.get('timezone', '')
|
||||
services_enabled = payload.get('services_enabled', [])
|
||||
ddns_provider = payload.get('ddns_provider', 'none')
|
||||
cloudflare_api_token = payload.get('cloudflare_api_token', '')
|
||||
duckdns_token = payload.get('duckdns_token', '')
|
||||
|
||||
errors.extend(self.validate_cell_name(cell_name))
|
||||
errors.extend(self.validate_password(password))
|
||||
@@ -168,28 +171,36 @@ class SetupManager:
|
||||
if self.is_setup_complete():
|
||||
return {'success': False, 'errors': ['Setup has already been completed.']}
|
||||
|
||||
# ── create admin user ──────────────────────────────────────────
|
||||
# ── create or update admin user ────────────────────────────────
|
||||
# The installer may have bootstrapped an admin account from a
|
||||
# generated password. The wizard's job is to set the real password,
|
||||
# so update it if the account already exists.
|
||||
ok = self.auth_manager.create_user(
|
||||
username='admin',
|
||||
password=password,
|
||||
role='admin',
|
||||
)
|
||||
if not ok:
|
||||
return {'success': False, 'errors': ['Failed to create admin user. The username may already exist.']}
|
||||
ok = self.auth_manager.update_password('admin', password)
|
||||
if not ok:
|
||||
return {'success': False, 'errors': ['Failed to set admin password.']}
|
||||
|
||||
# ── persist identity fields ────────────────────────────────────
|
||||
self.config_manager.set_identity_field('cell_name', cell_name)
|
||||
self.config_manager.set_identity_field('domain_mode', domain_mode)
|
||||
if domain_name:
|
||||
self.config_manager.set_identity_field('domain_name', domain_name)
|
||||
self.config_manager.set_identity_field('timezone', timezone)
|
||||
self.config_manager.set_identity_field('services_enabled', services_enabled)
|
||||
self.config_manager.set_identity_field('ddns_provider', ddns_provider)
|
||||
if cloudflare_api_token:
|
||||
self.config_manager.set_identity_field('cloudflare_api_token', cloudflare_api_token)
|
||||
if duckdns_token:
|
||||
self.config_manager.set_identity_field('duckdns_token', duckdns_token)
|
||||
|
||||
# NOTE: DDNS registration is deferred to Phase 3.
|
||||
# For now we just store ddns_provider in config.
|
||||
logger.info(
|
||||
'DDNS registration skipped (Phase 1). '
|
||||
'DDNS registration will happen in Phase 3. '
|
||||
f'ddns_provider={ddns_provider!r} stored in identity config.'
|
||||
'DDNS registration deferred to Phase 3. '
|
||||
f'ddns_provider={ddns_provider!r} domain_name={domain_name!r}'
|
||||
)
|
||||
|
||||
# ── mark setup complete (must be last) ─────────────────────────
|
||||
|
||||
@@ -1,36 +0,0 @@
|
||||
{
|
||||
"cell_name": "modified",
|
||||
"domain": "cell.local",
|
||||
"ip_range": "10.0.0.0/24",
|
||||
"network": {
|
||||
"dns_port": 53,
|
||||
"dhcp_range": "10.0.0.100-10.0.0.200",
|
||||
"ntp_servers": ["pool.ntp.org"]
|
||||
},
|
||||
"wireguard": {
|
||||
"port": 51820,
|
||||
"private_key": "test_key",
|
||||
"address": "10.0.0.1/24"
|
||||
},
|
||||
"email": {
|
||||
"domain": "cell.local",
|
||||
"smtp_port": 25,
|
||||
"imap_port": 143
|
||||
},
|
||||
"calendar": {
|
||||
"port": 5232,
|
||||
"data_dir": "/app/data/calendar"
|
||||
},
|
||||
"files": {
|
||||
"port": 8080,
|
||||
"data_dir": "/app/data/files"
|
||||
},
|
||||
"routing": {
|
||||
"nat_enabled": true,
|
||||
"firewall_enabled": true
|
||||
},
|
||||
"vault": {
|
||||
"ca_configured": true,
|
||||
"fernet_configured": true
|
||||
}
|
||||
}
|
||||
@@ -1,389 +0,0 @@
|
||||
# Personal Internet Cell - Network Configuration Guide
|
||||
|
||||
This guide explains how to configure networking for the Personal Internet Cell to provide internet access to WireGuard VPN clients.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Overview](#overview)
|
||||
2. [Network Architecture](#network-architecture)
|
||||
3. [Quick Setup](#quick-setup)
|
||||
4. [Detailed Configuration](#detailed-configuration)
|
||||
5. [Troubleshooting](#troubleshooting)
|
||||
6. [Advanced Configuration](#advanced-configuration)
|
||||
7. [Security Considerations](#security-considerations)
|
||||
|
||||
## Overview
|
||||
|
||||
The Personal Internet Cell provides a complete VPN solution with internet access. This requires proper configuration of:
|
||||
|
||||
- **IP Forwarding**: Allow traffic to pass through the server
|
||||
- **NAT (Network Address Translation)**: Translate private IPs to public IPs
|
||||
- **Routing**: Direct traffic from VPN clients to the internet
|
||||
- **Firewall Rules**: Control traffic flow and security
|
||||
|
||||
## Network Architecture
|
||||
|
||||
```
|
||||
Internet
|
||||
│
|
||||
▼
|
||||
[Host Server] (195.178.106.244)
|
||||
│
|
||||
├── [Docker Network] (172.20.0.0/16)
|
||||
│ └── [WireGuard Container] (cell-wireguard)
|
||||
│ └── [WireGuard Interface] (wg0: 10.0.0.1/24)
|
||||
│
|
||||
└── [VPN Clients] (10.0.0.2-10.0.0.254/24)
|
||||
└── [Internet Access via NAT]
|
||||
```
|
||||
|
||||
### Key Components
|
||||
|
||||
- **Host Interface**: `eth0` (or main network interface)
|
||||
- **WireGuard Interface**: `wg0` (10.0.0.1/24)
|
||||
- **Client Network**: `10.0.0.0/24`
|
||||
- **NAT Translation**: Client IPs → Host IP
|
||||
|
||||
## Quick Setup
|
||||
|
||||
### 1. Run the Network Configuration Script
|
||||
|
||||
```bash
|
||||
# Make the script executable (if not already done)
|
||||
chmod +x /opt/pic/scripts/setup-network.sh
|
||||
|
||||
# Run the configuration
|
||||
sudo /opt/pic/scripts/setup-network.sh setup
|
||||
```
|
||||
|
||||
### 2. Verify Configuration
|
||||
|
||||
```bash
|
||||
# Check status
|
||||
sudo /opt/pic/scripts/setup-network.sh status
|
||||
|
||||
# Test configuration
|
||||
sudo /opt/pic/scripts/setup-network.sh test
|
||||
```
|
||||
|
||||
### 3. Connect a VPN Client
|
||||
|
||||
Use the generated WireGuard configuration to connect a client. The client should now have internet access.
|
||||
|
||||
## Detailed Configuration
|
||||
|
||||
### IP Forwarding
|
||||
|
||||
IP forwarding allows the server to route packets between different network interfaces.
|
||||
|
||||
**Enable on Host:**
|
||||
```bash
|
||||
echo 'net.ipv4.ip_forward = 1' >> /etc/sysctl.conf
|
||||
sysctl -p
|
||||
```
|
||||
|
||||
**Enable in Container:**
|
||||
```bash
|
||||
docker exec cell-wireguard sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
|
||||
```
|
||||
|
||||
### NAT Configuration
|
||||
|
||||
NAT (Network Address Translation) allows VPN clients to access the internet using the server's public IP.
|
||||
|
||||
**Container NAT Rules:**
|
||||
```bash
|
||||
# Allow forwarding for WireGuard traffic
|
||||
iptables -A FORWARD -i wg0 -j ACCEPT
|
||||
iptables -A FORWARD -o wg0 -j ACCEPT
|
||||
|
||||
# NAT rule for internet access
|
||||
iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -o eth0 -j MASQUERADE
|
||||
```
|
||||
|
||||
**Host NAT Rules:**
|
||||
```bash
|
||||
# Allow traffic from WireGuard network
|
||||
iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -o eth0 -j MASQUERADE
|
||||
iptables -A FORWARD -i wg0 -j ACCEPT
|
||||
iptables -A FORWARD -o wg0 -j ACCEPT
|
||||
```
|
||||
|
||||
### Routing Configuration
|
||||
|
||||
**WireGuard Interface Setup:**
|
||||
```bash
|
||||
# Create WireGuard interface
|
||||
ip link add dev wg0 type wireguard
|
||||
|
||||
# Set private key
|
||||
wg set wg0 private-key /path/to/private-key
|
||||
|
||||
# Set listen port
|
||||
wg set wg0 listen-port 51820
|
||||
|
||||
# Add IP address
|
||||
ip addr add 10.0.0.1/24 dev wg0
|
||||
|
||||
# Bring interface up
|
||||
ip link set wg0 up
|
||||
|
||||
# Add peers
|
||||
wg set wg0 peer <public-key> allowed-ips 10.0.0.2/32
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
#### 1. VPN Connected but No Internet
|
||||
|
||||
**Symptoms:**
|
||||
- WireGuard shows connected
|
||||
- Can ping server (10.0.0.1)
|
||||
- Cannot access internet
|
||||
|
||||
**Solutions:**
|
||||
```bash
|
||||
# Check IP forwarding
|
||||
cat /proc/sys/net/ipv4/ip_forward
|
||||
# Should return 1
|
||||
|
||||
# Check NAT rules
|
||||
iptables -t nat -L POSTROUTING -n
|
||||
# Should show MASQUERADE rule for 10.0.0.0/24
|
||||
|
||||
# Check forwarding rules
|
||||
iptables -L FORWARD -n
|
||||
# Should show ACCEPT rules for wg0
|
||||
|
||||
# Restart network configuration
|
||||
sudo /opt/pic/scripts/setup-network.sh reset
|
||||
sudo /opt/pic/scripts/setup-network.sh setup
|
||||
```
|
||||
|
||||
#### 2. Cannot Connect to VPN
|
||||
|
||||
**Symptoms:**
|
||||
- WireGuard client cannot connect
|
||||
- No handshake in server logs
|
||||
|
||||
**Solutions:**
|
||||
```bash
|
||||
# Check WireGuard interface
|
||||
docker exec cell-wireguard wg show
|
||||
|
||||
# Check if port 51820 is open
|
||||
netstat -ulnp | grep 51820
|
||||
|
||||
# Check firewall rules
|
||||
ufw status
|
||||
iptables -L INPUT -n
|
||||
|
||||
# Check Docker port mapping
|
||||
docker port cell-wireguard
|
||||
```
|
||||
|
||||
#### 3. DNS Issues
|
||||
|
||||
**Symptoms:**
|
||||
- Can ping IP addresses
|
||||
- Cannot resolve domain names
|
||||
|
||||
**Solutions:**
|
||||
```bash
|
||||
# Check DNS configuration in client config
|
||||
# Should include: DNS = 8.8.8.8, 1.1.1.1
|
||||
|
||||
# Test DNS from container
|
||||
docker exec cell-wireguard nslookup google.com
|
||||
|
||||
# Check if DNS is being blocked
|
||||
docker exec cell-wireguard iptables -L -n | grep 53
|
||||
```
|
||||
|
||||
### Diagnostic Commands
|
||||
|
||||
```bash
|
||||
# Check network status
|
||||
sudo /opt/pic/scripts/setup-network.sh status
|
||||
|
||||
# Test connectivity from container
|
||||
docker exec cell-wireguard ping -c 3 8.8.8.8
|
||||
|
||||
# Check routing table
|
||||
docker exec cell-wireguard ip route show
|
||||
|
||||
# Check interface status
|
||||
docker exec cell-wireguard ip addr show wg0
|
||||
|
||||
# Check NAT rules
|
||||
docker exec cell-wireguard iptables -t nat -L -n
|
||||
|
||||
# Check forwarding rules
|
||||
docker exec cell-wireguard iptables -L FORWARD -n
|
||||
```
|
||||
|
||||
## Advanced Configuration
|
||||
|
||||
### Custom DNS Servers
|
||||
|
||||
To use custom DNS servers, modify the WireGuard client configuration:
|
||||
|
||||
```ini
|
||||
[Interface]
|
||||
PrivateKey = <private-key>
|
||||
Address = 10.0.0.2/32
|
||||
DNS = 1.1.1.1, 1.0.0.1, 8.8.8.8, 8.8.4.4
|
||||
|
||||
[Peer]
|
||||
PublicKey = <server-public-key>
|
||||
Endpoint = 195.178.106.244:51820
|
||||
AllowedIPs = 0.0.0.0/0
|
||||
PersistentKeepalive = 25
|
||||
```
|
||||
|
||||
### Split Tunneling
|
||||
|
||||
To allow only specific traffic through the VPN:
|
||||
|
||||
```ini
|
||||
[Peer]
|
||||
AllowedIPs = 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16
|
||||
# Only route private networks through VPN
|
||||
```
|
||||
|
||||
### Port Forwarding
|
||||
|
||||
To forward specific ports to VPN clients:
|
||||
|
||||
```bash
|
||||
# Forward port 8080 to client 10.0.0.2
|
||||
iptables -t nat -A PREROUTING -p tcp --dport 8080 -j DNAT --to-destination 10.0.0.2:8080
|
||||
iptables -A FORWARD -p tcp -d 10.0.0.2 --dport 8080 -j ACCEPT
|
||||
```
|
||||
|
||||
### Bandwidth Limiting
|
||||
|
||||
To limit bandwidth for VPN clients:
|
||||
|
||||
```bash
|
||||
# Install tc (traffic control)
|
||||
apt-get install iproute2
|
||||
|
||||
# Limit client 10.0.0.2 to 1Mbps
|
||||
tc qdisc add dev wg0 root handle 1: htb default 30
|
||||
tc class add dev wg0 parent 1: classid 1:1 htb rate 1mbit
|
||||
tc class add dev wg0 parent 1:1 classid 1:10 htb rate 1mbit ceil 1mbit
|
||||
tc filter add dev wg0 protocol ip parent 1:0 prio 1 u32 match ip dst 10.0.0.2 flowid 1:10
|
||||
```
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Firewall Rules
|
||||
|
||||
**Basic Security Rules:**
|
||||
```bash
|
||||
# Drop invalid packets
|
||||
iptables -A INPUT -m conntrack --ctstate INVALID -j DROP
|
||||
|
||||
# Allow established connections
|
||||
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
|
||||
|
||||
# Allow WireGuard traffic
|
||||
iptables -A INPUT -p udp --dport 51820 -j ACCEPT
|
||||
|
||||
# Allow SSH (if needed)
|
||||
iptables -A INPUT -p tcp --dport 22 -j ACCEPT
|
||||
|
||||
# Drop everything else
|
||||
iptables -A INPUT -j DROP
|
||||
```
|
||||
|
||||
### Client Isolation
|
||||
|
||||
To prevent clients from communicating with each other:
|
||||
|
||||
```bash
|
||||
# Block inter-client communication
|
||||
iptables -A FORWARD -i wg0 -o wg0 -j DROP
|
||||
```
|
||||
|
||||
### Logging
|
||||
|
||||
To log VPN traffic:
|
||||
|
||||
```bash
|
||||
# Log all WireGuard traffic
|
||||
iptables -A FORWARD -i wg0 -j LOG --log-prefix "WG-FORWARD: "
|
||||
iptables -A FORWARD -o wg0 -j LOG --log-prefix "WG-FORWARD: "
|
||||
|
||||
# Log NAT traffic
|
||||
iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -j LOG --log-prefix "WG-NAT: "
|
||||
```
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Real-time Monitoring
|
||||
|
||||
```bash
|
||||
# Monitor WireGuard connections
|
||||
watch -n 1 "docker exec cell-wireguard wg show"
|
||||
|
||||
# Monitor traffic
|
||||
watch -n 1 "docker exec cell-wireguard wg show wg0 transfer"
|
||||
|
||||
# Monitor NAT rules
|
||||
watch -n 1 "iptables -t nat -L POSTROUTING -n -v"
|
||||
```
|
||||
|
||||
### Log Analysis
|
||||
|
||||
```bash
|
||||
# Check system logs
|
||||
journalctl -u pic-network.service -f
|
||||
|
||||
# Check iptables logs
|
||||
tail -f /var/log/kern.log | grep WG-
|
||||
|
||||
# Check Docker logs
|
||||
docker logs cell-wireguard -f
|
||||
```
|
||||
|
||||
## Backup and Recovery
|
||||
|
||||
### Backup Configuration
|
||||
|
||||
```bash
|
||||
# Backup iptables rules
|
||||
iptables-save > /opt/pic/backups/iptables-backup-$(date +%Y%m%d).rules
|
||||
|
||||
# Backup WireGuard configuration
|
||||
cp /opt/pic/config/wireguard/wg_confs/wg0.conf /opt/pic/backups/wg0-backup-$(date +%Y%m%d).conf
|
||||
|
||||
# Backup network script
|
||||
cp /opt/pic/scripts/setup-network.sh /opt/pic/backups/setup-network-backup-$(date +%Y%m%d).sh
|
||||
```
|
||||
|
||||
### Restore Configuration
|
||||
|
||||
```bash
|
||||
# Restore iptables rules
|
||||
iptables-restore < /opt/pic/backups/iptables-backup-YYYYMMDD.rules
|
||||
|
||||
# Restore WireGuard configuration
|
||||
cp /opt/pic/backups/wg0-backup-YYYYMMDD.conf /opt/pic/config/wireguard/wg_confs/wg0.conf
|
||||
docker restart cell-wireguard
|
||||
```
|
||||
|
||||
## Support
|
||||
|
||||
If you encounter issues:
|
||||
|
||||
1. Check the troubleshooting section above
|
||||
2. Run the diagnostic commands
|
||||
3. Check the logs for error messages
|
||||
4. Verify your network configuration
|
||||
5. Test with a simple client configuration
|
||||
|
||||
For additional help, check the main Personal Internet Cell documentation or create an issue in the project repository.
|
||||
@@ -1,51 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Script to fix import statements in test files
|
||||
"""
|
||||
|
||||
import os
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
def fix_imports_in_file(file_path):
|
||||
"""Fix import statements in a test file"""
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
content = f.read()
|
||||
|
||||
# Fix relative imports to absolute imports from api package
|
||||
content = re.sub(r'from \.(\w+) import', r'from \1 import', content)
|
||||
content = re.sub(r'import \.(\w+)', r'import \1', content)
|
||||
|
||||
# Add path setup if not present
|
||||
if 'sys.path.insert' not in content and 'api_dir' not in content:
|
||||
path_setup = '''import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Add api directory to path
|
||||
api_dir = Path(__file__).parent.parent / 'api'
|
||||
sys.path.insert(0, str(api_dir))
|
||||
|
||||
'''
|
||||
# Insert after the first import line
|
||||
lines = content.split('\n')
|
||||
for i, line in enumerate(lines):
|
||||
if line.startswith('import ') or line.startswith('from '):
|
||||
lines.insert(i, path_setup.rstrip())
|
||||
break
|
||||
content = '\n'.join(lines)
|
||||
|
||||
with open(file_path, 'w', encoding='utf-8') as f:
|
||||
f.write(content)
|
||||
|
||||
print(f"Fixed imports in {file_path}")
|
||||
|
||||
def main():
|
||||
"""Fix all test files"""
|
||||
tests_dir = Path('tests')
|
||||
|
||||
for test_file in tests_dir.glob('test_*.py'):
|
||||
if test_file.name not in ['test_cli_tool.py', 'test_peer_registry.py']: # Already fixed
|
||||
fix_imports_in_file(test_file)
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -1,31 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Fix import statements in test files
|
||||
"""
|
||||
|
||||
import os
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
def fix_imports_in_file(file_path):
|
||||
"""Fix import statements in a test file"""
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
content = f.read()
|
||||
|
||||
# Replace 'from api.' with 'from .'
|
||||
content = re.sub(r'from api\.', 'from .', content)
|
||||
content = re.sub(r'import api\.', 'import .', content)
|
||||
|
||||
with open(file_path, 'w', encoding='utf-8') as f:
|
||||
f.write(content)
|
||||
|
||||
print(f"Fixed imports in {file_path}")
|
||||
|
||||
def main():
|
||||
tests_dir = Path('tests')
|
||||
|
||||
for test_file in tests_dir.glob('test_*.py'):
|
||||
fix_imports_in_file(test_file)
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
+10
-5
@@ -245,23 +245,28 @@ else
|
||||
log_ok "Repository cloned to ${PIC_DIR}"
|
||||
fi
|
||||
|
||||
# Ensure the pic user owns the directory
|
||||
chown -R "${PIC_USER}:${PIC_USER}" "$PIC_DIR"
|
||||
# Give the invoking user (or pic if run directly as root) ownership of the repo
|
||||
# so they can run `make update` and other git commands without sudo.
|
||||
REPO_OWNER="${SUDO_USER:-${PIC_USER}}"
|
||||
chown -R "${REPO_OWNER}:${REPO_OWNER}" "$PIC_DIR"
|
||||
# Allow all users to run git commands here regardless of who owns the files
|
||||
git config --system --add safe.directory "$PIC_DIR" 2>/dev/null || true
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Step 5 — Run make install
|
||||
# ---------------------------------------------------------------------------
|
||||
log_step 5 "Running 'make install'..."
|
||||
|
||||
# make install generates config, writes the systemd unit, and touches .installed.
|
||||
# We run it as the pic user (via sudo -u) so files get correct ownership, but
|
||||
# make install itself calls sudo internally where root is needed.
|
||||
cd "$PIC_DIR"
|
||||
|
||||
if ! make install 2>&1 | sed 's/^/ /'; then
|
||||
die "'make install' failed. Check the output above."
|
||||
fi
|
||||
|
||||
# make install runs as root so config/ and data/ get created root-owned.
|
||||
# Re-apply ownership to the invoking user so they can manage files without sudo.
|
||||
chown -R "${REPO_OWNER}:${REPO_OWNER}" "$PIC_DIR"
|
||||
|
||||
log_ok "'make install' complete"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Update the cell's DDNS record with the current public IP.
|
||||
|
||||
Called by: make ddns-update
|
||||
systemd timer (optional, see scripts/pic-ddns-update.timer)
|
||||
|
||||
Reads the DDNS token from data/api/.ddns_token (written by setup_cell.py).
|
||||
Exits 0 on success or if already up to date, non-zero on failure.
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
DDNS_URL = os.environ.get('DDNS_URL', 'https://ddns.pic.ngo/api/v1')
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
TOKEN_FILE = os.path.join(ROOT, 'data', 'api', '.ddns_token')
|
||||
IP_CACHE_FILE = os.path.join(ROOT, 'data', 'api', '.ddns_last_ip')
|
||||
|
||||
|
||||
def get_public_ip() -> str:
|
||||
return urllib.request.urlopen('https://api.ipify.org', timeout=5).read().decode().strip()
|
||||
|
||||
|
||||
def read_token() -> str:
|
||||
if not os.path.exists(TOKEN_FILE):
|
||||
print('ERROR: DDNS token not found. Run "make setup" to register.', file=sys.stderr)
|
||||
sys.exit(1)
|
||||
return open(TOKEN_FILE).read().strip()
|
||||
|
||||
|
||||
def read_last_ip() -> str:
|
||||
try:
|
||||
return open(IP_CACHE_FILE).read().strip()
|
||||
except FileNotFoundError:
|
||||
return ''
|
||||
|
||||
|
||||
def write_last_ip(ip: str) -> None:
|
||||
with open(IP_CACHE_FILE, 'w') as f:
|
||||
f.write(ip)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
try:
|
||||
public_ip = get_public_ip()
|
||||
except Exception as e:
|
||||
print(f'ERROR: Could not detect public IP: {e}', file=sys.stderr)
|
||||
return 1
|
||||
|
||||
last_ip = read_last_ip()
|
||||
if public_ip == last_ip:
|
||||
print(f'DDNS: IP unchanged ({public_ip}) — no update needed')
|
||||
return 0
|
||||
|
||||
token = read_token()
|
||||
data = json.dumps({'token': token, 'ip': public_ip}).encode()
|
||||
req = urllib.request.Request(
|
||||
f'{DDNS_URL}/update',
|
||||
data=data,
|
||||
headers={'Content-Type': 'application/json'},
|
||||
method='PUT',
|
||||
)
|
||||
try:
|
||||
resp = urllib.request.urlopen(req, timeout=10)
|
||||
result = json.loads(resp.read())
|
||||
if result.get('updated'):
|
||||
write_last_ip(public_ip)
|
||||
print(f'DDNS: Updated to {public_ip}')
|
||||
return 0
|
||||
else:
|
||||
print(f'ERROR: Unexpected response: {result}', file=sys.stderr)
|
||||
return 1
|
||||
except urllib.error.HTTPError as e:
|
||||
body = e.read().decode()
|
||||
print(f'ERROR: DDNS update failed ({e.code}): {body}', file=sys.stderr)
|
||||
return 1
|
||||
except Exception as e:
|
||||
print(f'ERROR: DDNS update failed: {e}', file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
+76
-1
@@ -185,7 +185,13 @@ def write_cell_config(cell_name: str, domain: str, port: int):
|
||||
'domain': domain,
|
||||
'ip_range': '172.20.0.0/16',
|
||||
'wireguard_port': port,
|
||||
}
|
||||
},
|
||||
'ddns': {
|
||||
'provider': 'pic_ngo',
|
||||
'api_base_url': DDNS_URL.replace('/api/v1', ''),
|
||||
'totp_secret': DDNS_TOTP_SECRET,
|
||||
'enabled': True,
|
||||
},
|
||||
}
|
||||
with open(cfg_path, 'w') as f:
|
||||
json.dump(config, f, indent=2)
|
||||
@@ -238,6 +244,74 @@ def ensure_session_secret():
|
||||
print('[CREATED] data/api/.session_secret')
|
||||
|
||||
|
||||
DDNS_URL = os.environ.get('DDNS_URL', 'https://ddns.pic.ngo/api/v1')
|
||||
DDNS_TOTP_SECRET = os.environ.get('DDNS_TOTP_SECRET', 'S6UMA464YIKM74QHXWL5WELDIO3HFZ6K')
|
||||
|
||||
|
||||
def register_with_ddns(cell_name: str) -> None:
|
||||
"""Register cell_name.pic.ngo with the DDNS server using TOTP auth.
|
||||
|
||||
Idempotent: if a token file already exists the registration is skipped.
|
||||
Skipped silently if DDNS_TOTP_SECRET is not set.
|
||||
"""
|
||||
token_path = os.path.join(ROOT, 'data', 'api', '.ddns_token')
|
||||
if os.path.exists(token_path):
|
||||
print('[EXISTS] DDNS registration — token already present')
|
||||
return
|
||||
|
||||
if not DDNS_TOTP_SECRET:
|
||||
print('[SKIP] DDNS_TOTP_SECRET not set — skipping DDNS registration')
|
||||
return
|
||||
|
||||
import urllib.request
|
||||
import urllib.error
|
||||
|
||||
# Detect public IP
|
||||
try:
|
||||
public_ip = urllib.request.urlopen(
|
||||
'https://api.ipify.org', timeout=5
|
||||
).read().decode().strip()
|
||||
except Exception as e:
|
||||
print(f'[WARN] Could not detect public IP: {e} — skipping DDNS registration')
|
||||
return
|
||||
|
||||
# Generate TOTP code (requires pyotp; if not available fall back gracefully)
|
||||
try:
|
||||
import pyotp
|
||||
otp = pyotp.TOTP(DDNS_TOTP_SECRET).now()
|
||||
except ImportError:
|
||||
# Try python3 -c as a subprocess fallback
|
||||
try:
|
||||
otp = subprocess.check_output(
|
||||
['python3', '-c', f"import pyotp; print(pyotp.TOTP('{DDNS_TOTP_SECRET}').now())"]
|
||||
).decode().strip()
|
||||
except Exception as e:
|
||||
print(f'[WARN] pyotp not available and fallback failed: {e} — skipping DDNS')
|
||||
return
|
||||
|
||||
data = json.dumps({'name': cell_name, 'ip': public_ip}).encode()
|
||||
req = urllib.request.Request(
|
||||
f'{DDNS_URL}/register',
|
||||
data=data,
|
||||
headers={'Content-Type': 'application/json', 'X-Register-OTP': otp},
|
||||
method='POST',
|
||||
)
|
||||
try:
|
||||
resp = urllib.request.urlopen(req, timeout=10)
|
||||
result = json.loads(resp.read())
|
||||
token = result['token']
|
||||
os.makedirs(os.path.dirname(token_path), exist_ok=True)
|
||||
with open(token_path, 'w') as f:
|
||||
f.write(token)
|
||||
os.chmod(token_path, 0o600)
|
||||
print(f'[CREATED] DDNS registration: {result["subdomain"]} ip={public_ip}')
|
||||
except urllib.error.HTTPError as e:
|
||||
body = e.read().decode()
|
||||
print(f'[WARN] DDNS registration failed ({e.code}): {body}')
|
||||
except Exception as e:
|
||||
print(f'[WARN] DDNS registration failed: {e}')
|
||||
|
||||
|
||||
def bootstrap_admin_password():
|
||||
import secrets as _secrets
|
||||
users_file = os.path.join(ROOT, 'data', 'api', 'auth_users.json')
|
||||
@@ -303,6 +377,7 @@ def main():
|
||||
write_caddy_config(ip_range, cell_name, domain)
|
||||
ensure_session_secret()
|
||||
bootstrap_admin_password()
|
||||
register_with_ddns(cell_name)
|
||||
|
||||
print()
|
||||
print('--- Setup complete! Run: make start ---')
|
||||
|
||||
@@ -1,559 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Comprehensive tests for Flask app endpoints
|
||||
"""
|
||||
|
||||
import unittest
|
||||
import sys
|
||||
import os
|
||||
import tempfile
|
||||
import shutil
|
||||
import json
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch, MagicMock
|
||||
|
||||
# Add api directory to path
|
||||
api_dir = Path(__file__).parent / 'api'
|
||||
sys.path.insert(0, str(api_dir))
|
||||
|
||||
class TestFlaskAppEndpoints(unittest.TestCase):
|
||||
def setUp(self):
|
||||
"""Set up test environment"""
|
||||
# Create temporary directories
|
||||
self.test_dir = tempfile.mkdtemp()
|
||||
self.data_dir = os.path.join(self.test_dir, 'data')
|
||||
self.config_dir = os.path.join(self.test_dir, 'config')
|
||||
os.makedirs(self.data_dir, exist_ok=True)
|
||||
os.makedirs(self.config_dir, exist_ok=True)
|
||||
|
||||
# Set environment variables
|
||||
os.environ['TESTING'] = 'true'
|
||||
os.environ['LOG_LEVEL'] = 'ERROR'
|
||||
|
||||
# Import and create app
|
||||
from app import app
|
||||
self.app = app
|
||||
self.client = app.test_client()
|
||||
|
||||
# Mock external dependencies
|
||||
self.patchers = []
|
||||
|
||||
# Mock subprocess.run
|
||||
subprocess_patcher = patch('subprocess.run')
|
||||
self.mock_subprocess = subprocess_patcher.start()
|
||||
self.mock_subprocess.return_value.returncode = 0
|
||||
self.mock_subprocess.return_value.stdout = b"test output"
|
||||
self.patchers.append(subprocess_patcher)
|
||||
|
||||
# Mock docker
|
||||
docker_patcher = patch('docker.from_env')
|
||||
self.mock_docker = docker_patcher.start()
|
||||
self.mock_docker_client = MagicMock()
|
||||
self.mock_docker.return_value = self.mock_docker_client
|
||||
self.patchers.append(docker_patcher)
|
||||
|
||||
# Mock file operations
|
||||
file_patcher = patch('builtins.open', create=True)
|
||||
self.mock_file = file_patcher.start()
|
||||
self.mock_file.return_value.__enter__.return_value.read.return_value = '{}'
|
||||
self.patchers.append(file_patcher)
|
||||
|
||||
def tearDown(self):
|
||||
"""Clean up test environment"""
|
||||
shutil.rmtree(self.test_dir)
|
||||
for patcher in self.patchers:
|
||||
patcher.stop()
|
||||
|
||||
def test_health_endpoint(self):
|
||||
"""Test /health endpoint"""
|
||||
response = self.client.get('/health')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('status', data)
|
||||
|
||||
def test_api_status_endpoint(self):
|
||||
"""Test /api/status endpoint"""
|
||||
response = self.client.get('/api/status')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('status', data)
|
||||
|
||||
def test_api_config_get_endpoint(self):
|
||||
"""Test GET /api/config endpoint"""
|
||||
response = self.client.get('/api/config')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIsInstance(data, dict)
|
||||
|
||||
def test_api_config_put_endpoint(self):
|
||||
"""Test PUT /api/config endpoint"""
|
||||
test_config = {'test': 'value'}
|
||||
response = self.client.put('/api/config',
|
||||
data=json.dumps(test_config),
|
||||
content_type='application/json')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
def test_api_config_backup_endpoint(self):
|
||||
"""Test POST /api/config/backup endpoint"""
|
||||
response = self.client.post('/api/config/backup')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('backup_id', data)
|
||||
|
||||
def test_api_config_backups_endpoint(self):
|
||||
"""Test GET /api/config/backups endpoint"""
|
||||
response = self.client.get('/api/config/backups')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIsInstance(data, list)
|
||||
|
||||
def test_api_config_restore_endpoint(self):
|
||||
"""Test POST /api/config/restore/<backup_id> endpoint"""
|
||||
response = self.client.post('/api/config/restore/test_backup')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
def test_api_config_export_endpoint(self):
|
||||
"""Test GET /api/config/export endpoint"""
|
||||
response = self.client.get('/api/config/export')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIsInstance(data, dict)
|
||||
|
||||
def test_api_config_import_endpoint(self):
|
||||
"""Test POST /api/config/import endpoint"""
|
||||
test_config = {'test': 'value'}
|
||||
response = self.client.post('/api/config/import',
|
||||
data=json.dumps(test_config),
|
||||
content_type='application/json')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
def test_api_services_bus_status_endpoint(self):
|
||||
"""Test GET /api/services/bus/status endpoint"""
|
||||
response = self.client.get('/api/services/bus/status')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('services', data)
|
||||
|
||||
def test_api_services_bus_events_endpoint(self):
|
||||
"""Test GET /api/services/bus/events endpoint"""
|
||||
response = self.client.get('/api/services/bus/events')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIsInstance(data, list)
|
||||
|
||||
def test_api_services_bus_start_endpoint(self):
|
||||
"""Test POST /api/services/bus/services/<service_name>/start endpoint"""
|
||||
response = self.client.post('/api/services/bus/services/test/start')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
def test_api_services_bus_stop_endpoint(self):
|
||||
"""Test POST /api/services/bus/services/<service_name>/stop endpoint"""
|
||||
response = self.client.post('/api/services/bus/services/test/stop')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
def test_api_services_bus_restart_endpoint(self):
|
||||
"""Test POST /api/services/bus/services/<service_name>/restart endpoint"""
|
||||
response = self.client.post('/api/services/bus/services/test/restart')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
def test_api_logs_services_endpoint(self):
|
||||
"""Test GET /api/logs/services/<service> endpoint"""
|
||||
response = self.client.get('/api/logs/services/test')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIsInstance(data, list)
|
||||
|
||||
def test_api_logs_search_endpoint(self):
|
||||
"""Test POST /api/logs/search endpoint"""
|
||||
search_data = {'query': 'test', 'level': 'INFO'}
|
||||
response = self.client.post('/api/logs/search',
|
||||
data=json.dumps(search_data),
|
||||
content_type='application/json')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIsInstance(data, list)
|
||||
|
||||
def test_api_logs_export_endpoint(self):
|
||||
"""Test POST /api/logs/export endpoint"""
|
||||
export_data = {'format': 'json', 'filters': {}}
|
||||
response = self.client.post('/api/logs/export',
|
||||
data=json.dumps(export_data),
|
||||
content_type='application/json')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('export_path', data)
|
||||
|
||||
def test_api_logs_statistics_endpoint(self):
|
||||
"""Test GET /api/logs/statistics endpoint"""
|
||||
response = self.client.get('/api/logs/statistics')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('total_entries', data)
|
||||
|
||||
def test_api_logs_rotate_endpoint(self):
|
||||
"""Test POST /api/logs/rotate endpoint"""
|
||||
response = self.client.post('/api/logs/rotate')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
def test_api_dns_records_endpoints(self):
|
||||
"""Test DNS records endpoints"""
|
||||
# GET
|
||||
response = self.client.get('/api/dns/records')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIsInstance(data, list)
|
||||
|
||||
# POST
|
||||
record_data = {'name': 'test.example.com', 'type': 'A', 'value': '192.168.1.1'}
|
||||
response = self.client.post('/api/dns/records',
|
||||
data=json.dumps(record_data),
|
||||
content_type='application/json')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
# DELETE
|
||||
response = self.client.delete('/api/dns/records',
|
||||
data=json.dumps(record_data),
|
||||
content_type='application/json')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
def test_api_dhcp_endpoints(self):
|
||||
"""Test DHCP endpoints"""
|
||||
# GET leases
|
||||
response = self.client.get('/api/dhcp/leases')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIsInstance(data, list)
|
||||
|
||||
# POST reservation
|
||||
reservation_data = {'mac': '00:11:22:33:44:55', 'ip': '192.168.1.100'}
|
||||
response = self.client.post('/api/dhcp/reservations',
|
||||
data=json.dumps(reservation_data),
|
||||
content_type='application/json')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
# DELETE reservation
|
||||
response = self.client.delete('/api/dhcp/reservations',
|
||||
data=json.dumps(reservation_data),
|
||||
content_type='application/json')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
def test_api_ntp_status_endpoint(self):
|
||||
"""Test GET /api/ntp/status endpoint"""
|
||||
response = self.client.get('/api/ntp/status')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('status', data)
|
||||
|
||||
def test_api_network_info_endpoint(self):
|
||||
"""Test GET /api/network/info endpoint"""
|
||||
response = self.client.get('/api/network/info')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('interfaces', data)
|
||||
|
||||
def test_api_dns_status_endpoint(self):
|
||||
"""Test GET /api/dns/status endpoint"""
|
||||
response = self.client.get('/api/dns/status')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('status', data)
|
||||
|
||||
def test_api_network_test_endpoint(self):
|
||||
"""Test POST /api/network/test endpoint"""
|
||||
test_data = {'target': '8.8.8.8', 'type': 'ping'}
|
||||
response = self.client.post('/api/network/test',
|
||||
data=json.dumps(test_data),
|
||||
content_type='application/json')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
def test_api_wireguard_endpoints(self):
|
||||
"""Test WireGuard endpoints"""
|
||||
# GET keys
|
||||
response = self.client.get('/api/wireguard/keys')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('public_key', data)
|
||||
|
||||
# POST generate peer keys
|
||||
response = self.client.post('/api/wireguard/keys/peer')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('public_key', data)
|
||||
|
||||
# GET config
|
||||
response = self.client.get('/api/wireguard/config')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('config', data)
|
||||
|
||||
# GET peers
|
||||
response = self.client.get('/api/wireguard/peers')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIsInstance(data, list)
|
||||
|
||||
# POST add peer
|
||||
peer_data = {'peer': 'test_peer', 'ip': '10.0.0.1', 'public_key': 'test_key'}
|
||||
response = self.client.post('/api/wireguard/peers',
|
||||
data=json.dumps(peer_data),
|
||||
content_type='application/json')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
# DELETE remove peer
|
||||
response = self.client.delete('/api/wireguard/peers',
|
||||
data=json.dumps({'peer': 'test_peer'}),
|
||||
content_type='application/json')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
# GET status
|
||||
response = self.client.get('/api/wireguard/status')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('status', data)
|
||||
|
||||
def test_api_peers_endpoints(self):
|
||||
"""Test peers endpoints"""
|
||||
# GET peers
|
||||
response = self.client.get('/api/peers')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIsInstance(data, list)
|
||||
|
||||
# POST add peer
|
||||
peer_data = {'peer': 'test_peer', 'ip': '10.0.0.1'}
|
||||
response = self.client.post('/api/peers',
|
||||
data=json.dumps(peer_data),
|
||||
content_type='application/json')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
# DELETE remove peer
|
||||
response = self.client.delete('/api/peers/test_peer')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
def test_api_email_endpoints(self):
|
||||
"""Test email endpoints"""
|
||||
# GET users
|
||||
response = self.client.get('/api/email/users')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIsInstance(data, list)
|
||||
|
||||
# POST create user
|
||||
user_data = {'username': 'test_user', 'email': 'test@example.com'}
|
||||
response = self.client.post('/api/email/users',
|
||||
data=json.dumps(user_data),
|
||||
content_type='application/json')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
# DELETE user
|
||||
response = self.client.delete('/api/email/users/test_user')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
# GET status
|
||||
response = self.client.get('/api/email/status')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('status', data)
|
||||
|
||||
def test_api_calendar_endpoints(self):
|
||||
"""Test calendar endpoints"""
|
||||
# GET users
|
||||
response = self.client.get('/api/calendar/users')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIsInstance(data, list)
|
||||
|
||||
# POST create user
|
||||
user_data = {'username': 'test_user', 'email': 'test@example.com'}
|
||||
response = self.client.post('/api/calendar/users',
|
||||
data=json.dumps(user_data),
|
||||
content_type='application/json')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
# DELETE user
|
||||
response = self.client.delete('/api/calendar/users/test_user')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
# GET status
|
||||
response = self.client.get('/api/calendar/status')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('status', data)
|
||||
|
||||
def test_api_files_endpoints(self):
|
||||
"""Test files endpoints"""
|
||||
# GET users
|
||||
response = self.client.get('/api/files/users')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIsInstance(data, list)
|
||||
|
||||
# POST create user
|
||||
user_data = {'username': 'test_user'}
|
||||
response = self.client.post('/api/files/users',
|
||||
data=json.dumps(user_data),
|
||||
content_type='application/json')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
# DELETE user
|
||||
response = self.client.delete('/api/files/users/test_user')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
# GET status
|
||||
response = self.client.get('/api/files/status')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('status', data)
|
||||
|
||||
def test_api_routing_endpoints(self):
|
||||
"""Test routing endpoints"""
|
||||
# GET status
|
||||
response = self.client.get('/api/routing/status')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('status', data)
|
||||
|
||||
# POST NAT rule
|
||||
nat_data = {'type': 'masquerade', 'interface': 'eth0'}
|
||||
response = self.client.post('/api/routing/nat',
|
||||
data=json.dumps(nat_data),
|
||||
content_type='application/json')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('rule_id', data)
|
||||
|
||||
# GET NAT rules
|
||||
response = self.client.get('/api/routing/nat')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIsInstance(data, list)
|
||||
|
||||
def test_api_vault_endpoints(self):
|
||||
"""Test vault endpoints"""
|
||||
# GET status
|
||||
response = self.client.get('/api/vault/status')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('status', data)
|
||||
|
||||
# GET certificates
|
||||
response = self.client.get('/api/vault/certificates')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIsInstance(data, list)
|
||||
|
||||
# POST generate certificate
|
||||
cert_data = {'common_name': 'test.example.com'}
|
||||
response = self.client.post('/api/vault/certificates',
|
||||
data=json.dumps(cert_data),
|
||||
content_type='application/json')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('certificate', data)
|
||||
|
||||
# GET CA certificate
|
||||
response = self.client.get('/api/vault/ca/certificate')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('certificate', data)
|
||||
|
||||
def test_api_containers_endpoints(self):
|
||||
"""Test containers endpoints"""
|
||||
# GET containers
|
||||
response = self.client.get('/api/containers')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIsInstance(data, list)
|
||||
|
||||
# POST start container
|
||||
response = self.client.post('/api/containers/test/start')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
# POST stop container
|
||||
response = self.client.post('/api/containers/test/stop')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('success', data)
|
||||
|
||||
# GET container logs
|
||||
response = self.client.get('/api/containers/test/logs')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIsInstance(data, list)
|
||||
|
||||
def test_api_services_status_endpoint(self):
|
||||
"""Test GET /api/services/status endpoint"""
|
||||
response = self.client.get('/api/services/status')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('services', data)
|
||||
|
||||
def test_api_services_connectivity_endpoint(self):
|
||||
"""Test GET /api/services/connectivity endpoint"""
|
||||
response = self.client.get('/api/services/connectivity')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIn('results', data)
|
||||
|
||||
def test_api_health_history_endpoint(self):
|
||||
"""Test GET /api/health/history endpoint"""
|
||||
response = self.client.get('/api/health/history')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIsInstance(data, list)
|
||||
|
||||
def test_api_logs_endpoint(self):
|
||||
"""Test GET /api/logs endpoint"""
|
||||
response = self.client.get('/api/logs')
|
||||
self.assertEqual(response.status_code, 200)
|
||||
data = json.loads(response.data)
|
||||
self.assertIsInstance(data, list)
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -36,6 +36,7 @@ import app as app_module
|
||||
|
||||
class TestAppMisc(unittest.TestCase):
|
||||
def setUp(self):
|
||||
app_module.app.config['TESTING'] = True
|
||||
# Patch managers to avoid side effects
|
||||
self.patches = [
|
||||
patch.object(app_module, 'network_manager', MagicMock()),
|
||||
|
||||
@@ -37,15 +37,12 @@ def _make_response(status_code=200, json_data=None, text=''):
|
||||
|
||||
|
||||
def _make_config_manager(ddns_cfg=None, domain_cfg=None):
|
||||
"""Return a mock config_manager whose get_identity() returns a useful dict."""
|
||||
"""Return a mock config_manager with a real configs dict."""
|
||||
cm = MagicMock()
|
||||
configs = {}
|
||||
if ddns_cfg is not None:
|
||||
identity = {'domain': {'ddns': ddns_cfg}}
|
||||
elif domain_cfg is not None:
|
||||
identity = {'domain': domain_cfg}
|
||||
else:
|
||||
identity = {}
|
||||
cm.get_identity.return_value = identity
|
||||
configs['ddns'] = ddns_cfg
|
||||
cm.configs = configs
|
||||
return cm
|
||||
|
||||
|
||||
@@ -83,6 +80,27 @@ class TestPicNgoDDNSRegister(unittest.TestCase):
|
||||
_, kwargs = mock_post.call_args
|
||||
self.assertNotIn('Authorization', kwargs.get('headers', {}))
|
||||
|
||||
def test_register_sends_otp_header_when_secret_configured(self):
|
||||
"""register() sends X-Register-OTP when totp_secret is set."""
|
||||
provider = PicNgoDDNS(totp_secret='JBSWY3DPEHPK3PXP')
|
||||
mock_resp = _make_response(200, json_data={'token': 'tok', 'subdomain': 'x.pic.ngo'})
|
||||
with patch('requests.post', return_value=mock_resp) as mock_post:
|
||||
provider.register('x', '1.2.3.4')
|
||||
_, kwargs = mock_post.call_args
|
||||
self.assertIn('X-Register-OTP', kwargs.get('headers', {}))
|
||||
otp = kwargs['headers']['X-Register-OTP']
|
||||
self.assertEqual(len(otp), 6)
|
||||
self.assertTrue(otp.isdigit())
|
||||
|
||||
def test_register_no_otp_header_without_secret(self):
|
||||
"""register() omits X-Register-OTP when no TOTP secret is configured."""
|
||||
provider = PicNgoDDNS()
|
||||
mock_resp = _make_response(200, json_data={'token': 't', 'subdomain': 'x'})
|
||||
with patch('requests.post', return_value=mock_resp) as mock_post:
|
||||
provider.register('x', '1.2.3.4')
|
||||
_, kwargs = mock_post.call_args
|
||||
self.assertNotIn('X-Register-OTP', kwargs.get('headers', {}))
|
||||
|
||||
|
||||
class TestPicNgoDDNSUpdate(unittest.TestCase):
|
||||
"""PicNgoDDNS.update() calls the correct URL with Authorization header."""
|
||||
|
||||
@@ -53,8 +53,11 @@ def empty_auth_manager(tmp_path):
|
||||
os.makedirs(data_dir, exist_ok=True)
|
||||
os.makedirs(config_dir, exist_ok=True)
|
||||
mgr = AuthManager(data_dir=data_dir, config_dir=config_dir)
|
||||
# The constructor creates the file with '[]' (empty list). We do NOT add
|
||||
# any user, so list_users() returns [] but the file is readable.
|
||||
# Explicitly create the file with an empty list to simulate the
|
||||
# "auth configured but no users" misconfiguration scenario.
|
||||
users_file = os.path.join(data_dir, 'auth_users.json')
|
||||
with open(users_file, 'w') as f:
|
||||
f.write('[]')
|
||||
assert mgr.list_users() == [], 'Expected empty user list'
|
||||
return mgr
|
||||
|
||||
|
||||
@@ -133,7 +133,8 @@ class TestGenerateCorefile(unittest.TestCase):
|
||||
self.assertIn('reload', content)
|
||||
|
||||
def test_returns_false_on_write_error(self):
|
||||
result = firewall_manager.generate_corefile([], '/nonexistent/path/Corefile')
|
||||
with unittest.mock.patch('builtins.open', side_effect=OSError('Permission denied')):
|
||||
result = firewall_manager.generate_corefile([], '/any/path/Corefile')
|
||||
self.assertFalse(result)
|
||||
|
||||
|
||||
|
||||
@@ -5,6 +5,7 @@ import sys
|
||||
import os
|
||||
import tempfile
|
||||
import unittest
|
||||
import unittest.mock
|
||||
from pathlib import Path
|
||||
|
||||
api_dir = Path(__file__).parent.parent / 'api'
|
||||
@@ -98,7 +99,8 @@ class TestWriteEnvFile(unittest.TestCase):
|
||||
self.assertTrue(result)
|
||||
|
||||
def test_returns_false_on_unwritable_path(self):
|
||||
result = ip_utils.write_env_file('172.20.0.0/16', '/nonexistent/deep/path/.env')
|
||||
with unittest.mock.patch('builtins.open', side_effect=OSError('Permission denied')):
|
||||
result = ip_utils.write_env_file('172.20.0.0/16', '/any/path/.env')
|
||||
self.assertFalse(result)
|
||||
|
||||
def test_contains_cell_network(self):
|
||||
|
||||
@@ -128,6 +128,12 @@ def test_anon_blocked_from_peer_routes(anon_client):
|
||||
assert r.status_code == 401
|
||||
|
||||
|
||||
def test_setup_routes_bypass_auth(anon_client):
|
||||
"""/api/setup/* must be reachable without a session — setup runs before any account exists."""
|
||||
r = anon_client.get('/api/setup/status')
|
||||
assert r.status_code != 401
|
||||
|
||||
|
||||
def test_anon_blocked_from_peer_dashboard(anon_client):
|
||||
r = anon_client.get('/api/peer/dashboard')
|
||||
assert r.status_code == 401
|
||||
|
||||
@@ -252,10 +252,11 @@ def test_complete_setup_returns_error_when_create_user_fails(
|
||||
setup_manager, mock_config_manager, mock_auth_manager, tmp_path):
|
||||
mock_config_manager.get_identity.return_value = {}
|
||||
mock_auth_manager.create_user.return_value = False
|
||||
mock_auth_manager.update_password.return_value = False
|
||||
with patch.dict(os.environ, {'DATA_DIR': str(tmp_path)}):
|
||||
result = setup_manager.complete_setup(_valid_payload())
|
||||
assert result['success'] is False
|
||||
assert any('admin' in e.lower() or 'user' in e.lower() for e in result['errors'])
|
||||
assert any('admin' in e.lower() or 'password' in e.lower() for e in result['errors'])
|
||||
|
||||
|
||||
# ── get_setup_status ──────────────────────────────────────────────────────────
|
||||
|
||||
+285
-71
@@ -1,6 +1,6 @@
|
||||
import React, { useState, useMemo } from 'react';
|
||||
import { useNavigate } from 'react-router-dom';
|
||||
import { Eye, EyeOff, CheckCircle, AlertCircle } from 'lucide-react';
|
||||
import { Eye, EyeOff, CheckCircle, AlertCircle, Globe } from 'lucide-react';
|
||||
import { setupAPI } from '../services/api';
|
||||
|
||||
// ── constants ─────────────────────────────────────────────────────────────────
|
||||
@@ -13,27 +13,36 @@ const DOMAIN_OPTIONS = [
|
||||
{
|
||||
value: 'pic_ngo',
|
||||
label: 'PIC.NGO subdomain',
|
||||
description: 'Get a free yourname.pic.ngo domain — managed automatically.',
|
||||
description: 'Get a free yourname.pic.ngo address — HTTPS and DDNS managed automatically.',
|
||||
},
|
||||
{
|
||||
value: 'custom',
|
||||
label: 'Custom domain',
|
||||
description: 'Bring your own domain. You will configure DNS records manually.',
|
||||
description: 'Use your own domain with Cloudflare, DuckDNS, or standard HTTP challenge.',
|
||||
},
|
||||
{
|
||||
value: 'lan_only',
|
||||
value: 'lan',
|
||||
label: 'LAN only',
|
||||
description: 'No public domain. Accessible only on your local network and via VPN.',
|
||||
},
|
||||
];
|
||||
|
||||
const DDNS_OPTIONS = [
|
||||
{ value: 'pic_ngo', label: 'pic.ngo (managed)', description: 'Automatic — no setup required.' },
|
||||
{ value: 'cloudflare', label: 'Cloudflare', description: 'Use Cloudflare DNS with API token.' },
|
||||
{ value: 'duckdns', label: 'DuckDNS', description: 'Free dynamic DNS via duckdns.org.' },
|
||||
{ value: 'noip', label: 'No-IP', description: 'Free dynamic DNS via noip.com.' },
|
||||
{ value: 'freedns', label: 'FreeDNS', description: 'Free DNS via freedns.afraid.org.' },
|
||||
{ value: 'manual', label: 'Manual / None', description: 'You will handle DNS updates yourself.' },
|
||||
const CUSTOM_METHOD_OPTIONS = [
|
||||
{
|
||||
value: 'cloudflare',
|
||||
label: 'Cloudflare DNS',
|
||||
description: 'DNS-01 via Cloudflare API. Your domain must use Cloudflare nameservers.',
|
||||
},
|
||||
{
|
||||
value: 'duckdns',
|
||||
label: 'DuckDNS',
|
||||
description: 'Free subdomain via duckdns.org with automatic DNS-01 challenge.',
|
||||
},
|
||||
{
|
||||
value: 'http01',
|
||||
label: 'HTTP-01 (any registrar)',
|
||||
description: 'Standard ACME challenge. Port 80 must be publicly reachable.',
|
||||
},
|
||||
];
|
||||
|
||||
const OPTIONAL_SERVICES = [
|
||||
@@ -55,35 +64,37 @@ function getAllTimezones() {
|
||||
try {
|
||||
return Intl.supportedValuesOf('timeZone');
|
||||
} catch {
|
||||
// Fallback list for older browsers
|
||||
return [
|
||||
'UTC',
|
||||
'America/New_York',
|
||||
'America/Chicago',
|
||||
'America/Denver',
|
||||
'America/Los_Angeles',
|
||||
'Europe/London',
|
||||
'Europe/Paris',
|
||||
'Europe/Berlin',
|
||||
'Asia/Tokyo',
|
||||
'Asia/Shanghai',
|
||||
'Australia/Sydney',
|
||||
'UTC', 'America/New_York', 'America/Chicago', 'America/Denver',
|
||||
'America/Los_Angeles', 'Europe/London', 'Europe/Paris', 'Europe/Berlin',
|
||||
'Asia/Tokyo', 'Asia/Shanghai', 'Australia/Sydney',
|
||||
];
|
||||
}
|
||||
}
|
||||
|
||||
function passwordStrength(pw) {
|
||||
if (!pw) return { label: '', color: '', width: '0%' };
|
||||
if (!pw) return { label: '', color: '', width: '0%', score: 0 };
|
||||
let score = 0;
|
||||
if (pw.length >= 12) score++;
|
||||
if (pw.length >= 16) score++;
|
||||
if (/[A-Z]/.test(pw)) score++;
|
||||
if (/[a-z]/.test(pw)) score++;
|
||||
if (/[0-9]/.test(pw)) score++;
|
||||
if (/[^A-Za-z0-9]/.test(pw)) score++;
|
||||
if (score <= 1) return { label: 'Weak', color: 'bg-red-500', width: '20%' };
|
||||
if (score === 2) return { label: 'Fair', color: 'bg-yellow-500', width: '40%' };
|
||||
if (score === 3) return { label: 'Good', color: 'bg-blue-500', width: '65%' };
|
||||
return { label: 'Strong', color: 'bg-green-500', width: '100%' };
|
||||
if (score <= 2) return { label: 'Weak', color: 'bg-red-500', width: '20%', score };
|
||||
if (score === 3) return { label: 'Fair', color: 'bg-yellow-500', width: '45%', score };
|
||||
if (score === 4) return { label: 'Good', color: 'bg-blue-500', width: '70%', score };
|
||||
return { label: 'Strong', color: 'bg-green-500', width: '100%', score };
|
||||
}
|
||||
|
||||
function meetsApiRequirements(pw) {
|
||||
return pw.length >= 12 && /[A-Z]/.test(pw) && /[a-z]/.test(pw) && /[0-9]/.test(pw);
|
||||
}
|
||||
|
||||
function getDomainMode(domainType, customMethod) {
|
||||
if (domainType === 'pic_ngo') return 'pic_ngo';
|
||||
if (domainType === 'lan') return 'lan';
|
||||
return customMethod || 'http01';
|
||||
}
|
||||
|
||||
// ── sub-components ────────────────────────────────────────────────────────────
|
||||
@@ -218,12 +229,14 @@ function Step1CellName({ value, onChange, onNext }) {
|
||||
}
|
||||
};
|
||||
|
||||
const isValid = CELL_NAME_RE.test(value);
|
||||
|
||||
return (
|
||||
<div>
|
||||
<StepHeader
|
||||
step={1}
|
||||
title="Name your cell"
|
||||
description="This is the internal identifier for your Personal Internet Cell. It appears in hostnames and logs."
|
||||
description="This becomes your cell's identity and subdomain. If you choose pic.ngo it will be reachable at name.pic.ngo."
|
||||
/>
|
||||
<div>
|
||||
<label className="block text-sm text-gray-400 mb-1.5" htmlFor="cell-name">
|
||||
@@ -241,13 +254,20 @@ function Step1CellName({ value, onChange, onNext }) {
|
||||
setServerError('');
|
||||
}}
|
||||
onKeyDown={e => e.key === 'Enter' && handleNext()}
|
||||
placeholder="e.g. homelab"
|
||||
placeholder="e.g. myhome"
|
||||
className="w-full bg-gray-800 border border-gray-600 rounded px-3 py-2 text-white text-sm focus:outline-none focus:border-blue-500 placeholder-gray-600"
|
||||
aria-describedby={error || serverError ? 'cell-name-error' : undefined}
|
||||
/>
|
||||
{isValid ? (
|
||||
<p className="mt-1.5 text-xs text-blue-400 flex items-center gap-1">
|
||||
<Globe className="h-3.5 w-3.5" />
|
||||
pic.ngo preview: <span className="font-mono font-medium ml-1">{value}.pic.ngo</span>
|
||||
</p>
|
||||
) : (
|
||||
<p className="mt-1 text-xs text-gray-500">
|
||||
Lowercase letters, numbers, hyphens. Must start with a letter. 2–31 characters.
|
||||
</p>
|
||||
)}
|
||||
<div id="cell-name-error">
|
||||
<FieldError message={error || serverError} />
|
||||
</div>
|
||||
@@ -263,11 +283,18 @@ function Step2Password({ password, confirm, onChangePassword, onChangeConfirm, o
|
||||
const [errors, setErrors] = useState({});
|
||||
|
||||
const strength = passwordStrength(password);
|
||||
const ready = meetsApiRequirements(password) && password === confirm;
|
||||
|
||||
const validate = () => {
|
||||
const e = {};
|
||||
if (!password) e.password = 'Password is required.';
|
||||
else if (password.length < 12) e.password = 'Password must be at least 12 characters.';
|
||||
if (!password) {
|
||||
e.password = 'Password is required.';
|
||||
return e;
|
||||
}
|
||||
if (password.length < 12) e.password = 'Must be at least 12 characters.';
|
||||
else if (!/[A-Z]/.test(password)) e.password = 'Must contain at least one uppercase letter.';
|
||||
else if (!/[a-z]/.test(password)) e.password = 'Must contain at least one lowercase letter.';
|
||||
else if (!/[0-9]/.test(password)) e.password = 'Must contain at least one digit.';
|
||||
if (!confirm) e.confirm = 'Please confirm your password.';
|
||||
else if (password !== confirm) e.confirm = 'Passwords do not match.';
|
||||
return e;
|
||||
@@ -279,14 +306,12 @@ function Step2Password({ password, confirm, onChangePassword, onChangeConfirm, o
|
||||
if (Object.keys(e).length === 0) onNext();
|
||||
};
|
||||
|
||||
const isReady = password.length >= 12 && password === confirm;
|
||||
|
||||
return (
|
||||
<div>
|
||||
<StepHeader
|
||||
step={2}
|
||||
title="Set admin password"
|
||||
description="This password protects access to your cell. Choose something strong and store it safely."
|
||||
description="This password protects access to your cell. At least 12 characters with uppercase, lowercase, and a digit."
|
||||
/>
|
||||
<div className="space-y-4">
|
||||
<div>
|
||||
@@ -313,7 +338,6 @@ function Step2Password({ password, confirm, onChangePassword, onChangeConfirm, o
|
||||
{showPw ? <EyeOff className="h-4 w-4" /> : <Eye className="h-4 w-4" />}
|
||||
</button>
|
||||
</div>
|
||||
{/* Strength bar */}
|
||||
{password.length > 0 && (
|
||||
<div className="mt-2">
|
||||
<div className="w-full bg-gray-700 rounded-full h-1">
|
||||
@@ -356,7 +380,7 @@ function Step2Password({ password, confirm, onChangePassword, onChangeConfirm, o
|
||||
<div id="pw-confirm-error"><FieldError message={errors.confirm} /></div>
|
||||
</div>
|
||||
</div>
|
||||
<NavButtons onBack={onBack} onNext={handleNext} nextDisabled={!isReady} />
|
||||
<NavButtons onBack={onBack} onNext={handleNext} nextDisabled={!ready} />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -386,27 +410,182 @@ function Step3Domain({ value, onChange, onNext, onBack }) {
|
||||
);
|
||||
}
|
||||
|
||||
function Step4DDNS({ value, onChange, onNext, onBack }) {
|
||||
function Step4DomainConfig({
|
||||
domainType, cellName,
|
||||
customDomain, onCustomDomain,
|
||||
customMethod, onCustomMethod,
|
||||
cloudflareToken, onCloudflareToken,
|
||||
duckdnsToken, onDuckdnsToken,
|
||||
onNext, onBack,
|
||||
}) {
|
||||
const [errors, setErrors] = useState({});
|
||||
|
||||
// ── pic_ngo: just show the derived domain ────────────────────────────────
|
||||
if (domainType === 'pic_ngo') {
|
||||
return (
|
||||
<div>
|
||||
<StepHeader
|
||||
step={4}
|
||||
title="DDNS provider"
|
||||
description="Which provider will keep your dynamic IP address up to date? Credentials are configured separately after setup."
|
||||
title="Your pic.ngo domain"
|
||||
description="Your cell will be reachable at the address below. HTTPS and DDNS are managed automatically."
|
||||
/>
|
||||
<div className="bg-gray-800/50 border border-gray-700 rounded-lg p-5 text-center mb-4">
|
||||
<p className="text-xs text-gray-500 mb-2">Your public address</p>
|
||||
<p className="text-2xl font-mono font-semibold text-white tracking-tight">
|
||||
{cellName || '…'}.pic.ngo
|
||||
</p>
|
||||
<p className="text-xs text-gray-400 mt-3">
|
||||
DNS and TLS certificates are provisioned automatically via the pic.ngo API.
|
||||
</p>
|
||||
</div>
|
||||
<p className="text-xs text-gray-500">
|
||||
Not the right name? Go back to step 1 to change your cell name.
|
||||
</p>
|
||||
<NavButtons onBack={onBack} onNext={onNext} />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
// ── custom domain ─────────────────────────────────────────────────────────
|
||||
const validateCustom = () => {
|
||||
const e = {};
|
||||
const dom = customDomain.trim();
|
||||
if (!dom) {
|
||||
e.domain = 'Domain name is required.';
|
||||
} else if (!/^[a-z0-9]([a-z0-9-]*[a-z0-9])?(\.[a-z0-9]([a-z0-9-]*[a-z0-9])?)+$/i.test(dom)) {
|
||||
e.domain = 'Enter a valid domain name (e.g. home.example.com).';
|
||||
}
|
||||
if (!customMethod) e.method = 'Select a TLS method.';
|
||||
if (customMethod === 'cloudflare' && !cloudflareToken.trim())
|
||||
e.token = 'Cloudflare API token is required.';
|
||||
if (customMethod === 'duckdns' && !duckdnsToken.trim())
|
||||
e.token = 'DuckDNS token is required.';
|
||||
return e;
|
||||
};
|
||||
|
||||
const handleNext = () => {
|
||||
const e = validateCustom();
|
||||
setErrors(e);
|
||||
if (Object.keys(e).length === 0) onNext();
|
||||
};
|
||||
|
||||
const isReady =
|
||||
customDomain.trim() &&
|
||||
customMethod &&
|
||||
(customMethod === 'http01' ||
|
||||
(customMethod === 'cloudflare' && cloudflareToken.trim()) ||
|
||||
(customMethod === 'duckdns' && duckdnsToken.trim()));
|
||||
|
||||
return (
|
||||
<div>
|
||||
<StepHeader
|
||||
step={4}
|
||||
title="Domain configuration"
|
||||
description="Enter your domain and choose how TLS certificates will be obtained."
|
||||
/>
|
||||
<div className="space-y-5">
|
||||
{/* Domain name */}
|
||||
<div>
|
||||
<label className="block text-sm text-gray-400 mb-1.5">
|
||||
Domain name <span className="text-red-400">*</span>
|
||||
</label>
|
||||
<input
|
||||
type="text"
|
||||
value={customDomain}
|
||||
onChange={e => {
|
||||
onCustomDomain(e.target.value.toLowerCase().trim());
|
||||
setErrors(p => ({ ...p, domain: '' }));
|
||||
}}
|
||||
placeholder="e.g. home.example.com"
|
||||
className="w-full bg-gray-800 border border-gray-600 rounded px-3 py-2 text-white text-sm focus:outline-none focus:border-blue-500 placeholder-gray-600"
|
||||
/>
|
||||
<FieldError message={errors.domain} />
|
||||
</div>
|
||||
|
||||
{/* TLS method */}
|
||||
<div>
|
||||
<label className="block text-sm text-gray-400 mb-1.5">
|
||||
TLS method <span className="text-red-400">*</span>
|
||||
</label>
|
||||
<div className="space-y-2">
|
||||
{DDNS_OPTIONS.map(opt => (
|
||||
{CUSTOM_METHOD_OPTIONS.map(opt => (
|
||||
<RadioOption
|
||||
key={opt.value}
|
||||
value={opt.value}
|
||||
label={opt.label}
|
||||
description={opt.description}
|
||||
selected={value === opt.value}
|
||||
onChange={onChange}
|
||||
selected={customMethod === opt.value}
|
||||
onChange={v => {
|
||||
onCustomMethod(v);
|
||||
setErrors(p => ({ ...p, method: '', token: '' }));
|
||||
}}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
<NavButtons onBack={onBack} onNext={onNext} nextDisabled={!value} />
|
||||
<FieldError message={errors.method} />
|
||||
</div>
|
||||
|
||||
{/* Cloudflare token */}
|
||||
{customMethod === 'cloudflare' && (
|
||||
<div>
|
||||
<label className="block text-sm text-gray-400 mb-1.5">
|
||||
Cloudflare API token <span className="text-red-400">*</span>
|
||||
</label>
|
||||
<input
|
||||
type="password"
|
||||
autoComplete="off"
|
||||
value={cloudflareToken}
|
||||
onChange={e => {
|
||||
onCloudflareToken(e.target.value);
|
||||
setErrors(p => ({ ...p, token: '' }));
|
||||
}}
|
||||
placeholder="Cloudflare API token with DNS:Edit permission"
|
||||
className="w-full bg-gray-800 border border-gray-600 rounded px-3 py-2 text-white text-sm focus:outline-none focus:border-blue-500 placeholder-gray-600"
|
||||
/>
|
||||
<p className="mt-1 text-xs text-gray-500">
|
||||
Create at Cloudflare Dashboard → My Profile → API Tokens. Needs Zone / DNS / Edit.
|
||||
</p>
|
||||
<FieldError message={errors.token} />
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* DuckDNS token */}
|
||||
{customMethod === 'duckdns' && (
|
||||
<div>
|
||||
<label className="block text-sm text-gray-400 mb-1.5">
|
||||
DuckDNS token <span className="text-red-400">*</span>
|
||||
</label>
|
||||
<input
|
||||
type="password"
|
||||
autoComplete="off"
|
||||
value={duckdnsToken}
|
||||
onChange={e => {
|
||||
onDuckdnsToken(e.target.value);
|
||||
setErrors(p => ({ ...p, token: '' }));
|
||||
}}
|
||||
placeholder="Your DuckDNS account token"
|
||||
className="w-full bg-gray-800 border border-gray-600 rounded px-3 py-2 text-white text-sm focus:outline-none focus:border-blue-500 placeholder-gray-600"
|
||||
/>
|
||||
<p className="mt-1 text-xs text-gray-500">
|
||||
Found at duckdns.org after login. The subdomain must already exist in your account.
|
||||
</p>
|
||||
<FieldError message={errors.token} />
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* HTTP-01 info */}
|
||||
{customMethod === 'http01' && (
|
||||
<div className="p-3 bg-yellow-950/40 border border-yellow-700/50 rounded-lg">
|
||||
<p className="text-xs text-yellow-300">
|
||||
<span className="font-semibold">Port 80 must be publicly reachable</span> from the
|
||||
internet for Let's Encrypt HTTP-01 validation. Ensure your router forwards port 80
|
||||
to this machine before completing setup.
|
||||
</p>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<NavButtons onBack={onBack} onNext={handleNext} nextDisabled={!isReady} />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -426,7 +605,6 @@ function Step5Services({ selected, onChange, onNext, onBack }) {
|
||||
description="Choose which services to enable. You can change this later in Settings."
|
||||
/>
|
||||
|
||||
{/* Optional services */}
|
||||
<div className="space-y-2 mb-6">
|
||||
{OPTIONAL_SERVICES.map(svc => {
|
||||
const checked = selected.includes(svc.key);
|
||||
@@ -452,7 +630,6 @@ function Step5Services({ selected, onChange, onNext, onBack }) {
|
||||
})}
|
||||
</div>
|
||||
|
||||
{/* Always-on services */}
|
||||
<div>
|
||||
<p className="text-xs font-medium text-gray-500 uppercase tracking-wider mb-2">
|
||||
Always enabled
|
||||
@@ -538,10 +715,18 @@ function ReviewRow({ label, value }) {
|
||||
}
|
||||
|
||||
function Step7Review({ fields, onBack, onSubmit, submitting, submitError }) {
|
||||
const domainLabel = DOMAIN_OPTIONS.find(o => o.value === fields.domain_type)?.label || fields.domain_type;
|
||||
const ddnsLabel = DDNS_OPTIONS.find(o => o.value === fields.ddns_provider)?.label || fields.ddns_provider;
|
||||
const serviceLabels = fields.services.length
|
||||
? fields.services.map(k => OPTIONAL_SERVICES.find(s => s.key === k)?.label || k).join(', ')
|
||||
const domainDisplay =
|
||||
fields.domain_type === 'pic_ngo' ? `${fields.cell_name}.pic.ngo` :
|
||||
fields.domain_type === 'lan' ? 'LAN only (no public domain)' :
|
||||
fields.custom_domain || '(not set)';
|
||||
|
||||
const tlsDisplay =
|
||||
fields.domain_type === 'pic_ngo' ? 'Automatic (pic.ngo)' :
|
||||
fields.domain_type === 'lan' ? '—' :
|
||||
CUSTOM_METHOD_OPTIONS.find(o => o.value === fields.custom_method)?.label || fields.custom_method;
|
||||
|
||||
const serviceLabels = (fields.services_enabled || []).length
|
||||
? fields.services_enabled.map(k => OPTIONAL_SERVICES.find(s => s.key === k)?.label || k).join(', ')
|
||||
: 'None selected';
|
||||
|
||||
return (
|
||||
@@ -554,9 +739,9 @@ function Step7Review({ fields, onBack, onSubmit, submitting, submitError }) {
|
||||
<div className="bg-gray-800/50 border border-gray-700 rounded-lg px-4 py-1 mb-2">
|
||||
<ReviewRow label="Cell name" value={fields.cell_name} />
|
||||
<ReviewRow label="Admin password" value="••••••••••••" />
|
||||
<ReviewRow label="Domain type" value={domainLabel} />
|
||||
{fields.domain_type !== 'lan_only' && (
|
||||
<ReviewRow label="DDNS provider" value={ddnsLabel} />
|
||||
<ReviewRow label="Domain" value={domainDisplay} />
|
||||
{fields.domain_type !== 'lan' && (
|
||||
<ReviewRow label="TLS / DNS" value={tlsDisplay} />
|
||||
)}
|
||||
<ReviewRow label="Optional services" value={serviceLabels} />
|
||||
<ReviewRow label="Timezone" value={fields.timezone} />
|
||||
@@ -591,7 +776,10 @@ export default function Setup() {
|
||||
const [password, setPassword] = useState('');
|
||||
const [passwordConfirm, setPasswordConfirm] = useState('');
|
||||
const [domainType, setDomainType] = useState('pic_ngo');
|
||||
const [ddnsProvider, setDdnsProvider] = useState('pic_ngo');
|
||||
const [customDomain, setCustomDomain] = useState('');
|
||||
const [customMethod, setCustomMethod] = useState('');
|
||||
const [cloudflareToken, setCloudflareToken] = useState('');
|
||||
const [duckdnsToken, setDuckdnsToken] = useState('');
|
||||
const [services, setServices] = useState(['email', 'calendar', 'files', 'webmail']);
|
||||
const [timezone, setTimezone] = useState(
|
||||
(() => { try { return Intl.DateTimeFormat().resolvedOptions().timeZone || 'UTC'; } catch { return 'UTC'; } })()
|
||||
@@ -601,39 +789,46 @@ export default function Setup() {
|
||||
const [submitting, setSubmitting] = useState(false);
|
||||
const [submitError, setSubmitError] = useState('');
|
||||
|
||||
const skipDdns = domainType === 'lan_only';
|
||||
const skipStep4 = domainType === 'lan';
|
||||
|
||||
const goNext = () => setStep(s => Math.min(s + 1, TOTAL_STEPS));
|
||||
const goBack = () => setStep(s => Math.max(s - 1, 1));
|
||||
|
||||
// Skip step 4 when LAN only
|
||||
const handleStep3Next = () => {
|
||||
if (skipDdns) setStep(5);
|
||||
else setStep(4);
|
||||
};
|
||||
const handleStep3Next = () => skipStep4 ? setStep(5) : setStep(4);
|
||||
const handleStep4Back = () => setStep(3);
|
||||
const handleStep5Back = () => {
|
||||
if (skipDdns) setStep(3);
|
||||
else setStep(4);
|
||||
};
|
||||
const handleStep5Back = () => skipStep4 ? setStep(3) : setStep(4);
|
||||
|
||||
const handleSubmit = async () => {
|
||||
setSubmitError('');
|
||||
setSubmitting(true);
|
||||
|
||||
const domainMode = getDomainMode(domainType, customMethod);
|
||||
const domainName =
|
||||
domainType === 'pic_ngo' ? `${cellName}.pic.ngo` :
|
||||
domainType === 'lan' ? '' :
|
||||
customDomain;
|
||||
|
||||
const payload = {
|
||||
cell_name: cellName,
|
||||
password,
|
||||
domain_type: domainType,
|
||||
...(skipDdns ? {} : { ddns_provider: ddnsProvider }),
|
||||
services,
|
||||
domain_mode: domainMode,
|
||||
domain_name: domainName,
|
||||
timezone,
|
||||
services_enabled: services,
|
||||
...(domainType !== 'lan' && {
|
||||
ddns_provider: domainType === 'pic_ngo' ? 'pic_ngo' : customMethod === 'http01' ? 'none' : customMethod,
|
||||
}),
|
||||
...(customMethod === 'cloudflare' && { cloudflare_api_token: cloudflareToken }),
|
||||
...(customMethod === 'duckdns' && { duckdns_token: duckdnsToken }),
|
||||
};
|
||||
|
||||
try {
|
||||
await setupAPI.complete(payload);
|
||||
setDone(true);
|
||||
setTimeout(() => navigate('/login', { replace: true }), 2000);
|
||||
} catch (e) {
|
||||
setSubmitError(
|
||||
e?.response?.data?.errors?.join(' ') ||
|
||||
e?.response?.data?.error ||
|
||||
'Setup could not be completed. Please check your entries and try again.'
|
||||
);
|
||||
@@ -642,7 +837,14 @@ export default function Setup() {
|
||||
}
|
||||
};
|
||||
|
||||
const allFields = { cell_name: cellName, domain_type: domainType, ddns_provider: ddnsProvider, services, timezone };
|
||||
const reviewFields = {
|
||||
cell_name: cellName,
|
||||
domain_type: domainType,
|
||||
custom_domain: customDomain,
|
||||
custom_method: customMethod,
|
||||
services_enabled: services,
|
||||
timezone,
|
||||
};
|
||||
|
||||
if (done) {
|
||||
return (
|
||||
@@ -659,7 +861,6 @@ export default function Setup() {
|
||||
return (
|
||||
<div className="flex items-center justify-center min-h-screen bg-gray-950 px-4 py-10">
|
||||
<div className="w-full max-w-lg bg-gray-900 border border-gray-700 rounded-xl p-8 shadow-2xl">
|
||||
{/* Page title */}
|
||||
<div className="mb-6">
|
||||
<h1 className="text-xl font-bold text-white">Personal Internet Cell</h1>
|
||||
<p className="text-sm text-gray-400 mt-0.5">First-time setup</p>
|
||||
@@ -684,7 +885,20 @@ export default function Setup() {
|
||||
<Step3Domain value={domainType} onChange={setDomainType} onNext={handleStep3Next} onBack={goBack} />
|
||||
)}
|
||||
{step === 4 && (
|
||||
<Step4DDNS value={ddnsProvider} onChange={setDdnsProvider} onNext={goNext} onBack={handleStep4Back} />
|
||||
<Step4DomainConfig
|
||||
domainType={domainType}
|
||||
cellName={cellName}
|
||||
customDomain={customDomain}
|
||||
onCustomDomain={setCustomDomain}
|
||||
customMethod={customMethod}
|
||||
onCustomMethod={setCustomMethod}
|
||||
cloudflareToken={cloudflareToken}
|
||||
onCloudflareToken={setCloudflareToken}
|
||||
duckdnsToken={duckdnsToken}
|
||||
onDuckdnsToken={setDuckdnsToken}
|
||||
onNext={goNext}
|
||||
onBack={handleStep4Back}
|
||||
/>
|
||||
)}
|
||||
{step === 5 && (
|
||||
<Step5Services selected={services} onChange={setServices} onNext={goNext} onBack={handleStep5Back} />
|
||||
@@ -694,7 +908,7 @@ export default function Setup() {
|
||||
)}
|
||||
{step === 7 && (
|
||||
<Step7Review
|
||||
fields={allFields}
|
||||
fields={reviewFields}
|
||||
onBack={goBack}
|
||||
onSubmit={handleSubmit}
|
||||
submitting={submitting}
|
||||
|
||||
Reference in New Issue
Block a user