Personal Internet Cell

🌟 Overview

The Personal Internet Cell is a production-grade, self-hosted, decentralized digital infrastructure that empowers you to:

• Host your own services: Email, calendar, contacts, files, DNS, DHCP, NTP
• Secure mesh networking: Connect with trusted peers via WireGuard VPN
• Advanced routing: VPN gateway, NAT, firewall, exit nodes, and bridge routing
• Enterprise security: Self-hosted CA, certificate management, trust systems
• Modern management: RESTful API, enhanced CLI, and comprehensive monitoring
• Event-driven architecture: Service orchestration and real-time communication

---

🚀 Key Features

🔧 Core Services

• Network Services: DNS, DHCP, NTP with dynamic management
• VPN & Mesh: WireGuard-based peer federation with dynamic IP updates
• Digital Services: Email (SMTP/IMAP), Calendar/Contacts (CalDAV/CardDAV), File Storage (WebDAV)
• Security: Self-hosted Certificate Authority, Age/Fernet encryption, trust management
• Container Orchestration: Docker-based service management and deployment

🏗️ Architecture Highlights

• BaseServiceManager: Unified interface across all 10 service managers
• Event-Driven Service Bus: Real-time service communication and orchestration
• Centralized Configuration: Type-safe validation, backup/restore, import/export
• Comprehensive Logging: Structured JSON logs with rotation, search, and export
• Enhanced CLI: Interactive mode, batch operations, service wizards
• Health Monitoring: Real-time health checks and performance metrics

📊 Production Features

• Service Orchestration: Automatic service dependency management
• Configuration Management: Schema validation, versioning, and migration
• Error Handling: Standardized error handling and recovery mechanisms
• Testing: Comprehensive test suite with 77%+ coverage
• Documentation: Complete API documentation and usage guides

---

📋 Table of Contents

1. Quick Start
2. Architecture
3. Service Managers
4. API Reference
5. CLI Guide
6. Configuration
7. Security
8. Development
9. Testing
10. Deployment
11. Contributing
12. License

---

🚀 Quick Start

Prerequisites

• Debian/Ubuntu host (apt-based). All other dependencies are installed automatically.
• 2 GB+ RAM, 10 GB+ disk space
• Open ports: 53 (DNS), 80/443 (HTTP/S), 3000 (API), 8081 (Web UI), 51820/udp (WireGuard)

1. Install

git clone <repo-url> pic
cd pic

# Install all system dependencies (docker, python3, python3-cryptography, etc.)
make check-deps

# Default cell (name=mycell, domain=cell, VPN=10.0.0.1/24, port=51820)
make setup
make start

# Custom cell — use when installing a second cell on a different host
CELL_NAME=pic1 VPN_ADDRESS=10.1.0.1/24 make setup && make start

make check-deps installs python3, python3-cryptography, docker, docker-compose, curl, openssl, git via apt and adds the current user to the docker group.

make setup generates WireGuard keys, writes configs, and creates all data directories.

make start builds and brings up all 12 Docker containers.

2. Access

Service	URL
Web UI	http://<host-ip>:8081
API	http://<host-ip>:3000
Health	http://<host-ip>:3000/health

On a WireGuard client: http://mycell.cell (or whatever your cell name is).

3. Local dev (no Docker)

pip install -r api/requirements.txt
python api/app.py                          # API on :3000

cd webui && npm install && npm run dev     # React UI on :5173 (proxies API to :3000)

---

🛠️ Management Commands

# First install
make check-deps          # install all system packages via apt
make setup               # generate keys, write configs
make start               # start all 12 containers

# Daily operations
make status              # container status + API health
make logs                # follow all logs
make logs-api            # follow logs for one service (api, dns, wg, mail, caddy, ...)
make shell-api           # open a shell inside a container

# Deploy latest code
make update              # git pull + rebuild + restart

# Full wipe and reinstall (useful on test machine)
make reinstall           # stop, wipe config/data, setup, start fresh

# Remove everything
make uninstall           # stop + remove images; prompts whether to also wipe config/data

# Maintenance
make backup              # tar config/ + data/ into backups/
make restore             # list available backups
make clean               # remove containers/volumes, keep config/data

# Tests
make test                # run all tests
make test-coverage       # tests + HTML coverage report

---

🔗 Connecting Two Cells (PIC Mesh)

Two PIC instances can form a mesh — full site-to-site WireGuard tunnels with automatic DNS forwarding so each cell's services are reachable from the other.

Install the second cell

# On the second host (different VPN subnet; port 51820 is fine — different machine)
CELL_NAME=pic1 VPN_ADDRESS=10.1.0.1/24 make setup && make start

Exchange invites (two pastes, two clicks)

1. On Cell A → open Web UI → Cell Network → copy the invite JSON.
2. On Cell B → Cell Network → paste into "Connect to Another Cell" → click Connect.
3. On Cell B → copy its invite JSON.
4. On Cell A → paste Cell B's invite → click Connect.

Both cells now have:

• A site-to-site WireGuard peer (AllowedIPs = remote cell's VPN subnet).
• A CoreDNS forwarding block so *.pic1.cell resolves across the tunnel.

The Connected Cells panel shows live handshake status (green = online).

Same-LAN tip

If both cells share the same external IP (behind NAT), the auto-detected endpoint in the invite will be the public IP. Replace it with the LAN IP before clicking Connect so traffic stays local:

{ "endpoint": "192.168.31.50:51820", ... }

---

🏗️ Architecture

Service Manager Architecture

All services inherit from BaseServiceManager, providing:

• Unified Interface: Consistent methods across all services
• Health Monitoring: Standardized health checks and metrics
• Error Handling: Centralized error handling and logging
• Configuration: Common configuration management patterns

Event-Driven Service Bus

# Services communicate via events
service_bus.register_service('network', network_manager)
service_bus.register_service('wireguard', wireguard_manager)
service_bus.publish_event(EventType.SERVICE_STARTED, 'network', data)

Service Dependencies

wireguard → network
email → network, vault
calendar → network, vault
files → network, vault
routing → network, wireguard
vault → network

---

🔧 Service Managers

Core Network Services

• NetworkManager: DNS, DHCP, NTP with dynamic zone management
• WireGuardManager: VPN configuration, peer management, key generation
• PeerRegistry: Peer registration, IP updates, trust management

Digital Services

• EmailManager: SMTP/IMAP email with user management
• CalendarManager: CalDAV/CardDAV calendar and contacts
• FileManager: WebDAV file storage with user directories

Infrastructure Services

• RoutingManager: NAT, firewall, advanced routing (exit/bridge/split)
• VaultManager: Certificate authority, trust management, encryption
• ContainerManager: Docker orchestration and container management
• CellManager: Overall cell configuration and service orchestration

---

📡 API Reference

Core Endpoints

# Service Status
GET /api/services/status
GET /api/services/connectivity

# Configuration Management
GET /api/config
PUT /api/config
POST /api/config/backup
POST /api/config/restore/<backup_id>

# Service Bus
GET /api/services/bus/status
GET /api/services/bus/events
POST /api/services/bus/services/<service>/start

# Logging
GET /api/logs/services/<service>
POST /api/logs/search
POST /api/logs/export

Service-Specific Endpoints

# Network Services
GET /api/dns/records
POST /api/dns/records
GET /api/dhcp/leases
GET /api/ntp/status

# WireGuard & Peers
GET /api/wireguard/peers
POST /api/wireguard/peers
GET /api/wireguard/status

# Digital Services
GET /api/email/users
GET /api/calendar/users
GET /api/files/users

# Routing & Security
GET /api/routing/status
POST /api/routing/nat
GET /api/vault/certificates

---

💻 CLI Guide

Enhanced CLI Features

# Interactive Mode
python api/enhanced_cli.py --interactive

# Batch Operations
python api/enhanced_cli.py --batch "status" "services" "health"

# Configuration Management
python api/enhanced_cli.py --export-config json
python api/enhanced_cli.py --import-config config.json

# Service Wizards
python api/enhanced_cli.py --wizard network
python api/enhanced_cli.py --wizard email

# Health Monitoring
python api/enhanced_cli.py --health
python api/enhanced_cli.py --logs network

Service Management

# Show status
python api/enhanced_cli.py --status

# List services
python api/enhanced_cli.py --services

# Peer management
python api/enhanced_cli.py --peers

# Service logs
python api/enhanced_cli.py --logs wireguard

---

⚙️ Configuration

Configuration Management

# Export configuration
curl -X GET http://localhost:3000/api/config

# Update configuration
curl -X PUT http://localhost:3000/api/config \
  -H "Content-Type: application/json" \
  -d '{"cell_name": "mycell", "domain": "mycell.cell"}'

# Backup configuration
curl -X POST http://localhost:3000/api/config/backup

Service Configuration

Each service has its own configuration schema:

• Network: DNS zones, DHCP ranges, NTP servers
• WireGuard: Interface settings, peer configurations
• Email: Domain settings, user accounts, mailboxes
• Calendar: User accounts, calendar sharing
• Files: Storage quotas, user directories
• Routing: NAT rules, firewall policies, routing tables

---

🔒 Security

Certificate Management

• Self-hosted CA: Issue and manage TLS certificates
• Certificate Lifecycle: Generate, renew, revoke certificates
• Trust Management: Direct and indirect trust relationships
• Age Encryption: Modern encryption for sensitive data

Network Security

• WireGuard VPN: Secure peer-to-peer communication
• Firewall & NAT: Granular access control
• Service Isolation: Docker containers for each service
• Input Validation: All API endpoints validate input

Data Protection

• Encrypted Storage: Sensitive data encrypted at rest
• Secure Communication: TLS for all API endpoints
• Access Control: Role-based access for services
• Audit Logging: Comprehensive security event logging

---

🛠️ Development

Project Structure

PersonalInternetCell/
├── api/                    # Backend API server
│   ├── base_service_manager.py    # Base class for all services
│   ├── config_manager.py          # Configuration management
│   ├── service_bus.py             # Event-driven service bus
│   ├── log_manager.py             # Comprehensive logging
│   ├── enhanced_cli.py            # Enhanced CLI tool
│   ├── network_manager.py         # DNS, DHCP, NTP
│   ├── wireguard_manager.py       # VPN and peer management
│   ├── email_manager.py           # Email services
│   ├── calendar_manager.py        # Calendar services
│   ├── file_manager.py            # File storage
│   ├── routing_manager.py         # Routing and NAT
│   ├── vault_manager.py           # Security and trust
│   ├── container_manager.py       # Container orchestration
│   ├── cell_manager.py            # Overall cell management
│   ├── peer_registry.py           # Peer registration
│   └── app.py                     # Main API server
├── webui/                  # React frontend
├── config/                 # Configuration files
├── data/                   # Persistent data
├── tests/                  # Test suite
└── docker-compose.yml      # Container orchestration

Running Locally

# Install dependencies
pip install -r api/requirements.txt

# Start the API server
python api/app.py

# Run tests
python api/test_enhanced_api.py

# Start frontend (if available)
cd webui && bun install && npm run dev

Service Development

from base_service_manager import BaseServiceManager

class MyServiceManager(BaseServiceManager):
    def __init__(self, data_dir='/app/data', config_dir='/app/config'):
        super().__init__('myservice', data_dir, config_dir)
    
    def get_status(self) -> Dict[str, Any]:
        # Implement service status
        pass
    
    def test_connectivity(self) -> Dict[str, Any]:
        # Implement connectivity test
        pass

---

🧪 Testing

Test Suite

# Run all tests
python api/test_enhanced_api.py

# Test specific components
python -m pytest api/tests/test_network_manager.py
python -m pytest api/tests/test_service_bus.py

# Coverage report
coverage run -m pytest api/tests/
coverage html

Test Coverage

• BaseServiceManager: 100% coverage
• ConfigManager: 95%+ coverage
• ServiceBus: 95%+ coverage
• LogManager: 95%+ coverage
• All Service Managers: 77%+ overall coverage

---

🚀 Deployment

Docker Deployment

# Production deployment
docker-compose -f docker-compose.prod.yml up -d

# Development deployment
docker-compose up --build

System Requirements

• CPU: 2+ cores
• RAM: 2GB+ (4GB recommended)
• Storage: 10GB+ (SSD recommended)
• Network: Stable internet connection

Monitoring

# Health check
curl http://localhost:3000/health

# Service status
curl http://localhost:3000/api/services/status

# Service connectivity
curl http://localhost:3000/api/services/connectivity

---

🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Implement your changes
4. Add tests for new functionality
5. Submit a pull request

Development Guidelines

• Follow the existing code style
• Add comprehensive tests
• Update documentation
• Use the BaseServiceManager pattern
• Implement proper error handling

---

📄 License

MIT License - see LICENSE file for details.

---

📚 Documentation

• Quick Start Guide: Get up and running quickly
• API Documentation: Complete API reference
• Comprehensive Improvements: Detailed architecture overview
• Enhanced API Improvements: Technical implementation details

---

🌟 The Personal Internet Cell - Your self-hosted, production-grade digital infrastructure!