roof/pic
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add security fixes, port hardening, and expanded QA coverage

Browse Source 
Security fixes:
- Replace debug=True with env-driven FLASK_DEBUG in app.py
- Add _safe_path helper and path-traversal protection to all 6 file routes
  in file_manager.py
- Add peer_name regex and input validation (public_key, name, endpoint_ip)
  in wireguard_manager.py
- Stop returning private key from GET /api/wireguard/keys; return only
  public_key + has_private_key boolean
- Fix is_local_request() XFF bypass by checking remote_addr only, ignoring
  X-Forwarded-For
- Remove duplicate get_all_configs / get_config_summary methods from
  config_manager.py

DevOps:
- Bind 6 internal service ports to 127.0.0.1 in docker-compose.yml
  (radicale, webdav, api, webui, rainloop, filegator)
- Move WebDAV credentials to env vars (WEBDAV_USER, WEBDAV_PASS)
- Pin flask, flask-cors, requests, cryptography, docker to secure minimum
  versions in requirements.txt

QA (560 tests, 0 failures):
- tests/test_wireguard_endpoints.py: 18 new endpoint tests
- tests/test_file_endpoints.py: 24 new endpoint tests incl. path traversal
- tests/test_container_manager.py: expanded from 2 to 30 tests
- tests/test_config_backup_restore_http.py: 25 new tests (new file)
- tests/test_config_apply.py: 9 new tests (new file)

Docs:
- Rewrite README.md with accurate architecture, ports, env vars, security notes
- Rewrite QUICKSTART.md with verified commands

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Dmitrii Iurco
2026-04-25 13:08:24 -04:00
parent eb817ffdc5
commit a338836bb8
13 changed files with 1861 additions and 681 deletions
Download Patch File Download Diff File Expand all files Collapse all files
QUICKSTART.md
+239 -358
View File
@@ -1,358 +1,239 @@
# Personal Internet Cell - Quick Start Guide
## 🚀 Getting Started
This guide will help you get your Personal Internet Cell up and running with the new production-grade architecture in minutes.
### Prerequisites
- **Docker and Docker Compose** installed
- **Python 3.10+** (for CLI and development)
- **Ports available**: 53, 80, 443, 3000, 51820
- **Administrative access** (for WireGuard and network services)
- **2GB+ RAM, 10GB+ disk space**
### Step 1: Initial Setup
```bash
# Clone or download the project
git clone https://github.com/yourusername/PersonalInternetCell.git
cd PersonalInternetCell
# Start all services with Docker (Recommended)
docker-compose up --build
# Or run locally
pip install -r api/requirements.txt
python api/app.py
```
### Step 2: Verify Installation
```bash
# Check if API is responding
curl http://localhost:3000/health
# Check service status
curl http://localhost:3000/api/services/status
# Use the enhanced CLI
python api/enhanced_cli.py --status
```
### Step 3: Explore Services
```bash
# Show all services
python api/enhanced_cli.py --services
# Check health data
python api/enhanced_cli.py --health
# Interactive mode
python api/enhanced_cli.py --interactive
```
## 📋 Enhanced CLI Commands
### Basic Management
```bash
# Service status
python api/enhanced_cli.py --status
python api/enhanced_cli.py --services
# Health monitoring
python api/enhanced_cli.py --health
# Service logs
python api/enhanced_cli.py --logs network
python api/enhanced_cli.py --logs wireguard
```
### Configuration Management
```bash
# Export configuration
python api/enhanced_cli.py --export-config json
python api/enhanced_cli.py --export-config yaml
# Import configuration
python api/enhanced_cli.py --import-config config.json
# Configuration wizard
python api/enhanced_cli.py --wizard network
python api/enhanced_cli.py --wizard email
```
### Batch Operations
```bash
# Execute multiple commands
python api/enhanced_cli.py --batch "status" "services" "health"
# Interactive mode with tab completion
python api/enhanced_cli.py --interactive
```
## 🌐 Accessing Services
Once running, you can access:
- **API Server**: http://localhost:3000
- **API Health**: http://localhost:3000/health
- **Service Status**: http://localhost:3000/api/services/status
- **Configuration**: http://localhost:3000/api/config
- **Service Bus**: http://localhost:3000/api/services/bus/status
- **Logs**: http://localhost:3000/api/logs/services/network
## 🔧 Configuration
### Cell Configuration
The cell uses a centralized configuration system with schema validation:
```bash
# View current configuration
curl http://localhost:3000/api/config
# Update configuration
curl -X PUT http://localhost:3000/api/config \
-H "Content-Type: application/json" \
-d '{
"cell_name": "mycell",
"domain": "mycell.cell",
"ip_range": "10.0.0.0/24",
"wireguard_port": 51820
}'
```
### Service Configuration
Each service has its own configuration schema:
```bash
# Network configuration
python api/enhanced_cli.py --wizard network
# Email configuration
python api/enhanced_cli.py --wizard email
# WireGuard configuration
python api/enhanced_cli.py --wizard wireguard
```
### Network Configuration
The cell uses the following network ranges:
- **Cell Network**: 10.0.0.0/24 (configurable)
- **DHCP Range**: 10.0.0.100-10.0.0.200 (configurable)
- **WireGuard Port**: 51820/UDP (configurable)
- **API Port**: 3000 (configurable)
## 🔗 Adding Peers
### 1. Generate WireGuard Keys (on peer cell)
```bash
wg genkey | tee private.key | wg pubkey > public.key
```
### 2. Add Peer to Your Cell
```bash
# Using the enhanced CLI
python api/enhanced_cli.py --batch "add-peer bob 203.0.113.22 $(cat public.key)"
# Or via API
curl -X POST http://localhost:3000/api/wireguard/peers \
-H "Content-Type: application/json" \
-d '{
"name": "bob",
"ip": "203.0.113.22",
"public_key": "your_public_key_here"
}'
```
### 3. Configure Routing Rules
```bash
# Allow peer to access your LAN
curl -X POST http://localhost:3000/api/routing/peers \
-H "Content-Type: application/json" \
-d '{
"peer_name": "bob",
"peer_ip": "203.0.113.22",
"allowed_networks": ["10.0.0.0/24"],
"route_type": "lan"
}'
# Allow peer to use your cell as exit node
curl -X POST http://localhost:3000/api/routing/exit-nodes \
-H "Content-Type: application/json" \
-d '{
"peer_name": "bob",
"peer_ip": "203.0.113.22",
"allowed_domains": ["google.com", "github.com"]
}'
```
## 🔍 Troubleshooting
### Services Not Starting
```bash
# Check Docker logs
docker-compose logs
# Check individual service
docker-compose logs api
docker-compose logs wireguard
# Check service status via API
curl http://localhost:3000/api/services/status
```
### API Issues
```bash
# Test API health
curl http://localhost:3000/health
# Check service connectivity
curl http://localhost:3000/api/services/connectivity
# View API logs
python api/enhanced_cli.py --logs api
```
### Network Issues
```bash
# Test DNS resolution
nslookup google.com 127.0.0.1
# Check network service status
curl http://localhost:3000/api/dns/status
curl http://localhost:3000/api/network/info
# Test network connectivity
curl -X POST http://localhost:3000/api/network/test \
-H "Content-Type: application/json" \
-d '{"target": "8.8.8.8"}'
```
### WireGuard Issues
```bash
# Check WireGuard status
curl http://localhost:3000/api/wireguard/status
# Test WireGuard connectivity
curl -X POST http://localhost:3000/api/wireguard/connectivity \
-H "Content-Type: application/json" \
-d '{"target_ip": "203.0.113.22"}'
# View WireGuard logs
python api/enhanced_cli.py --logs wireguard
```
### Configuration Issues
```bash
# Validate configuration
curl http://localhost:3000/api/config
# Backup and restore
curl -X POST http://localhost:3000/api/config/backup
curl -X POST http://localhost:3000/api/config/restore/backup_id
# Export/import configuration
python api/enhanced_cli.py --export-config json
python api/enhanced_cli.py --import-config config.json
```
## 📁 File Structure
```
PersonalInternetCell/
├── docker-compose.yml # Main orchestration
├── api/ # API server and service managers
│ ├── base_service_manager.py # Base class for all services
│ ├── config_manager.py # Configuration management
│ ├── service_bus.py # Event-driven service bus
│ ├── log_manager.py # Comprehensive logging
│ ├── enhanced_cli.py # Enhanced CLI tool
│ ├── network_manager.py # DNS, DHCP, NTP
│ ├── wireguard_manager.py # VPN and peer management
│ ├── email_manager.py # Email services
│ ├── calendar_manager.py # Calendar services
│ ├── file_manager.py # File storage
│ ├── routing_manager.py # Routing and NAT
│ ├── vault_manager.py # Security and trust
│ ├── container_manager.py # Container orchestration
│ ├── cell_manager.py # Overall cell management
│ ├── peer_registry.py # Peer registration
│ ├── app.py # Main API server
│ └── test_enhanced_api.py # Comprehensive test suite
├── config/ # Configuration files
│ ├── cell.json # Cell configuration
│ ├── network.json # Network service config
│ ├── wireguard.json # WireGuard config
│ └── ...
├── data/ # Persistent data
│ ├── api/ # API data
│ ├── dns/ # DNS zones
│ ├── email/ # Email data
│ ├── calendar/ # Calendar data
│ ├── files/ # File storage
│ ├── vault/ # Certificates and keys
│ └── logs/ # Service logs
└── webui/ # React frontend (if available)
```
## 🔒 Security Notes
- **Self-hosted CA**: The cell generates and manages its own certificates
- **WireGuard keys**: Generated automatically with secure key management
- **Service isolation**: All services run in isolated Docker containers
- **Encrypted storage**: Sensitive data encrypted using Age/Fernet
- **Trust management**: Peer trust relationships with cryptographic verification
- **Configuration validation**: All configuration validated against schemas
## 🆘 Getting Help
### Diagnostic Commands
```bash
# Comprehensive status check
python api/enhanced_cli.py --status
# Service health check
python api/enhanced_cli.py --health
# Service logs
python api/enhanced_cli.py --logs network
# Configuration validation
curl http://localhost:3000/api/config
# Service connectivity test
curl http://localhost:3000/api/services/connectivity
```
### Common Issues
1. **Port conflicts**: Ensure ports 53, 3000, 51820 are available
2. **Permission issues**: Run with appropriate privileges for network services
3. **Configuration errors**: Use the configuration wizard for guided setup
4. **Service dependencies**: Check service bus status for dependency issues
## 🚀 Next Steps
After basic setup, consider:
1. **Customizing your cell name** and domain configuration
2. **Adding trusted peers** for mesh networking
3. **Configuring email services** with your domain
4. **Setting up file storage** and user management
5. **Implementing backup strategies** for configuration and data
6. **Exploring advanced routing** features (exit nodes, bridge routing)
7. **Setting up monitoring** and alerting for service health
## 📚 Additional Resources
- **[API Documentation](api/API_DOCUMENTATION.md)**: Complete API reference
- **[Comprehensive Improvements](COMPREHENSIVE_IMPROVEMENTS_SUMMARY.md)**: Architecture overview
- **[Enhanced API Improvements](ENHANCED_API_IMPROVEMENTS.md)**: Technical details
- **[Project Wiki](Personal%20Internet%20Cell%20–%20Project%20Wiki.md)**: Detailed project information
---
**🌟 Happy networking with your Personal Internet Cell!**
# Quick Start
This guide walks through a first-time PIC installation from a clean Linux host.
---
## Prerequisites
- Linux host with the WireGuard kernel module (`modprobe wireguard` to verify)
- Docker Engine and Docker Compose installed
- Python 3.10+ (needed for `make setup` only)
- 2 GB+ RAM, 10 GB+ disk
---
## 1. Clone the repository
```bash
git clone <repo-url> pic
cd pic
```
---
## 2. Configure the environment
Copy the example environment file and edit it:
```bash
cp .env.example .env
```
Open `.env` and set at minimum:
```
WEBDAV_PASS=changeme
```
`WEBDAV_PASS` must be set before starting — the WebDAV container will fail to start without it.
All other variables have working defaults. See the Configuration section in [README.md](README.md) for the full list.
---
## 3. Run setup
`make setup` installs system dependencies, generates WireGuard keys, and writes all required config files under `config/`:
```bash
make check-deps # installs docker, python3-cryptography, etc. via apt
make setup # generates keys and writes configs
```
To customise the cell identity at setup time, pass overrides on the command line:
```bash
CELL_NAME=myhome CELL_DOMAIN=cell VPN_ADDRESS=10.0.0.1/24 WG_PORT=51820 make setup
```
`VPN_ADDRESS` must be an RFC-1918 address (e.g. `10.0.0.1/24`).
---
## 4. Start the stack
```bash
make start
```
This builds the `cell-api` and `cell-webui` images and starts all 13 containers. The first run takes a few minutes while images are pulled and built.
Check that everything came up:
```bash
make status
```
You should see all containers in the `Up` state and the API responding at `http://localhost:3000/health`.
---
## 5. Open the web UI
Open a browser and go to:
```
http://<host-ip>:8081
```
If you are running locally:
```
http://localhost:8081
```
The sidebar contains: Dashboard, Peers, Network Services, WireGuard, Email, Calendar, Files, Routing, Vault, Containers, Cell Network, Logs, Settings.
---
## 6. Set cell identity
Go to **Settings** in the sidebar.
Set your:
- **Cell name** — a short identifier, e.g. `myhome`
- **Domain** — the TLD your cell will use internally, e.g. `cell`
- **VPN IP range** — the CIDR for WireGuard peers, e.g. `10.0.0.0/24`
After saving, the UI will show a banner asking you to apply the changes. Click **Apply Now**. The containers will restart briefly to pick up the new configuration.
---
## 7. Add a WireGuard peer
Go to **WireGuard** in the sidebar.
1. Click **Add Peer**.
2. Enter a name for the peer (e.g. `laptop`).
3. The API generates a key pair and assigns the next available VPN IP automatically.
4. Click the QR code icon to display the peer config as a QR code.
5. Scan the QR code with a WireGuard client (Android, iOS, or the WireGuard desktop app).
The peer config sets your cell as the DNS server. Once connected, `*.cell` names resolve through the cell's CoreDNS.
To manage peers from the command line:
```bash
make list-peers
make add-peer PEER_NAME=phone PEER_IP=10.0.0.3 PEER_KEY=<base64-pubkey>
```
---
## 8. Day-to-day operations
```bash
# Follow logs from all services
make logs
# Follow logs from a single service
make logs-api
make logs-wireguard
make logs-caddy
# Check container status and API health
make status
# Open a shell inside a container
make shell-api
make shell-dns
```
---
## 9. Backup
Before making significant changes, create a backup:
```bash
make backup
```
This archives `config/` and `data/` into `backups/cell-backup-<timestamp>.tar.gz`.
To list available backups:
```bash
make restore
```
To restore manually:
```bash
tar -xzf backups/cell-backup-YYYYMMDD-HHMMSS.tar.gz
make start
```
Backup and restore is also available in the UI under **Settings**.
---
## 10. Updating PIC
```bash
make update
```
This runs `git pull`, then rebuilds and restarts all containers. If `config/` is missing (e.g. after a fresh clone), it runs `make setup` automatically.
---
## Troubleshooting
**Containers not starting**
```bash
make logs
make logs-api
```
Look for errors related to missing config files or port conflicts.
**Port 53 already in use**
On Ubuntu/Debian, `systemd-resolved` listens on port 53. Disable it:
```bash
sudo systemctl disable --now systemd-resolved
sudo rm /etc/resolv.conf
echo "nameserver 8.8.8.8" | sudo tee /etc/resolv.conf
```
Then run `make start` again.
**WebDAV container exits immediately**
`WEBDAV_PASS` is not set in `.env`. Set it and run `make start` again.
**WireGuard container fails to load kernel module**
Ensure the WireGuard kernel module is available:
```bash
sudo modprobe wireguard
```
On some minimal installs you may need to install `wireguard-tools` and the kernel headers for your running kernel.
**API returns 503 or UI shows "Backend Unavailable"**
The Flask API may still be starting. Wait 10–15 seconds after `make start` and refresh. If it persists:
```bash
make logs-api
```
**Config changes not taking effect**
After changing identity or service settings in the UI, a yellow banner appears at the top of the page. Click **Apply Now** to restart the affected containers.