feat: proper logging architecture — Docker rotation, persisted service logs, verbosity config

docker-compose.yml:
- Add json-file logging driver (max-size: 10m, max-file: 5) to all 13 containers
- Docker now owns container stdout/stderr rotation automatically
- Add ./data/logs:/app/api/data/logs volume to API — service logs now persist across restarts

log_manager.py:
- Remove container log collection hack (Docker handles it natively)
- Add set_service_level(service, level) — change log level at runtime without restart
- Add get_service_levels() — return current per-service levels
- Simplify get_all_log_file_infos to return only service log files

app.py:
- Add GET /api/logs/verbosity — return current per-service log levels
- Add PUT /api/logs/verbosity — update levels at runtime, persist to config/log_levels.json
- Load persisted log level overrides at startup from log_levels.json
- Simplify rotate endpoint (service logs only, container logs owned by Docker)

wireguard_manager.py:
- get_keys(): return empty strings if key files don't exist (prevents get_status crash
  when wg0.conf is missing at startup and falls through to generate_config)

Logs page (4 tabs):
- API Service Logs: structured JSON logs from Python managers, with search/filter/rotate panel
- Container Logs: live docker logs (read via existing /api/containers/<name>/logs endpoint)
- Verbosity Config: per-service level dropdowns, apply immediately + persist
- Health History: existing health poll table

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-04-21 02:07:57 -04:00
parent 7b39331417
commit f848a1d056
6 changed files with 313 additions and 289 deletions
+44 -18
View File
@@ -125,6 +125,16 @@ service_log_configs = {
for service, config in service_log_configs.items():
log_manager.add_service_logger(service, config)
# Apply any persisted log level overrides
_levels_file = os.path.join(os.path.dirname(__file__), 'config', 'log_levels.json')
if os.path.exists(_levels_file):
try:
with open(_levels_file) as _f:
for _svc, _lvl in json.load(_f).items():
log_manager.set_service_level(_svc, _lvl)
except Exception:
pass
# Start service bus
service_bus.start()
@@ -619,17 +629,11 @@ def get_log_statistics():
@app.route('/api/logs/rotate', methods=['POST'])
def rotate_logs():
"""Manually rotate logs (API service log or container log)."""
"""Manually rotate an API service log file."""
try:
data = request.get_json(silent=True) or {}
name = data.get('name') # e.g. 'wireguard' or 'container_cell-api'
kind = data.get('kind', 'service') # 'service' or 'container'
if kind == 'container':
container_name = name[len('container_'):] if name and name.startswith('container_') else name
log_manager.rotate_container_log(container_name)
else:
log_manager.rotate_logs(name)
service = data.get('service') # None = rotate all
log_manager.rotate_logs(service)
return jsonify({"message": "Logs rotated successfully"})
except Exception as e:
logger.error(f"Error rotating logs: {e}")
@@ -637,23 +641,45 @@ def rotate_logs():
@app.route('/api/logs/files', methods=['GET'])
def get_log_file_infos():
"""List all stored log files (service + container) with sizes."""
"""List service log files with sizes."""
try:
return jsonify(log_manager.get_all_log_file_infos())
except Exception as e:
logger.error(f"Error listing log files: {e}")
return jsonify({"error": str(e)}), 500
@app.route('/api/logs/containers/<container_name>', methods=['GET'])
def get_stored_container_logs(container_name):
"""Collect latest docker logs into file and return last N lines."""
@app.route('/api/logs/verbosity', methods=['GET'])
def get_log_verbosity():
"""Return current per-service log levels."""
try:
tail = int(request.args.get('tail', 100))
log_manager.collect_container_logs(container_name)
lines = log_manager.get_container_log_lines(container_name, tail)
return jsonify({'container': container_name, 'lines': lines})
return jsonify(log_manager.get_service_levels())
except Exception as e:
logger.error(f"Error getting stored container logs for {container_name}: {e}")
logger.error(f"Error getting log verbosity: {e}")
return jsonify({"error": str(e)}), 500
@app.route('/api/logs/verbosity', methods=['PUT'])
def set_log_verbosity():
"""Update log levels for one or all services. Body: {service: level} map."""
try:
data = request.get_json(silent=True) or {}
for service, level in data.items():
log_manager.set_service_level(service, level)
# Persist to config so levels survive API restarts
levels_file = os.path.join(os.path.dirname(__file__), 'config', 'log_levels.json')
os.makedirs(os.path.dirname(levels_file), exist_ok=True)
current = {}
if os.path.exists(levels_file):
try:
with open(levels_file) as f:
current = json.load(f)
except Exception:
pass
current.update(data)
with open(levels_file, 'w') as f:
json.dump(current, f, indent=2)
return jsonify({"message": "Log levels updated", "levels": log_manager.get_service_levels()})
except Exception as e:
logger.error(f"Error setting log verbosity: {e}")
return jsonify({"error": str(e)}), 500
# Network Services API