fix: logging verbosity now actually applies + per-service log levels
Unit Tests / test (push) Successful in 12m34s

Root causes fixed:
- Dead LOG_LEVEL globals() lookup pinned root logger at INFO regardless of
  PIC_LOG_LEVEL env or config; replaced with _resolve_root_log_level() +
  apply_root_log_level() which sets both root logger and all attached handlers
  at startup and on runtime re-apply.
- set_service_level() only set the named 'pic.<service>' logger; bare module
  loggers (e.g. 'caddy_manager') were never reached, so per-service log files
  stayed 0 bytes. Fixed via _SERVICE_MODULE_LOGGERS map covering all managers.
- Log viewer GET /api/logs had no level filter; added ?level= query param.
- Per-service log levels lived in an out-of-band config/api/log_levels.json
  side-file with no validation; migrated into ConfigManager under a new
  'logging' section ({python:{root,services}, containers:{caddy,coredns,
  wireguard,mailserver,api}}) with get/set helpers, invalid-level rejection,
  and one-time migration from the old file on first load.

New capabilities:
- Container log levels: Caddy (injects global log { level X } + hot reload),
  CoreDNS (DEBUG enables log plugin, else errors-only), WireGuard/mailserver
  via pending_restart path.
- PUT /api/logs/verbosity accepts {python, containers} dict; returns per-entry
  applied:hot|pending_restart status.
- Webui Logs page gains two-section Verbosity tab (Python services + Container
  services) with needs-restart badges.
- managers.py wires per-service loggers before manager instantiation and
  re-applies persisted levels from ConfigManager; legacy log_levels.json read
  removed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-06-10 19:14:01 -04:00
parent 89aed4efe0
commit 13074f56cb
15 changed files with 726 additions and 158 deletions
+30 -23
View File
@@ -45,6 +45,22 @@ config_manager = ConfigManager(
service_bus = ServiceBus()
log_manager = LogManager(log_dir='./data/logs')
# Attach per-service file loggers BEFORE any manager is instantiated. Managers
# log during __init__ via self.logger ('picell.<svc>'); without the handlers in
# place first, those early records would be lost from the per-service log files.
_service_log_configs = {
'network': {'level': 'INFO', 'formatter': 'json', 'console': False},
'wireguard': {'level': 'INFO', 'formatter': 'json', 'console': False},
'email': {'level': 'INFO', 'formatter': 'json', 'console': False},
'calendar': {'level': 'INFO', 'formatter': 'json', 'console': False},
'files': {'level': 'INFO', 'formatter': 'json', 'console': False},
'routing': {'level': 'INFO', 'formatter': 'json', 'console': False},
'vault': {'level': 'INFO', 'formatter': 'json', 'console': False},
'api': {'level': 'INFO', 'formatter': 'json', 'console': True},
}
for _svc, _cfg in _service_log_configs.items():
log_manager.add_service_logger(_svc, _cfg)
# ServiceRegistry depends only on config_manager; create it early so
# NetworkManager and CaddyManager can derive subdomains from manifests
# instead of hardcoding service names.
@@ -112,30 +128,21 @@ service_store_manager.egress_manager = egress_manager
setup_manager = SetupManager(config_manager=config_manager, auth_manager=auth_manager,
network_manager=network_manager)
# Service logger configuration
_service_log_configs = {
'network': {'level': 'INFO', 'formatter': 'json', 'console': False},
'wireguard': {'level': 'INFO', 'formatter': 'json', 'console': False},
'email': {'level': 'INFO', 'formatter': 'json', 'console': False},
'calendar': {'level': 'INFO', 'formatter': 'json', 'console': False},
'files': {'level': 'INFO', 'formatter': 'json', 'console': False},
'routing': {'level': 'INFO', 'formatter': 'json', 'console': False},
'vault': {'level': 'INFO', 'formatter': 'json', 'console': False},
'api': {'level': 'INFO', 'formatter': 'json', 'console': True},
}
for _svc, _cfg in _service_log_configs.items():
log_manager.add_service_logger(_svc, _cfg)
# Apply persisted per-service log levels from ConfigManager (single source of
# truth — the logging section of cell_config). This runs AFTER managers are
# instantiated so it overrides their default INFO and reaches the module loggers.
try:
_logging_cfg = config_manager.get_logging_config()
for _svc, _lvl in _logging_cfg['python']['services'].items():
log_manager.set_service_level(_svc, _lvl)
except Exception:
pass
# Apply any persisted log level overrides (stored in the mounted config volume)
import json as _json
_levels_file = os.path.join(CONFIG_DIR, 'log_levels.json')
if os.path.exists(_levels_file):
try:
with open(_levels_file) as _lf:
for _s, _l in _json.load(_lf).items():
log_manager.set_service_level(_s, _l)
except Exception:
pass
# Let generate_corefile keep the configured CoreDNS log level sticky across all
# regenerations, not just verbosity-triggered ones.
firewall_manager.set_coredns_level_resolver(
lambda: config_manager.get_logging_config()['containers'].get('coredns', 'INFO')
)
service_bus.start()