Reliability audit: backups, monitoring, integrity

Method

Read-only access across every host using key authentication, plus a review of hypervisor backup jobs, backup storage and the monitoring database. Every change made during the audit was logged with a rollback path, and no secrets went into the report.

Fixed during the audit

• Backup coverage: found the containers that weren’t in a backup schedule and added a nightly snapshot job for them, with daily, weekly and monthly retention. Confirmed the first dumps landed the next morning.
• Monitoring coverage: compared the service inventory against Uptime Kuma and added 10 missing monitors, all wired to Telegram alerts.
• Isolated service monitoring: the business app sits in an isolated network segment the monitor can’t reach, by design. Replaced the polling check with a push heartbeat from the hypervisor. (How that first went is INC-0003.)
• Backup integrity: ran zstd -t against the latest backup of every critical guest. All passed.

Output

A prioritized report with 10 findings. Each one has an owner, a risk rating, a rollback plan and a next action. The remaining items are hardening work, worked in order of risk.

Lesson

An audit is only useful if every finding has an owner, a priority and a next action. “We should look at that someday” isn’t a finding.