Database Monitoring

A developer at a SaaS company pushed a code change that added a new feature, user search by name. The feature worked in development with 100 test users. In production, with 8 million users, the query was doing a full table scan. It held a lock on the users table for 30 seconds per execution.

Within an hour of deployment, the database connection pool was exhausted. Every web server was waiting for connections. The entire platform went down. Not just search, everything. Login, dashboard, API, webhooks, all of it was blocked on a database that couldn't respond because every connection was occupied by a query scanning 8 million rows.

The database is the single most common root cause of production outages. When the database is slow, everything is slow. When the database is down, everything is down. Monitoring it isn't optional. It's your first line of defense.

Query performance:

• Slow query count, queries exceeding a threshold (e.g., > 100ms)
• Query throughput, total queries per second
• Average query duration, across all queries
• Top N slow queries, the specific queries hurting performance

Connection management:

• Active connections, currently executing queries
• Idle connections, open but not doing work
• Connection pool utilization, percentage of max connections in use
• Connection wait time, how long new connections wait for a slot

Replication health (if using replicas):

• Replication lag, how far behind the replica is in seconds
• Replication errors, broken replication that needs intervention
• Replica connection count, is the replica handling its fair share?

Resource utilization:

• Buffer/cache hit ratio, percentage of reads served from memory vs disk (should be > 95%)
• Disk I/O, read/write throughput and IOPS
• Table/index size growth, gradual disk consumption
• Lock contention, queries waiting on locks held by other queries