Why Your Uptime Percentage Misses the Outage

Your video call dropped at 20:40. The monitor covering that evening reports 99.9% uptime. Both numbers come from the same network, and the second one will never explain the first.

Uptime percentage is an average of a binary series: up, up, up, down, up. Averages lose runs, and the run is what ends a call or kills an SSH session. Throughput is a separate question, and you measure it with the iPerf3 app or the command-line binary.

What 99.9% buys you

Now put one dropped call next to that. Thirty seconds of dead air in a 30-day month is 0.0012% of the month, so the monitor reports 99.9988% and the status page rounds it to 100%. The arithmetic is right. It is also blind to the only event you asked about.

Reason one: the check interval

A monitor samples. Between two probes it has no data, and it fills that space with whatever the next probe returns.

Take a 90-second outage and a monitor that probes every 5 minutes. The probe has to fire while the outage is in progress, which happens on 90 seconds out of every 300. Seven times out of ten the outage opens and closes between two successful probes and leaves nothing behind.

The three times out of ten a probe does land inside, the record is wrong the other way. Most monitors count downtime from the failed check to the next good one, so 90 seconds of dead air enters the log as 5 minutes. Recorded downtime is a multiple of your interval. It is never the real duration.

Two defaults worth knowing, because most homelabs run one of them:

• UptimeRobot free checks every 5 minutes.
• Uptime Kuma ships with a 60-second heartbeat and no retries.

Retries move the line further out. Three retries at 60 seconds means nothing shorter than about three minutes will ever page you, whatever the dashboard says afterwards.

Reason two: the average

99.9% of a month is 43 minutes. That is one 43-minute outage, or 43 outages of a minute each, or 2,592 drops of one second. Same percentage, three different networks.

The first one wakes you at 03:00 and you fix it. The third one never shows up as an incident and ruins everything that holds a connection open. TCP hides a lost second with a retransmit. A video call does not hide twenty.

Reason three: what you never probed

The percentage covers the things you told it to check. A container you stood up in April and forgot has no number at all, and a missing number never looks bad.

This is the one that bites homelabs, because you type the check list by hand while the lab keeps growing. Generate the list instead: docker ps --format '{{.Names}}' on the host, or the service names in your compose file, beat a list you wrote once and stopped updating.

Read the run, not the average

One number replaces the percentage: the longest run of consecutive failed probes. It answers the question you have, which is how long you were dark, and averaging cannot touch it because it is a maximum.

Alert on the same thing. Two failures in a row, three at the most, beats any threshold on a percentage. By the time an outage is long enough to move a monthly average, whatever you were doing has already failed.

Measure it with what you already have

Any box that stays on will do. A Raspberry Pi in the corner is ideal, because it keeps probing while the machine you use is asleep.

# one line per probe, appended, never overwritten
while :; do
  ping -c1 -W1 1.1.1.1 >/dev/null 2>&1 && s=1 || s=0
  printf '%s %s\n' "$(date -u +%FT%TZ)" "$s" >> ~/net.log
  sleep 5
done

The interval drifts by however long the probe took, which does not matter at this resolution.

One portability trap costs people an evening: -W means different units on the two systems you are likely to use.

On macOS -W1 waits one millisecond, so every probe fails and your log fills with a fake outage. Use -t1 there, which bounds the whole command in seconds.

Now read the log two ways:

# longest run of consecutive failures
awk '$2==0 {n++; if (n>m) {m=n; t=$1}} $2==1 {n=0}
     END {print m, "probes =", m*5, "s, ending", t}' ~/net.log

# the percentage, from the same file
awk '{total++; up+=$2} END {printf "%.4f%%\n", 100*up/total}' ~/net.log

The percentage is the number you show someone. The run is the number you fix.

Picking the interval

Take the shortest outage you care about and halve it. Two probes have to fall inside the gap for a two-strike alert to fire, so half the target length is the working rule. Something that drops calls lives in the tens of seconds, so probe every 10 to 15 seconds. A stalled download tolerates minutes, and 60 seconds covers it.

Traffic is not the constraint. A probe every 5 seconds is 17,280 packets a day, under 3 MB on the wire. One minute of streaming video moves ten times that.

Probe four things, not one

A single target blends every layer between you and it. Four targets take them apart:

1. Your gateway. Is the LAN alive?
2. The first hop past it. Is the ISP still handing you off?
3. A public anchor such as 1.1.1.1. Is the internet reachable?
4. The service you use.

A run that appears on all four at once is your link. A run on the fourth alone is theirs.

ICMP has a limit worth stating: it tells you the path is alive, not that the service works. A host that answers ping while its app returns 500s reads as 100% up. Probe the port or the URL for the service and keep ICMP for the path.

When the gap lands at the same hour

A run that shows up at 20:00 every evening is congestion, not a failure. Once you have timestamps you can test the same route for throughput at that hour and again at 03:00. Same hardware, same endpoint, different result, and you have a pattern the support desk has to answer. The full procedure is in How to Prove Your ISP Is Throttling You, and Measuring Jitter and Packet Loss covers the case where the link stays up and still ruins the call.

On a Mac the same method runs in the menu bar: Net Ping Monitor pings the hosts you give it, alerts after N failures in a row, and draws the gaps on a timeline. $5.99, one time, no account.

Do this tomorrow

1. Start the loop against your gateway and one public anchor.
2. Leave it for a week.
3. Read the longest run first, the percentage second.
4. Move your alert to two failures in a row.

If the longest run comes back as a single probe, the outage was shorter than your interval and you still do not know its length. Halve the interval and give it another week.