Error budgets: exact consumption, alerting, and decision policy

An error budget is the amount of unreliability permitted by an SLO over its declared population and window. For a 99.9% request SLO the allowed bad-event ratio is 0.001. Budget remaining is an accounting result; what a team does at a given level is an explicit policy negotiated between service owners and stakeholders.

Burn rate normalizes the observed bad-event ratio by the allowed ratio. It enables alerts that work across SLO targets, but a burn threshold becomes actionable only with a long window, a short confirmation window, event-volume guards, missing-data semantics, and a stated budget fraction. A deploy freeze, exception, rollback, or SLO review is governance—not arithmetic.

Модель и предпосылки

• At 30% bad events under 99.9%, burn is 0.30 / 0.001 = 300x. If sustained uniformly, a 30-day budget exhausts in 0.1 day = 2.4 hours.
• Google’s example 14.4x alert over 1 hour with a 5-minute short window corresponds to about 2% of a 30-day budget; 6x over 6 hours with a 30-minute short window corresponds to about 5%.
• The release pipeline queries a policy gate. The gate records the budget calculation, service risk, exception owner, expiry, and rollback plan rather than silently blocking all changes.

Проверяемые утверждения

• C1. Burn rate is observed bad-event ratio divided by allowed bad-event ratio; 30% against 0.1% is 300x, not 14.4x.
• C2. At 300x, a uniformly consumed 30-day budget exhausts in 30/300 days = 2.4 hours.
• C3. Multi-window alert factors encode a chosen budget fraction and detection/reset trade-off; the documented 14.4x and 6x examples are not universal outage labels.
• C4. An error-budget policy is agreed organizational behavior; the Google policy is an example and a freeze is not automatic for every team.
• C5. Budget calculation depends on a valid SLI population, window, data source and missing-data policy.

Исполняемые сценарии

▶Walkthrough: calculate-budget-consumption (not found)

Calculate budget consumption. Eligible and good events produce a dimensionally consistent budget result for the declared window.

▶Walkthrough: three-hundred-x-burn (not found)

Correct 300x fast burn. Thirty percent bad events under a 99.9% SLO is 300x and would exhaust a uniform 30-day budget in about 2.4 hours.

▶Walkthrough: release-policy-decision (not found)

Release policy decision. A release is allowed, constrained, or rejected through a pre-agreed policy with a recorded exception path.

▶Walkthrough: missing-data-fail-closed (not found)

Missing-data policy. Incomplete SLI data is exposed as uncertainty and cannot silently replenish the budget.

Ошибки проектирования

• Не называйте 30% error rate 14.4x при SLO 99.9%; это 300x.
• Не путайте 2% месячного бюджета, потраченные за час, с полным исчерпанием бюджета.
• Не замораживайте reliability fixes вместе с feature work и не допускайте бессрочный override без owner/expiry.
• Не вычисляйте budget remaining по неполным данным молча.

Границы гарантии

• Rolling and calendar windows produce different reset behavior and must be named.
• Low traffic and correlated failures need event-count guards and sometimes synthetic or longer-window evidence.
• Budget policy cannot substitute for incident mitigation, security response, legal duties, or safety requirements.

Связанные материалы

[CONCEPT]sli-slo-sla [CONCEPT]incident-management [CONCEPT]post-mortems

Официальные и первичные источники

1. Google SRE Workbook: Alerting on SLOs
2. Google SRE Workbook: Example Error Budget Policy
3. Google SRE Workbook: Implementing SLOs
4. Google SRE Book: Embracing Risk