What Is an SLO? Why SLOs Matter for Software Teams

When people talk about reliability engineering, SLO (service-level objective) is usually one of the first terms that comes up. Google's book Site Reliability Engineering even says that if a team does not have SLOs, there is no need for SRE, because SRE work is prioritized around SLOs to make sure the team can meet them effectively.

We briefly mentioned SLOs in How Should Software Engineers Do Monitoring?. In this article, we will discuss SLOs in more depth: what an SLO is, how it relates to the commonly mentioned SLI and SLA, and how teams should think about setting SLOs.

What Are SLI, SLO, and SLA?

Before defining SLOs directly, let's step back and think about the nature of software products.

Before reading on, try writing down your own answers to these questions:

• What is the purpose of a software product? Think about software you have built at work or in personal projects. Why was it worth spending time on?
• How should a software team judge whether it has reliably achieved that purpose? If someone asks how reliable your team's product is, how would you answer? What would your answer be based on?
• As a software engineer, suppose you want to propose spending more time on technical improvements and refactoring. How would you prove that this work is valuable?

In a software team, the first question is usually answered by the product manager, although good engineers should have their own view as well. The second and third questions are questions senior engineers need to face in day-to-day work. SLOs exist to help answer those two questions.

SLI Comes Before SLO

SLO stands for service-level objective. For engineers, an SLO is a set of objectives the service needs to meet. Common examples include "API request success rate reaches 99.9%" or "90% of API requests complete within 10 ms." Different software products will have different SLOs, and engineers are responsible for defining those SLOs and making sure they are met.

Unfortunately, some engineering teams set SLOs by casually picking targets just to satisfy management. Others copy SLOs from other products without thinking. To avoid both problems, teams need to first identify meaningful SLIs before setting SLOs. Otherwise, the SLOs may be arbitrary or blindly copied from elsewhere.

SLI stands for service-level indicator. In plain terms, it describes what matters for the software product.

Different products care about different things. For financial or accounting software, correctness is usually the most important property. Even a small error may be unacceptable, so these products often sacrifice some speed in exchange for correctness.

For social media products, speed is often more important because users may leave if the experience feels slow. Correctness requirements are usually lower. If a post's like count is not perfectly up to date at every moment, it usually does not cause serious harm.

That is why SLIs are usually defined in collaboration with product managers. The team needs to write down what matters most to the product. Common indicators include speed, latency, availability, durability, correctness, and completeness.

Setting SLOs Based on SLIs

Once the team has written down the important SLIs, many readers may still feel that something is missing. If a product manager says availability is the most important thing for the product, most engineers will immediately ask, "What counts as high availability?"

For example, is 99% availability high enough? Or does it need to be 99.9%? Is an API response time of 100 ms considered low latency? Or does it need to be under 50 ms?

The O in SLO, objective, answers this question. SLOs are goals set based on SLIs. Indicators tell the team what matters; objectives tell the team how good the system needs to be to protect that important indicator. Even if two systems share the same indicator, their targets may differ. Even within the same system, the same indicator may have different targets in different contexts. Later sections can go deeper into how teams should set suitable targets.

Returning to the earlier question, "How should a software team judge whether it has reliably achieved the purpose of the software?" The answer is that the team first needs to define reliability-related SLOs. Only when those SLOs are met can the team say it has reliably achieved the product's purpose.

SLA: The Promise to Customers

For engineering teams, SLOs are usually enough. From a product or business perspective, however, teams often go one step further and define SLAs, or service-level agreements.

An SLA is usually an external commitment. For example, suppose your team is building a competitor to AWS S3. If you want to convince customers to use your product instead of S3, you might say, "Our product has higher availability than AWS S3." The customer will naturally ask, "How can you guarantee that?"

SLAs address that concern. If you simply claim that your storage system has 99.999% availability, customers probably will not be convinced. But if you say, "If availability does not reach 99.999%, we will refund all fees," customers are much more likely to believe the commitment.

AWS S3's SLA (link) says that if availability falls below 99%, customers receive a 10% service credit; if availability falls below 95%, they receive a full refund.

Most teams set SLOs stricter than SLAs because doing so gives the team a buffer. For example, if the SLA is 99.9%, the internal SLO might be 99.95%. This means the SLO will be missed before the SLA is violated. When the SLO is missed, the team still has time to fix the issue before directly breaching the SLA and owing compensation.

Do Not Measure Only Averages

In practice, teams usually look at percentiles such as P50, P90, P95, and P99. P50 means the 50th percentile, also known as the median. P99 means the 99th percentile, which means that 99 out of 100 requests are below that value.

For example, if the P50 response time is 80 ms, that means half of requests are faster than 80 ms and the other half are slower than 80 ms. Similarly, if the P95 response time is 100 ms, then 5% of requests take longer than 100 ms.

Percentiles are recommended over averages because averages do not describe distribution clearly. Suppose 5% of requests are extremely fast while another 5% are very slow. The average may still look fine, but percentiles will make the slow 5% visible. That makes it easier for the team to notice the problem.

This is especially important for products with large traffic. For those products, 5% of users may mean millions of people. The team needs to make sure not only P95 but even P99 can meet the target; otherwise, the cost can be significant.

Read More

After understanding what an SLO is, many people will naturally ask, "How should a team set good SLOs for its product?" In the E+ member article, we discuss this further through concrete examples.

Support ExplainThis

If you found this content valuable, please consider supporting our work with a one-time donation of whatever amount feels right to you through this Buy Me a Coffee page.

Creating in-depth technical content takes significant time. Your support helps us continue producing high-quality educational content accessible to everyone.