What do I do if there are no benchmarks whatsoever? How do I plan my SLAs in this case?

When faced with a complete lack of historical benchmarks, SLA planning must pivot from historical system data to business requirements and iterative discovery. Here is a practical framework to approach this:

Drive from Business and User Experience: Start by asking "What is acceptable to the end-user?" instead of "What can the system handle?". Define thresholds based on user tolerance (e.g., standard 2-3 second UI load limits) and critical business flows.

Establish SLIs and Internal SLOs First: Identify key Service Level Indicators (Latency, Error Rate, Throughput). Before committing to external SLAs, establish internal Service Level Objectives (SLOs) as a safety buffer.

Create a Day-Zero Baseline: Conduct exploratory load and stress tests in a production-like environment. This provides an initial snapshot of the current architecture's capabilities and breaking points.

Adopt Industry Standards: Utilize established industry benchmarks for similar systems (e.g., API response times under 200ms) as a temporary starting point.

Iterative Maturation (Start Loose, Tighten Later): Set conservative, wider SLAs initially. Monitor production telemetry for the first 30-60 days, collect real-world data, and progressively tighten the SLAs as you establish reliable internal benchmarks.

If there are no existing benchmarks, I wouldn’t simply pick an SLA out of thin air.

I’d start by defining what “acceptable” means from the business and user perspective:

• Identify the critical user journeys and business transactions.
• Talk to product/business stakeholders about what response time they consider acceptable.
• Look at production/user data if available — current response times, traffic patterns, error rates, etc.
• If production data isn’t available, establish a **baseline** by running performance tests against the current system.
• Use that baseline to set initial SLOs, while clearly treating them as **provisional targets**, not absolute truths.
• Then validate those targets through load testing and, more importantly, real-world feedback after release.

For example, if there is no benchmark for a checkout API, I might initially propose something like “95% of requests should complete within 2 seconds under expected load.” But I’d document *why* that number was chosen and review it with the business rather than presenting 2 seconds as a universal standard.

The key is: an SLA should be derived from business/user expectations and evidence, not from a number that sounds good.

And once we have production data, the SLA/SLO should evolve with the system rather than remaining a one-time performance target.
 

For planning without benchmarks, we can:
a) Check if there is any telemetry data such as datadog. Any previous runs. Telemetry metric correlated with load.
b) Ask the business end-users directly, check if there is any historical data or report.
c) Integrate NeoLoad and qTest and refer to qTest report.
d) Use AI agents to run automated sampling tests in production that measures the SUT at various scenarios. Example, run Tricentis AIDA and review the AIDA reports. 

Good Day ​@PolinaKr ,

Here is my response:
 

Without benchmarks, don't promise a number. Promise a process, and let the data earn the number.

1. Work backwards from a latency budget.
Start with what users tolerate (e.g., 3s for a page load) and split it across layers: network, API, business logic, DB, third-party calls. This gives you a derived target per component before you've run a single test.

2. Do the dependency math.
Your availability can't exceed that of your dependencies. If you rely on a payment gateway (99.9%) and an auth provider (99.9%), your ceiling is about 99.8% before your own failures. Don't sign an SLA the architecture can't support.

3. Tier by journey criticality.
Checkout and login get strict targets. Reports and admin screens get relaxed ones. A single blanket SLA is the fastest way to over-promise.

4. Stage the commitment: Observe > Target > Commit.
- Observe (weeks 0-4): Run baseline load tests (e.g., NeoLoad) plus synthetic monitoring in production. Track p50/p95/p99, not averages, because averages hide the pain.
- Target (weeks 4-8): Publish internal SLOs only, with margin (if p95 measures 1.4s, set the SLO at 2s).
- Commit (after 1-2 release cycles): Convert the SLOs that held steady into customer-facing SLAs, keeping a 10-20% buffer.

5. Use error budgets to keep it honest.
A 99.9% SLO allows roughly 43 minutes of downtime per month. Agree upfront on what happens when the budget burns (e.g., feature freeze, reliability work). This makes the SLA a living agreement, not a document that gets filed away.

6. Document your assumptions.
Record the load profile, data volume, and environment behind each number, and set a review date. When assumptions change, the SLA gets revisited.

Bottom line: with no benchmarks, create the evidence first (load tests, synthetic checks, dependency limits), commit only to what you've observed, and leave room to tighten. Under-promise, measure, then improve.

Thanks,

Ramanan

As Bryan Cole explains in the webinar, SLAs are often a business-level definition of performance. As users, we expect certain actions to be completed within a specific amount of time — fast enough not to negatively impact the business. Therefore, the first and most important step is to understand end-user expectations and translate them into measurable performance requirements.

I would start by adding non-functional requirements (NFRs) to the application development backlog, defining the expected response times for key user actions based on business needs.

Another important starting point is telemetry. I would analyze the performance characteristics of previous product versions using production environment data. These observations can provide an initial baseline and help us understand how performance evolves over time.

But what if there is no historical data at all? In that case, I would run performance tests as early as possible to establish the first baseline. I would then use the results to refine the performance requirements in collaboration with business stakeholders, ensuring that the targets reflect actual business needs rather than arbitrary numbers.

Tools such as NeoLoad can help us design and execute performance tests, collect relevant metrics, and analyze the results.

My key takeaway is that the absence of benchmarks should not prevent us from defining SLAs. We should start with business expectations, establish a baseline through measurement, and continuously refine our targets as we gain more data and a better understanding of the application's performance.

Thank you very much, everyone, for participating in the challenge!
And we have a winner! 🎊🎉
​@dirishere , congratulations!

I'll contact you shortly to arrange delivery of your prize. 😊