What do I do if there are no benchmarks whatsoever? How do I plan my SLAs in this case?

When faced with a complete lack of historical benchmarks, SLA planning must pivot from historical system data to business requirements and iterative discovery. Here is a practical framework to approach this:

Drive from Business and User Experience: Start by asking "What is acceptable to the end-user?" instead of "What can the system handle?". Define thresholds based on user tolerance (e.g., standard 2-3 second UI load limits) and critical business flows.

Establish SLIs and Internal SLOs First: Identify key Service Level Indicators (Latency, Error Rate, Throughput). Before committing to external SLAs, establish internal Service Level Objectives (SLOs) as a safety buffer.

Create a Day-Zero Baseline: Conduct exploratory load and stress tests in a production-like environment. This provides an initial snapshot of the current architecture's capabilities and breaking points.

Adopt Industry Standards: Utilize established industry benchmarks for similar systems (e.g., API response times under 200ms) as a temporary starting point.

Iterative Maturation (Start Loose, Tighten Later): Set conservative, wider SLAs initially. Monitor production telemetry for the first 30-60 days, collect real-world data, and progressively tighten the SLAs as you establish reliable internal benchmarks.

If there are no existing benchmarks, I wouldn’t simply pick an SLA out of thin air.

I’d start by defining what “acceptable” means from the business and user perspective:

• Identify the critical user journeys and business transactions.
• Talk to product/business stakeholders about what response time they consider acceptable.
• Look at production/user data if available — current response times, traffic patterns, error rates, etc.
• If production data isn’t available, establish a **baseline** by running performance tests against the current system.
• Use that baseline to set initial SLOs, while clearly treating them as **provisional targets**, not absolute truths.
• Then validate those targets through load testing and, more importantly, real-world feedback after release.

For example, if there is no benchmark for a checkout API, I might initially propose something like “95% of requests should complete within 2 seconds under expected load.” But I’d document *why* that number was chosen and review it with the business rather than presenting 2 seconds as a universal standard.

The key is: an SLA should be derived from business/user expectations and evidence, not from a number that sounds good.

And once we have production data, the SLA/SLO should evolve with the system rather than remaining a one-time performance target.
 

For planning without benchmarks, we can:
a) Check if there is any telemetry data such as datadog. Any previous runs. Telemetry metric correlated with load.
b) Ask the business end-users directly, check if there is any historical data or report.
c) Integrate NeoLoad and qTest and refer to qTest report.
d) Use AI agents to run automated sampling tests in production that measures the SUT at various scenarios. Example, run Tricentis AIDA and review the AIDA reports.