Where Do You Start? Finding Real Test Coverage in SAP ABAP Changes
The Coverage Illusion
A global oil and gas company was preparing a major SAP upgrade. The release contained 27,000 changes, and the team had 9,000 tests ready to go. By any traditional measure, that looked like solid coverage.
Then they ran an AI-powered impact analysis. It showed that only 8% of the changed SAP objects were actually being verified by those tests. The other 92% of the change risk would have gone live untested.
The team caught it in time, focused on the highest-risk areas, closed the gaps, and released successfully. The lesson was learned, though. Thousands of tests can still miss almost everything that changed, because traditional testing isn’t connected to what each transport touches.
This isn’t an unusual case. Tricentis research shows that 51% of SAP teams say updating their systems puts business processes at risk, and 75% rely on hypercare after go-live to limit disruption. Hypercare means war rooms, extended monitoring, and extra people on standby. It is a symptom rather than a strategy. When testing doesn’t give teams confidence before a release, they end up paying for that confidence afterward with people.
Why SAP Is Especially Hard to Test
The volume of change hitting SAP landscapes has never been higher. Companies are moving to S/4HANA and RISE, AI is speeding up the pace of code and configuration changes, and SAP sits at the center of a growing web of connected systems like Salesforce, Workday, and Snowflake.
A single change rarely stays contained. It can ripple through finance, supply chain, procurement, manufacturing, and every third-party system connected to them.
Four characteristics make SAP harder than most environments:
- Constant change. Continuous S/4HANA and RISE updates run alongside ongoing ECC maintenance, and every support pack and transport can break something.
- Deep custom ABAP. Most of the real business risk lives in thousands of custom programs and enhancements, not in standard SAP code.
- Dense integrations. IDocs, RFCs, and middleware connections mean one change can cascade across the landscape, far beyond the object you touched.
- A full-stack surface. Front end, back end, configuration, and data migration are four separate disciplines, and testing has to cover all of them.
The result is a familiar triangle. Delivering fast means cutting test time. Managing risk means running more tests. Controlling cost means running fewer. Teams can’t optimize all three, so they make trade-offs, often without realizing it, and end up with slower releases, escaped defects, and rising costs.
Prediction vs. Proof
Most SAP teams handle a release in one of two ways:
- The first is to run full regression every cycle, regardless of what changed. It feels safe, but it’s slow and expensive, and it gets worse as the volume of change grows.
- The second is to use a change impact analysis tool that traces object dependencies and produces a list of tests to run. That’s faster, but the list is a prediction. Nobody verifies that those tests actually exercised the changed code. When a team says “we tested it,” what they often mean is “we ran something that was supposed to cover it.”
SeaLights takes a different approach. Instead of guessing from dependencies, it observes what each test really touches and matches that against what changed. The difference shows up clearly in the numbers. A release that would require 3,000 executables under full regression drops to around 2,000 with dependency-based analysis. With SeaLights analyzing the actual code, it comes down to about 400 executables that are genuinely at risk, a reduction of roughly 85% compared with testing everything. And for each of those, SeaLights points to the specific tests that cover them.
How SeaLights Works
SeaLights relies on two components.
The code scanner is an agent that runs inside your environment and connects to your SAP systems over RFC. It first maps your existing landscape to create a baseline, then continuously listens for new transports. Every time a transport arrives, it analyzes the delta and records exactly what changed, whether that’s code, configuration, a table, or an enhancement.
The test listener connects to your testing activity across manual, regression, and UAT testing. It doesn’t just note that “a test related to VA01 ran.” It records the specific objects each test touched while it was running.
SeaLights’ machine learning model combines both data streams into an object-to-test mapping, a precise record of which tests exercise which parts of your SAP system. That mapping makes four things possible:
- Seeing untested changes. You can see exactly which changed objects no test has touched, even when you believed your test suite already covered that area.
- Finding gaps between test types. If a change is covered by your regression pack but not by manual or automated tests, SeaLights shows it, so teams can focus on the gaps instead of re-covering everything.
- Measuring change coverage. You know whether a release is 100% covered or 80% covered, and you can direct effort to the remaining 20%.
- Recommending tests. Before anything runs, SeaLights can scan your test repositories and recommend which tests to execute, prioritizing the areas that are both changed and heavily used. After tests have run, it uses execution history to recommend what you can safely skip next time.
SeaLights integrates with the test repositories SAP teams already use, including qTest, Tosca, SAP Cloud ALM, SAP Solution Manager, Worksoft, OpenText, Azure DevOps, and Xray. Teams without any of these can import test cases from Excel. It supports SAP on-premise and private cloud landscapes, runs in the cloud, and deploys in days.
Part of a Larger Platform
SeaLights is part of the broader Tricentis platform, alongside Tosca for test automation, qTest for test management, NeoLoad for performance testing, and Tricentis Data Integrity for data testing. SeaLights is offered through SAP as SAP Change Impact Analysis by Tricentis, part of the SAP solution extension partnership. Tricentis says it is the only vendor with a full set of SAP-certified testing solutions covering automation, performance, data integrity, and change impact analysis.
The product draws on more than 20 years of enterprise testing work and over 1,800 enterprise customers. Tricentis built it after seeing the same problem repeatedly across that customer base. Teams were chasing coverage they didn’t need while missing the risks they couldn’t see.
Getting Started
Tricentis offers three ways to evaluate SeaLights:
- A discovery session. This is a 30-minute conversation tailored to your SAP testing landscape.
- A 20-minute product webinar. It shows how SeaLights shortens test cycles and gives teams real coverage data.
- A proof of value. Tricentis connects SeaLights to your actual environment, so within a couple of weeks you see your own coverage, gaps, and savings.
