This article was written by Michael Fritzius and published on the TestProject Blog
It’s no secret that software is getting complex faster than we can manually test it. Automation is now a necessity. What’s not as well known is that the way most companies automate tests can’t keep up either. It takes time to define, write, run and maintain each automated test, which creates a bottleneck, and it results in a decreasing percentage of coverage over time. What if instead of writing each test, we can detect bugs as quickly as we can send test data through a system? And what if we didn’t have a bunch of automated tests to create and maintain?
Let me introduce you to Model Based Testing.
What is Model Based Testing (MBT)?
MBT is where you create a mockup of the system being tested, along with a way to generate random, yet realistic, test data. The mockup’s job is to try to produce the same output as the actual system. Then the two outputs are compared.
Differences mean one of two things:
- The model needs to be updated.
- The actual system isn’t behaving as expected.
Let’s take a look at a simple example to further describe this.
Building the Data Generator
Having good test data is the first half of this method. But we also don’t want the effort to be spent creating a bunch of data sets. Instead, let’s make a data generator. The generator’s job will be to make data that the system would expect. It can’t be gibberish, otherwise we’ll be using a bunch invalid data, hence the “random yet realistic” description before.
Let’s say your system expects a person record in JSON format, with a name, address, city, state and ZIP code. The generator would then create valid JSON, but could choose to include (or not) each piece of info. The generator can be created with a switch statement, along with picking random values out of arrays of data (state, names, ZIP codes, etc.).
Building the Model
For the data generator example above, let’s say the system checks to see that a valid person record was provided, in the following ways:
- Accept names less than or equal to 50 characters.
- Accept addresses less than or equal to 35 characters.
- Accept valid combinations of city, state, ZIP.
- Throw appropriate error for any other case.
Even if the system doing these checks is actually a collection of small microservices, that’s fine. A model can be built fairly simply, and take the form of a single if/else statement.
Comparing Results
The final step is comparing the outputs of the model versus the system. If you gave a record with a city of St. Louis, and a ZIP code of 00000 (which is invalid), an accurate model would expect an error. If the actual system doesn’t, then you may have a bug.
It’s possible to also create a kind of heatmap to see which parts were different when they were compared. Then, as dozens or hundreds of comparisons start showing a common theme of the city and ZIP code being involved, the more apparent it’ll become that something weird is happening in that logic.
Advantages and Tips
The biggest advantage to this approach is that the more complex the system is, the better. The chances of both the model and the system being wrong in the exact same way are near impossible, so you have a much greater assurance that a zero-difference comparison means that that part of the system is behaving correctly.
Plus, because your model and generator can each live in a giant program, you can use something like Cucumber to generate code for you. This makes understanding the code—along with the maintenance—much easier, since you can read exactly what the code will be doing.
Is Model Based Testing Right for You?
It might just be. If you’re hitting that “maximum safe speed” with your automation, and feel like doing much more will cause you to cut corners, then it may be time to “shift into a lower gear” and adopt this so you can go faster later.
Check out the video I created on the subject:
What are your thoughts on Model Based Testing? Please share in the comment section below!
