Hi, I thought I would share my container for comparing NeoLoad results against a baseline. 😀
In addition to using SLAs to verify that a test has performed well, one can also use NeoloadCompare, which utilizes NeoLoad Web's API to compare the test result against a baseline.
You can choose to compare the average response time for all transactions in the test or percentile.
The program only looks at transactions in actions, so you need to use this in your userpath. (Transactions in the init part will be ignored)
The program runs after a performance test has been executed in the pipeline.
The following parameters need to be sent:
- Workspace (Name of workspace)
- BaselineTest (Name of the baseline test without #)
- Scenario (name of the scenario being run)
- Percentage (After how many percent difference from the baseline the test should fail)
- Element avgDuration/percentile90/percentile95/percentile99/no (if 'no' is specified, no comparison will be made)
The program will then, through calls to the NeoLoad Web API, retrieve the latest scenario run in the specified workspace and then compare the value of avgDuration (average response time)
against the same value in the specified baseline test for each transaction. If any of the test transactions have increased by more than the specified percentage compared to the baseline, the test will be marked as failed.