This article was written by Jette Pedersen and published on the TestProject Blog
Well, the most important thing about quality assurance is to ensure that we do not put bugs into production… Or is it? 🤔
That might have been what I was thinking 9 years ago when I started doing QA full time. Our perception of quality assurance changes over the years, along with the development in the area. This is definitely what has happened to me. Today my focus involves many other things than not putting too many bugs into production. This is what I will tell you about in this article.
Before I started working as a QA full time, I was a developer for 11 years. I was use to testing my own code, and hated when bugs slipped into production. So, when 9 years ago started I as a QA engineer on a scrum team as the only tester, my main focus was to make sure no bugs hit production 🐞 This has definitely changed as the years have gone by. Today I am no longer part of only one team. I always support at least two teams, and it is rarely me that is doing the testing. Today I spent more time coaching on how to test than doing actual hands-on testing.
Where to start
When my colleagues and I start a new task, my focus starts in a completely different place than 9 years ago. But what is it that has changed my perspective?
At my workplace, we have changed our focus. We have now realized it is much more important to find out what the users want, before we built it. This is done through a lot of experimentation.
The first thing I think about is therefore to determine if all requirements are known to us. If you are not doing heavy documentation, this is where you need to clarify the expectations of the feature/product. When they are clear, then it is time to put the focus on the users. In our discovery phase we always ask ourselves these questions: Will they use it, will they buy it, will our stakeholders support it, can we build it.
Risk Assessment
The next thing on my mind would be to make a risk assessment. A good overview for this could be to make a Risk radar of the products/features, where each function is rated against impact and probability.
You are then able to see in which area of the risk radar is the feature you will be working with. Here’s an example of a Risk Radar:
The risk assessment gives you guidance on how much time you should spend on testing, how you should be testing, and where the automated tests should be written (unit test, integration test or E2E tests). It is important to have this overview so you do not spend more time testing than you have to, and still get satisfying quality. Here are a couple of examples:
- If the feature is of low risk (all the green functions) then you do not have to spend much time on thorough testing and testing can be done by the person developing the feature.
- If the feature is of medium risk (all yellow functions), you should spend more time discussing how to make the feature e.g. set up a kick-off with different colleagues. You should also talk more about how to test it. A rule of thumb is that if the risk is not low, then it would be best if another colleague than the developer who makes the feature will do the testing.
- If it is a high-risk feature (all red functions) then you should consider doing a technical kick-off. A QA should be consulted and it would be best practice to do a pair test session with a QA. Alternatively set up a test session where more colleagues participate.
- If the feature is left of the dotted line, then it is recommended that an end to end testing is implemented, if this is possible.
UI changes
The next thing that I would then think about is, what UI changes will there be to this feature. Would we for example need to do cross browser testing? Do we need new E2E tests?
Performance
Performance is also a topic that is very high on my radar. You need to ensure that what is built meets the performance standard for the company.
The company cannot afford to have a slow site. Users do not have much patience, and if the site does not load quickly, they will not use it, no matter how cool features we have built for them.
Security
Security has always been something that we as QAs should think about, but it has become even more important now to apply with the GDPR laws. This can be a tricky thing, but nevertheless, this is an area that today should have a high focus for everybody.
Limit the blast radius
Before code gets pushed into production, you also need to think about what can be done to mitigate bugs, in case there are flaws found in the code. This is what we call: “Limit the blast radius”. This can be done with feature toggles and with phased rollout.
By using feature toggles it is possible to turn off a piece of code in seconds. This secures that the users do not experience troubles for a longer period of time.
The same goes with phased rollout. Here you can turn on the code for a specific percentage of users, and then monitor if flaws are seen in the logs. If not then you can turn up the percentage.
Technical dept
Yet another thing to call out is focus on technical debt. If you develop new features in an area with technical debt, you should be more aware. It is more likely to make more mistakes, and spend more time coding and testing. The best thing to do is to reduce the technical dept while working in the area. And reflect it on the Risk radar afterwards.
Integrations to other systems
Integrations into other systems are also something to be aware of. When working in a specific area of the product, where there are integrations to other systems, you need to make sure to inform persons with interest. You also need to make sure to align on the test effort and the quality delivered in the given area.
Monitoring
The last thing you need to talk about before going live with new code is if you need specific monitoring of the new feature which have been built. This could be done in Grafana, Kibana or it could be specific trickers that are watched by our Site Operation colleagues.
Business areas where loss of revenue is a risk, should always be monitored with high alerts 🚨
Conclusion
As you can see there are many areas, besides testing, in delivering high quality software. The user perspective, requirements, risk assessment, “Limit the blast radius”, UI changes, performance, security, technical depth, integration and monitoring are also important aspects of the process.
I hope this will be of inspiration to you when you are doing quality assurance or when you are coaching your colleagues in how to ensure good quality 🏆