The Testing Mindset Myth

  • 19 January 2021
  • 3 replies
  • 305 views
The Testing Mindset Myth
Userlevel 2
Badge +1

I spent a significant amount of my career in the role of a dedicated testing specialist. I worked at Microsoft leading a variety of cross-company initiatives when we had 10,000 testers at the company. We worked closely with development teams and, at the time, made a difference in product quality. For most of that time, developers generally did a small bit of testing, but our test teams performed the vast majority of all testing efforts, including extensive suites of verification tests enhanced with a flurry of diagnostic tools and a healthy dose of exploratory testing and investigation.

Somewhere in my tester journey, Microsoft moved from a mix of testers with and without programming experience to a model where all testers knew how to program, and I was a fly on the wall when we came up with the title for the infamous SDET role. There were many good (and some not so good) reasons for this move that are outside the focus of this post. One thing I remember from some folks opposed to the move was a notion that testers who knew how to program could not do effective testing because they were “skewed” by their understanding of how the code works. The feeling of these people was that the testing mindset is separate from the developer mindset 🧠 – that if you had one, you couldn’t have the other.

My experiences showed a different story. While I certainly ran across testers who tried too hard to solve every testing challenge with code, most readily and easily took on the role of tester and did deep exploration and investigation in order to understand and evaluate risk. Often, as expected, they would write programs or utilities to accelerate their testing. Their programming skills didn’t hurt their testing – their programming skills enhanced their testing. The idea of a skewed mindset was a myth.

For the last several years, I’ve been working on teams without dedicated testers. While these teams often have a few folks (including myself) who have some responsibility for helping the development team do better testing, the developers are responsible for fully testing their code – not just for functional correctness, but for all aspects of quality.

A few people have stated to me recently that developers are unable to perform testing because “they lack the mindset for skilled testing”. This statement – like the statement I heard many years ago about skewed mindsets is flawed and incorrect.

🧠👉 Watch this live webinar recording: Mindsets, Skill Sets, and Developers that can Test 👈👨‍💻

Mindsets and Skill Sets

Some voices in software development believe that building a product demands a certain mindset, while testing it demands a different mindset.

The book, Mindset by Carol S. Dweck says that mindset shapes how you learn and relate to others – and states that you have one of two mindsets – a fixed mindset, where you believe your abilities are unchangeable, and a growth mindset, where you believe the abilities you’re born with are a starting point for learning and advancing.

Peter Drucker coined the term, knowledge work to differentiate work that requires problem-solving and critical thinking from rote factory work. Authors such as Daniel Pink (A Whole New Mind) have elaborated on the impact of creative thinking in knowledge workers.

Software development and software testing are knowledge work. Both require multiple skillsets, constant learning, and problem-solving. While developing and testing software may require different problem-solving approaches or skill sets, they are both part of the same (growth) mindset, and the best people I know in software can switch between their development/builder and tester/investigator skill sets rapidly and fluidly. While I agree that it’s not easy – with deliberate and frequent practice shifting between these skill sets, most knowledge workers I’ve worked with can – and have become quite successful in developing this flow.

The Creators Skill Set 

I studied music composition as both an undergraduate and graduate student 🎶 Like building applications and writing, composition requires creation, bound by rules – which are sometimes intentionally broken in order to achieve the creator’s intent – as well as deep analysis and evaluation. As a composer, I constantly had to switch from putting notes on the page to looking at the passage or entire piece as a whole to make sure it achieved my intent. This analysis ranged from verifying notes and voicing were correct to examining every nuance of dynamics and orchestration. Like all metaphors, this one breaks down at some point, but composition skills require both creative (builder) and analysis skill sets (I took multiple courses in each), and the act of composition includes ample amounts of both.

For most of the last decade, I’ve spent a significant amount of my time helping developers expand their testing knowledge. I’ve watched hundreds of developers grow their knowledge about testing, and then almost immediately apply what they have learned. To be fair, I’ve also seen them make many of the same mistakes that new testers also make, but they’re eager to learn and constantly adjust. Many of these developers are at least as good at testing as the best testers I know. With no prompting or mindset shifts, most find a way to fluidly shift from the microscope view to the 10,000-foot view and back again. 

What I’ve found in nearly every case is that initially, developers lack the skill sets that experienced testers have mastered. I’ve also seen that developers are completely capable of learning a wide variety of deep testing and investigation skills. I’ve also found that teams where developers have testing skills deliver more quickly, more predictably, and with higher quality.

Making the Shift

I’m not advocating for any software team to eliminate the testing role (although I believe testing expertise is much more suited embedded into the team than in a dedicated test team). But I am firmly against the idea that developers shouldn’t do testing or can’t be good testers. I’ve proved that myth wrong dozens if not hundreds of times and colleagues in the industry have shared at least as many examples. Developers are capable of testing, can do testing and we should expect them to do testing. The expert testers of today would be well served by helping them learn and improve these skills.

We can start by recognizing that we’re all knowledge workers and that a big part of our job satisfaction and growth comes from learning and taking on harder and more challenging tasks. We should also recognize that software development requires a variety of unique skills, but not a change in mindset. We don’t all have to master every skill, but learning, growth, and skill development is something we all can do.

We just have to put our minds to it 😉

I challenge you to whip through this recording of a webinar I hosted last week and form your own opinions 🧠

 


3 replies

Userlevel 4
Badge +4

The testing mindset is NOT a myth. It’s an important fact of every serious tester’s life. I can’t explain Alan’s opposition to the idea that testers and developers think differently, except that perhaps he is engaging in wishful thinking. Perhaps he is a coder and he just really reaaaaallly wants testing to be a nice little task that sits fully within the proclivities of every serious developer. 

But it manifestly does not. I experience this on every project. Even recently on a project for Tricentis. The developers focus on getting things done, not on being cautious and critical. When I have been a developer (and I did recently prototype a tool for Tricentis) I have wished I had a tester-- because I simply didn’t have time to test much before I had to show off a big demo. See https://www.tricentis.com/blog/why-i-need-a-tester. The same dynamic exists for production code and productions coders, only with higher stakes.

The frustrating thing about Alan’s essay here, is that he uses the word “myth” in the title, and then denies that it is a myth in his text. Note this key passage:  “While I agree that it’s not easy – with delibnerate and frequent practice shifting between these skill sets, most knowledge workers I’ve worked with can – and have become quite successful in developing this flow.”

So, mindsets are a myth except not everyone escapes the builder’s mindset? It’s hard to escape the builder’s mindset? And the only people who can escape the builder’s mindset are those who diligently and deliberately practice? It sounds the midnset division is a myth in the same way that obesity is a myth (disclosure: obesity is not a myth). 

I disagree with Alan that the builder’s mindset is escapable in any practical sense. I have not seen Alan talk about the cognitive aspects of testing. Maybe he knows a lot about cognitive psychology and biases. Maybe he understands sociological aspects of identity construction and the complicated dynamics of testing communities interacting with developer communities. If so, I’d love to hear him talk about those things instead of seemingly dismiss them.

He mentions the concerns expressed by testers at Microsoft. I had a lot of tester friends at Microsoft at one point. I taught and spoke at Microsoft numerous times. Most of them left when it turned against the culture of skilled testing. Has Microsoft’s reputation for quality had an upswing at all over the years? Not that I can tell. Alan says he saw developers do deep testing. Really? I wonder what he thinks deep testing is? What Alan writes and speaks about is very code oriented, it seems to me. I’m a coder, too. But when I talk about testing I put that aside, because coding is not testing. Automation frameworks are not testing. Test cases are not testing.

He writes that he is not against the tester role “but I am firmly against the idea that developers shouldn’t do testing or can’t be good testers.” This is a straw man argument, because I don’t know that anyone since 1990 has seriously suggested that developers should not do testing at all, or couldn’t in principle be good testers if they were not burdened by the need to be a good developer (and by the identity of “builder” as opposed to “critic”). For all the sound and fury, perhaps Alan does not differ from my view of this subject after all. Except for one big thing: the division between the testing mindset and the builder’s mindset is a real thing that he agrees does exist (he just thinks it can be transcended if you try, like, super hard), so can we please explore the implications of that? Instead of pooh-poohing it, let’s understand what is happening in the mind of someone thinking critically instead of hopefully about technology. Let’s do that well.

The effect of articles like these is to give aid and comfort to people who want to extinguish the culture of educated and disciplined software testers.

 

Userlevel 5
Badge +3

I’ve been a developer for over a decade and then a tester for over a decade. Outside of work, I’ve been a keen home cook also. The above 2 posts seem to give strong opinions & perspectives on both good + bad ends of the mindset experience… but I stand firmly in the middle here.

 

  • Can or should all testers know programming to the level of design patterns and architecture concepts? Should they keep continuously up to date on rampant development languages & tools?
  • Can or should all developers be certified in ISTQB, CSTP etc.? Should they know security testing, performance engineering, usability, user experience and accessibility testing?
  • Should no-one with developer + tester mindsets be able to make fresh pasta?

 

I believe it’s admirable for developers to learn testing concepts & develop a tester mindset AND believe it equally admirable for testers to learn a developer mindset. To which extent and to what extreme, likely depends on lots of other factors: the individual’s goals, the team size & role make-up & company culture… *if* a developer has the relevant training and experience as to be as capable as a professional tester, then this *can* work & I’ve first hand experience of this (the same visa-versa). Whether this is an ideal setup is separate question...

Userlevel 2
Badge +1

 

I believe it’s admirable for developers to learn testing concepts & develop a tester mindset AND believe it equally admirable for testers to learn a developer mindset. ...

 

This actually sums up nicely one of the main reasons this community has the word “shift” in it’s name. The idea of testing mindset shifting left and right in the development to production process. End of the day, no real right or wrong answer and love that people want to learn more and explore.

Reply