Is Software Testing Dead? Not Yet.

There is a story circulating around the world of software development that comes in many different flavors. Here’s a common version:

(A) Thanks to modern development methods, software quality is drastically improving.

(B) Thanks to the Facebook effect, customers are willing to live with software that is much lower quality.

(C) Thanks to DevOps, we notice problems in production very quickly and can “roll back” before most customers ever have a problem.

Therefore, we do not need software testers.

To some extent, I am willing to grant that statements A, B, and C are correct.

It’s the “therefore” part that throws me for a loop.

Another version of the story comes from Arlo Belshee. In his blog post Scaling Agile the Easy Way, Belshee states that scenario A requires “technical fluency”—skills such as unit testing, pair programming, refactoring, and continuous integration. Arlo goes on to say that programming teams who reach technical fluency can reach a point where they essentially make no mistakes—about one mistake per one-hundred developer days or one per team “fortnight” (two weeks). Most of these errors are caught by continuous integration within one hour.

If you only have one bug per couple of months split through CI, what do you need testers for?

The third version of this story, and perhaps the version that I agree most with, is that since the agile movement “blurred the lines” between dev and test, the next logical step is to eliminate the lines completely, with anyone doing any role.

So we have people I respect, like Elisabeth Hendrickson, who once ran under the label of “test obsessed,” who now uses phrases on her job description like “Although my title involves the words ‘Quality Engineering’, on a day-to-day basis my actual job looks more or less what any other engineer at Pivotal Labs does in that I pair with other team members on coding and testing, I just pay a little more attention to questions around test strategy ...”

I am not trying to be critical of Elisabeth. Not only is she responding to market conditions, she is also correctly observing that it is more powerful to be multi-skilled than single, and to focus on the value delivered—the software—over one portion of the process. I do not fault her.

While I might grieve for software testing, I have to admit; no eight year old ever woke up and said “I want to be a tester when I grow up!”

If the role went away, so what?

It means that everyone will be doing a little software testing, all the time, so there will be need for instructors and trainers. Perhaps fewer of us, perhaps more educated, but the teaching of testing will continue. Let the profession fade. If I work hard enough, I’ll probably be all right.

Then, one day in February, my eight-year-old daughter woke up in the morning and said, “I want to be a tester when I grow up.”

All of a sudden, I do want to protect the profession.

An Analysis
I can’t disagree with Arlo’s impressions of scenario A above—a high-functioning team can get to the point where they make very few mistakes. In that, he’s dead-on.

Though, it does seem prudent to exercise some caution around that word mistake. Consider:

•  The third-party software that has mistakes embedded in it that new code trips
• The generic language, like, say “JavaScript,” whose very definition is open to interpretation. The developer made no “mistake” on the latest version of Chrome, yet the code looks broken on previous version of Safari.
• The operating system vendor whose new version of the operating system changes the screen real estate and rendering of your web page
• New platforms, like tablet, and platform-specific interpretations, like what do with font sizes when the device moves from landscape mode to portrait mode
• The open source code library a developer just “upgraded” the entire project for
• The new, yet to be invented, device your biggest customer’s CEO’s nephew just got for Christmas, who expects it to work. Right. Now.

All of these are problems in the software—real problems I have seen on real projects in the past twelve months. (The last one was actually a tablet, but not much of an exaggeration. The client company was “not supporting tablet,” right up until they realized customers were using the software on tablets anyway to manage real financial transactions, which the client was unwilling to let go wrong.)

Arlo’s writings, as he has expressed them, don’t quite handle these cases. (For that matter, scenarios B and C (Facebook effect and DevOps) don’t consider cases where quality still matters, like financial transactions.) The companies I have interviewed or worked with, that have reached “technical fluency,” tend to deal with these problems by having specialists that specialize in ... testing.

So, no, Virginia, I don’t think that testing as a role will “go away.” Not anytime soon.

Other Ways Testers Contribute
The testing skillset is one that not everyone has. It is a kind of work others would prefer not to do, and when they have to do it, they do it poorly.

It is the kind of thing where I can knock on a company’s door and say, “Hey, I do that. I’m good at it. I like doing it. If I did it for you, you could go faster. Here are my rates.”

As long as people respond to that offer, there will be a market for testing.

Will it change? Blur? Become a focus and less of a role? Be called something entirely different? Do we need new models, new ways of framing and thinking about the work?

Perhaps. Perhaps we do. Yes.

Last month, the good folks at SQE made me an offer I couldn’t refuse: Come join the editorial staff at Stickyminds.com to help chronicle the adventure of re-inventing testing by finding and publishing stories written by those working for change. And do a little blogging.

How could I say no?

For the time being at least, I’m doubling down on test.

Party over? I think it’s just getting started.

If you want to re-invent testing, change it, have a crazy idea—or, better yet, an experience!—this is your chance to publish it on StickyMinds. Drop me an email at [email protected] for pitch guidelines.