Review By: Andrew L. Pollner
07/08/2010Software Test Automation is divided in two sections. Part 1 deals with best practices to successfully use automation. Part 2 recounts experiences from various practitioners on the success—to varying levels—of automation at various customer accounts.

Chapter 1 establishes a baseline of definitions and distinguishes testing from test automation. Problems and limitations of automation are addressed early to set the stage for later chapters’ successful implementation guidelines. Chapter 2 explains why “capture/replay is not test automation.” This should come as no surprise to current test automators, and should be attentively read by test managers who see automation as a “silver bullet.” Chapter 3 describes effective scripting techniques and the similarity these bear to traditional software application programming. Concepts include creating sharable scripts, data-driven and keyword-driven techniques.

Chapter 4 describes various comparisons which can be achieved, their appropriate use, and the benefits each brings. Chapter 5 describes testware architecture--the components required, with recommended grouping to facilitate managing all of the pieces automation uses and creates over time. Chapter 6 covers automating processes surrounding your core automation—automating the initial conditions required before an automated test case is run, and automating any cleanup activity immediately following a run.

Chapter 7 focuses on keeping the momentum in automation by building maintainable tests. Chapter 8 helps quantify the success of testing and automation through measurement and metrics. Chapter 9 describes how to select and prioritize tests to run. Chapter 10 provides guidelines on how to select an appropriate test tool given the environment to be tested. Included here are the build vs. buy decision, a tool evaluation questionnaire, and a tool user reference questionnaire. Chapter 11 focuses on implementation and suggests ways to incrementally introduce automation. Also addressed are the people’s roles, expectations, and management considerations.

Part 2 provides information from actual case studies in test automation which provide great insight into the many issues and obstacles which can—and sometimes do—derail automation initiatives. These accounts present a realistic and varied flavor for what others have gone through to succeed in automation.

Overall, Software Test Automation is a complete and comprehensive reference guide that is approachable to tester, developer, and QA manager alike.

A large part of the book does not talk specifically about actually using the test-automation tools themselves. Although this may lead one to think that the authors do not adequately cover the topic, the exact opposite is true. What many testers do not understand, and what the authors make clear, is that successful automation doesn’t come from purely focusing on the test tools themselves. Test automation is a subset of testing and as such requires the thorough planning and organization detailed in this comprehensive reference book.

The authors dispel myths as they describe test automation. In the first of two sections, techniques for test automation are outlined along with corresponding definitions. From establishing the context of automation to including what should and what should not be considered automation, the book continues by describing various scripting techniques, comparison guidelines, testware architecture, and what it takes to build maintainable tests. Part one concludes with metrics, tool selection, and tailoring automation to your organization.

The authors describe how the basic structured programming techniques such as variable naming conventions, looping constructs, and modularization are equally important to ensure test-automation-development success. Creating automated tests shares aspects of software development, yet as the end goals are different, so are the means to the end. Test automation tools are powerful development environments with many testing-specific features.

Discussing testing tools in detail is difficult given the various differences and capabilities that competing products exhibit. To avoid what may appear as a product endorsement, the authors have successfully described automation-specific ideas in a generalized, unbiased manner with pseudo-code examples. This provides a framework from which tool-specific implementations can be adopted.

The book’s second section contains various accounts regarding the implementation of test automation. While this section lends credibility to the potential—and pitfalls—of automation, it provides little in the way of applying the described techniques. The strength of this section lies in the realization of what automation entails for a variety of organizations wishing to automate testing.

The book is easy to read, informative, and complete. I would caution against reading it cover-to-cover as there may be too much information to process. The material is presented in a manner that allows one to selectively apply ideas presented by chapter with no penalty. This is consistent with the step-by-step approach each chapter provides allowing novices and experts alike the opportunity to easily digest and apply the information presented.

Software Test Automation is a thorough and useful reference book for anyone contemplating using automation and for those not receiving the return on investment (ROI) they had hoped for from test automation tools.