Managing Acceptance Criteria Plans

Some of the hardest and most crucial instances in any project execution are the user acceptance test phases. Usually, the acceptance criteria are not defined at the beginning of the project—or even if they are, they are eventually dismissed. Many project teams assume they have the same point of view the end-users will have when running test cases, and this is a big misconception.

Even if the specifics will change, the acceptance criteria should be outlined at the start of the project to set up the exit criteria from the beginning. And just before starting the user acceptance process, the team should agree on which test cases must be executed and what the expected results are. This will help in aligning your criteria with the end-user’s.

The key is to have the same bunch of test cases as the end-user to be executed. If the end-user is managing his own test cases list, there will arise a lot of unpleasant issues that may lead to going over budget, having to redo work, and in the worst case scenario, a conflicting customer relationship.

Coming up with a test plan will help you avoid those issues. Be sure to answer some important questions:

• When will the test cases be executed? Set up milestones.
• What is the bunch of functionalities to be verified? Define a clear scope.
• How will functionalities be tested?
• What are the expected results?
• What are the criteria that will mean the delivered scope is approved and accepted?

The first step of designing the test plan is to agree with the end-user about what is a defect and which kind of defects and impairments you are going to handle.

The table below outlines some of the most typical and useful descriptions of defects to help you in performing this classification.

Note that this classification system is based on severity levels relating to the likely business impact. There arises two different points of view to classify any defect, and project managers have to keep both in mind: business continuity and project health.

Defects may largely impact business continuity, but the project manager shouldn’t forget that defects may also severely impact project goals. If defects and impairments must be fixed, it means the project team will spend more effort and time on the project. It is important to gauge these possibilities from the beginning of the project and lay out the right plan to support issues like this with the minimum deviation from initial goals.

Once you have performed the defects classification, you must set up the clearest acceptance criteria that meet your customer’s quality expectations as well as your project goals. In project management, you have to keep in mind that any decision made and each statement you write must be aligned with the project and company goals. This is typically the hardest task project managers deal with to ensure a project’s success.

I recommend writing your own criteria based on each project, but the following is a good example of basic acceptance and exit criteria.

For each functionality to be executed:

• There are no open Severity 1 defects for any functionality.
• There are no open Severity 2 defects for any functionality.
• There is a maximum of two Severity 3 defects and three Severity 4 defects for a high-complexity functionality.
• There is a maximum of one Severity 3 defect and two Severity 4 defects for a medium-complexity functionality.
• There is a maximum of three Severity 4 defects for a medium-complexity functionality.
• At least 90 percent of the planned and executed test cases are in Pass status.

Once you have defined the list of severity defects and stated the most suitable acceptance criteria, you identify the complexity of each functionality and then match them with the acceptance criteria you agreed on with the end-user.

The project team and the end-user must have the same point of view in terms of what kind of defects can be handled, severity levels, and impact. This gives the team a plan when performing user acceptance tests to avoid discrepancies and unnecessary functional or technical discussions. Having a thorough and clearly documented process for evaluating acceptance and exit criteria that you previously agreed on with the end-user will help you handle expectations and plan on results.