Test Methodology

Rapid Testing Framework

This diagram is a roadmap of the major issues and elements of the Rapid Software Testing methodology. I use it as for managing and navigating the testing story. In Rapid Software Testing, the testing story is a dominant heuristic for keeping things on track. When I speak of the testing story, I'm talking about a narrative structure that represents the logic of the test process. I'm speaking literally of a story. In prospect, it's the general form of our plan. In retrospect, it maps to the true story of what happened-- if what happened was good testing.

Diagram created by James Bach and Michael Bolton

Test Estimation Landscape

This diagram lays out all the major issues and elements of a good test estimation done in the Rapid Software Testing style. When I am solving an estimation problem, I walk myself through it. There are some elements here that are far from self-explanatory. In the upcoming guide to Rapid Software Testing™, all will be explained.


Diagram created by James Bach, Michael Bolton and thanks to Mary Alton for visual design consulting.



Military Scouting as Exploratory Testing

This diagram comes from a book about World War I. It shows the progress of a scout starting in the lower right corner and progress into enemy territory. Notice that the scout makes ad hoc decisions about where to go and what to do. Notice how the scout must be ready to pick up on any little sign and draw inferences from them as he goes. This is software testing, friends. This is what good testers do.


Diagram from Freedom's Triumph: The Why, When and Where of the European Conflict Published by The Magazine Circulation Company, Inc. 1919



This is an unordered repository of a few of the test methodology documents that exemplify our approach to testing.

Heuristics of Testability

This is a list of ideas for what makes a product more testable. It can help testers and developers improve the testability of a product so that testing goes faster and takes less effort.

Session-Based Test Management

Exploratory testing (sometimes referred to as "ad hoc" testing) is a creative, intuitive process. Everything testers do is optimized to find bugs fast, so plans often change as testers learn more about the product and its weaknesses. Session-based test management is one method to organize and direct exploratory testing. It allows us to provide meaningful reports to management while preserving the creativity that makes exploratory testing work. This page includes an explanation of the method as well as sample session reports, and a tool we developed that produces metrics from those reports.

SQA for New Technology Projects

When you're doing a 1.0 product, you can't rely much on experience to guide your SQA process. You also don't have regression test suites or any other specialized test materials that you can reuse. Meanwhile, the product itself is probably changing at a high rate. It's poorly documented and you may not be the first to know about major changes.

This document is a set of ideas for dealing with that situation. It begins with the idea that you have to change your thinking from a task orientation to a risk orientation.

General Functionality and Stability Test Procedure

(for Microsoft Windows 2000 Application Certification)

I produced this procedure for Microsoft to help them do a better job of assuring that applications that claim to be Windows 2000 compatible really are compatible. The procedure itself is documented in 6 pages. As far as I know it is the first published exploratory testing procedure. It's used along with a second non-exploratory procedure (which is 400 pages long!) to perform the certification test. What's interesting about that is the fact that my 6 pages represent about one third of the total test effort.

Satisfice Test Context Model

This is version 1.0 of a model I use to help me analyze software test projects. It depicts the major elements in the context of testing that should influence choices about test strategy, test logistics, and testing products.

Satisfice Heuristic Test Strategy Model

This model is a comprehensive set of lists that help a tester think through test strategy. I use this model to organize my thoughts about all the elements of test design. By referring to this model, I am able to rapidly generate lots of ideas for how to test anything. This is a classic example of "guideword heuristics.

Test Plan Evaluation Model

This is a model I use when I'm reviewing and critiquing a test plan. It lays out what are, in my opinion, all the interesting issues to consider concerning a test plan and associated documents. One of the interesting features of this model is a set of test project heuristics.

Test Plan Building Process

This is an experimental process for evolving a good test plan. I'm still experimenting with it. It's an example of a "forward-backward" process, where you proceed concurrently on all tasks, rather than linearly through each task in a predefined order. It's also yet another example of a heuristic approach to testing. This procedure doesn't tell you what to do, so much as suggest what to think about.

A Context-Driven Approach to Automation in Testing

First published: James Bach's Blog, 02/16

This article, by James Bach and Michael Bolton, is a comprehensive look at the role of automation in the testing process. We claim that testing cannot be automated, and that it is toxic to the craft of testing to claim that it can be. We explain a productive way to think about testing that is assisted by tools.

Investigating Bugs: A Testing Skills Study

This is an analysis of what one particular team actually did to investigate bugs. Their stated process did not match their actual process, and that's a well known phenomenon in social science. This is why just writing down what people say they do and calling that a "process" is a bad idea that has led to an amazing amount of waste. Instead, we must learn to use the methods of participant-observer studies, as I demonstrate in this article.




Upcoming Events



May 01-03 CLASS FULL
Broadcasting from Orcas Island, Washington, USA
ONLINE CLASS: Rapid Software Testing Applied with James Bach

June 12-14 CLASS FULL
Broadcasting from Orcas Island, Washington, USA
ONLINE CLASS: Rapid Software Testing Applied with James Bach

June 23
Bangalore, India
Tester MeetUP: Invited Speaker, organized by The Test Tribe

June 25
Bangalore, India
1 day workshop: "Testing Day with James Bach," organized by The Test Tribe

June 26-28
Bangalore, India
Corporate Training: Rapid Software Testing

July 10-12
Eindhoven, Netherlands
Public Class: Rapid Software Testing, organized by Improve Quality Services

July 24-26
Broadcasting from Orcas Island, Washington, USA
ONLINE CLASS: Rapid Software Testing Applied with James Bach

August 14-16
Houston, Texas
Corporate Training: Rapid Software Testing

September 04-06
Sydney, Australia
Public Class: Rapid Software Testing, organized by test-ed.com.au

September 09-11
Brisbane, Australia
Public Class: Rapid Software Testing, organized by test-ed.com.au

September 24
Bloomington, Minnesota
Tester MeetUp: Invited Speaker, organized by The Nerdery

September 25-27
Bloomington, Minnesota
Corporate Class: Rapid Software Testing

(click here to see the whole schedule...)