Lean Medical Device Startup: Test Driven Development

Lean manufacturing, now common in the medical device industry, originated in the automotive industry. Companies that truly embrace lean practices dramatically reduce costs and inventory levels while improving product product quality.

Stage-gate and requirements-driven product development, pervasive due to FDA’s QSR, has roots in PRTM’s PACE process and Robert Cooper’s Winning at New Products. A well-designed product development process shortens new product development timelines and improves the likelihood of product success.

Medical device companies reap tremendous benefits from borrowing the best practices of other industries. Unfortunately, most of us spend our whole careers in medical devices with limited knowledge of other industry practices. Meanwhile, as Marc Andreesen wrote last summer, “Software is eating the world.”

It’s not just the dramatic decline in the costs of memory, processing and communications that is fueling the software revolution. Key to software success is a radically new approach to product development best summarized in Eric Ries’ fantastic book, The Lean Startup and Kent Beck’s classic Extreme Programming Explained.

While I don’t expect scalpels to be replaced with software anytime soon, we in medical devices can learn a lot from Lean Startup software practices. Today’s topic: test-driven development (TDD).

For most medical device companies, product verification tests are developed after the product is designed, often during the mad rush to complete the verification and validation process before some management-imposed deadline. For most of the history of software development, the paradigm was the same: code-test-fix-verify. Extreme programming reversed that paradigm, insisting on developing the test first. That’s right. Before developing a single line of functional code, today’s programmer first spends the time and effort to develop and qualify the verification test.

TDD has been well described elsewhere, so I won’t repeat it here. It’s definitely worth the time to read a few general descriptions of test-driven development for software:
http://en.wikipedia.org/wiki/Test-driven_development
http://jamesshore.com/Agile-Book/test_driven_development.html
http://www.agiledata.org/essays/tdd.html
http://digitalcommons.calpoly.edu/cgi/viewcontent.cgi?article=1034&context=csse_fac

The benefits of TDD in software are many:

• Sharpens the specification to something testable, and forces the engineer to really understand the desired product performance.
• Can be used over and over on different design iterations, providing data on the comparative performance of different designs.
• Speeds debugging. Bugs are identified and fixed early in the process, instead of being hunted down during a later system integration.
• Reduces wasted development effort, as only enough product is developed to pass tests.
• Speeds overall development time, by reducing errors and reducing over-coding. Practitioners of lean manufacturing will recognize this as a classic lean strategy.
• Ultimately serves as the verification test, and a regression test.

In software, TDD is used selectively. In particular, TDD is applied to unit-level software development, where full functional testing is not needed. Functional testing and system integration testing are critical complementary test techniques.

TDD cannot be applied blindly to medical device development. Software tests can be run in minutes (or less), so they can be repeated hundreds of times as the software product is developed. Software tests are automated, and new tests are simply added to a software company’s automated test queue, which often runs every day or multiple times per day. On the other hand, medical device tests take days or weeks, and usually require large sample sizes for statistical significance. Repeating them many times is simply not practical.

That said, I do think that a few key principles can be applied to medical device development, and test-development-first often makes sense.

First, TDD is as much a problem analysis and specification analysis technique, as it is a test technique. Making the test forces critical thinking about the design problem.

Second, test-development-first changes the order of work but does not add work. In medical devices, we have to develop the verification tests eventually. It’s just the order that’s unusual.

Third, test development is a capital investment that speeds overall development time and reduces waste. It is analogous to the lean manufacturing technique of setup time reduction, where an investment in reducing set-up time enables smaller batch sizes (or single-piece flow) and improved production cycle times.

Fourth, TDD can be applied at the component or subsystem level just before developing each component or subsystem. Instead of jumping in to catheter design, you might first develop your kink-test, tracking/pushability test and tensile test. While you can’t run a large sample size of every design iteration through each test, you can surely use the tests as screening tools on smaller sample sizes.

Why not try TDD for medical devices?  Software companies had to overcome many obstacles to implement TDD, and medical device companies will face the same issues:

• Developers are naturally more interested in designing products than tests, so tests get de-prioritized, delayed, or handed-off to test engineers. TDD improves the overall development process, but the benefits to individual engineers are harder to see.
• Ingrained culture is hard to change. TDD requires a discipline that is challenging to learn and maintain. TDD is new, and not yet taught in engineering school. With lean manufacturing, small batch sizes and strategic bottlenecks originally seemed counterintuitive, but now are widely understood and accepted. TDD presents the same challenge to established practices.
• TDD must be applied creatively and selectively. Engineers and managers must identify the appropriate uses of the technique.

I think these obstacles can and must be overcome. It’s our job as managers to find ways to improve our product development practices. The software industry is leading the way with its radical rethinking of many product development techniques. The medical device industry has a lot to learn.  With appropriate adoption of TDD, medical device companies may attain the benefits the software industry already enjoys – sharper specifications, shorter development cycles, reduced product development waste, a faster V&V process, and ultimately better products.

Related articles

• TDD is Kanban for Code (threeriversinstitute.org)