My colleague Chris recently noted: “the right way to do things is often a pain in the butt.” No question that most engineers see protocols as a pain in the butt – yet another file to sherpa through the document approval process.
There’s an important logic behind the practice of doing protocols. Imagine doing an experiment on humans (aka a clinical study) without one. But “good product development practice” isn’t the only reason star medical device engineers write protocols. Believe it or not, star medical device engineers view protocol writing as a key element of team leadership and team effectiveness.
Let me explain.
Stakes are high in clinical studies – mistakes can be disastrous. Experimenting on people is serious business, and trials are time-consuming and expensive. Clinical protocols help ensure that the benefits and risks of the experiment are balanced, that patients are protected, and that good science is performed. Sound protocols ensure that the trial results are replicable. Protocol review identifies and eliminates methodological errors (e.g. not controlling for a potentially confounding variable) and identifies and reduces mistakes in execution (e.g. forgetting to capture critical data at the appropriate follow-up intervals). Even retrospective clinical data analysis is typically performed under a protocol, for all the same reasons.
While the human stakes of bench and pre-clinical experiments are not as high as in clinical trials, mistakes can still be disastrous. You don’t want to mess up design and process verification studies that determine whether new devices are safe and effective to be used on humans. Likewise, wrong results in spec-defining or concept selection studies can send engineering efforts in the wrong direction for months or years. If you want replicable, methodologically sound, well-executed bench and pre-clinical experiments, you’re better off writing a protocol. It’s that simple.
Star medical device engineers get all that, but they see protocols as something more. Star engineers view protocol writing as a critical practice to improve product development team performance. How?
- Leverage team expertise. Star medical device engineers seek better, faster ways of doing things, and want to incorporate the perspectives of both engineering and non-engineering teammates. At Fractyl, I’m currently designing a experiment to quantify our device-tissue effect . I’ve gotten a bunch of great ideas from my team on how to capture a lot of data with a small number of samples. Another colleague is helping me with the protocol statistics. I can’t do it all myself.
- Gain team buy-in. Common understanding of an experimental design fosters later understanding of the results. It’s easy to interpret outcomes when you understand the experiment. At infraReDx, our algorithm development results were all the more compelling because the team already understood our ex vivo experimental protocol. On the other hand, we all know clinical studies that started at risk, before FDA buy-in, where positive results ultimately did not support regulatory approval. Failure to obtain up-front buy-in can have significant repercussions.
- Seek reciprocity. When you ask a colleague to review your protocol, she’s likely to ask you to review hers. Star engineers want their colleagues to run great experiments – the whole team benefits. They wouldn’t dream of asking to review colleagues’ protocols without offering theirs for review in return.
- Delegate effectively. A clearly drawn experiment, leveraging well-defined test methods, can be delegated to other team members with confidence. Star engineers delegate effectively, to get more done.
It takes more than a group of great individuals to develop a great medical device. It takes a great team. Star medical device engineers adopt work practices that turn groups of brilliant individuals into brilliantly effective teams. More than just a good product development practice, protocol writing is a key play in a star engineer’s team playbook.
Jay Caplan on Medical Devices 