Usability‑Engineering Projects: What Are the Common Misunderstandings?

At the beginning of this month, Ilkka Juuso and Ilpo Pöyhönen published a book entitled Medical-Device Usability: Human Factors for Medical Device Development, which simplifies the process regarding the application of usability to medical devices for new developers by providing a thought-out walkthrough of what is involved and how the expectations from the essential standards affect their work and why.

When I spoke with Ilkka this past week, I asked him, "What are the most common misunderstandings developers encounter when starting a usability‑engineering project?" Here is his complete answer:

By far the easiest mistake to make is to think that usability in medical devices is just about an interface that is pleasant to look at and use. That will be a part of it, but far more important is the safety of that interface. An aesthetically appealing user interface is nice, an interface with all the controls in intuitive places is a great start, but what actually matters the most is that it is difficult to use the interface in the wrong way, leading to real harm to patients. That may sound counterintuitive from a general usability perspective – after all, usability typically talks about the ease of all use – but safety and the safe use of a device really is paramount when it comes to health. In an optimal situation, of course, all these rings align so that a safe interface is also intuitive and appealing, and thus hopefully even safer to use. In our book, we approach usability engineering from the viewpoint of safety, but also bring in lessons from the field of general usability engineering, which is useful to a discussion of medical devices.

The second mistake to avoid is keeping usability engineering as an isolated activity, run once and then forgotten about, without much impact on the development or maintenance of the medical device. In the book, we show how usability connects to the overall quality management and risk management activities of the organization, and how usability can actually inform the whole path from an initial product concept to a finished device and beyond. Our message here is that usability engineering is not just something you have to do, it’s something you should want to do because of how it can add to your knowledge of your user, use environment, and the use of your device – and bring benefits to all of these.

For those working in the medical device industry, do you agree with Ilkka's perspective regarding common mistakes? In your organization, does usability connect directly to quality and risk management?

Do Managers Truly Understand How to Measure Critical Success Factors (CSFs)?

Earlier this month, I spoke with James H. Dobbins about his latest book, Critical Success Factors: How to Effectively Identify, Measure, and Apply CSFs, which equips managers with a practical framework to accurately identify, measure, and apply their critical success factors (CSFs). Unlike previous efforts that relied on broad surveys and statistical analyses intended to generate generalized lists of CSFs, this approach honors the foundational definition of CSFs and provides actionable guidance tailored to each manager’s unique context.

During our conversation, I asked James, "What are the biggest mistakes managers make when trying to measure critical success factors (CSFs)?" Here is his complete answer:

The biggest mistake managers make when trying to measure critical success factors (CSFs) is failing to understand what CSFs are in the first place. Managers turn to the results of research by others, projects often done by academics in the pursuit of an advanced degree. The research objective is to produce a list of generalized CSFs.  Once the list is published, the research is completed, the degree is granted, and the researcher goes on with their life. No measures are suggested. Many researchers worldwide, all with the same objective, have published different sets of CSFs. There is no guidance on which list to use. The biggest problem is that all the research was conducted using surveys of large numbers of managers, and there has never been a valid list of CSFs produced from surveys. Using surveys to identify CSFs has several fundamental flaws.  Some, noted in the Introduction to the book, are:

• The assumption that project managers know how to identify their CSFs,
• The failure to recognize that CSFs are contextually relevant to the manager, and therefore, the elimination of anything not generalizable, is contrary to the definition of CSF.
• The failure to understand that Critical Success Factors are, in fact, critical for a specific manager, and that all must be done well.  CSFs are not something you pick from a menu, like a survey result, hoping you chose the right ones.
• The absence of any longitudinal studies. There was no follow-up with any of the managers surveyed to validate the study's results or to determine whether and how they utilized the CSFs identified in the survey.
• There was no attempt to identify measures for the identified CSFs to help managers track their success in meeting the CSFs.  

Does your company actively use and apply critical success factors (CSFs) in its operations? Does your organization rely on surveys to determine which CSFs to prioritize? And do you agree with the mistakes James H. Dobbins identifies in this area?