COVID-19 and HFE: Human Factors Development and Usability Testing during Physical Distancing
Updated: Jul 22, 2021
With the constantly changing COVID-19 situation around the world, this is an unprecedented time for everyone. A time that is filled with uncertainty for many, in our personal and professional lives. Businesses are facing challenges and hardships around the globe. Some have had to completely change course on development projects, staffing situations have changed, and resources and priorities have shifted, as shelter in place and physical distancing has become the new daily normal.
Is FDA relaxing requirements overall in response to COVID-19?
In response to the current public health emergency the U.S. Food and Drug Administration (FDA) has issued guidances, such as, FDA Guidance on Conduct of Clinical Trials of Medical Products during COVID-19 Public Health Emergency, providing general considerations to manufacturers to assure safety of trial participants while maintaining good clinical practice (GCP) compliance and trial integrity, as well as new policies around Emergency Use Authorizations (EUAs), to help expedite the availability of development and approval of diagnostic tests. Therefore, many within the medical device industry have started to wonder and question if and how these new and “relaxed” expectations of the FDA for some areas might potentially impact human factors (HF) engineering (HFE) requirements for medical device development as well, more specifically (summative) human factors (validation) usability testing.
While there are many key factors (and activities) within the human factors engineering (HFE) process, understanding the usability, safety, and effectiveness of your device, are most critical. Gathering key insights about device users, use environments, use tasks, and user interface design (touchpoints), enables the measurement of your device's capacity to ensure it has meet users’ needs and performs safe and effectively according to intended use. This is done, alongside other critical activities, e.g., use-related risk analysis, through the (iterative) use of usability testing, formative and summative. And while formative usability testing can take on many shapes and sizes, using various different methods, tools, and technologies, summative HF usability testing follows a rather standardized and traditional process and format. A process and format in which the most critical key aspect remains to be the live face-to-face observation of representative (intended) users performing critical (and essential) tasks in realistic use scenarios, enabling the capture and collection of comprehensive data sets with regard to safe and effective natural use, including use errors and problems, as well as associated root causes.
In a time of physical distancing though, this usability testing format becomes challenging. People have started searching for alternatives, as well as FDA guidance on how to conduct traditional HF validation usability testing during the pandemic, in order to 1) ensure overall device development progression, while 2) still meeting existing FDA HFE requirements and expectations. Questions arise on how “relaxed” FDA would view HF validation usability test data sets acquired from alternative testing methods, e.g., remote usability testing, while we are adhering to physical distancing during a global pandemic.
While the FDA’s viewpoint does not exclusively eliminate HF validation usability testing to be performed remotely, at this time, FDA has also not issued an official statement and/or guidance on amending and/or “relaxing” human factors requirements and expectations with respect to the HFE processes and its activities, or for that matter, its validation usability testing requirements and corresponding datasets either. So, what does this mean and what to do in a time of a pandemic, in which physical distancing is the new norm and traditional usability testing is no longer a straight forward option?
HFE and usability testing during physical distancing
When it comes to the wonderfully diverse world of usability testing, new technologies and advances in cross-functional and co-located teamwork have enabled endless possibilities in usability testing. The usability tool kit has vastly increased, which has allowed us not only to employ usability testing in a more flexible and spontaneous manner, but also enabled us to more quickly collect user input and product performance without lengthy planning, massive budgets, or big resources on hand. With video-conferencing tools like Skype and Zoom, eye-tracking technologies, interface touchpoint software, as well as observational studies via webcam, usability testing is more than capable to adjust and adapt in a time of a pandemic. These methods and technologies can easily be applied, even during a time of physical distancing, and can enable product teams to stay on track with their development progression and milestones with regard to collecting user input and usability performance. And while employing such methods and technologies during formative testing is commendable and even advantageous in some scenarios, the same methods and technologies could pose challenges and limitations for certain devices during summative validation testing, possibly even risk study integrity.
While some products are more easily tested in a remote or alternative test setting, some simply cannot avoid the face-to-face traditional usability test setting. In such cases it would be most advisable to consider postponement of the HF validation usability test activity, saving resources and budget, and instead use this time to take an in-depth look at all test materials, ensuring all to be in order and aligned with regulatory requirements and testing needs, while we wait for the need of physical distancing to decrease.
However, should you wish to go forward with your HF validation usability testing, and possibly even consider a remote and/or alternative approach, you should meet with FDA (Q-sub meeting) and seek alignment and agreement prior to proceeding with your testing activity. You should also consider adapting your study protocol to allow for implementation of public health and safety guidelines issued by the Centers for Disease Control and Prevention (CDC) with respect to COVID-19, to ensure study personnel and participants’ safety (and comfort) during testing. Amending study protocols to enable adherence to physical distancing and implementation of appropriate cleaning and disinfecting protocols, could all be part (but unknown at this point) of the discussion on how to conduct HF validation usability testing during a pandemic whilst ensuring the safety of everyone involved.
Ensuring FDA HFE compliance and overall device development progression
Alternatively, it may also be worth to take another look at your current development status and milestones, the ones already completed, as well as the ones that are still outstanding.
Are you truly ready for HF validation usability testing?
Have packaging, labeling, and instructional materials (if applicable) been reviewed, tested, and finalized? Are they also ready for validation testing?
Is there an opportunity to conduct additional remote usability testing on such materials to ensure final state of such?
Is the device to be tested in its final state? Are there potential improvements to be done on testing prototypes that could decrease the risk of device error during testing and improve likelihood of a successful testing outcome?
Have you mitigated all use-related risks and implemented all necessary design controls?
Is your HFE documentation in order?
Are you able to easily compile your usability engineering file, as well as well as your human factors engineering report once validation is completed?
These are all things to be considered and questions to be asked, not only to ensure FDA and IEC 62366 human factors and usability compliance, but also to avoid device development disruptions and additional costs due to potential HF validation development and usability testing repeats (due to unpreparedness).
No need to idle during this time – HFE is more than just usability testing
There are many things that can be done that are not directly impacted by the need for physical distancing, especially for manufacturers at the beginning of their development project. This time can be used by ensuring the development and implementation of robust HFE processes and documentation, including a regulatory path strategy. This encompasses a clear and comprehensive understanding of human factors regulatory requirements and standards, and how such translate into the device development process, its activities, and its documentation within the design history and usability file, and consequently the overall submission file. In addition, while in person, face-to-face activities might not be viable options right now, or only executable in limited fashion; activities such as gap, task, and use-related risk analyses are ideal tasks to complete in a time of physical distancing. Some other activities and documents you should secure during early development stages include (but are not limited to):
define and develop a robust regulatory pathway approach and corresponding human factors plan/strategy/approach (meet with FDA and applicable regulatory bodies and seek alignment of such approach prior to device development commencement);
complete gap analyses, meta analyses, video observational research studies, telephone and/or online surveys, and/or any other appropriate physical distancing compliant form of research method/tool and identify user needs/use(r) requirements and ensure incorporation of such in overall product requirements and corresponding documentation;
identify and define device user profiles, use environments, tasks, and training (when applicable) and develop a detailed use specification;
establish a comprehensive risk management approach, incorporating use-related risk analyses, within such ensure the identification of known and foreseeable hazards and hazardous use scenarios and corresponding critical and essential tasks; and
develop and evaluate early prototypes, low- and/or high-fidelity prototypes, which can be tested in informal test settings, without the element of in-person face-to-face involvement, and instead can be evaluated via remote usability testing tools.
This is also a great time to learn and incorporate new approaches, e.g. agile usability testing and SCRUM like approaches, which lean on agile software development methodologies, in which individual user interface elements are identified, designed, developed, and evaluated in shortened increments. Such methods can allow for more transparency and visibility, be more adaptable and accommodating to changes, increase project efficiency, and potentially shorten time to market while saving costs. (Contact HFUX Research if you’d like to learn more.)
But not just companies in the early stages of development can use this time efficiently. For companies well underway into development, possibly even approaching verification and validation phases of the development process; this time can be used most efficiently to verify that design inputs meet design outputs, as well as ensure all corresponding documentation is in place, aligned, and comprehensively reviewed. Activities and documentation critical to these phases include (but are not limited to):
formative evaluations have been appropriately documented, and critical findings have been identified and incorporated in design, and when applicable, previously unidentified use-related risks occurred and discovered during testing, have been added to the use-related risk analysis for further mitigation;
risk mitigations and design controls have been adequately identified, developed, implemented, and evaluated (satisfactory to FDA expectations);
user interface characteristics have been documented in corresponding user interface specifications;
labeling, packaging, and instructions for use (IFU) have been developed and comprehensively reviewed and evaluated in line with device design, to ensure optimal usability and use safety and effectiveness; and
training approach and corresponding training materials are identified, developed, and evaluated (when applicable), as such not only have to be included in the submission package, but most critically, have to be incorporated and evaluated in human factors validation usability testing.
Finally, for companies already in final design validation stages, possibly even in the process of designing and planning HF validation usability testing, this time could be used to take advantage of executing more time-consuming tasks, such as FDA review of HF validation usability test protocol and materials. This typically takes up to 90 days and is an ideal activity to complete right now. Prior to review ensure that you have identified ideal study design and developed a strong test protocol, appropriate to your applicable device. Preparing a comprehensive test protocol for HF validation usability testing is key to study outcome. Some other key elements to consider during HF validation usability testing preparation include:
selection and inclusion of use-related hazardous use scenarios and corresponding critical tasks in test scenarios;
preparation and development of key study materials such as recruitment screeners, test moderator guides, and test materials to be used during testing;
submission for Institutional Review Board (IRB) approval;
starting the recruitment process in time to ensure appropriate time allocation to enable identification and recruitment of eligible study participants;
identification of testing geographies and securing of corresponding test sites within;
completing internal dress rehearsal/pilot test(s) to ensure test protocol effectiveness and correct time allocation to all applicable test tasks and activities; and
ensure all HF documentation is in order and ready to be integrated in FDA HFE Report and overall design history and usability file.
While FDA has not issued any official statement or amended policies and/or guidelines with respect to changing their viewpoint on expectations of human factors (engineering) processes, activities, and documentation (including usability testing and corresponding data collection and reporting of such), this time does not have to be spent idling, as discussed above, there are plenty of things to focus on and complete in order to stay on track with your human factors development and not lose momentum in overall product development progression.
For further questions, or if you’d like to learn more and find out how HFUX Research can help and support you in staying on track with your HF development during this time, please go to www.hfuxresearch.com – or contact us directly at firstname.lastname@example.org.