Johnson, C. M., Johnston, D., Crowley, P. K., Culbertson, H., Rippen, H. E., Damico, D. J., Plaisant, C., EHR Usability Toolkit: A Background Report on Usability and Electronic Health Records (AHRQ Publication No. 11-0084-EF)
(August, 2011), Agency for Healthcare Research and Quality, Rockville, MD
(last updated on 2012/09/30)
Abstract:
Report prepared for Agency for Healthcare Research and Quality by: Westat, 1600 Research Boulevard, Rockville, MD 20850-3129. under contract # HHSA290200900023I
Executive Summary:
The reports funded by the Agency for Healthcare Research and Quality (AHRQ) and developed under the “Use of Dense Display of Data and Principles in Primary Care Health Information Technology (IT) Systems” project identified key shortcomings in the usability of certified electronic health record (EHR) products and the ways that health care organizations implement them.1,2 Notably, these shortcomings included a lack of standard practices, best-practice sharing, and systematic processes. Key recommendations from these reports include establishment of usability as a core aspect of EHR certification and development of objective criteria that reflect best practices in EHR usability.
To begin exploration of improving EHR usability, AHRQ contracted with Westat to develop, test, refine, and disseminate a Web-based toolkit for primary care providers to assess how well the usability of health IT systems support them in delivering safe, effective, and efficient care. This toolkit would also provide objective yet practical means to assess whether the usability of EHRs support such primary care functions as proper diagnosis, identification of high-risk patients, tracking of health parameters over time, and population health management. Finally, the toolkit should assess how well EHRs adhere to critical “best practice” usability principles as well as help to identify flaws in usability that might adversely impact the quality of care. Thus, three important objectives of this project are the following:
1. Develop and refine a toolkit for primary care providers that supports rapid yet meaningful usability evaluations of EHRs.
2. Disseminate the toolkit as a means to increase attention on the importance of EHR usability, to promote use of evidence-based usability evaluation methods, and to stimulate collaboration among entities developing and implementing EHR systems.
3. Inform EHR accreditation efforts on usability, including those by the National Institute of Standards and Technology.
To support these objectives, the project identified and convened a panel of experts from the fields of usability testing and assessment, biomedical and clinical informatics, human factors, cognitive psychology, human-computer interaction, tool development, and end-users of EHRs. Members of the expert panel (listed in Appendix A of this report) were called upon for their experience with existing toolkits and resources in the area and with measures and metrics for measuring usability of EHR systems, and to provide feedback on the design, testing, and dissemination of the toolkit.
This report provides the basis for recommendations that will guide the development of an initial Web-based toolkit that supports primary care providers by identifying usability issues in their current EHRs. To ensure our recommendations are based on the most current evidence, we have reviewed and analyzed the literature on usability issues in health care information systems and on usability evaluation methods.
The toolkit will reflect the complex attributes of primary care settings—such as diversity of EHR users, complexity of shared tasks, differences in clinical encounters, and the variety of patients and conditions. The toolkit will also need to address usability issues often encountered by primary care EHR users including challenges in information processing, workflow interruption, and increases in cognitive burden stemming from poorly designed interfaces and EHR functions. Central to our approach, then, is ensuring that the attributes of primary care settings and the evidence of usability issues in EHRs inform selection of usability evaluation methods for inclusion in the Web-based toolkit.
The authors searched the literature to identify the range of usability evaluation methods that could be used to identify problems in EHR systems. These methods include heuristic evaluation, cognitive walkthrough, remote evaluation, laboratory testing, and usability questionnaires. In addition, the authors reviewed other human factors methods that can be used to evaluate information systems. These are predictive modeling, risk assessment, and workload evaluation methods. These human factors methods are included because these methods can help to identify usability problems within a system. Based on the scope of the project, guidance from AHRQ and the Technical Expert Panel, and factors identified through consideration of EHR capabilities, usability problems, and attributes of primary care settings, high-level criteria for selecting methods for inclusion in the initial toolkit were developed. Two important criteria include the efficiency of a particular method (i.e., how easy it is to use and how quickly it can be applied) and the ability for primary care practices to administer a method independently from human factors experts. When assessing the advantages and disadvantages of existing usability evaluation methods with these and other criteria, it was determined that usability questionnaires would be the most practical, rapid, and useful methods for the initial toolkit.
To meet the goals of this project, the report recommends the development of a Web-based toolkit consisting of three main components: assessment, education, and tailoring. The assessment component will consist of core usability questions most relevant to EHRs. The educational component will provide a description of the usability problem and why it is relevant to an EHR. It may also include links to resources to help toolkit users understand usability and a summary sheet that could be shared with vendors. The tailoring component will provide a summary that providers can share with vendors and EHR certifying bodies to complement their user-centered design programs. 63 pages
