New clinical usability survey (download)

November 2023

‘Usability is like love. You have to care, you have to listen, and you have to be willing to change.'

- Jeffrey Zeldman (Web Designer)

Usability surveys are commonly used to collect clinician feedback about EHR interface usability. Such surveys measure user experience and perception of a particular system by measuring a diverse set of constructs such as satisfaction, efficiency, effectiveness, learnability, usefulness, ease of use, and interface and information quality.

Arguably the most popular survey is the System Usability Scale (SUS), which (the author suggests) ‘provides a quick, reliable tool for measuring usability’ (Brooke, 1996). The SUS consists of a 10-item questionnaire with a 5-point Likert response scale, from ‘Strongly disagree’ to ‘Strongly agree’. Despite the SUS being highly generic—it is used to evaluate hardware, software, mobile devices, websites and applications—it is often used for EHR interface evaluation. One excellent example of its use is the UK Emergency Department EHR usability paper by Bloom et al. (2021).

Twenty years after the SUS was published, and modelled on it, the 5-item Clinical Software Usability Scale (cSUS) was created at the 2015 Digital Health Summer School (Baw & Hoeksma, 2016). Clinical evaluation has been introduced but this is limited to high-level clinical impact, with no granular evaluation of clinical practice. Furthermore, the fifth item asks about multiple factors (four of them) but only outputs a single score.

Most recently, the UK EPR usability survey was distributed by KLAS (Storey & Clune, 2022). Here, 11 items are used—but again, there is minimal clinical specificity and even then, only at the highest level.

By far the most prominent limitation of these usability surveys is their lack of granular specificity to clinical practice. Even these surveys, which purport to be EHR-specific, are marginal modifications of generic usability surveys. They are not clinical usability surveys. They cannot adequately evaluate EHR interfaces, let alone inform their design.

User-centred design (UCD) addresses these limitations by situating the user at the heart of the survey design process. This approach generates a profound understanding of users and their contexts of use, engaging with them through iterative design, prototyping and testing cycles (Norman and Draper, 1986). As Braithwaite et al. (2015) argue, ‘Users’ participation and analysis of their work are key elements... [focusing] on the actual work done as opposed to 'work-as-imagined.’

Participatory Design (PD), an evolution of UCD, actively involves users in the design process. PD transforms users into co-designers, allowing their knowledge and perspectives to shape the final design outcome. PD incorporates rich, context-specific details that might otherwise be overlooked, increasing the potential for the survey to resonate with user needs and realities (Robertson and Simonsen, 2012).

The new clinical usability survey presented here (see image) is a product of a thorough UCD and PD approach. It is for use by clinicians and focuses on how well an EHR interface supports clinical practice—it is not concerned with technical aspects such as interoperability or connectivity speed. It is generalisable to any outpatient EHR interface.

The survey is downloadable (with its scoring system) by clicking on the image.

Baw, M. and Hoeksma, J. (2016). Clinical Software Usability Survey (cSUS). [Online]. Available at: https://openhealthhub.org/t/clinical-software-usability-questions/456

Bloom, B. M. et al. (2021). Usability of electronic health record systems in UK EDs. Emergency Medicine Journal, 38 (6), pp.410–415.

Braithwaite, J., Wears, R. L. and Hollnagel, E. (2015). Resilient health care: turning patient safety on its head. International Journal for Quality in Health Care, 27 (5), pp.418–420.

Brooke, J. (1996). SUS: A quick and dirty usability scale. In: Patrick W. Jordan, B. Thomas, Ian Lyall McClelland, Bernard Weerdmeester (Ed). Usability evaluation in industry. Broca Raton, FL, USA: CRC Press.

Norman, D. A. and Draper, S. W. (1986). User Centered System Design: New Perspectives on Human-computer Interaction. Boca Raton, FL, USA: CRC Press.

Robertson, T. and Simonsen, J. (2012). Participatory Design: an introduction. In: Routledge International Handbook of Participatory Design. Milton Park, UK: Routledge. pp.21–38.

Storey, T. and Clune, C. (2022). NHS Acute Supplier Presentation: the KLAS NHS EPR usability survey. [Online]. Available at: https://pixl8-cloud-techuk.s3.eu-west-2.amazonaws.com/prod/public/a7291891-62c7-436d-b909ce30b16f579e/NHS-Acute-Supplier-6-30-22-Presentation.pdf

Click on the image to download the clinical usability survey