Clinician feedback: quantitative

Following a pilot survey, an online mixed methods clinical usability (CU) survey of Integrated Sexual Health (ISH) clinicians was distributed nationally. Respondents rated 14 clinical usability statements on a 7-point Likert scale, analysed with Tableau™. This was a national survey of 118 clinicians using the four main sexual health EHR interfaces, plus those using EHR interfaces not specifically designed for sexual health.

Respondents were asked to give a score of 1 (strongly unfavourable) to 7 (strongly favourable) to these 14 clinical usability statements:

1. It is easy to get a general overview of the patient’s history

2. There is good use of graphics

3. It is easy to make sense of laboratory results

4. It disrupts the sense of flow in the patient conversation

5. Information is laid out in a way that helps my clinical intuition

6. I find myself having to repeat questions over and over

7. The more often a patient is seen, the more accurate their records become

8. The screen layout reflects clinical real life

9. The way it works helps my memory work more effectively

10. I enjoy using it

11. I wish I didn’t have to double-click so much

12. Better accessibility to previous history would make the clinical conversation more meaningful

13. It is easy to search for clinical information

14. Poor information layout is a cause of fatigue and frustration

The image below shows feedback in the form of Net Favourability Scores (NFS), which are calculated as [% ‘favourable’ minus ‘% ‘unfavourable’] for each EHR type. Taking all EHRs as one collective group, the NFS tell the following story:

• NFS are overwhelmingly negative across all EHRs: 60 (86%) of the 70 NFS are negative

• Negative NFS are typically heavily negative (range -92% to -4%, mean -51%, median -50%)

• NFS are positive for far fewer heuristics: of 7 (10%) of the 70 NFS are negative

• Positive NFS are typical slightly positive (range +4% to +33%, mean +23%, median +24%)

• EHR4 and ‘Other EHR’ appear to have marginally less negative NFS overall

Quantitative findings from this clinical usability survey revealed all EHRs to be heavily biased towards unfavourable. EHR1 and EHR3 performed worst. EHR4 and 'Other EHRs' performed best, with the polarisation of their responses suggesting some clinical usability aspects might work well for some users (or clinics) but frustrate others. EHR2 was somewhere in between.

These findings paint a picture of clinician users struggling to make clinical sense of the information at high (overview, summary) as well as low (granular, detail) level.

A final iteration of this clinical usability survey is discussed (and downloadable) in another post.

November 2023

Click on image to download the full poster