Defining the realworld reproducibility of visual

Authors: Graham D Cole Niti M Dhutia Matthew J ShunShin Keith Willson James Harrison Claire E Raphael Massoud Zolgharni Jamil Mayet Darrel P Francis

Publish Date: 2015/07/04

Volume: 31, Issue: 7, Pages: 1303-1314

PDF Link

Abstract

Left ventricular function can be evaluated by qualitative grading and by eyeball estimation of ejection fraction EF We sought to define the reproducibility of these techniques and how they are affected by image quality experience and accreditation Twenty apical fourchamber echocardiographic cine loops Online Resource 1–20 of varying image quality and left ventricular function were anonymized and presented to 35 operators Operators were asked to provide 1 a onephrase grading of global systolic function 2 an “eyeball” EF estimate and 3 an image quality rating on a 0–100 visual analogue scale Each observer viewed every loop twice unknowingly a total of 1400 viewings When grading LV function into five categories an operator’s chance of agreement with another operator was 50  and with themself on blinded representation was 68  Blinded eyeball LVEF reestimates by the same operator had standard deviation SD of difference of 76 EF units with the SD across operators averaging 83 EF units Image quality defined as the average of all operators’ assessments correlated with EF estimate variability r = −0616 p  001 and visual grading agreement r = 058 p  001 However operators’ own single quality assessments were not a useful forewarning of their estimate being an outlier partly because individual quality assessments had poor withinoperator reproducibility SD of difference 178 Reproducibility of visual grading of LV function and LVEF estimation is dependent on image quality but individuals cannot themselves identify when poor image quality is disrupting their LV function estimate Clinicians should not assume that patients changing in grade or in visually estimated EF have had a genuine clinical changeClinicians are sometimes surprised that a patient moves between normal and impaired left ventricular function with just reassessment of the same acquired images Outside of research qualitative grading of ventricular function using portable hardware with limited functionality is common 1 2 An alternative is the similarly speedy “eyeball” EF 3 in which the recommended formal Simpson’s calculation 4 is not carried out but a judgment is made from the images alone It is apparent that this practice occurs not only in clinical practice but also in recruitment for landmark randomized controlled trials REVERSE 5 and MADITCRT 6 for example have disclosed the histograms of EF values from recruitment centers which suggest that the majority were eyeball estimatesPatients undergoing echocardiography for clinical reasons may have images that would not be of the quality typically displayed as published examples 7 of the technique Whilst previous studies have shown that visual estimation and formal calculation of EF have a strong relationship 8 9 10 it is not known whether the reproducibility of qualitative grading of LV function and visual estimation of ejection fraction is resilient to imperfect image qualityThe use of bedside echocardiography as an extension of the clinical examination is desirable 2 and increasingly affordable 1 Improved access makes serial reassessment during the same episode of care possible This portable hardware often has limited functionality leaving operators to judge LV function on visual appearance without access to the full panel of measurementsCurrent guidelines already discourage shortcut estimation of LV function 4 Whether these techniques should be universally discouraged for all cases regardless of image quality and for all operators regardless of experience and accreditation status is unknownIn this study in a cohort of patients undergoing routine clinical inpatient or outpatient echocardiography we defined the reproducibility of qualitative grading and estimation of EF and quantified the impact of image quality experience and accreditationWe selected 20 anonymous apical fourchamber echocardiograms acquired using a General Electric Vivid I General Electric Hatfield UK or Philips ie33 Philips Guildford UK The cine loops as seen by operators are shown in Online Resources 1–20 Two of the authors GDC DPF reviewed the studies to ensure that there was a range of image quality and LV function across the studies Each echocardiogram was duplicated so that there was the appearance of 40 studies The studies were ordered randomly in a Powerpoint presentation and viewed by study participants unaware of the duplication

Keywords: