deathswww.dajiba.com

DEBATEOpenAccessDebate:shouldweusevariableadjustedlifedisplays(VLAD)toidentifyvariationsinperformanceingeneralsurgeryStephenONeill,StephenJ.
WigmoreandEwenM.
Harrison*AbstractBackground:Therecentpushforthepublicationofindividualsurgeonoutcomesunderpinspublicinterestinsafersurgery.
Conventional,retrospectiveassessmentofsurgicalperformancewithoutcontinuousmonitoringmayleadtodelaysinidentifyingpoorperformanceorrecognitionofpracticesthatleadtobebetterthanexpectedperformance.
Discussion:Thevariablelifeadjusteddisplay(VLAD)isnotnew,yetisnotwidelyutilisedinGeneralSurgery.
Itsconstructionissimpleandifcaveatsareappreciatedtheinterpretationisstraightforward,allowingforcontinuoussurveillanceofsurgicalperformance.
Summary:Whilelimitationsinthedetectionofvariationsinperformanceareappreciated,theVLADcouldrepresentamoreusefultoolformonitoringperformance.
BackgroundTherecentpushforthepublicationofindividualsur-geonoutcomesunderpinspublicinterestinsafersur-gery.
Conventional,retrospectiveassessmentofsurgicalperformancewithoutcontinuousmonitoringmayleadtodelaysinidentifyingpoorperformanceorrecognitionofpracticesthatleadtobebetterthanexpectedper-formance.
Thevariablelifeadjusteddisplay(VLAD)isnotnew,yetisnotwidelyutilisedinGeneralSurgery.
Itsconstructionissimpleandifcaveatsareappreciatedtheinterpretationisstraightforward,allowingforcontinuoussurveillanceofsurgicalperformance.
Whilelimitationsinthedetectionofvariationsinperformanceareappre-ciated,theVLADcouldrepresentamoreusefultoolformonitoringperformance.
DiscussionVLADTheVLADwasestablishedbyLovegroveetal.
[1]todemonstratethedifferencebetweenobservedandex-pectedmortalityoveraspecifiedperiodoftimeinCardiacSurgery.
TheVLADissometimescalledtheexpected-observedcumulativesum(CuSum)plot[2].
Itisagraphthatplotsthecumulativedifferenceinob-servedmortalityfromexpectedmortalityonthey-axisagainstindividualcasesinthechronologicalorderthattheyoccuronthex-axis.
ThereforeaVLADforamor-talityratethatisequaltowhatisexpectedwillendatzero,whileaVLADforamortalityrateabovewhatisexpectedisseenasafallingline,andviceversa.
ThiseasilyinterpretablevisualsummaryexplainswhytheVLADispopularamongstclinicians[3].
However,thisapparentstrengthoftheVLAD,canalsobeviewedasaweaknessduetothestrongtemptationtoviewob-servedminusexpectedoutcomesas'livessaved'or'liveslost',whichisinappropriate.
Anexample:expectedmortalityof5%Consideranexampleinasurgicalcontextwheretheprobabilityofdeathforagivenprocedureis0.
05or5%(Fig.
1a).
Eachconsecutiveprocedureperformedisassignedabinaryvalue,whichis0ifthereisnodeathand1ifthepatientdied.
Ascoreiscalculatedfromthepredictedriskofdeathforthatprocedure,whichinthisexampleis0.
05.
TheVLADscoreiscalculatedbysub-tractingtheobservedoutcome(either0or1)fromtheexpectedoutcome(inthiscase0.
05).
Thereforefora*Correspondence:mail@ewenharrison.
comDepartmentofClinicalSurgery,UniversityofEdinburgh,RoyalInfirmaryofEdinburgh,EdinburghEH164SA,UK2015ONeilletal.
OpenAccessThisarticleisdistributedunderthetermsoftheCreativeCommonsAttribution4.
0InternationalLicense(http://creativecommons.
org/licenses/by/4.
0/),whichpermitsunrestricteduse,distribution,andreproductioninanymedium,providedyougiveappropriatecredittotheoriginalauthor(s)andthesource,providealinktotheCreativeCommonslicense,andindicateifchangesweremade.
TheCreativeCommonsPublicDomainDedicationwaiver(http://creativecommons.
org/publicdomain/zero/1.
0/)appliestothedatamadeavailableinthisarticle,unlessotherwisestated.
ONeilletal.
BMCSurgery(2015)15:102DOI10.
1186/s12893-015-0087-0procedureresultinginadeaththescorewouldbe0.
05minus1,whichisequaltoadownwardincrementof0.
95.
Whileforaprocedurethatresultedinsurvival,thescorewouldbe0.
05minus0,whichisequaltoapositiveincrementof0.
05.
If20caseswereperformedwherebytheexpectedmortalityandobservedmortalitywasequal,theninthe19caseswheretherewasnodeath,thesurgeonwouldhave19upwardincrementsof0.
05,whichisequalto0.
95.
Thiswouldbebalancedbytheoneexpecteddeaththatisobserved,whichwouldresultinadownwardincrementof0.
95andthelineontheVLADwouldreturntozero.
Therefore,insummary:VLAD=Cumulative(Expectedoutcome-observedoutcome)Expectedoutcomeistheprobabilityofdeathe.
g.
0.
05Observedoutcomewheresurvival=0anddeath=1AdvantagesofVLADAVLADissimpletoconstructandcanbeeasilygener-atedwithoutanyspecialiststatisticalknowledgeorsoft-ware[4].
TheVLADfacilitatestargetedandcontinuousrealtimeoutcomesurveillance.
ThisallowstheVLADtoincludeasurgeon'sentirecaseload,whichprovidesabetterperspectiveofoverallperformance.
Comparedwiththepracticeofretrospectiveassessment,thiscon-tinuoussurveillancemechanismofferstheopportunitytoidentifyandaddressthecausesofunexpectedresultsatanearlierstage.
Thismaymitigateon-goingpoorperformanceorhighlightbetterthanexpectedper-formance[5].
FunnelplotsarenotdesignedforrealFig.
1aVLADfortensimulatedsurgeons(blacklines)performing200caseswithactualmortalityequaltothepopulationrisklevelof5%.
Thebluelinesare95%controllimitssetfor10,000similarplots.
b10,000simulationsofaVLADforasurgeonwithanactualmortalityrateof2%(redlines,200shown)withapopulationrisklevelof1%(blacklines,200shown).
Themeanisthethickblacklineandbluelinesare95%controllimits.
c10,000simulationsofaVLADforasurgeonperforming200caseswithanactualmortalityrateequaltothepopulationrisklevelof10%for94casesbutthenhaving6deathsinarowbeforeresumingtheirinitialrisk(greenlines,200shown).
Theblacklines(200shown)arethepopulationlevelriskof10%,themeanisthethickblacklineandbluelinesare95%controllimits.
d10,000simulationsofaVLADforasurgeonperforming200caseswithanactualmortalityrateequaltothepopulationrisklevelof10%for94casesbutthenchangingtoanincreasedactualrisklevelof12.
5%(yellowlines,200shown).
Theblacklines(200shown)arethepopulationlevelriskof10%,themeanisthethickblacklineandbluelinesare95%controllimits.
Plotsavailablefrom:http://www.
datasurg.
net/vladONeilletal.
BMCSurgery(2015)15:102Page2of4timemonitoringsotheabilityoftheVLADtobeusedasacontinuoussurveillancetoolisadistinctadvan-tage[3].
RiskadjustmentsWhenusingtheVLAD,anappropriateadjustmentforoperativeriskiscriticalforensuringaccurateassess-ments.
Definingariskofdeathspecifictoeachindivid-ualmaybemorerobustthandefiningthesameriskofdeathforallindividualsundergoingoneprocedure.
Outcomesarethereforeadjustedforriskbydifferentmodelsthatestimatetheriskofdeathforeachpatientbasedontheirindividualcharacteristicsandco-morbidities.
However,cautionmustbeobservedinap-plyingriskadjustments[3].
Assurgicalmortalityratesdecrease,riskscoresneedtobeupdatedtorepresentthecurrentstandardofpractice[3].
Tsangetal.
[6]showedinpaediatriccardiologyhowoverarelativelyshorttimeperiodriskmodelscouldrapidlybecomeoutofdate.
Noriskmodelisperfectandtheremaybein-herentweaknessesinthemethodusedtoriskadjust.
Forexample,thepartialriskadjustmentinsurgery(PRAiS)modelfailstoadjustforcertainco-morbidconditionsandslightlyunderestimatesriskforthehighestriskpatients.
InarecentpublicationbyPageletal.
[7]thisweaknessinPRAiSledtoanegativeimpres-sionofperformanceinoneUKcentrethatwasinvolvedinrealtimemonitoringofrisk-adjustedpaediatriccar-diacsurgeryoutcomesusingtheVLAD.
ControllimitsTheVLADlackscontrollimits,whichcanmakeitdiffi-culttoassessthepossiblecontributionofrandomvari-ationtoperformance[8].
Italsomeansthatidentifyingtheappropriatetimetotakeactionbasedonobservedresultsisnotquantitativelydetermined.
ThishasledtocriticismthattheVLADislimitedinitsabilitytoiden-tifymortalityratechangeswithadequatespeed[3].
However,sinceVLADsshowthechangeinoutcomesovertime,onemaynotwishtowaittohit'significance'beforereflectingonanapparenttrend.
Thisapproachcouldleadtothelossoflivesthatmighthavebeensaved,andirretrievabledamage(maybewrongly)toasurgeon'scareerwhensomeinsightorretrainingmayhavehelped[9].
Assuch,theVLADshouldnotbecon-sideredastatisticalevaluation[1].
Despitethis,controllimits,whicharesometimescalledrockettails,canoftenappliedtotheVLADtoactasalertthresholds[8].
WalterA.
Shewhart,thein-ventoroftheindustrialcontrolcharttechnique,usedthreestandarddeviationscontrollimitsbutinhealth-carethesecontrollimitsareoftensetatthe5%level.
Althoughthiscut-offisarbitraryitcanbeconsideredasthepointwhentheprobabilitythatdifferencesbetweenexpectedandobservedoutcomesareunlikelytobeduetochancealone[8].
Nevertheless,aswithanycontrollimit,ifcontrollimitsareappliedtotheVLAD,careneedstobetaken,asapparentvariationinper-formancemaybehighlightedwhencontrollimitsarecrossedsimplyasaresultofrandomvariation[8].
Anoften-citedanalogyistheuseofmetaldetectorstoscreenpassengersatairports.
Inthissituationthesensitivityofthedetectorcanbevaried.
Lowsensitivityrunstheriskthataprohibitedmetalitemsuchasagunwillpassundetected.
Highsensitivityreducestheriskoffailingtodetectagun,butincreasesthenumberofpassengerswhoarenotcarryingmetalwhowillbepulledbychanceoutofline.
Wherethelimitsofdetectionshouldbesetdependonthecircumstancesoftheoutcome,itsseriousnessandtheneedtodetectoutliers.
InFig.
1a,typicalVLADswerecreatedbysimulationusingRforstatisticalprogramming(versionversion3.
1.
1)forsurgeonswithanactualmortalityrateexactlythesameasthatofthebaselineriskacrosstheentirepopulation.
Despitethesesurgeonsworkingattheexpectedpopulationmortalityrate,thereisapparentvariationseenasaresultoftheprocessofrandomvari-ation.
ItwouldthereforebeexpectedintheseVLADsthatonesurgeonintwentymaybeaboveorbelowthe95%controllimitatanygiventimeandthereforepotentiallysubjecttoareview.
Usingsimilarsimulations,onemayalsoconsiderthechanceofasurgeonorunitwithamortalityratehigherthanexpectedbeingdetected.
Thistranslatestothenumberofcasesthatrequiretobeperformedbeforetheaberrantpracticeisidentified.
Forexample,withanexpectedmortalityof1%,by200casesonly23%ofsurgeonswithanactualmortalityrateof2%willhavecrosseda95%controllimit(Fig.
1b).
Thisfocusesthemindastowhatsizeofdifferencefromnormalpracticeshouldactuallybeconsidereddifferent.
Wehavecreatedaweb-applicationthatcanbeusedtoexplorethesefiguresfurther(http://www.
datasurg.
net/vlad).
LimitationsofVLADAnothercriticismoftheVLADisthatagoodrunofre-sultsmaymaskasubsequentpoorrun,whichwillmeanthatanexcessofmortalitiesareneededtocrossthecontrollimitandtriggerareview[5].
InFig.
1c,surgeonsaresimulatedwithanactualmortalityrateequivalenttothatofthepopulationmortalityratefor94casesbutthentheyhaveapoorrunof6deathsinarow.
Duetothepreviousgoodrun,only32%ofsur-geonswillcrossthelower95%controlintervalcontrollimitatthispoint.
TherearealsopotentiallimitationswiththeVLADfordetectingmoreconsistentchangesinpracticeinanestablishedsystem.
ThistypeofchangemayoccurdueONeilletal.
BMCSurgery(2015)15:102Page3of4tosurgeonperformancebutcouldalsopotentiallyoccursecondarytoanysignificantchangeinthehealthcareenvironment(e.
g.
criticalcareprovision).
InFig.
1d,surgeonsaresimulatedwithanactualmortalityrateequivalenttothatofthepopulationratefor100cases.
Atcase100,theactualmortalityratechangestoahigherlevel,butthisnew"changepoint"isnotdetectedgiventhewidercontrollimitsatthistime.
Thesefigurescanalsobeexploredfurtherusingtheaforementionedweb-application(http://www.
datasurg.
net/vlad).
Onemethodtopreventgoodrunsmaskingsubse-quentpoorperformanceistopreventtheVLADfrombecomingpositivesothatonlyrunsofworseningout-comeareexaminedbutthismayleadtoexcesstrigger-ingandunneededreviewsofperformance[5].
Alternativeplotssuchastherisk-adjustedCuSumandrisk-adjustedexponentiallyweightedmovingaver-agealsoovercometheselimitationsbutmaybemorecomplextoconstruct.
Therisk-adjustedCuSumplotutilizesasequentialsamplingtechniquetotestthehy-pothesisthattheriskofdeathisincreasedanddoesn'tallowforaccumulationofcreditforgoodperformanceasthestatisticaltestisboundedbythelowerlimitofzero.
Therisk-adjustedexponentiallyweightedmovingaverageplotisarunningestimateofthemeanoutputofaprocess,wherethemostrecentobservationsaregivenexponentiallymoreweightthanhistoricallydis-tantobservations[10].
UseofVLADinGeneralSurgeryAlthoughithastakentime,examplesoftheuseofVLADsinGeneralSurgeryarebeginningtoemerge.
Collinsetal.
[3]retrospectivelyperformedananalysisofthedatabaseoftheScottishAuditofGastro-OesophagealCancerser-vicesusingaVLAD.
WhileRobertsetal.
[5]recentlypub-lishedthefirstreal-time,risk-adjustedVLADofasinglecentre'soutcomeafterIvor-Lewisoesophagectomyforoesophagealcancer.
Guestetal.
[4]appliedtheVLADtosinglesurgeon'soutcomesfollowingoesophagogastricre-sectionsforcancercomparedwiththosepredictedbythePortsmouthpredictormodification(P-POSSUM)score.
Guestetal.
[4]alsowentontosuggestthattheVLADwasapotentiallyusefultoolintheprocessofrevalidationforsurgeons.
ThiscouldfurtherextendtheapplicabilityofVLADinthecontextinGeneralSurgery,ascouldtheuseoftheVLADtomonitorotherperformanceoutcomessuchaspost-operativecomplications.
EvenforthehighestriskproceduresinGeneralSurgery(e.
g.
uppergastrointes-tinalcancerresection),theelectivemortalityrateisnowonaverage<5%[11].
Thereforeothermarkers(e.
g.
failuretorescue,infectionandanastomoticleak)couldbepar-ticularlyimportantintheGeneralSurgicalsetting.
How-ever,beforethiscanhappendueconsiderationofdataquality,definitionofoutcomes,casemixandinstitutionalfactorsthataffectoutcomewillbeimportant.
SummaryIneffortstoimprovepatientsafetythemonitoringofsurgicalperformanceisbecomingmorewidespread.
Asgeneralsurgerydatawillbeincreasinglyplacedinthepublicdomainitisimportantthatgeneralsurgeonstakeanactiveroleinthisprocess.
DifferentmethodsofmonitoringsurgicalperformanceneedtobeexaminedbythegeneralsurgicalcommunityandtheuseofVLADscouldcontributesignificantlytoidentifyingvar-iationsinperformance.
CompetinginterestsTheauthorsdeclarethattheyhavenocompetinginterests.
Authors'contributionsSONdraftingofthemanuscript.
SJWconceptionanddesign,andrevisionofthemanuscriptcriticallyforimportantintellectualcontent.
EMHconceptionanddesign,acquisitionofdata,analysisandinterpretationofdata,andrevisionofthemanuscriptcriticallyforimportantintellectualcontent.
Allauthorsreadandapprovedthefinalmanuscript.
Received:8June2015Accepted:20August2015References1.
LovegroveJ,ValenciaO,TreasureT,Sherlaw-JohnsonC,GallivanS:Monitoringtheresultsofcardiacsurgerybyvariablelife-adjusteddisplay.
Lancet1997,350(9085):1128-1130.
2.
SmithPC,MossialosE,PapanicolasI:PerformanceMeasurementforHealthSystemImprovement:Experiences,ChallengesandProspects:CambridgeUniversityPress;2010.
3.
CollinsGS,JibawiA,McCullochP:Controlchartmethodsformonitoringsurgicalperformance:acasestudyfromgastro-oesophagealsurgery.
EurJSurgOncol2011,37(6):473-480.
4.
GuestRV,ChandrabalanVV,MurrayGD,AuldCD:ApplicationofVariableLifeAdjustedDisplay(VLAD)torisk-adjustedmortalityofesophagogastriccancersurgery.
WorldJSurg2012,36(1):104-108.
5.
RobertsG,TangCB,HarveyM,KadirkamanathanS:Real-timeoutcomemonitoringfollowingoesophagectomyusingcumulativesumtechniques.
WorldJGastrointestSurg2012,4(10):234-237.
6.
TsangVT,BrownKL,SynnergrenMJ,KangN,deLevalMR,GallivanS,UtleyM:Monitoringrisk-adjustedoutcomesincongenitalheartsurgery:doestheappropriatenessofariskmodelchangewithtimeAnnThoracSurg2009,87(2):584-587.
7.
PagelC,UtleyM,CroweS,WitterT,AndersonD,SamsonR,McLeanA,BanksV,TsangV,BrownK:Realtimemonitoringofrisk-adjustedpaediatriccardiacsurgeryoutcomesusingvariablelife-adjusteddisplay:implementationinthreeUKcentres.
Heart2013,99(19):1445-1450.
8.
Sherlaw-JohnsonC,MortonA,RobinsonMB,HallA:Real-timemonitoringofcoronarycaremortality:acomparisonandcombinationoftwomonitoringtools.
IntJCardiol2005,100(2):301-307.
9.
deLevalMR,FrancoisK,BullC,BrawnW,SpiegelhalterD:Analysisofaclusterofsurgicalfailures.
Applicationtoaseriesofneonatalarterialswitchoperations.
JThoracCardiovascSurg1994,107(3):914-923;discussion923-914.
10.
CookDA,DukeG,HartGK,PilcherD,MullanyD:Reviewoftheapplicationofrisk-adjustedchartstoanalysemortalityoutcomesincriticalcare.
Criticalcareandresuscitation:journaloftheAustralasianAcademyofCriticalCareMedicine2008,10(3):239-251.
11.
HealthandSocialCareInformationCentre.
NationalOesophago-gastricCancerAudit2013.
ONeilletal.
BMCSurgery(2015)15:102Page4of4