Methodologicalnnt

RESEARCHARTICLEOpenAccessNumberneededtotreat(NNT)inclinicalliterature:anappraisalDiogoMendes1,2*,CarlosAlves1,2andFranciscoBatel-Marques1,2AbstractBackground:Thenumberneededtotreat(NNT)isanabsoluteeffectmeasurethathasbeenusedtoassessbeneficialandharmfuleffectsofmedicalinterventions.
SeveralmethodscanbeusedtocalculateNNTs,andtheyshouldbeapplieddependingonthedifferentstudycharacteristics,suchasthedesignandtypeofvariableusedtomeasureoutcomes.
WhetherornotthemostrecommendedmethodshavebeenappliedtocalculateNNTsinstudiespublishedinthemedicalliteratureisyettobedetermined.
TheaimofthisstudyistoassesswhetherthemethodsusedtocalculateNNTsinstudiespublishedinmedicaljournalsareinlinewithbasicmethodologicalrecommendations.
Methods:Thetop25high-impactfactorjournalsinthe"Generaland/orInternalMedicine"categorywerescreenedtoidentifystudiesassessingpharmacologicalinterventionsandreportingNNTs.
Studieswerecategorizedaccordingtotheirdesignandthetypeofvariables.
NNTswereassessedforcompleteness(baselinerisk,timehorizon,andconfidenceintervals[CIs]).
ThemethodsusedforcalculatingNNTsinselectedstudieswerecomparedtobasicmethodologicalrecommendationspublishedintheliterature.
Datawereanalyzedusingdescriptivestatistics.
Results:Thesearchreturned138citations,ofwhich51wereselected.
Mostweremeta-analyses(n=23,45.
1%),followedbyclinicaltrials(n=17,33.
3%),cohort(n=9,17.
6%),andcase–controlstudies(n=2,3.
9%).
Binaryvariablesweremorecommon(n=41,80.
4%)thantime-to-event(n=10,19.
6%)outcomes.
Twenty-sixstudies(51.
0%)reportedonlyNNTtobenefit(NNTB),14(27.
5%)reportedbothNNTBandNNTtoharm(NNTH),and11(21.
6%)reportedonlyNNTH.
Baselinerisk(n=37,72.
5%),timehorizon(n=38,74.
5%),andCI(n=32,62.
7%)forNNTswerenotalwaysreported.
BasicmethodologicalrecommendationstocalculateNNTswerenotfollowedin15studies(29.
4%).
Theproportionofstudiesapplyingnon-recommendedmethodswasparticularlyhighformeta-analyses(n=13,56.
5%).
Conclusions:Aconsiderableproportionofstudies,particularlymeta-analyses,appliedmethodsthatarenotinlinewithbasicmethodologicalrecommendations.
Despitetheirusefulnessinassistingclinicaldecisions,NNTsareuninterpretableifincompletelyreported,andtheymaybemisleadingifcalculatingmethodsareinadequatetostudydesignsandvariablesunderevaluation.
Furtherresearchisneededtoconfirmthepresentfindings.
Keywords:Numbersneededtotreat,Evidence-basedmedicine,Epidemiologicmethods,Datainterpretation,Statistical,Meta-analysis,Randomizedcontrolledtrial,Cohortstudies,Case–controlstudiesBackgroundTheconceptof"numberneededtotreat"(NNT)wasintroducedinthemedicalliteraturebyLaupacisetal.
in1988[1].
NNTisanabsoluteeffectmeasurewhichisinterpretedasthenumberofpatientsneededtobetreatedwithonetherapyversusanotherforonepatienttoencounteranadditionaloutcomeofinterestwithinadefinedperiodoftime[1,2].
ThecomputationofNNTisfoundedonthecumulativeincidenceoftheoutcomepernumberofpatientsfollowedoveragivenperiodoftime,beingclassicallycalculatedbyinvertingabsoluteriskreduction(ARR)(alsocalledriskdifference[RD])betweentwotreatmentoptions[1,2].
SomecharacteristicsareinherentlyassociatedwiththeconceptofNNT.
Theresultingvalueisspecifictoasin-glecomparisonbetweentwotreatmentoptionswithinasinglestudy,ratherthananisolatedabsolutemeasureof*Correspondence:diogomendes26@gmail.
com1AIBILI–AssociationforInnovationandBiomedicalResearchonLightandImage,CHAD–CentreforHealthTechnologyAssessmentandDrugResearch,AzinhagadeSantaComba,Celas,3000-548Coimbra,Portugal2UniversityofCoimbra,SchoolofPharmacy,LaboratoryofSocialPharmacyandPublicHealth,Coimbra,PortugalTheAuthor(s).
2017OpenAccessThisarticleisdistributedunderthetermsoftheCreativeCommonsAttribution4.
0InternationalLicense(http://creativecommons.
org/licenses/by/4.
0/),whichpermitsunrestricteduse,distribution,andreproductioninanymedium,providedyougiveappropriatecredittotheoriginalauthor(s)andthesource,providealinktotheCreativeCommonslicense,andindicateifchangesweremade.
TheCreativeCommonsPublicDomainDedicationwaiver(http://creativecommons.
org/publicdomain/zero/1.
0/)appliestothedatamadeavailableinthisarticle,unlessotherwisestated.
Mendesetal.
BMCMedicine(2017)15:112DOI10.
1186/s12916-017-0875-8clinicaleffectofasingleintervention.
Thus,NNTisspe-cifictotheresultsofagivencomparison,nottoapar-ticulartherapy[3].
Inaddition,threeotherfactors,beyondtheefficacyorsafetyoftheinterventionandthecomparator,influenceNNT:baselinerisk(i.
e.
,controleventrate[CER]),timeframe,andoutcomes[3].
TheuseofNNThasbeenvaluableindailyclinicalpractice,namelyatassistingphysiciansinselectingtherapeuticinterventions[4,5].
Further,thismetrichasthepotentialforuseasasupportivetoolinbenefit-riskassessmentsandinhelpingregulatorsmakedecisionsondrugregulation[6–8].
TheConsolidatedStandardsofReportingTrials(CONSORT)statementrecommendstheuseofbothrelativeandabsolutemeasuresofeffectforrandomizedcontrolledtrials(RCTs)withbinaryandtime-to-eventoutcomes[9,10].
TheBritishMedicalJournal(BMJ)re-quiresthat,wheneverpossible,absoluteratherthanrela-tiverisksandNNTswith95%confidenceintervals(CIs)aretobereportedinRCTs[11].
Yet,fewauthorsexpresstheirfindingsintermsofNNTorARR[12–14].
Relativeeffectmeasures,suchasrelativerisk(RR)oroddsratio(OR),aremorecommonlyseeninthescientificlitera-ture[14,15].
Despitetheunquestionableusefulnessofrelativeeffectmeasures,theydonotreflectbaselinerisks,makingitimpracticabletodiscriminatelargefromsmalltreatmenteffects,andleadingsometimestomis-leadingconclusions[15–17].
AlthoughtheNNTwasoriginallyconceivedtobeusedinRCTs[1],theconcepthasbeenusedtoexpresstreat-mentdifferencesincomparativestudieswithotherde-signs,includingsystematicreviewsandmeta-analyses,andobservationalstudies(cohortandcase–controlstud-ies)[18–23].
Notethattheterms"numberneededtotreattobenefit"(NNTB)and"numberneededtotreattobeharmed"(NNTH)wereproposedtodistinguishbetweenbeneficialandharmfuloutcomes,respectively[24].
Furthermore,"numberneededtobeexposed"(NNE)hasbeenproposedtoapplytheconceptofNNTinobservationalstudies,inwhichthefocusisexposureratherthantreatment[22].
NNEBandNNEHcanbeusedtodescribethenumberneededtobeexposedforonepersontobenefitorbeharmed[22].
Inordertosimplify,thetermNNTisusedthroughoutthispaper.
ThecalculationofNNTshouldbebasedupontheuseofmethodsthatalignwiththecharacteristicsofagivenstudy,suchastheresearchdesignandthetypeofvariable(e.
g.
,binary,timetoevent,orcontinuous)usedtoexpresstheoutcomeofinterest[19,22,25–32].
Theuseofinad-equatemethodsmayleadtoerroneousresults[12,29,30,33,34].
Apreviousresearchstudyanalyzingarticlespub-lishedinfourmajormedicaljournalsfoundthatNNTsweremiscalculatedin60%ofRCTsinvolvingvaryingfollow-uptimes[29].
Theauthorsofanotherpaperconcludedthat50%oftheRCTsreportingNNTsderivedfromtime-to-eventoutcomesappliedinadequatecalcula-tionmethods[12].
Moreover,only34%ofRCTspresentedthecorrespondingCIsforpoint-estimateNNTs[12].
Theapplicationofinadequatemethodswithinotherresearchdesigns,suchasusingpooledRDsinmeta-analyses[35,36]orunadjustedincidenceratesinobservationalstudies[22,34],hasalsobeenpointedout.
ThemaingoalofthisstudyistoassesswhetherthemethodsusedtocalculateNNTinstudiespublishedinmedicaljournalsareinlinewithbasicmethodologicalrecommendations.
MethodsStudiesreportingNNTinmedicaljournalsIdentificationandselectionofstudiesPubMedwassearchedforpapersreportingNNTesti-matesthatwerepublishedbetween2006and2015inthetop25high-impactfactorjournalsinthecategoryof"Generaland/orInternalMedicine,"accordingtotheScienceCitationIndex(Additionalfile1:TableS1)[37].
Thesearchwasrestrictedtothesejournalsbecausetheyaremorelikelytoinfluenceclinicians'perceptionsonthebenefitsandharmsofmedicines[38].
Nofurtherlimitswereusedinthesearchstrategy(Additionalfile1:TableS2).
Titlesandabstractsofallretrievedcitationswerescreenedbytwoindependentreviewers(DMandCA)toidentifypo-tentiallyrelevantpublications.
Fulltextswereretrievedforrelevantcitations.
Discrepancieswereresolvedbymajoritydecision(twoofthree)involvingathirdinvestigator(FBM).
Studieswereincludediftheymetthefollowinginclu-sioncriteria:(1)haveacontrolgroup;(2)assesstheef-fectofapharmacologicalinterventiononbeneficialand/orharmfuloutcomes;(3)expressatleastoneresultingeffectbymeansoftheNNT.
Studiesassessingmedicalinterventionsotherthanpharmacologicalinterventions(e.
g.
,surgicaltechniques,dietaryinterventions,lifestylemodifications)werenotincluded.
DataextractionGeneralcharacteristicsofincludedstudiesDataelementsextractedtodescribegeneralstudychar-acteristicsincluded:(1)studyreference(authorsandjournalname);(2)yearofpublication;(3)country(deter-minedbythefirstauthor'saffiliation);(4)studydesign;(5)numberofincludedstudies(forsystematicreviewsandmeta-analyses);(6)numberofparticipants;(7)studyduration(i.
e.
,lengthofparticipants'follow-upinlongi-tudinalstudies);(8)disease/conditionofthestudiedpopulation;(9)pharmacologicalinterventions(includingcomparators);(10)primaryoutcome(includingitsclassi-ficationasanefficacyand/orsafetyoutcome).
Diseases/conditionswereclassifiedusingtheMedicalDictionaryMendesetal.
BMCMedicine(2017)15:112Page2of13forRegulatoryActivities(MedDRA),v.
18.
0,accordingtotheSystemOrganClass(SOC)[39].
CharacteristicsofNNTsinincludedstudiesDatawerecollectedfromincludedstudiestodescribeandcharacterizeNNTsaswellastoallowforfurtheras-sessmentofcalculatingmethods,accordingtoalistofpre-definedqueries(Additionalfile1:TableS3andTableS4).
WhenthemethodologyusedtocalculateNNTswasnotdescribedinthemethodssectionoftheincludedstudies,informationfromtheresultsorthedis-cussionsections,namelystatementsgiveninthetext,wasusedtoidentifythecalculatingmethods.
MethodsrecommendedtocalculateNNTMethodologicalrecommendationsAsummaryofbasicandgeneralrecommendationswassetupbasedupontheevidencereportedintheCochraneHandbookforSystematicReviewsofInterven-tions[31],inathoroughreviewperformedbyBenderaboutmethodstoobtainNNTsfordifferentstudyde-signs[25],andalsoinanotherreviewthatfocusedonobservationalstudies[21].
Inaddition,alimited,non-systematicliteraturesearchwasperformedinPubMedtoidentifypaperslaterpublishedthatcouldcomplementthisevidence(Additionalfile1:TableS5).
Systematicreviewandmeta-analysisTheNNTshouldbecalculatedbasedupontheuseofarelativeeffectbe-causerelativeeffectstendtobemorestableacrossriskgroupsthanabsolutedifferences[19,31,40,41].
TheRRandOR,obtainedwithinfixedorrandomeffectsregres-sionmodels,appeartobereasonablyconstantacrossdifferentbaselinerisks[19].
ThepooledRRorORcanbeusedtocalculateindividualizedNNTsfordifferentbaselinerisks(i.
e.
,π0theriskcontrolgroup),usingfor-mulas(1)or(2)[19,25,31].
Further,expressingRRorORasavarietyofNNTsacrossarangeofdifferentbase-lineriskshasbeenrecommended[18,31,36].
NNT11RRπ0;forRRNNT1RR1π0;forRR>11NNT11ORπ0OR1OR1π0;forORNNT1OR1π0OROR11π0;forOR>12RandomizedcontrolledtrialsInRCTswithabinaryoutcomeandadefinedperiodoftimeduringwhichallpatientsarefollowed,theNNTisestimatedbasedupontheuseofsimpleproportionsofpatientswiththeout-come(i.
e.
,π0theriskcontrolgroupandπ1theriskintreatmentgroup),accordingtoformula(3)[1,2]:NNT1π1π01RD3InRCTswithtime-to-eventoutcomes,thetimeoffollow-upisnotequalforallpatients.
Simplepropor-tionsshouldnotbeusedtoestimateNNTsbecausetheydonotaccountforvaryingfollow-uptimes[25,29].
Insuchstudies,theKaplan-Meierapproachcanbeusedtoestimateproportionsofpatientswiththeoutcomeofinterestovertime[26].
TheNNTcanthenbecalculatedbyinvertingtheRDbetweencumulativeincidences(i.
e.
,survivalprobabilitiesS1(t)fortreatmentgroupsandS0(t)forcontrolgroup)atagivenpointoftime(t),asshowninformula(4)[26]:NNT1S1tS0t4Further,thehazardratio(HR),estimatedbymeansoftheCoxregressionmodel,canbeusedtoestimatetheNNTiftheassumptionofproportionalhazardsisfulfilledandS0(t)isavailable,asdescribedinformula(5)[26]:NNT1S0tHRS0t5ObservationalstudiesDuetothelackofrandomization,theestimationoftreatmenteffectsinobservationalstud-iesrequiresadjustmentforconfoundingfactors[22].
Regression-basedmethods,namelymultiplelogisticre-gression,orpropensityscoremethodscanbeperformedtoestimateadjustedrelativeeffects[21].
TheNNTshouldalsobeadjustedandnotbasedoncruderiskdif-ferenceswithoutadjustment[22].
Case–controlstudiesIncase–controlstudies,multiplelogisticregressionisusuallyperformedtoestimateadjustedORasarelativeeffectmeasure[22,23].
TheNNTcanbecalculatedbycombiningtheadjustedORwiththeriskincontrolorunexposedgroup(usuallycalledtheunexposedeventrate[UER])[22,27].
Incase–controlstudiestheUERisobtainedfromanexter-nalsource(forexample,controlsinRCTsorunexposedsubjectsincohortstudies)[27].
Formula(2),whereπ0=UER,shouldbeusedtocalculateadjustedNNTfromadjustedOR.
IftherelativeeffectmeasureisadjustedRR,thenformula(1)shouldbeapplied.
CohortstudiesIncohortstudiesusingregression-basedmethods,twogeneralapproachescanbeusedtoestimateMendesetal.
BMCMedicine(2017)15:112Page3of13NNT.
ThefirstapproachisbasedupontheuseofadjustedOR,estimatedbymeansofmultiplelogisticregression[22].
AdjustedNNTisobtainedwiththeapplicationofadjustedORtoUER,asdescribedinformula(2).
However,thisap-proachshouldonlybeusedifthereisasmallvariationoftherisksaroundthemean[23].
Themeanriskofunex-posedsubjects(UER),whichisestimatedbymeansofthelogisticregressionmodel,canbeusedtocalculateadjustedNNTforthecorrespondingconfounderprofile.
AnothermethodthatcanbeusedistocalculateNNTforsomefixedconfounderprofiles[22].
Inthesecondapproach,NNTiscalculatedbytakingthereciprocaloftheaverageRDovertheobservedconfoundervalues,estimatedbymeansofmultiplelogisticregression[23].
Ingeneral,theapproachbasedupontheaverageRDshouldbeapplied[23].
Fortime-to-eventoutcomes,NNTcanbeestimatedasthereciprocalofthedifferencebetweentwomarginalprobabilities,withinagivendurationoffollow-up,usinganadjustedsurvivalmodel(e.
g.
,theCoxproportionalhazardsregressionmodel)[21,42–44].
Incohortstudiesusingpropensityscoremethods,NNTcanbeestimatedbyinvertingRD,whichisdirectlyestimatedbycomparingtheprobabilityoftheoutcomebetweentreatedanduntreatedsubjectsinthematchedsampleinpropensityscorematching[21].
Iftheout-comeistimetoevent,NNTisgivenbythereciprocalofthedifferenceestimatedfromKaplan-Meiersurvivalcurvesintreatedanduntreatedsubjectswithinagivendurationoffollow-up[21].
AdherencetomethodologicalrecommendationsThemethodsusedtocalculateNNTsinstudiesfrommedicaljournalswerecomparedtobasicmethodologicalrecommendations.
Theadherenceofcalculatingmethodstomethodologicalrecommendationswasassessed,consid-eringthestudydesignandthetypeofvariableusedtomeasureoutcomesofinterest.
DataanalysisDatawereanalyzedusingdescriptivestatistics.
Dataana-lyseswereperformedusingMicrosoftExcel2013.
ResultsFigure1presentsthesearchstrategyflowchart.
From138publications,51wereselectedafterexcludingstud-iesnotfulfillingtheinclusioncriteria.
Table1presentsasummaryofthemaincharacteristicsofincludedstudies,namelythecharacteristicsofvari-ablesandeffectmeasuresusedtoassesseffectsofinter-ventionsandthecompletenessofdataaroundNNTestimates.
AdetaileddescriptionofthecharacteristicsofeachstudyisprovidedinAdditionalfile1:TableS6.
GeneralcharacteristicsofincludedstudiesThemajorityofstudiesreportingNNTswereidentifiedfromtheJournaloftheAmericanMedicalAssociation(JAMA,n=17,33.
3%)andTheLancet(n=14,27.
5%)(Additionalfile1:TableS7).
Themediannumberofpapersperyearwas5.
5(rangingfrom1in2009to7in2011,2012,and2014).
TheincludedstudiesweremorefrequentlyauthoredbyresearchersfromtheUSA(n=21,41.
2%),UK(n=6,11.
8%),andCanada(n=6,11.
8%).
Twenty-threepublications(45.
1%)weresystematicre-viewsandmeta-analyses,while17wereindividualRCTs(33.
3%),9cohortstudies(17.
6%),and2case–controlstudies(3.
9%).
Themorefrequentlystudieddiseases/conditionswere"infectionsandinfestations"(n=7,13.
7%),"cardiacdisorders"(n=7,13.
7%),and"psychi-atricdisorders(n=7,13.
7%).
Theprimaryoutcomesofmoststudiesassessedonlyefficacy(n=30,58.
8%)ofinterventions.
Safetywasassessedasthesoleprimaryoutcomein11studies(21.
6%).
Theremaining10studies(19.
6%)assessedbothefficacyandsafetyasaprimaryoutcome.
Theprimaryoutcomewasbinaryin41studies(80.
4%)andtimetoeventin10studies(19.
6%).
InadditiontoNNTestimates,themajorityofstudies(n=42,82.
4%)alsousedrelativeeffectmeasurestoex-presstreatmentdifferences.
TheRR(n=18,35.
3%)andOR(n=16,31.
4%)werethemostcommonlyused.
CharacteristicsofNNTsinincludedstudiesNNTswereestimatedonlyforprimaryoutcomesin28studies(54.
9%),forprimaryandalsosecondaryout-comesin21studies(41.
2%),andonlyforsecondaryout-comesin2studies(3.
9%).
NNTswereusedtoassessonlybenefitsofinterventionsin26studies(51.
0%),bothbenefitsandharmsin14studies(27.
5%),andonlyharmsin11studies(21.
6%).
ThetypeofNNTpresentedinmoststudieswasaperson-basedNNT(n=40,78.
4%).
Aperson-time-basedNNTwaspresentedin11studies(21.
6%).
Thecompletenessofdatapresentedaroundthepoint-estimateNNTwasassessed.
Thebaselinerisk(i.
e.
,CER)waspresentedin37studies(72.
5%),adefinedtimehori-zonin38studies(74.
5%),andCIsin32studies(62.
7%).
AssessmentofmethodsusedtocalculateNNTsMethodsusedtocalculateNNTsinincludedstudieswerecomparedtobasicmethodologicalrecommendations(Table2).
AdetaileddescriptionofdatausedtoassessthecompletenessofinformationandtheappropriatenessofmethodsusedtocomputeNNTsinincludedstudiesisavailableinAdditionalfile1:TableS8.
ThemethodologyusedtocalculateNNTwasclearlydefinedinthemethodssectionofthepublicationsin28Mendesetal.
BMCMedicine(2017)15:112Page4of13studies(54.
9%).
Themethodologywasnotpresentedinthemethodssectionoftheremaining23studies(45.
1%),butitcouldbeidentifiedusinginformationfromothersectionsofthepublications.
Overall,basicmethodologicalrecommendationswerefollowedtocalculateNNTin36studies(70.
6%).
Asum-maryofthecharacteristicsofstudiesthatdidnotfollowbasicmethodologicalrecommendations(n=15,29.
4%)isprovidedinTable3.
NNTwascalculatedastheinverseoftheRDbetweengroupsin39studies(76.
5%)(13meta-analyses,17RCTs,and9cohortstudies).
Ofthosestudies,17usedsimpleproportions,12usedpooledRDs,4usedaverageRDs,and6usedcumulativeincidencerates.
Simplepro-portionswerecorrectlyusedin14studies(13RCTsand1cohortstudy)andinappropriatelyusedin3studies(1meta-analysis,1RCT,and1cohortstudy).
PooledRDswerealwaysinadequatetothestudyde-sign(12meta-analyses).
TheaverageRDmethodwasconsideredtohavebeencorrectlyusedinall4stud-ies(4cohortstudies).
Cumulativeincidencerateswereadequatelyusedinall6studies(3cohortstud-iesand3RCTs).
Theresultofarelativeeffectmeasure(e.
g.
,OR,RR)wasappliedtoaCERtocalculateNNTin12studies(23.
5%)(10meta-analysesand2case–controlstudies).
Theuseofthismethodologyinthosestudieswasinlinewithbasicmethodologicalrecommendations.
DiscussionThepresentstudyprovidesanoverviewabouttheuseoftheNNTinmedicalresearchduringthelastdecade.
Theadherenceofselectedstudiestobasicmethodo-logicalrecommendationswasreviewed.
ThistopicisparticularlyrelevantgiventhattheNNTconcepthasbeenextendedtoderiverelatedmetricswithpotentialforuseinbenefit-riskassessments,namelyforclinicaldecisionmakingordrugregulatorypurposes.
Anex-ampleisprovidedbyimpactnumbers,whichgiveapopulationperspectivetotheNNT[45,46].
Impactnumbersareusefultodescribethepublichealthburdenofadiseaseandthepotentialimpactofatreatment[6].
Twomeasuresofimpactnumbersareparticularlyinter-esting:thenumberofeventspreventedinthepopulation(NEPP)andthepopulationimpactnumberofeliminat-ingariskfactorovertimet(PIN-ER-t)[6,47,48].
CliniciansandotherinvestigatorsshouldbeawarethatthecalculationandinterpretationofNNTsdependonspecificstudycharacteristics,particularlythedesignandoutcomevariables.
Theuseofinadequatecalculatingmethodsmayleadtobiasedresultsandmisleadingcon-clusions[22,29,35,49].
Fig.
1FlowofstudiesthroughthereviewprocessMendesetal.
BMCMedicine(2017)15:112Page5of13Table1Characteristicsoftheincludedstudiesandofthenumberneededtotreat(NNT)CharacteristicsMeta-analysis(n=23)RCT(n=17)Cohort(n=9)Nestedcase–control(n=2)Overall(n=51)JournalJAMA9(39.
1%)4(23.
5%)2(22.
2%)2(100.
0%)17(33.
3%)Lancet6(26.
1%)7(41.
2%)1(11.
1%)0(0.
0%)14(27.
5%)AmJMed2(8.
7%)0(0.
0%)2(22.
2%)0(0.
0%)4(7.
8%)Other6(26.
1%)6(35.
3%)4(44.
4%)0(0.
0%)16(31.
4%)CountryUSA13(56.
5%)2(11.
8%)6(66.
7%)0(0.
0%)21(41.
2%)UK4(17.
4%)2(11.
8%)0(0.
0%)0(0.
0%)6(11.
8%)Canada1(4.
3%)2(11.
8%)1(11.
1%)2(100.
0%)6(11.
8%)Other5(21.
7%)11(64.
7%)2(22.
2%)0(0.
0%)18(35.
3%)Disease/conditionInfectionsandinfestations4(17.
4%)2(11.
8%)1(11.
1%)0(0.
0%)7(13.
7%)Cardiacdisorders3(13.
0%)3(17.
6%)1(11.
1%)0(0.
0%)7(13.
7%)Psychiatricdisorders4(17.
4%)3(17.
6%)0(0.
0%)0(0.
0%)7(13.
7%)Other12(52.
2%)9(52.
9%)7(77.
8%)2(100.
0%)30(58.
8%)PrimaryoutcomeofstudyEfficacy12(52.
2%)16(94.
1%)2(22.
2%)0(0.
0%)30(58.
8%)Safety2(8.
7%)1(5.
9%)6(66.
7%)2(100.
0%)11(21.
6%)Efficacyandsafety9(39.
1%)0(0.
0%)1(11.
1%)0(0.
0%)10(19.
6%)Typeofvariable(primaryoutcome)Binary22(95.
7%)a13(76.
5%)5(55.
6%)1(50.
0%)41(80.
4%)Timetoevent1(4.
3%)4(23.
5%)4(44.
4%)1(50.
0%)10(19.
6%)RelativeeffectmeasureYesRelativerisk11(47.
8%)b5(29.
4%)2(22.
2%)0(0.
0%)18(35.
3%)aOddsratio9(39.
1%)b4(23.
5%)2(22.
2%)1(50.
0%)16(31.
4%)aHazardratio1(4.
3%)3(17.
6%)3(33.
3%)0(0.
0%)7(13.
7%)Rateratio0(0.
0%)0(0.
0%)1(11.
1%)1(50.
0%)2(3.
9%)No3(13.
0%)5(29.
4%)1(11.
1%)0(0.
0%)9(17.
6%)OutcomeexpressedwithNNTPrimaryoutcome6(26.
1%)14(82.
4%)7(77.
8%)1(50.
0%)28(54.
9%)Secondaryoutcome0(0.
0%)2(11.
8%)0(0.
0%)0(0.
0%)2(3.
9%)Primaryandsecondaryoutcomes17(73.
9%)1(5.
9%)2(22.
2%)1(50.
0%)21(41.
2%)NNTforbenefitorharmBenefit8(34.
8%)15(88.
2%)3(33.
3%)0(0.
0%)26(51.
0%)Harm2(8.
7%)1(5.
9%)6(66.
7%)2(100.
0%)11(21.
6%)Benefitandharm13(56.
5%)1(5.
9%)0(0.
0%)0(0.
0%)14(27.
5%)TypeofNNTcalculatedinthestudyPerson-basedNNT21(91.
3%)a13(76.
5%)5(55.
6%)1(50.
0%)40(78.
4%)Person-time-basedNNT2(8.
7%)4(23.
5%)4(44.
4%)1(50.
0%)10(21.
6%)CompletenessofNNTestimateControleventrateYes13(56.
5%)17(100.
0%)6(66.
7%)1(50.
0%)37(72.
5%)No10(43.
5%)0(0.
0%)3(33.
3%)1(50.
0%)14(27.
5%)Mendesetal.
BMCMedicine(2017)15:112Page6of13Themajorityofstudiesincludedinthepresentreviewaimedatassessingprimarilyonlytheefficacyofmedicalinterventions.
TheNNTwasusedmoreoftentoassessonlybenefits(51.
9%)ratherthanonlyharms(21.
2%).
Thisfindingwasexpected,consideringwhatiscom-monlyseeninthemedicalliterature.
Aprevioussystem-aticreviewincludingmeta-analysespublishedovera5-yearperiodfoundthatonly14%ofstudieswerede-signedtoinvestigatedrugsafetyasprimaryoutcome[38].
Inanotherstudycomprisingsystematicreviewswithabsoluteeffectestimates,itwasfoundthattheNNTwasmostlyusedtoassessbeneficialoutcomesratherthanharmfulevents[14].
Overall,includedstudiesreportedmorefrequentlyre-sultsforbinaryoutcomesthanfortime-to-eventoutcomes.
Thisfindingcontrastswiththeresultsofapreviousreviewinwhichnearly55%ofincludedstudiesreportedNNTsfortime-to-eventoutcomes[12].
How-ever,thatreviewincludedonlyRCTs[12],whilethepresentstudyincludedseveralresearchdesigns.
Relativemeasuresofeffectwereusedtoexpresstreat-mentdifferencesinthemajorityofincludedstudies(82.
4%).
Thesefindingsareinlinewiththeconclusionsofarecentsurveyof202systematicreviews[14].
Ofthose,themajorityincludedmeta-analyseswithestima-tionofrelativeeffects(92.
1%),whileabsoluteeffectesti-mateswereprovidedin36.
1%[14].
Aspreviouslymentioned,theconceptofNNTre-quiresthedescriptionofadefinedperiodoftimeandvarieswithbaselinerisk(alsocalledCER).
Nevertheless,thetimehorizonwaslackinginmorethanonefourth(25.
5%)ofstudies.
TheNNTisuninterpretableifthetimeoffollow-upduringwhichcumulativeoutcomein-cidencesaremeasuredisnotprovided[34].
Inaddition,baselineriskscouldnotbeascertainedinnearly28%ofstudies.
Previousfindingsindicatethat56.
2%ofstudiesreportingabsoluterisksdonotpresentthesourceofbaselineriskestimates[14].
Lastly,morethanonethird(37.
3%)ofstudiesincludedinthepresentreviewdidnotreporttheCIforthepoint-estimateNNT.
Thisresultisinlinewithpreviousfindings[12].
Thus,amoderatelyhighproportionofpaperspublishedinjournalswithhighimpactfactorinthecategoryof"Generaland/orInternalMedicine"misusetheNNTmetric.
Asseenacrossthearticlesreviewedhere,severalap-proacheshavebeenusedtoderiveNNTsfrommeta-analyses.
However,in13outof23meta-analyses(56.
5%)theapproachwasconsideredinadequate.
Ofthesemeta-analyses,onecalculatedthereciprocalofsimplepropor-tions(usingtotalnumbersofbothpatientswithoutcomeandexposedpatientscomingfromallincludedstudies).
Usingsimpleproportions,i.
e.
,treatingthedataasiftheyallcomefromasingletrial,tocalculateNNTsisnotcorrect,asthismethodispronetobiasduetoSimpson'sparadox[35,50].
Theother12meta-analysesinvertedpooledRDs,butthismethodshouldalsobeavoided[19,31,36,51].
AbsoluteRDsareusuallynotconstantandhomogeneousacrossdifferentbaselineeventrates;therefore,theyarerarelyappropriateforcal-culatingNNTsfrommeta-analyses[19,31,36,51].
Moreover,theeffectsofseculartrendsondiseaseriskandtimehorizonprecludetheuseofpooledRDs,astheycanresultinmisleadingNNTs[36,51].
Relativeef-fectmeasures(suchasRRandOR)areusuallymorestableacrossriskgroupsthanareabsolutedifferences.
Thus,pooledestimatesofrelativeeffectmeasuresshouldbeusedratherthanabsoluteRDstoderiveNNTsfrommeta-analyses[19,31,36].
CliniciansshouldpreferablyusefixedeffectsOR,randomeffectsORorRR,andthepatientexpectedeventrate(PEER)toindividualizeNNTwhenapplyingresultsfrommeta-analysesinclinicalpractice[4,19].
MostRCTs(94.
1%)followedbasicmethodologicalrec-ommendationstocalculateNNTs.
ItisnoteworthythatthemajorityofincludedRCTs(13outof17)analyzedbinaryoutcomes.
Studieswithfixedtimesoffollow-upareusuallynotpronetomiscalculationofNNTbecausecumulativeincidencesequalsimpleproportionsatthestudyend[29].
However,previousstudiessuggestedthatNNTsaremiscalculatedinatleasthalfofRCTswithtime-to-eventoutcomes[12,29].
Inthepresentreview,oneoutfourRCTswithvaryingfollow-uptimesappliedanon-recommendedmethodtoTable1Characteristicsoftheincludedstudiesandofthenumberneededtotreat(NNT)(Continued)TimehorizonYes10(43.
5%)17(100.
0%)9(100.
0%)2(100.
0%)37(72.
5%)No13(56.
5%)0(0.
0%)0(0.
0%)0(0.
0%)14(27.
5%)ConfidenceintervalsYes16(65.
2%)c8(47.
1%)8(88.
9%)1(50.
0%)32(62.
7%)No8(34.
8%)9(52.
9%)1(11.
1%)1(50.
0%)19(37.
3%)aThevariablefortheprimaryoutcomeofonemeta-analysisisbinary,andpooledOR(95%CI)wascalculated.
However,aperson-time-basedNNTwascalculatedbytakingthereciprocalofRDbetweenpooledeventratesper1000patient-years(Preiss2011)bOnestudyreportedrelativerisk(RR)andoddsratio(OR)(Maheretal.
2011)cConfidenceintervalwasprovidedwithNNTonlyfortheprimaryoutcomeinastudyreportingNNTforseveraloutcomes(Greenetal.
2007)Mendesetal.
BMCMedicine(2017)15:112Page7of13Table2Assessmentofmethodologyusedtocalculatenumberneededtotreat(NNT)inincludedstudiesMeta-analysis(n=23)RCT(n=17)Cohort(n=9)Nestedcase–control(n=2)Overall(n=51)MethodologyusedtocalculateNNTisdefinedinthemethodssectionofthestudyYes1982.
6%)0(0.
0%)7(77.
8%)2(100.
0%)28(54.
9%)No417.
4%)17(100.
0%)2(22.
2%)0(0.
0%)23(45.
1%)GeneralcharacteristicsofthemethodologyusedtocalculateNNTinthestudyReciprocalofriskdifferenceSimpleproportions1(4.
3%)14(82.
4%)2(22.
2%)0(0.
0%)17(33.
3%)CumulativeIR0(0.
0%)3(17.
6%)3(33.
3%)0(0.
0%)6(11.
8%)PooledRD12(52.
2%)0(0.
0%)0(0.
0%)0(0.
0%)12(23.
15)AverageRD0(0.
0%)0(0.
0%)4(44.
4%)0(0.
0%)4(7.
8%)Relativeeffectmeasure10(43.
5%)0(0.
0%)0(0.
0%)2(100.
0%)12(23.
5%)MethodologyusedtocalculateNNTisinlinewithbasicrecommendations(overall)Yes10(43.
5%)16(94.
1%)8(88.
9%)2(100.
0%)37(70.
6%)No13(56.
5%)1(5.
9%)1(11.
1%)0(0.
0%)15(29.
4%)MethodologyusedtocalculateNNTisinlinewithbasicrecommendations(detailed)BinaryvariablesYes9(39.
1%)13(76.
5%)5(55.
6%)1(50.
0%)28(54.
9%)ReciprocalofriskdifferenceSimpleproportions0(0.
0%)13(76.
5%)1(11.
1%)0(0.
0%)14(27.
5%)CumulativeIR0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)PooledRD0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)AverageRD0(0.
0%)0(0.
0%)4(44.
4%)0(0.
0%)4(7.
8%)Relativeeffectmeasure9(39.
1%)0(0.
0%)0(0.
0%)1(50.
0%)10(19.
6%)No13(56.
5%)0(0.
0%)0(0.
0%)0(0.
0%)13(25.
5%)ReciprocalofriskdifferenceSimpleproportions1(4.
3%)0(0.
0%)0(0.
0%)0(0.
0%)1(2.
0%)CumulativeIR0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)PooledRD12(52.
2%)0(0.
0%)0(0.
0%)0(0.
0%)12(23.
5%)AverageRD0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)Relativeeffectmeasure0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)Time-to-eventvariablesYes1(4.
3%)3(17.
6%)3(33.
3%)1(50.
0%)8(15.
7%)ReciprocalofriskdifferenceSimpleproportions0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)CumulativeIR0(0.
0%)3(17.
6%)3(33.
3%)0(0.
0%)6(11.
8%)PooledRD0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)AverageRD0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)Relativeeffectmeasure1(4.
3%)0(0.
0%)0(0.
0%)1(50.
0%)2(3.
9%)No0(0.
0%)1(5.
9%)1(11.
1%)0(0.
0%)2(3.
9%)ReciprocalofriskdifferenceSimpleproportions0(0.
0%)1(5.
9%)1(11.
1%)0(0.
0%)2(3.
9%)CumulativeIR0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)PooledRD0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)AverageRD0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)Relativeeffectmeasure0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)0(0.
0%)IRincidencerate,RCTrandomizedcontrolledtrial,RDriskdifferenceMendesetal.
BMCMedicine(2017)15:112Page8of13Table3Characteristicsoftheincludedstudiesinwhichbasicrecommendationswerenotfollowedtocalculatethenumberneededtotreat(NNT)StudyVariableBaselineriskTimehorizonConfidenceintervalMethodologyusedtocomputeNNTdefinedinmethodssectionMethodusedtocomputeNNTSourceofdatausedtocomputeNNTCommentsSystematicreviewandmeta-analysisJonas2014BinaryNoNoYesYesNNT=1/RDPooledRDApooledRDwascalculatedfortwooutcomes.
Durationofincludedtrialsrangedfrom12to52weeksfortheoutcomeanydrinking,andfrom12to24weeksforheavingdrinkingHempel2012BinaryNoNoYesYesNNT=1/RDPooledRDThepooledRD(obtainedfrommeta-analysis)ledtoalossoffollow-uptime.
Mosttrialseitherdidnotspecifythefollow-upperiod,ortheassessmentwasexplicitlylimitedtothetimeofantibioticstreatmentLeucht2012BinaryYesYesYesYesNNT=1/RDPooledRDTheoutcomeisassessedbetween7and12monthsoffollow-up;ameanstudydurationisindicatedforeachoutcomewithNNTcalculatedfromabsoluteRDpooledfromthemeta-analysisShah2012BinaryNoNoYesYesNNT=1/RDPooledRDThestudycomprehendsthecalculationandcomparisonofNNTforseveraltreatments.
However,NNTsarenotcomparablebecausetheywerecalculatedfrompooledRDsandtimesoffollow-upvaryconsiderablyacrossstudiesincludedinthemeta-analysis(10daysto48weeks)Preiss2011BinaryYesYesNoNoNNT=1/RDPooledRDThevariablefortheprimaryoutcomeofthestudyisbinary,andpooledOR(95%CI)wascalculated.
However,NNTwascalculatedbytakingthereciprocalofRDbetweenpooledeventratesper1000patient-years.
Person-time-basedNNTwaspresentedandinterpretedasthenumberofpersonsneededtotreatover1yearShamliyan2011BinaryYesNoYesYesNNT=1/RDPooledRDSeveralantiviraltreatmentswerecomparedbasedonestimatesofNNT.
However,studieswithdifferenttimesoffollow-upforantiviraltreatmentswereusedtopoolabsoluteRD.
ThetimehorizonfactorislostCoker2010BinaryYesYesYesNoNNT=1/RDPooledRDThepooledRDwasobtainedfora14dayfollow-updurationinallstudiesincludedinthemeta-analysis.
However,RDvariesconsiderablyacrossthestudiesincludedinthemeta-analysis(rangingfrom8%to27%)Testa2008BinaryNoNoYesYesNNT=1/RDPooledRDPooledRDwasusedtocalculateNNT.
Thefollow-upofincludedstudiesrangedfrom"inhospital"to6monthsBridge2007BinaryYesNoYesYesNNT=1/RDPooledRDDerSimonianandLairdrandom-effectsmodelwasusedtoobtainapooledestimateoftheRD(95%CI).
NNTwascalculatedasthereciprocalofRD.
Thedurationoffollow-upandthebaselineriskvariedconsiderablyacrossincludedstudiesDentali2007BinaryYesNoNoYesNNT=1/RDSimpleproportionsRawtotalsofpatientsfromeachstudywereaddedtogethertoestimateproportionsandcalculateRD,i.
e.
,treatingdataasifallwerefromonestudy(Simpson'sparadox).
Further,thebaselineriskrangedconsiderablyacrossincludedstudies(e.
g.
,0.
2–4.
0%forpulmonaryembolism)Mendesetal.
BMCMedicine(2017)15:112Page9of13Table3Characteristicsoftheincludedstudiesinwhichbasicrecommendationswerenotfollowedtocalculatethenumberneededtotreat(NNT)(Continued)Rovers2006BinaryYesYesNoNoNNT=1/RDPooledRDAlthoughitisnotclearlystatedinthemethodssection,thediscussionofthestudysuggeststhattheauthorscalculatedpooledRDbymeansofthemeta-analysisBongartz2006BinaryNoYesYesYesNNT=1/RDPooledRDNNTcalculatedfortreatmentperiodsof6–12monthsand3–12months,usingMantel-HaenszelfixedestimateofabsoluteRDincasesinwhichanORofatleast1.
5wasdetectedSpiegel2006BinaryNoNoNoYesNNT=1/RDPooledRDApooledRDwascalculatedfortwocomparisons.
Durationofincludedtrialsrangedfrom6to78weeksforonecomparisonandfrom12to24weeksforanothercomparisonRandomizedcontrolledtrialShepherd2008TimetoeventYesYesNoNoNNT=1/RDSimpleproportionsNNTcalculatedas1/RDusingfinalratesofeventandcitingamediantimeoffollow-upof4.
8years(NNT=14inpatientswithdiabetesandchronickidneydisease).
However,aKaplan-Meiercurveisprovidedinthestudy,whichshouldhavebeenused(sincethemedianfollow-upislowerthanthe5-yearsobjective,atleastsomepatientsdidnotcompletethefollow-up).
FromtheKaplan-Meiercurve,wewouldhave20.
3%and14.
0%patientswiththeoutcomeintheatorvastatin10mgand80mg/day,respectively,at4.
8yearsoffollow-upandanNNT=15.
8RetrospectivecohortstudyGraham2010TimetoeventYesYesYesYesNNT=1/RDSimpleproportionsNNTwascalculatedusingRDbetweenunadjustedincidencerates.
AdjustedincidenceratesfromtheKaplan-Meiercurvesshouldhavebeenused.
Forexample,at1yearoffollow-up,NNTforthecompositeendpointwouldbe92fromKaplan-Meiercurves,ratherthan60person-yearsfromunadjustedincidencerates.
Theauthorsinterpretedperson-yearsasnumberofpersonstreatedover1year,whichisnotexactlythesameNNTnumberneededtotreat,ORoddsratio,RDriskdifferenceMendesetal.
BMCMedicine(2017)15:112Page10of13calculateNNT(see,e.
g.
,[52]).
InthatRCT,theeffectoftwodosesofatorvastatin(80mgor10mgdaily)wastested,forthefirstoccurrenceofamajorcardiovascularevent(i.
e.
,atime-to-eventoutcome),inpatientswithcoronaryarterydisease(CAD)andtype2diabetes,withandwithoutchronickidneydisease[52].
Patientswerefollowedforvaryingtimes(median,4.
8years).
AlthoughKaplan-Meiercurveshavebeenestimated,theauthorsusedsimpleproportionsofpatientswiththeoutcometocomputeNNT(e.
g.
,forpatientswithdiabeteswithoutCAD,1/([62/441]–[57/444])=82)andconcludedthat82patientswereneededtotreatwith80mg/dayversus10mg/daytopreventonemajorcardiovasculareventover4.
8years[52].
Usingthecumulativeincidencespro-videdinKaplan-Meiercurves(12.
5%for80mgand13.
3%for10mg),NNTwouldhavebeenestimatedat125overthesametimehorizon.
Thisexampleillustrateshowtheuseofsimpleproportionscanleadtomislead-ingvaluesofNNT.
Simpleproportionsshouldbeusedonlyifallpatientsarefollowedfortheentirestudyperiod,astheyequalcumulativeincidencesestimatedbytheKaplan-Meierapproach[30].
Sincefollow-uptimesusuallyvaryinRCTs,simpleproportionsarenotvalidestimatesofcumulativeincidences.
Incaseswherefollow-upisshortandmostlycomplete,simplepro-portionsandKaplan-Meierincidencesarealmostsimilar[30].
Asthepresentstudyassessedresultsfromresearchpublishedsince2006,twodifferentmethodologieswereconsideredadequateforcalculatingNNTfromRCTswheretheoutcomeistimetoanevent[26,53,54].
Morerecently,however,theauthorsofastudycomparingtheriskdifferenceapproach(reciprocalofriskdifferencesestimatedbysurvivaltimemethods)andtheincidencedifferenceapproach(reciprocalofincidenceratesdiffer-ences)concludedthatthemethodsbasedonincidenceratesoftenleadtomisleadingNNTestimatesandrec-ommendedtheuseofsurvivaltimemethodstoestimateNNTsinRCTswithtime-to-eventoutcomes[28].
Theincidencedifferenceapproachstillcanbeusedinthecaseofsmallbaselinerisks,strongtreatmenteffects,andexponentiallydistributedsurvivaltimes[28].
Neverthe-less,Girerdetal.
arguedthatthetwomethodsmeasuredifferentthings,butbotharevalidandprovidecom-plementaryinformationregardingtheabsoluteeffectofanintervention,highlightingthattheincidencerateapproachassessesperson-yearsratherthanpersons[55].
Thiscalculatingmethodestimatesthenumberofperson-times(e.
g.
patient-years),nottheabsolutenumberofpersons,neededtoobserveoneless(oronemore)eventinthetreatmentgroupthaninthecontrolgroup[28,29,54–56].
Thisestimateisdiffer-entfromthe"classical"person-basedNNT,andthere-foremaybedifficulttointerpret[56].
Forexample,100patient-yearsdonotnecessarilymean100indi-vidualpatientstreatedover1year(or50patientstreatedfor2years).
Athoroughexplanationofper-son-basedNNT,person-time-basedNNT,andevent-basedNNT(formultiplerecurrentoutcomeevents)isprovidedelsewhere[29,57].
Withregardtoobservationalstudies,onecohortstudydidnotfollowmethodologicalrecommendations[58].
Inthatstudy,Kaplan-MeiercurvesandCoxproportionalHRsfortimetoevent,adjustedforconfoundingfactors,withpioglitazoneasreference,wereusedtotesttheef-fectofrosiglitazoneonseveralcardiovascularadverseevents[58].
However,theauthorsappliedunadjustedin-cidenceratedifferencestocalculateNNTs,insteadofusingadjusteddata.
Forexample,at1yearoffollow-up,theNNTforacompositecardiovascularendpointwouldbe92fromKaplan-Meiercurvesratherthanthe60person-yearsobtainedbytheauthors.
Further,theau-thorsinterpretedperson-yearsasnumberofpersonstreatedover1year,whichisnotexactlythesame.
Ade-tailedreviewanddiscussionofmethodsusedtocalcu-lateNNTsfromobservationalstudiesisprovidedelsewhere[21–23].
Thepresentstudywasnotprimarilyaimedatidentifyingallpaperswithmethodologicalrecom-mendationsforcalculatingNNTs.
Forthisreason,asystematicreviewofliteraturewasnotperformedtoidentifysuchpapers.
Thisisapotentiallimitationofthestudy.
Nevertheless,theliteratureusedasthesourceofevidencewasprobablyadequateforthecomplexityoftheassessment.
Thestudyfocusedontheadherenceofcalculatingmethodstobasicmeth-odologicalrecommendations,ratherthantomorecomplexmethodologicalandstatisticalissues.
There-fore,estimatesofNNTreportedbystudiesthatfollowedbasicmethodologicalrecommendationsarenotnecessarilycorrect.
Therearepossiblyotherrea-sonsthatcanstillleadtobiasedestimates,butwhichcouldnotbeassessedwithanacceptableef-fort.
Inaddition,themagnitudeoferrorproducedinstudiesthatdidnotfollowbasicmethodologicalrec-ommendationstocalculateNNTswasnottested.
Asidefromsomeexamplesprovidedinthediscus-sion,thecalculationofcorrectNNTswasnotsoughtforstudiesthatdidnotfollowrecommendations.
Lastly,thestudywaslimitedtothetop25high-impactfactorjournalsinthe"Generaland/orInternalMedicine"category.
Whetherornotthere-sultsinotherfieldsarelikelytoshowsimilarresultsdeservesfurthertesting.
Thepresentresultsillustratethatthesemetricshavenotalwaysbeenadequatelycalculated.
Fromtheclini-cians'pointofview,thismaycausesomeconcerns,sincethesemetricscanbeusedtosupportclinicalMendesetal.
BMCMedicine(2017)15:112Page11of13decision-makingprocesses,includingtheprescriptionofmedicines.
Therefore,cliniciansneedtorelyonthemethodologicalappropriatenessofsuchcalculations.
ConclusionsTheNNThelpstoquantifythemagnitudeofeffectsofmedicalinterventionsinanabsolutescale,thereforebringingaddedvaluetodecisionsondrugutilizationforclinicians,regulators,andotherstakeholders.
However,theyshouldbeawarethatthecalculationandinterpret-ationoftheNNTdependonthecharacteristicsofagivenstudy,namelythedesignandoutcomevariables.
Moreover,theymustacknowledgethatanNNTisspe-cifictoagivencomparison.
Therefore,baselinerisks,clearlydefinedoutcomes,timehorizons,andconfidenceintervalsshouldbeprovided.
ThepresentationofanNNTalone,i.
e.
,withoutitscontext,wouldbeambigu-ousandlessusefulfordecisionmaking.
Thisstudyshowedthat,althoughtheconceptofNNTwasintroducedseveralyearsago,therearebasicmeth-odologicalrecommendationsstillnotbeingfollowed,particularlyinmeta-analyses,leadingtomiscalculatedandmisinterpretedresults.
Furtherresearchisneededtoconfirmthepresentfindingsandtoexploretheinflu-enceofothermethodologicalaspectsthatmayimpactthecalculationoftheNNTinclinicalstudies.
AdditionalfileAdditionalfile1:TableS1.
Listofthe25journalsof"Generaland/orInternalMedicine"withhigherimpactfactorin2015.
TableS2.
Searchstrategyusedtoidentifystudiesreportingnumberneededtotreat(NNT),performedinPubMedon24August2016.
TableS3.
ListofqueriesusedtodescribeandcategorizeNNTinselectedstudies.
TableS4.
ListofqueriesusedtoassessmethodologiesusedtocalculateNNTinselectedstudies.
TableS5.
Searchstrategyusedtoidentifystudiesinvestigatingmethodsforcalculatingnumberneededtotreat(NNT),performedinPubMedon24August2016.
TableS6.
Maincharacteristicsofincludedstudies.
TableS7.
Numberofpublicationsreportingnumberneededtotreat(NNT)values,accordingtostudydesignandjournal.
TableS8.
DescriptionofdatausedtoassessthecompletenessofinformationandtheappropriatenessofmethodsusedtocomputeNNTsintheincludedstudies.
(DOCX78kb)AbbreviationsARR:Absoluteriskreduction;BMJ:BritishMedicalJournal;CAD:Coronaryarterydisease;CER:Controleventrate;CI:Confidenceinterval;CONSORT:ConsolidatedStandardsofReportingTrials;HR:Hazardratio;JAMA:JournaloftheAmericanMedicalAssociation;MedDRA:MedicalDictionaryforRegulatoryActivities;NEPP:Numberofeventspreventedinthepopulation;NNE:Numberneededtobeexposed;NNEB:Numberneededtobeexposedtobenefit;NNEH:Numberneededtobeexposedtobeharmed;NNT:Numberneededtotreat;NNTB:Numberneededtotreattobenefit;NNTH:Numberneededtotreattobeharmed;OR:Oddsratio;PEER:Patientexpectedeventrate;PIN-ER-t:Populationimpactnumberofeliminatingariskfactorovertimet;RCT:Randomizedcontrolledtrial;RD:Riskdifference;RR:Relativerisk;SOC:SystemOrganClass;UER:UnexposedeventrateAcknowledgementsNotapplicable.
FundingThisstudywasnotfinanciallysupportedbyanyinstitution.
AvailabilityofdataandmaterialsNotapplicable.
Authors'contributionsDMconceivedthestudy,collectedthedata,analyzedthedata,andwrotethepaper.
CAandFBMconceivedthestudy,analyzedthedata,andreviewedthepaper.
Allauthorsreadandapprovedthefinalmanuscript.
CompetinginterestsTheauthorsdeclarethattheyhavenocompetinginterests.
ConsentforpublicationNotapplicable.
EthicsapprovalandconsenttoparticipateNotapplicable.
Publisher'sNoteSpringerNatureremainsneutralwithregardtojurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations.
Received:22December2016Accepted:15May2017References1.
LaupacisA,SackettDL,RobertsRS.
Anassessmentofclinicallyusefulmeasuresoftheconsequencesoftreatment.
NEnglJMed.
1988;318(26):1728–33.
2.
CookD,SackettD.
Thenumberneededtotreat:aclinicallyusefulmeasureoftreatmenteffect.
BMJ.
1995;310:452–4.
3.
McAlisterFA.
The"numberneededtotreat"turns20—andcontinuestobeusedandmisused.
CMAJ.
2008;179(6):549–53.
4.
StrausSE,GlasziouP,RichardsonWS,HaynesRB.
Evidence-basedmedicine:howtopracticeandteachit.
London:ChurchillLivingstone;2011.
5.
CitromeL,KetterTA.
WhendoesadifferencemakeadifferenceInterpretationofnumberneededtotreat,numberneededtoharm,andlikelihoodtobehelpedorharmed.
IntJClinPract.
2013;67(5):407–11.
6.
Mt-IsaS,HallgreenCE,WangN,etal.
IMI-PROTECTbenefit-riskparticipants.
Balancingbenefitandriskofmedicines:asystematicreviewandclassificationofavailablemethodologies.
PharmacoepidemiolDrugSaf.
2014;23(7):667–78.
7.
MendesD,AlvesC,Batel-MarquesF.
Numberneededtoharminthepost-marketingsafetyevaluation:resultsforrosiglitazoneandpioglitazone.
PharmacoepidemiolDrugSaf.
2015;24(12):1259–70.
8.
MendesD,AlvesC,BatelMF.
Testingtheusefulnessofthenumberneededtotreattobeharmed(NNTH)inbenefit-riskevaluations:casestudywithmedicineswithdrawnfromtheEuropeanmarketduetosafetyreasons.
ExpertOpinDrugSaf.
2016;15(10):1301–12.
9.
AltmanDG,SchulzKF,MoherD,CONSORTGROUP(ConsolidatedStandardsofReportingTrials),etal.
TherevisedCONSORTstatementforreportingrandomizedtrials:explanationandelaboration.
AnnInternMed.
2001;134(8):663–94.
10.
MoherD,HopewellS,SchulzKF,etal.
CONSORT2010explanationandelaboration:updatedguidelinesforreportingparallelgrouprandomisedtrials.
BMJ.
2010;340:c869.
11.
BMJ.
ForAuthors,ResourcesforAuthors,ArticleTypes,Research.
bmj.
com.
http://www.
bmj.
com/about-bmj/resources-authors/article-types/research.
Accessed13Oct2016.
12.
HildebrandtM,VervlgyiE,BenderR.
CalculationofNNTsinRCTswithtime-to-eventoutcomes:aliteraturereview.
BMCMedResMethodol.
2009;9:21.
13.
NuovoJ,MelnikowJ,ChangD.
Reportingnumberneededtotreatandabsoluteriskreductioninrandomizedcontrolledtrials.
JAMA.
2002;287(21):2813–4.
14.
Alonso-CoelloP,Carrasco-LabraA,Brignardello-PetersenR,etal.
Systematicreviewsexperiencemajorlimitationsinreportingabsoluteeffects.
JClinEpidemiol.
2016;72:16–26.
15.
CitromeL.
Relativevs.
absolutemeasuresofbenefitandrisk:what'sthedifferenceActaPsychiatrScand.
2010;121(2):94–102.
Mendesetal.
BMCMedicine(2017)15:112Page12of1316.
KlawiterEC,CrossAH,NaismithRT.
Thepresentefficacyofmultiplesclerosistherapeutics:isthenew66%justtheold33%Neurology.
2009;73(12):984–90.
17.
MendesD,AlvesC,Batel-MarquesF.
Benefit-riskoftherapiesforrelapsing-remittingmultiplesclerosis:testingthenumberneededtotreattobenefit(NNTB),numberneededtotreattoharm(NNTH)andthelikelihoodtobehelpedorharmed(LHH):asystematicreviewandmeta-analysis.
CNSDrugs.
2016;30(10):909–29.
18.
McQuayHJ,MooreRA.
Usingnumericalresultsfromsystematicreviewsinclinicalpractice.
AnnInternMed.
1997;126(9):712–20.
19.
FurukawaTA,GuyattGH,GriffithLE.
Canweindividualizethe'numberneededtotreat'Anempiricalstudyofsummaryeffectmeasuresinmeta-analyses.
IntJEpidemiol.
2002;31:72–6.
20.
MooreRA,GavaghanDJ,EdwardsJE,etal.
Poolingdatafornumberneededtotreat:noproblemsforapples.
BMCMedResMethodol.
2002;2:2.
21.
AustinPC,LaupacisA.
Atutorialonmethodstoestimatingclinicallyandpolicy-meaningfulmeasuresoftreatmenteffectsinprospectiveobservationalstudies:areview.
IntJBiostat.
2011;7(1):6.
22.
BenderR,BlettnerM.
Calculatingthe"numberneededtobeexposed"withadjustmentforconfoundingvariablesinepidemiologicalstudies.
JClinEpidemiol.
2002;55(5):525–30.
23.
BenderR,KussO,HildebrandtM,GehrmannU.
EstimatingadjustedNNTmeasuresinlogisticregressionanalysis.
StatMed.
2007;26(30):5586–95.
24.
AltmanDG.
Confidenceintervalsforthenumberneededtotreat.
BMJ.
1998;317(7168):1309–12.
25.
BenderR.
Numberneededtotreat(NNT).
In:ArmitageP,ColtonT,editors.
Encyclopediaofbiostatistics.
Chichester:Wiley;2005.
p.
3752–61.
26.
AltmanDG,AndersenPK.
Calculatingthenumberneededtotreatfortrialswheretheoutcomeistimetoanevent.
BMJ.
1999;319(7223):1492–5.
27.
BjerreLM,LeLorierJ.
Expressingthemagnitudeofadverseeffectsincase–controlstudies:"thenumberofpatientsneededtobetreatedforoneadditionalpatienttobeharmed".
BMJ.
2000;320(7233):503–6.
28.
BenderR,KrompM,KieferC,SturtzS.
Absoluterisksratherthanincidenceratesshouldbeusedtoestimatethenumberneededtotreatfromtime-to-eventdata.
JClinEpidemiol.
2013;66(9):1038–44.
29.
SuissaD,BrassardP,SmiechowskiB,SuissaS.
Numberneededtotreatisincorrectwithoutpropertime-relatedconsiderations.
JClinEpidemiol.
2012;65(1):42–6.
30.
SuissaS.
Thenumberneededtotreat:25yearsoftrialsandtribulationsinclinicalresearch.
RambamMaimonidesMedJ.
2015;30:6(3).
31.
DeeksJJ,HigginsJPT,AltmanDG.
Chapter9:Analysingdataandundertakingmeta-analyses.
In:HigginsJPT,GreenS,editors.
Cochranehandbookforsystematicreviewsofinterventionsversion5.
1.
0.
Oxford:TheCochraneCollaboration;2011.
32.
daCostaBR,RutjesAW,JohnstonBC,etal.
Methodstoconvertcontinuousoutcomesintooddsratiosoftreatmentresponseandnumbersneededtotreat:meta-epidemiologicalstudy.
IntJEpidemiol.
2012;41(5):1445–59.
33.
SuissaS.
Calculationofnumberneededtotreat.
NEnglJMed.
2009;361(4):424–5.
34.
StangA,PooleC,BenderR.
Commonproblemsrelatedtotheuseofnumberneededtotreat.
JClinEpidemiol.
2010;63(8):820–5.
35.
CatesCJ.
Simpson'sparadoxandcalculationofnumberneededtotreatfrommeta-analysis.
BMCMedResMethodol.
2002;2:1.
36.
SmeethL,HainesA,EbrahimS.
Numbersneededtotreatderivedfrommeta-analyses—sometimesinformative,usuallymisleading.
BMJ.
1999;318:1548–51.
37.
Thomson-Reuters.
Journalcitationreports.
http://thomsonreuters.
com/en/products-services/scholarly-scientific-research/research-management-and-evaluation/journal-citation-reports.
html.
Accessed24Aug2016.
38.
AlvesC,Batel-MarquesF,MacedoAF.
Datasourcesondrugsafetyevaluation:areviewofrecentpublishedmeta-analyses.
PharmacoepidemiolDrugSaf.
2012;21(1):21–33.
39.
BrownEG,WoodL,WoodS.
Themedicaldictionaryforregulatoryactivities(MedDRA).
DrugSaf.
1999;20(2):109–17.
40.
SchmidCH,LauJ,McIntoshMW,CappelleriJC.
Anempiricalstudyoftheeffectofthecontrolrateasapredictoroftreatmentefficacyinmeta-analysisofclinicaltrials.
StatMed.
1998;17(17):1923–42.
41.
EngelsEA,SchmidCH,TerrinN,etal.
Heterogeneityandstatisticalsignificanceinmeta-analysis:anempiricalstudyof125meta-analyses.
StatMed.
2000;19(13):1707–28.
42.
AustinPC.
Absoluteriskreductionsandnumbersneededtotreatcanbeobtainedfromadjustedsurvivalmodelsfortime-to-eventoutcomes.
JClinEpidemiol.
2010;63(1):46–55.
43.
LaubenderRP,BenderR.
Estimatingadjustedriskdifference(RD)andnumberneededtotreat(NNT)measuresintheCoxregressionmodel.
StatMed.
2010;29(7–8):851–9.
44.
LaubenderRP,BenderR.
AnoteoncalculatingasymptoticconfidenceintervalsfortheadjustedriskdifferenceandnumberneededtotreatintheCoxregressionmodel.
StatMed.
2014;33(5):798–810.
Correction:StatMed.
2014;33(5):810–1.
45.
HellerRF,DobsonAJ,AttiaJ,PageJ.
Impactnumbers:measuresofriskfactorimpactonthewholepopulationfromcase–controlandcohortstudies.
JEpidemiolCommunityHealth.
2002;56(8):606–10.
46.
AttiaJ,PageJ,HellerRF,DobsonAJ.
Impactnumbersinhealthpolicydecisions.
JEpidemiolCommunityHealth.
2002;56(8):600–5.
47.
HellerRF,EdwardsR,McElduffP.
Implementingguidelinesinprimarycare:canpopulationimpactmeasureshelpBMCPublicHealth.
2003;3:7.
48.
HellerRF,BuchanI,EdwardsR,etal.
Communicatingrisksatthepopulationlevel:applicationofpopulationimpactnumbers.
BMJ.
2003;327(7424):1162–5.
49.
HuttonJL.
Numberneededtotreat:propertiesandproblems.
JRStatSocAStatSoc.
2000;163(3):403–19.
50.
AltmanDG,DeeksJJ.
Meta-analysis,Simpson'sparadox,andthenumberneededtotreat.
BMCMedResMethodol.
2002;2:3.
51.
MarxA,BucherHC.
Numbersneededtotreatderivedfrommeta-analysis:awordofcaution.
ACPJClub.
2003;138(2):A11–2.
52.
ShepherdJ,KasteleinJP,BittnerVA,etal.
Intensivelipidloweringwithatorvastatininpatientswithcoronaryarterydisease,diabetes,andchronickidneydisease.
MayoClinProc.
2008;83(8):870–9.
53.
LubsenJ,HoesA,GrobbeeD.
Implicationsoftrialresults:thepotentiallymisleadingnotionsofnumberneededtotreatandaveragedurationoflifegained.
Lancet.
2000;356(9243):1757–9.
54.
MayneTJ,WhalenE,VuA.
Annualizedwasfoundbetterthanabsoluteriskreductioninthecalculationofnumberneededtotreatinchronicconditions.
JClinEpidemiol.
2006;59(3):217–23.
55.
GirerdN,RabilloudM,DuarteK,RoyP.
Numberneededtotreatfromabsoluteriskandincidencerate:howtomakeapplesandorangescomparableJClinEpidemiol.
2014;67(2):236–8.
56.
BenderR,KrompM,KieferC,SturtzS.
Estimationofnumbersneededtotreatshouldbebasedonabsoluterisks.
JClinEpidemiol.
2014;67(2):238–9.
57.
SuissaS.
NumberneededtotreatinCOPD:exacerbationsversuspneumonias.
Thorax.
2013;68(6):540–3.
58.
GrahamDJ,Ouellet-HellstromR,MaCurdyTE,etal.
Riskofacutemyocardialinfarction,stroke,heartfailure,anddeathinelderlyMedicarepatientstreatedwithrosiglitazoneorpioglitazone.
JAMA.
2010;304(4):411–8.
Weacceptpre-submissioninquiriesOurselectortoolhelpsyoutondthemostrelevantjournalWeprovideroundtheclockcustomersupportConvenientonlinesubmissionThoroughpeerreviewInclusioninPubMedandallmajorindexingservicesMaximumvisibilityforyourresearchSubmityourmanuscriptatwww.
biomedcentral.
com/submitSubmityournextmanuscripttoBioMedCentralandwewillhelpyouateverystep:Mendesetal.
BMCMedicine(2017)15:112Page13of13