purposednsfail

AvailabilityandLatencyofWorldWideWebInformationServersCharlesL.
VilesandJamesC.
FrenchUniversityofVirginiaABSTRACT:Duringa90-dayperiodin1994,wemeasuredtheavailabilityandconnectionlatencyofHTTP(hypertexttransferprotocol)informationservers.
Thesemeasurements'weremadefromasiteintheEasternUnitedStates.
Thelistofserversincluded189serversfromEuropeand324serversfromNorthAmerica.
Ourmeasurementsindicatethatonaverage,5.
0percentofNorthAmericanserversand5.
4per-centofEuropeanserverswereunavailablefromthemeasurementsiteonanygivenday.
Asseenfromthemeasurementsite,theday-to-dayvariationinavailabil-itywasmuchgreaterfortheEuropeanserversthanfortheNorthAmericanservers.
Themeasurementsalsoshowawidevariationinavailabilityforindividualin-formationservers.
Forexampleomorethan80percentofallNorthAmericanserverswereavailableatleast95percentofthetime,but5percentoftheserverswereavailablelessthan80percentofthetime.
Thepattemofunavailabilitysuggestsastrongcorrelationbetweenunavailabilityandgeographiclocation.
Me-dianconnectionlatencyfromthemeasurementsitewasinthe0.
2-0.
5srangetootherNorthAmericansitesandinthe0.
4-2.
5srangetoEuropeansites,depend-inguponthedayoftheweek.
LatenciesweremuchmorevariabletoEuropethantoNorthAmerica.
ThemagnitudeofthelatenciessuggesttheadditionofanO1995TheUSENIXAssociation,ComputingSystems,Vol.
8.
No.
1.
Winter19956tMGETmethodtoHTTPtohelpalleviatelargeTCPset-uptimesassociatedwiththeretrievalofwebpageswithembeddedimages.
Thedatashowthat97percentand99percentofallsuccessfulconnectionsfromthemeasurementsitetoEuropeandNorthAmericarespec-tivelyweremadewithinthefirst10s.
Thissuggeststheestablishmentofclient-sidetime-outintervalsmuchshorterthanthoseusedfornormalTCPconnectionestablishment.
CharlesL.
VilesandJamesC.
FrenchI.
IntroductionTheWorldWideWeb(WWWorW3orWeb)[Berners-Leeetal.
1994,Berners-Leel9941hypertextparadigm,combinedwiththeavailabilityofgoodpublic-domainserverandbrowsingsoftware,hasenabledatrueexplosionofinformationresources.
Bymanyaccounts,bothanecdotalandobjective,thesizeoftheV/eb,intermsofnumberofservers[Beebee1994,Gray1994],numberofresources[FletcherI994,McBryan1994],andnetworktraffrc[Merit1994]hasincreasedexponentiallysincetheWeb'sconceptionatCERNintheearly1990's.
SynonymouswiththeWebisaninformationtransferprotocol(HypertextTransferProtocolorHTTP[Berners-Lee1993.
3)),amark-uplanguagewithwhichtocomposedocuments(HyperTextMarkupLanguageorHTML[Berners-LeeandConnolly19931),andamethodtoaddressinformationresources(UniversalRe-sourceIdentifiersorURIs[Bemers-Lee1993.
5]).
Complementarytechnologytoallowextensibletypingofinformationresources(MIMEtypes[Borenstein1992])hasalsobeeninstrumentalinthegrowthandpopularityoftheWeb.
Itseemsap-parentthattheWorldWideWebanditstechnologyismorethanapassingfancyandrepresentsafundamentalchangeinthewayinformationcanbeprovidedandusedontheInternet.
IfweimagineidealperformanceontheWeb,twomeasuresofinterestareavailabilityandlatency.
Ideally,wewouldlikeeverysitetobe100percentavail-ableandthelatencybetweentheselectionofahyperlinkandtheappearanceoftheinformationthatlinkrepresentstobeundetectable.
Forexposition'ssakewecantalkofthe100-100Web:100percentavailabilityforallserversand100millisecondlatencytoeveryserver.
Anythinglessthan100msecisperceivedasinstantaneousbymosthumansandisadesigncriteriainthecurrentdevelop-mentofHTTP[Berners-Lee1993.
31.
InSection3,wedefineexactlywhatwemeanbylatencyandavailability.
While100percentavailabilityofindividualserversisarealisticgoal,100per-centavailabilityofallserversisnot.
Failureshappen.
100milliseconduser-levellatenciesareunlikelyinthegeneralcase,givencurrentphysicalnetworksandpro-tocols.
Inthisstudy,weattempttoidentifyhowfartheactualWebisfromtheideal100-100'Web.
Wecharacterizethelatencyandavailabilityofalargegroup(>500)ofWebserversdistributedthroughouttheworld,butconcentratedinEu-ropeandNorthAmerica,asmeasuredfromanEasternUnitedStatessilerwithtypicalIntemetconnectivity.
Theimportantcontributionsofthispaperinclude[.
Weemphasizethisphmsetounderscorethefactthatsomeoftheresultsreportedheremustbeintelpretedwithrespecttoùemeasurementsite.
AvailabilityandLatencyofWWWInformationServers63.
AcharactenzationofthetypicalconnectionlatenciestoEuropeanandNorthAmericanWebserversfromaNorthAmericansite.
.
Acharacterizationoftheavailabilityof'TheWeb'overanextended(90-day)timeperiod,aswellastheavailabilityofparticularserversoverthesameperiod.
.
Theobservationthatrelativelyshort(10-20s)client-sidetime-outintervalswouldsignificantlyimproveworst-caseresponsetimesonexpansionofahyperlinkwithoutperceptiblyaffectingavailability.
.
Presentationofconvincingempiricalevidencethatthe95-500Webisamorerealisticgoalthanthe100-100Web.
.
AcallfortheadditionofanewmethoddefinitionforHTTPanalogoustotheMGETavailableinsomeFTPclientimplementations.
ThismethodwouldbeveryusefulforefficientretrievalofWebpagesthatcurrentlyre-quiremultipleGETstoassembleintheirentirety.
Inaddition,wefeeltheinformationpresentedherewillbeusefulinthedevel-opmentofnewserverandclientsoftwareandinthedesignofdistributedinfor-mationretrievalsystems,particularlythosethatrequireexhaustivesearchofallparticipatingsites.
Theorganizationofthepaperisasfollows.
WedescribesomeoftheotherInternetmonitoringactivitiesweareawateofinSection2,payingparticularat-tentiontothoseinvolvingtheWeb.
InSection3,wedescribeourexperimentalmethodology.
InSection4wepresentourresults,followedbyadiscussioninSec-tion5.
WeconcludewithashortsynopsisofourcontributionsinSection6.
2.
RelatedWork2.
1.
MeasurementActivitiesontheWorldWideWebWeknowofonlyafewmeasurementactivitiesontheWebasofthewritingofthispaper.
MostinvolvetheuseofautomatedprogramsthatcrawlovertheWebinsearchofnewdocumentsorservers.
2Someoftheseactivitieshaveasastated2.
Theseprogramsareoftencalledwebrobotsorspiders.
AdescriptionofcunentlyknownobotsismaintainedbyMartijnKosterathttp://www.
nexor.
co.
ulVmaVdoc/robots/robots.
htrnl.
Avoluntarystandardfortheuseandconstructionofwebrobotsisalsoavailableatthesamesite,Thestandardattemptstoensurethatrobots'donoharm'toindividualserversortheWebitself.
CharlesL.
VilesandJamesC.
French64goaltheestimationofthesizeoftheWebeitherinnumberofdocumentsornumberofservers[Beebee1,994,Gray1994],butmostareusedtocollectdoc-umentsforindexingandsearchingpurposes[Eichmann1994,Fletcher1994,MauldinandLeavitt1994,McBryan19941.
Websizeestimatesareaside-effectofthecollectioneffort.
Otherrobotsareusedtoaidinthemaintenanceofhy-pertextinfostructures[Fielding1994].
Amongotherthings,the'WWWMMrobot[Tronche1994.
32]wasbuilttoestimatethelatencyofsingleandmulti-linkpathsintheV/eb.
Latencywasdefinedasthetimeittookfromthedocumentrequestuntilthereceiptofthefirstbyteofthedocument[Tronche1994.
331.
Ourdefi-nitionincludesonlythetimetosetuptheconnectionandsendtherequest.
NodataorsummaryofresultsfromW'WWMMwasavailableasofthewritingofthispaper.
PadmanabhanandMogulll994lproposemodificationstotheevolvingHy-pertextTransferProtocoltoimproveuser-levellatency.
BraunandClaffy[1994]characteizedHTTPtrafficatapopularserverandexaminedhowcachingheavilyaccesseddocumentsatsitesclosertotherequestorwouldimprovebandwidthatthemainserver.
2.
2.
OtherInternetMeasurementActívitiesAvarietyofperformancedataabouttheNSFNETbackboneiscollectedeachmonth,summariesofwhicharemadepubliclyavailableatftp://nic.
merit.
edu/nsfnet/statistics.
Thisdataincludesone-waydelaysbetweenbackbonenodesandtrafficbreakdownsbyport,country,networkandday.
UsingthisdataClaffyetal.
[1993]describedgeneraltrendsintheNSFNETsT1back-boneupuntilitsretirementinlate1992.
TheIntemetDomainSurvey[Lottorl992,Lotot1'994],isalong-runningactivitytoestimatethesizeoftheInternetbycountingthenumberofhostsandnetworksintheDomainNameSystem.
Eachmonth,aflowanalysisoftrafficonUsenetnewsgloupsispublished[DECNetworkSystemsLaboratory1994).
Thisinformationincludesstatisticsonthesizeofarticles,sizeofnewssites,trafficdistributionbynewsgroup,andthetopnewssitesbytrafficvolume.
Usingpackettraces,Caceresetal.
U9911characterizedtheattributesofbothinteractiveandbulk-transferapplicationsinwide-areaTCP/IPconversations.
Nolatencyorfailuremeasurementsweremade.
Danzigetal.
[1'992]foundthatUDP-basedDNStrafficconsumedconsiderablymorenetworkbandwidththanwasstrictlynecessary,andattributedtheexcesstraffictobuggyimplementationsofnameserversandresolvers.
InpublicFTParchives,Maffeist19931foundthatac-cesstofilesgenerallyexhibitedhighlocality,withafewfilesbeingaccessedmostAvailabilityandLtencyofWWWInformationServers6566ofthetime.
Mostfiles(99percent)werelessthanlMBinsizeandtransfersofflleslessthan100Kmadeup80percentoftotalfileretrievals.
Muchhasbeendoneinexaminingend-to-enddelaysontheInternet[AgrawalaandSanghi1992,BoIot1993,Mills1983,Sanghietal.
1993]butmostofthisworkhasbeenatthepacketlevel,notattheconnectionlevel.
3.
DescriptionofMonitoringActivitiesInthissection,wefirstsupplysomegeneralbackgroundinformationontheWeb.
Weleavethedetailstothecitedworks.
Wethendefinethemeasurementsofinter-est,followedbythedescriptionofourexperiments.
3.
1.
Background3.
1.
1.
WorldWideWeb(WWWorW3orWeb)TheWorldWideWebprovidesahypertextenvironmentwhereusersareabletoshareinformationregardlessofitsphysicallocation.
Linksinahypertextdoc-umentmayleadtomanytypesofinformationresourceslocatedthroughoutfheInternet.
TheWebconceptwasdevelopedatCERN,theEuropeanParticlePhysicsLaboratory,butwasquicklyembracedglobally.
Moredetaileddescriptionsofthe\Veb,itsdesigngoals,anditscurrentandpotentialcapabilitiesareavailableelse-wherelBerners-Leeetal.
1994,Berners-Lee1994].
3.
1.
2.
HTTPTheHypertextTransferProtocol(HTTP)isanevolvingprotocolfortheexchangeofhypertextinformationoverwideareanetworks[Berners-Lee1993.
3]andisthenativeclient/serverprotocolfortheV/orldWideV/eb.
HTTPisanapplica-tionlevelprotocolthatrunsontopofthelayerfourprotocol(nominally,TCP).
HTTPtreatsdocumentsasobjectsanddefinesasetofmethodsthatcanbein-vokedontheobjects.
Thesemethodssupportsearchandretrievalandaredesignedtobeextensibletoencompassotherfunctionality,includingupdateandannotation.
HTTPisstatelessandisdesignedtobeaslightweightaspossibleinordertosup-portshortresponsetimes.
Thereare14methodsinthecurrentproposedstandardbutonlyasubsetoftheseareimplementedinmostservers:GET,POST,PUTDELETE,andHEAD.
BecauseHTTPisstateless,eachmethodrequestishandledasaseparateffansaction.
Theserverterminatestheconversationwiththeclientafterperformingeachmethod.
TheresultisaseparateTCPconnectionforeachmethodrequest.
CharlesL.
VilesandJamesC.
French3.
2.
MeasurementDefinitions3.
2.
1.
ConnectionInourexperiments,weattemptedtocontactalargesetofHTTPservers.
EachcontactattemptwasaTCPstreamsocketconnectiontotheportwheretheserverwaslistening(generally,butnotalways,port80).
Themainloopofthemeasure-mentprogramisdepictedinpseudocodeinFigure1.
TheResolve-AddressroutineincludesresolutionofthehostnamethroughDNS,andtheTCP-ConnectroutineincludesthebuildingoftheTCPstreamsocket.
Tominimizeserver-sidesystemdelays,anonsensemethodrequestwassenttotheserverinsteadofthenormalHTTP"GET"[Bemers-Lee1993.
3].
AknownHTTPmethodwouldoftenrequiretheservertogotodisktoresolvetherequest,thusintroducingadditionalsystemdelaysintothelatencymeasurement.
Themethodrequestthatwassentwas''TESTCOMMANDForlnfo-->}rttp://wacs.
cs.
virginia.
edu/-clv2nlwebtest.
html''.
TheargumenttoTESTCOMMANDwastheURIforadocumentdescribingthepurposeoftheexperiment.
Wefoundthatthisforestalledaflurryofe-mailfromcuriousserveradministrators.
ForeachcontacttoanHTTPserver,twomeasurementsweretaken:theres-olutionoftheconnectionattempt(successfulorunsuccessful),andthetimetoeitherestablishtheconnectionorgetafailure.
WewereabletodistinguishDNSfailuresfromthoseduetootherproblems,butwewereunabletofurtherdistin-guishthosefailuresinthe"other"category.
Fromthisdata,wepresenttwokindsofperformancemetrics,availabilityandlatency.
WhiTemorehostsbegínreadhostandport;starttimer;address=ResoTve_Address(host);successfuT=TCP_Connect(address,port);send_NonsenseRequest();stop-timer;recordsuccessandtimervaLue;cTean-upconnection;endToopFigure1.
Pseudo-codeforthemainloopinthemeasurementprogram.
AvailabilityandLatencyofWWWInformationServers673.
2.
2.
AvailabilityWedefinetwotypesofavailability,WAvailandSAvail.
NumberSuccessfulContactsatúWAvail(t):TotalContactsAttemptedúWAvailisameasureoftheproportionofV/ebseryersthatareactiveandservinginformationatanyparticulartime.
Ideally,measurementsofWAvailshouldbemadebycontactingallserverssimultaneously.
Inpractice,suchamethodwouldbeill-advised,sincethemeasurementactivitywouldlikelybiasthemeasurementitself.
Sequentialcontacts,asweremadeinthisstudy,allowonlyonependingcontactandthusdonotcausecongestionatthemeasurementsite.
Inthiswork,allmeasurementsofWAvailwerecalculatedbycontactingthetargetsetofHTTPserversoverashorttimeperiod(about30minutes)ratherthaninstantaneously.
Wedefineserveravailabilityintermsofaparticularserverandatimeperiodratherthanaparticulartime.
SAvail(s,h,tz):NumberofsuccessfulcontactstoserversbetweenhandtzTotalnumberofcontactstoserversbetweenhandtzInthispaper,wepresentasingleestimateofSAvailforeachserver,withú1andt2settobethestartandstopdaysofthelong-termmeasurementperiod.
3.
2.
3.
LatencyLatencyisthetimeittakestoresolvealogicalname,setupaconnection,andtransmitarequesttoaninformationserver,giventheserver'slogicalname.
Weconsiderthislatencytobealowerboundonthewaitthatinteractiveusersexpe-riencewhentheyclickonahyperlinkandwaitforthefirstbyteofadocumenttoappear.
Thisisbecauseourlatencydoesnotincludethetimespentwaitingforaservertofetchandreturntherequesteddocument.
3.
3.
TheExperiments3.
3.
1.
TheListofHTTPServersInourexperiments,werepeatedlycontactedasetof542HTTPservers.
ThissetofserverswasobtainedfromapublishedlistofWorldWideWebserversavail-ableontheWeb.
ThislistwasgeneratedbyaWeb-walkingautomatoncalledthe'WorldWideWebWandererlftay1994).
Theautomatonstartswithacollectionofknowndocumentsandconductsadepth-firstsearchoftheWeb.
ThestatedgoaloftheautomatonistoestimatethetotalsizeoftheWebintermsofthenumberofservers.
ItonlyexpandsonHTTPlinks,ignoringlinkslikeftp,wais,telnet,andCharlesL.
VilesandJamesC.
French68others.
InJanuaryoft994,thislistcontained623sites(inOctober1994,4600sites).
WeculledtheJanuarylistdownto542sites,eliminatingallsiteswithcor-rupteddomainnamesorthatwereotherwiseinaccessible.
Whentheexperimentstarted,all542siteswereupandavailable.
Fordataanalysis,wesplitthelistofsitesintothreegroups,Europe(EU),NorthAmerica(NA),and"Other.
"TheEu-ropeangroup(189servers)containedallserverswithEuropeancountrycodesintheirdomainnames.
Thislistwasverifiedusingpackettfaces[Sun1993.
31].
TheNorthAmericanlist(324servers)comprisedall.
edu,.
mil,.
org,and.
comsitesplusallCanadiansites.
The"Other"group(29servers)containedamiscellaneousgroupofserversfromtheFarEast,Australia,andCentralandSouthAmerica.
3.
3.
2.
MeasurenlentPeriodMeasurementstookplaceovertwotimeperiods.
WemeasuredWAvailandlatencyevery2hoursforthefirst7daysandevery4hoursforthenext5daysovertheperiodfromFebruary14,1994toFebruary25,1994.
Forthe90-dayperiodfromMarch1.
,1994toMay29,1994,wemeasuredWAvailandlatencytwiceadayatapproximately11AMand11PMEastemTime.
Forbothtimeperiods,wAvailwasmeasuredforthesetofserversandlatencywasmeasuredforeachserver.
Attheconclusionofthelongertimeperiod,wealsowereabletomeasureSAvailfortheentire90-dayperiodforeachserver.
Duetolocalproblemsatthemeasurementsite,ontwodaysoverthelongertimeperiod,onlyasinglemeasurementwasmade.
Atanyparticularmeasurementtime,aruntholghallserversconsistedofcon-tactingeachserveronthelistinsequence.
Arunnormallytookabout30minutes.
Thelistofsewerswasrandomizedbeforeeachruntoensurethattheconnectionattempttoanyparticularserveroccurredataslightlydifferenttimeoneachrun.
Thiswastoavoidregularlyscheduledactivitiesontheserversidethatmightbiasthemeasurementsforthatserver.
Forexample,wewantedtoavoidrepeatedlycontactingamachineinthemiddleofrunningitsdailybackups'3.
3.
3.
MeasurementEnvironmentForalloftheIZ-dayperiodandthefirst15daysofthe90-dayperiod,measule-mentsweremadefromaSPARCstationIPCrunningSunOS4.
3.
1with32l:;{Bofmemoryandanattacheddisk.
Forthelast75daysofthe90-dayperiod,mea-surementsweremadefromadifferentbutsimilarlyconfiguredSPARCstationIPC.
NetworkconnectionswithintheUniversityofVirginiaarefiber-opticbased,withaTl(I.
544mBit/sec)linktothewiderIntemet.
Non-localtrafficgenerallytakesthreeT1hopsandabout15msectogettotheNSFNETT3backbone.
TheUni-versityofVirginiarunsthreeDNSserverswhosecachesarepurgedeachnight.
AvailabilityandLatencyofWWWInformationServers694.
ResultsIntheresultsthatfollow,wepresentthetimeofdayintermsofthemeasurementsite,notthedestinationserver.
Thus,a'midnight'measurementforaEuropeanserverisactuallyfourtosixhourslaterintermsoftheserver'slocaltime.
Be-causeofthesmallnumberofserversintheOthergroup,resultsarepresentedonlyfortheEuropeandNorthAmericagroups.
Itisveryimportanttorealizethatallresultspresentedbelowarefrommea-surementsmadeatanEasternUnitedStatessite.
Anyinterpretationoftheresultsmustbemadeaccordingly.
Toavoidverbosityandmisinterpretation,wewillusethefollowingconventions.
WhenspeakingofmeasurementsmadeofEuropeanservers,wewilluse"NA-to-EU".
Similarly,forNorthAmericanservermeasure-ments,wewilluse"NA-to-Nlt''.
4.
1.
Availability4.
1.
1.
FirstMeasurementPeriod:l2-dayIntensiveInFigure2wepresentmeasurementsofWAvailforthel2-dayperiodfromFebru-ary14toFebruary25.
NA-to-NAisatthetop,andNA-to-EUisatthebottom.
BothNA-to-EUandNA{o-NAmeasurementsshowdailyminimaandmaximainavailability,occurringmoreorlessatihesamelocalservertime.
Minimalavail-abilitynormallyoccurredintheearlytomid-morningforNorthAmericaandaroundmidnightforEurope.
AvailabilitywashighestintheeveninginNorthAmericaandintheearlyafternooninEurope.
Thisbehaviorisnotsurprising,sinceserversgoingdownovernightmightnotgetre-booteduntiltheiradminis-tratorsarriveforworkthefollowingday.
Momingisalsothetimewhenmanyad-ministratorsbringserversdownforconfigurationchangesandothermaintenancetasks.
Webavailabilitywasroughly95percentoverthelifeoftheexperiment.
Figure2alsoshowsaslightweekenddropinavailabilityforbothgroupsofservers.
Itisdifficulttosaywhetherthisisaconsistentphenomenon,sinceonlyoneweekendperiodisshown.
4.
1.
2.
SecondMeasurementPeriod:90-dayLongTermInFigures3and4wepresenttheresultsofeverysingleconnectionattemptmadetoeveryserverinNorthAmericaandEuroperespectivelyinthesec-ondmeasurementperiod,March1toMay29,1994.
Eachrowintheimagesrepresentstheresultsofthe178connectionattemptsmadetoasingleserver.
Theattemptsarepresentedinchronologicalorderfromlefttoright.
ThereisCharlesL.
VilesandJamesC.
French70NorthAmericatoNorthAmerica0.
80uMonThuFriSatSunMonTueWedThuDayofWeekMonruewedrhuFfiir;iit"i"'iruewedrhuFfisatFigure2.
Webavailabilityfromthemeasurementsiteover12days.
Verticalgridlinesrepresentthestartoftheday.
noverticalrelationshipinFigures3and4,astheserversarepresentedinal-phabeticalorderoftheirdomainnames.
Whiteareasinthefiguresrepresenttimesthataserverwasdownandblackareasmeanaserverwasup.
Awhitelineindicatesthataserverwasdownforconsecutiveconnectionattempts.
Thelengthofthelinethenrepresentsthenumberofconsecutiveattemptsaserverwasdown.
Thetotalareainblack(white)foreitheroftheimagesrepresentstheoverallavailability(down-time)ofthegroupofserversoverthelifeoftheexperiment.
ForNA-to-NA,thistumsouttobe95.
0percent(5.
0percent)andforNA-to-8U,94.
6percent(5.
4percent).
Anotherinterestingobservationisinthepattern'{o.
so=:(dQo.
so=NorthAmericatoEuropeAvailabilityandLatencyofWWWInformationServers7loE(úc(úo!
-o->6o(l).
o(ú!
o-(úooU'(,()U)c)o-o(ú-c.
aooè()U)'l4590DayFigure4.
SummaryofEuropeanHTTPserveravailabilityover90days.
WeidentifredsixNorthAmericanserversandoneEuropeanserverthatweredownfor40ormoreconsecutivedaysandweredownattheendofthemeasure-mentperiod.
TheseserversaremarkedalongtherightmarginofFigures3and4.
Webelievetheseserversaredeadorhavemovedtoadifferentaddress.
InTable1,wesummarizesomeofthefailureinformationfromFigures3and4.
ThemostinterestingadditionalinformationtobegleanedfromthistableisthattheDNSfailureratewasaboutfivetimeshigherwhenresolvingEuropeandomainnamesthanwhenresolvingNorthAmericandomainnames.
WhenDNSfailuresareremoved,thefailurerates(andthusoverallavailability)forbothconti-nentsareverysimilar.
Toseeifthereweresomegeographicrelationshiptoserverdown-time,weclusteredalloftheEuropeanservershierarchicallybythenetworkpathtotheserver.
Thispathwasdeterminedusingpackettracesgeneratedbytraceroute[Sun1993.
31].
Ingeneral,theclusteringconsistedofthefirsttwoorthreeEu-ropeannetworkstraversedtothedestinationserver.
Withina"network"cluster,wesortedhierarchicallyusingdomainnames.
Theeffectofthisclusteringisaroughgeographicsorting,whereserversaregroupedfirstbythemajornetworktowhichtheyareattachedandsecondbytheorganizationtowhichtheybelong.
There-sult,depictedinFigure5,showssomeclearverticalrelationshipsbetweenserveldowntimes.
Themostobviousexampleofthiswasontheja.
netpathatday34ofAvailabilityandLatencyofWWWInformationServers73Table1.
SummaryofFailures.
EuropeNorthAmericaBothRunsSitesConnectionsDNSFailuresTotalFailuresDNSFailRate(Vo)OtherFailRate(7o)TotalFailRate(Vo)1781893364221.
818010.
64.
75.
4178324576726028680.
14.
95.
01785139r3r427846690.
34.
85.
1theexperiment,whereallservers(exclusivelyUKbasedinthisexperiment)con-nectedviathisnetworkwereunavailablefortwoconsecutiveruns.
ThisbehaviorpointstoanetworkfailureornetworkcongestionthateffectivelypartitionedtheUKserversfromthemeasurementsite.
Otherexamplesofthisgeographiclocalitydante.
netdante.
net+euro-hep.
netes.
net>(dfn.
deIinfn.
il)eu.
neticp.
net+ebone.
neticp.
net>ebone.
net+ft,neticp.
net>dante.
neticp.
net*nordu.
net-denel.
dkcp.
net+nordu.
net+(uninelt.
noIicp.
nel>sunel.
seja.
net145DayFigure5.
SummaryofEuropeanHTTPserveravailabilitywithrowsclusteredbyphysicalnetworkstraversed.
Serverswithinaclusterweresortedhierarchicallybydomainname.
Thepathforeachclusterisshowntotheleft,andthecoun-triesrepresentedineachclusteraredepictedtotheright.
CharlesL.
VilesandJamesC.
Frenchchdelfrdelitdelielnlielatfrnldknolfiseuk74appearonthedante.
netpath(3runsaroundday46):thedante.
neturo-hep.
netpath(1runatdays30and64):theeu.
netpath(1runeachatseveraldays);andtheicp.
net/nordu.
net/uninett.
nopath(onerunatdays43and80).
Forgeographi-callycloseservers,downperiodsdonotappeartobeindependentevents.
WedidnotattempttoclustertheNorthAmericanserversinthisfashion.
InFigure6(left),wetrackWAvailoverthelifeoftheexperimentforNA-to-NA(top),NA{o-EU(center)andNA-to-All(bottom)serversrespectively.
WAvailforNA-to-NAwasaround95percentfortheentireexperiment,withonlytwodipsbelow91percentandasinglepeakabove98percent.
Thissteadybehav-iorisrepresentedinthehistogramfotWAvail(Figure6top,right),whichshowsaverytightcentraltendencyaround95percentandveryfewoutliers.
ForNA-to-EU,WAvailwasalsoaround95percent,butwasmorevariable,withseveraldipsbelow90percentandtwoadjacentdipsbelow80percent.
ThedifferencebetweentheNA-to-NAandNA-to-EUavailabilitymeasurementswassmall,andthetwomeasureswerefoundtobestatisticallyindistinguishableusingtheWilcoxonranksumtest.
Variabilitywasespeciallyhighbetweendays20and60.
Europeanservewerealso100percentavailableforonerun.
Some,butnotall,ofthesedipsconespondtothegeographicallycorrelateddownperiodsdepictedinFigure5.
Forexarnple,thetwodipsbelow80percentareinfactthetwomeasure-mentperiodstheUKwasunavailableviatheja.
netpathfromthemeasurementsite.
ThewidervariabilityinWAvailisreflectedinthehistogramsforNA-to-EU(Figure6middle,right),whichshowawiderdistributionthanNA-to-NAaswellasmoreoutliers.
Figure7showshistogramsandcumulativehistogramsofSAvailforNorthAmericanservers(top),Europeanservers(middle),andallservers(bottom).
ForbothNA-to-NAandNA-to-EU,asignificantportionoftheserversshowverygoodavailability:80percentofNorthAmericanserversand70percentoftheEuropeanserverswereavailablefromNorthAmerica95percentofthetimeorbetter.
However,atleast5percentofbothNorthAmericanandEuropeanserverswereavailablefromNorthAmericalessthan80percentofthetime.
Evenac-countingforthesevenservers(sixNAandoneEU)thatdiedormoved,thereisasmallbutsignificantgroupofserverswithpooravailability.
Thisdatasug-geststhatthissmallgroupmightberesponsibleforalargeportionoftheWebdowntimepresentedearlier.
InTable2,weshowthatoverallavailabilityanddown-timewouldbebetterifweremovesomeofthissmallgroupofpoorper-formingservers.
Forexample,ifweremovethebottom5percent(asmeasuredbySAvail)oftheEUandNAservers,overalldowntimeontheWebdropsfrom5.
0percentto2.
5percentinNorthAmericaandfrom5.
4percentto4.
1percentinEurope,asmeasuredfromNorthAmerica.
AvailabilityandLatencyofWWWInformationSemers75NorthAmerica'Io.
r0.
0@-o.
70.
80.
9t.
00.
70306090DayEurope__!
lrrr,I0.
0@ogo6090"-0.
70.
80.
91.
0----t---IIIII7IIIt0.
0@-"0306090-'-o.
70.
80.
91.
0DayWAvailFigure6.
Webavailability(WAvail)fromthemeasurementsiteover90days.
Longitudinaltracksareontheleftandfrequencydistributionsareontheright.
4.
2.
LatencyThedistributionoflatenciesforsuccessfulconnectionsinatypicalrunispre-sentedinFigure8.
InthecaseofbothNAto-NAandNA-to-EU,thedistributionspresentedareremarkablytypical,e.
g.
oneNA-to-NArunisverysimilartoanyotherNA-to-NArun.
Thespreadofthedistributionschange{withtimeofday,butthelong-tailedshapewascharacteristicofallruns.
76CharlesL.
VilesandJamesC.
French6ào.
r0.
7Day'ìo.
r=YllttIIttlttt----l-----llltltltttlIIIIII----t-IIIIII----t-IIIrttlttt--r--r--T--Ttttrltttrttttttt:-ts-ttttltttltttlltto.
2'9,.
o,o:40:6o:sSAvailNorlhAmerica1.
00.
8'0.
69-o.
lLo.
2Europe0.
0]*".
-*}-0.
00.
20,0.
0.
00.
2llttlttl--r--r--T---r--tttttttt--i--;--i--i--lttl--tsjrrrrlrrrrl--i---i---T---i-j/llllJ|0.
40:SAvail0.
2(ú91o.
r0,8(úu.
o9o.
l!
0.
0l-0.
00.
40.
60.
8SAvail1.
00.
20.
40.
6SAvailAil1.
0o.
2'9o.
r'0.
69o.
lJ-o.
20.
00.
00.
20.
40.
0.
81.
0SAvailFigure7.
HTTPserveravailability(SAvail)fromthemea-surementsiteover90days.
Frequencydistributionsareontheleft,andcumulativefrequencydisributionsareontheright.
Withsuchlong-tailedbehavior,thesamplemedianandsamplemean\ryerefarapart.
Asanexample,themeanandmedianfortheNA-to-NArundepictedinFigure8(top)were1.
19sand0.
27srespectiveland3.
65sand1.
48srespec-tivelyfortheNA-to-EUrun.
Infact,inallcasesthesamplemeanwasconsider-ablylargerthanthesamplemedian.
Wefeltthatthesamplemedianwasabetterindicatoroftypicalconnectionlatencythanthesamplemean,sincethevastma-jorityofconnectionsshowedlatenciesbelowthesamplemean.
Forthisreason,weusethemedianasourpresentationstatistic.
0.
40.
60.
81.
0SAvailtllt--T--T--tttttttt--+--+ttltttttII--r--IIII--l-*-IIIItttttttt--r--T--r--r--lrrrrlrrrrJ--i--i--;--;--lrrrrl--F-Jrrrr'l__L__L__l___l_trrttJJl!
,,Jtttt-r--llIIttll-t---ftlIItltltlttttt--r--T.
--T--'T--tttt__L-_!
__-L___t__rttttt--t---f'--+--tttttt--i---1--T--ltrAvailabilityandLatencyofWWWInformationSeryers77Thble2.
OverallAvailabilitywithPoorPerformingServersRemoved.
Europepercentavailableercentdown)NorthAmericapercentavailable(percentdown)AllServersDeadServersRemovedBoomSVoRemovedBattomlOVoRemovede4.
6(s.
4)9s.
0(s.
0)e5.
9(4.
1)96.
5(3.
5)9s.
0(s.
0)e6.
4(3.
6)97.
s(2.
s\98.
1(1.
e)0.
2Eeooo!
0.
10.
0NorthAmericatoNorthAmericamean=1.
19smedian=0.
27sNorthAmecatoEuropemean=3.
65smedian=1.
48srbl'sSeconds0.
26EcooIo.
t0.
0t'o15SecondsFigure8.
TypicaldailatencydistributionsoNorthAmerica(top)andEurope(bottom)fromthemeasurementsite.
Thisisday3lofthelong-termexperiment.
CharlesL.
VilesandJamesC.
French784.
2.
1.
FirstMeasurementPeriod:l2-dayIntensiveFigure9showshowconnectionlatencyvariedbytimeofday.
ForbothNorthAmericanandEuropeanservers,periodicbehaviorwasobserved.
ForNA-to-NAconnections,latencieswereshortestintheearlymorning,between2and6AINI{,andlongestinthemid-aftemoon,between2and6PM.
ForNA-to-EUconnec-tions,latencieswereshortestinthelateeveningtoearlymorning,andlongestinthelatemorningtoearlyafternoon.
Giventhetimedifferencebetweenthetwocontinents,itappearsthatlafencytoaparticularsiteiscorrelatedwithlocalservertime.
Theperiodicbehaviorwasparticularlyevidentontheweekdaysandlesssoontheweekend.
NA{o-NAlatencieswereconsiderablylessthanNA-to-EUlaten-cies.
ThisisnotsurprisinggiventhatthemeasurementsitewasinNorthAmerica.
4.
2.
2.
SecondMeasurementPeriod:90-dayLongTermInFigure10,wepresentthemedianwith25percentand75percentquartilesforbothgroupsofserversandbothtimesofday.
EuropeanandNorthAmericanserversbothshowhighervariabilityinthe75percentquartilethanthemedianor25percentquartile.
Thisunderscoresthetypicallong-tailedlatencydistributionswementionedearlier.
Ifwecomparethefirst45daystothelast45days,thenitappearsforbothgroupsandbothtimesthatlatencieswerehigherandmorevari-ableinthefirsthalfoftheexperimentthanthelasthalf.
Wewereunabletotracethisphenomenontoanylocalevent(e.
g.
achangeinhardware).
InFigure1.
1.
,wemakeseveralcomparisonsofmedianlatenciesoverthe90-dayperiod.
Ontheleft,wecompareNA-to-EUandNA-to-NAat11AM(top)and11PM(bottom)Easterntime.
Forbothtimes,NA-to-NAlatenciesaremarkedlylowerthanNA-to-EUlatencies.
Forbothtimesofday,NA-to-NAlatencieswerefoundtobesignificantlyloweratthe0.
01levelthanNA{o-EUlatenciesusingaWilcoxonraksumtest.
Thisistobeexpected,notonlybecausedistancesfromNorthAmericatoEuropearegenerallygreaterthandistanceswithinNorthAmerica,butbecauseinter-continentnetworkbandwidthisgenerallylessthanintra-continentbandwidth.
TheNA-to-EUlatenciesalsoshowamarkedincreasebetweenDays15and40(Marchl5-April8)oftheexperiment.
Weareunsureastowhattoattributethisactivityotherthansometemporarynetworkphenomenonthatmayhavereducedbandwidthandconnectivityforatime.
TheseincreasesinlatencyalsocorrespondedwithaslightdecreaseinNA-to-EUavailability(Fig-ure6center-left).
Figure11alsoshowshowlatencyvarieswiththedayoftheweek.
Attop-leftofFigure11,thisisclearlyillustratedbytheperiodicbehavioroftheNA{o-EUlatencies,withdipsoccurringontheweekendsandpeaksoccurringmidweek.
WeseethesamebehaviorforNA-to-NA,justonadifferentscale.
AvailabilityandLatencyofWWWInformationServers79o'1OcoooU)'of,.
.
f0.
4l-I,.
LMonTueWedThuFriSátSunMonTueWedThuFriSatDay1.
00.
8t,ìcco0.
6C)oU)0.
4o.
2^^rt.
rrrII"'"MonTueWedThuFriSatSunMonTueWedThuFriSalDayFigure9.
MedianconnectionlatencytoNorthAmerica(top)andEurope(bottom)overL2days.
Verticalgndlinesrepre-sentthestartoftheday(00:00AMeasternstandardtime).
NorthAmericatoNorthAmericarrrrtttttrtrtrllllllrrtttrrlllll__!
__I__-t---t---LJ--J-_-l---L-_L_-rrrttttttltttttlllllllttrllllllll__.
!
___r___r_r___L___l_-_r_t___r__ttrttttlllltltllllllllrttrllllllltlttttttttl--Tr---r---r-i---l---l-i---tttttttllllttttlllllllrrttltttttrr--r--a---t---t-Jt-r--r--.
1---1---r---T--1-rrrrllrrrI\|rIIrIrr-IIllrri\r-r^IIIIlrl/V\IIrr,\r\rI\l--:t--RlFt--ì!
-,1-l-l-l-,-ìl-/--.
frr,--ìr+-IIItttttttlttttrttNorthAmericatoEuropetttlllJ_--l---tltttttttttl-l-r-l-L-lrl\|l\lt,\rl\l,lrI\rI\1r1-1r1-v\/rrlltlt---l---F--+L1---l---l-ttrllllllllrrttlllllllttrllllllllIIILIIIt_IIIITIIIEIIIIIIrIIILIIIIi-IIIrIII80CharlesL.
VilesandJamesC.
FrenchNorthAmericatoEurope-11AMI6a,E'co4ooU)20I61tooo20I6a,lCcooo(t,20203040506070Day304050DayDayFigure10.
25percent(bottomline),median(middleline),wtd75percent(topline)quartilesforlatencyofsuccessfulconnections.
Thetoptwofiguresshowthelatencyat11AMtoNorthAmericaandEurope,andthebottomtwofiguresshowthelatencyat11PM(bothtimesaeeasternstandardtime).
NorthAmericatoNorthAmerica-11AMNorthAmericatoEurope-11PM8tt6l-il,FtoINorthAmericatoNorthAmerica-11PMAvailabilityandLatencyofWWWInformationServers81Dooooco{-S-NA-EU(lrAM)NA-NA(11PM).
','.
'lj.
1l',r,.
[\/'V.
.
v^VU17\vI:-iiill.
f/.
r.
.
---t-11.
-NA-EU(11PM)'.
.
.
NA-NA(rlPM)_NA-NA(rlAM)NA-NA(11PM)-NA-EU(114M)NA-NA(11PM)EOc-oooocOrJoooooco{dJ20406080Day20406080DayOc-oooocO{6J20406080Day20406080Day0L0Figure11.
Comparisonofmedianconnectionlatenciesbylocationofservers(left)andtimeofday(right).
OntherightofFigure1.
1,wecompareNA-to-NA(op)andNA{o-EU(bot-tom)latenciesat11AMand11PM.
NA-to-NAlatenciesareverysimilar,butNA-to-EUlatenciesdifferforthetwotimes.
ThisisnotsurprisingwhenwelookbacktothemoreintensivemeasurementsdepictedinFigure9.
TherewecanseethatlatenciesatlatemorningandlateeveningareinfactverysimilarforNA-to-NA,butdifferforNA-to-EUduetothetimedifference.
Inthelong-termexperiment,wedidnotsamplethecurvedepictedinFigure9atasufficientlyhighratetopickupitsinherentperiodicity.
IntheNA-to-NAcase,thetwosampleshaveoccurredontheupslopeanddownslopeofthecurve,effectivelyeliminatingtheobservationofanyhighfrequencybehavior.
IntheNA-to-EUcase,wherethecurveiseffectivelyphase-shiftedbyseveralhoursfromtheNA-to-NAcurve,thesamplesoccuratdifferentrelativepointsalongthecurve,andsoweseesomeoftheperiodicbehaviorofFigure9evidencedintheNA-to-EUplotsofFigure11.
WeshowthedistributionoflatenciesforallsuccessfulandfailedconnectionsinFigure12.
NA-to-NAandNA{o-EUmeasurementsaredisplayedseparately.
Theseplotsrepresentallofthelatencydata,178runstoallsites,over90days.
CharlesL.
VilesandJamesC.
French82ttttttttttt--L-_r__I___t__ttttttttlttt__L__I__I__J-__tttttttttttt0.
15'0.
10EcooIo.
ou0.
000.
156.
0.
10Ecooo*'0.
050.
000.
156.
0.
10oooìr0.
050.
000.
1560.
10Ecooo;r0.
050.
00m406080Latenoy(seconds)406080Lateney(seconds)20406080Lalency(seconds)020406080100Latèncy(seoonds)NA-NASuccessfulNA-NAFailedNA-EUSuccessful100NA-EUFailed--l---I--t---III----I--l---Ir--T--rrlt+--+--.
t+--rlr!
--1__ttt--T--T--tt--t--+--lt20406080Lalency(seconds)20406080Lalency(sêconds)1.
00^0.
956Ec$o.
sor0.
850.
801.
00.
80.
6ooog0.
4o.
20.
01.
00^0.
95Ecf;o.
so6ll0.
850.
801.
00.
8Eo.
eoo9no0.
20.
0020406080looLalency(seconds)100om406080100Latency(seconds)Figure12.
Frequencydistributions(left)andcumulativefre-quencydistributions(right)forallconnectionattemptsover90days.
Thetoptworowsshowsuccessfulandfailedcon-nectionsrespectivelyforNorthAmericanservers,andthebottomtworowsshowsimilardataforEuropeanservers.
lttttttt-i------T--tttt_r_tttttttt_L__t_ttttttttIII--LIII-t--IIa--IIItt--L--r--III---l.
--III---t--IIII--L--IIìII--T--I__1__II-_l__IIII--r--I__r__II__L__IIIII.
LIIIJ-IIIltrtrttr--L__t__rtttt-_L--r--ttttttrIl-ILIl-i-IFI----T--tt--+--+--Jrrl--r--rirrl-.
=+-|--j---T--tl--f--t--ltTAvailabilityandLatencyofWWWInformationServers83Ifweexaminethehistogramplots(left),wecanmakeseveralobservations.
First,thedistributionoffailedconnectionsisbimodal,withalargegroupoffailuresoc-curringinthefirstfewsecondsandasecondgroupfailingat75ot90seconds.
Asmallpoftionofbothofthesegloups(seeTable1)areDNSfailures,butmostaregeneratedafternameresolution.
Webelievefailuresthatresolvedquicklywerethosethatwererejectedbythedestinationsite,eitherbecausetheserverwasdownorwasheavilyloaded,orthemachinewasdown.
Almostallofthefailuresclus-teredat75and,90secondsareduetotime-outsgeneratedbytheTCPservicerunningontheclientmachine.
Thesecondobservationisthatthevastmajorityofsuccessfulconnectionsoccurquickly.
Infact,thedatarepresentedbythecumulativedistributionsofFig-ure12showthat99.
1percentofallNA{o-NAconnectionsoccurinthefirst10s,and99.
8percentin20s.
ForNA-to-EU,thenumbersareslightlylower,96.
7pefcentand98.
9percentfor10sand20s,butstillveryhigh.
Thisdatasuggeststhatclientperformancewouldbegreatlyimprovedbyestablishingaclient-sidetime-outintervaldrasticallylowerthanthatoftensuppliedbytheTCPservice.
Forexample,atime-outperiodof10sforNA-to-NAconnectionattemptsand20sforNA-to-EUattemptswouldhavecausedthelossofonlyabout1percentofthesuccessfulconnections,whilesavingtensofsecondsinuserwait-timeforeachconnectionattempt.
ThedistributionofsuccessfulconnectionsforNA-to-NAshowstwoadditionalpeaksafterthemainbodyofthedistribution,oneat2sandanotherat6s.
NA-to-EUalsohasanadditionalpeakat6s.
Thoughnotshownbytheplots,asmallnumberofsitesareresponsibleforthesepeaks.
Forexample,the6speakforNA-to-NAisalmostentirelycomposedofconnectionstoabout20ofthe324NorthAmericanservers.
Weattributethislocalitytoagatewayorotherphysicalbottleneckbetweenthemeasurementsiteandthedestinationservers.
5.
DiscussonInthissectionweaddressinmoredetailsomeoftheissuesthatourmeasurementstraveraised.
5.
1.
HTTPandLatencYOneoftheoriginaljustificationsforHTTPwasaprotocollightweightenoughtoensure100millisecondresponsetimes[Berners-Lee1993].
Ourmeasure-mentsclearlyshowthatonlyahandfulofserversshowedmedianlatencyinthe100mseclange,andmanyserversshowedmedianlatenciesanorderofmag-nitudegreater.
ThisinitselfisnotanindictmentofHTTP:theselatenciesareCharlesL.
VilesadJamesC.
French84afactoflifeontheInternet.
Medianone-waymessagelatencybetweennodesonthecuffentNSFNETT3backbonecanbeasgreatas50nLsec[Merit1994].
Round-tripmessagelatencyfromoursiteontheEastCoastofNorthAmericatotheWestCoastofNorthAmericaisconsistentlygreaterthan100msecforICMPECHO-REQUESTmessagesgeneratedbythepingprogram[Sun1993.
30].
Thesemeasurementsaregoodestimatesoftheinherentnetworklatencybetweentwosites.
GiventhatsettingupaTCPconnectiontakesaminimumofthreemessages[Postel1981],withdatausuallynotbeingtransmitteduntilthefourthmessage,bythetimeaTCPconnectionissetup(andbeforeanyHTTPactionoccurs),la-tencyfromtheuser'sperspectiveisoftenwellabove100msec.
SinceTCPisthedefactostandardforreliablecommunicationontheInternet,connectionlatencyforWebserversisboundedbyboththecurrentphysicalinfrastructureandtheprotocolsusedtodelivertheinformation.
5.
2.
TheCaseforMGETThestatelessnatureofHTTPhasitsdrawbacks.
ManyWebpagesintegratetextandimagesinordertoeffectivelypresenttheinformationofinterest.
SinceHTMLincludesimagesbyreference,aclientmustcreateseparateTCPconnectionsforthetextofthedocumentandforeachincludedimage.
Forexample,apagewiththreeincludedimagesmustpaytheTCPconnectionlatencypricefourtimes.
Asthemeasurementspresentedinthispaperattest,thispriceisnon-trivial.
Ideally,thedocumenttextandimageswouldallbetransmittedduringthesametransac-tion.
Unfortunately,thiswouldrequiretheservertoparsethedocumentinordertoascertainwhatifanyincludedreferencestherewere.
WhilethiscapabilityispresentinsomeHTTPservers,therecanbeaconsiderableserver-sideperfor-mancecostassociatedwiththeparsing.
AreasonablecompromisesolutionwouldbetoextendHTTPbycreatinganewmethodcalledMGET,analogoustothecapabilityofthesamenameimplementedinsomeFTPclients.
TheWebclientwouldthenissueatmosttwoHTTPmethodrequests:thefirstwouldbeaGETforthedocumenttext,andthesecondrequestwouldbeaMGETforallimagesincludedbyreferenceinthefirstdocument.
Thissolutionobviatesparsingattheserver,limitsthenumberofTCPconnectionsforanyWebpagetotwo,andretainsthestatelessnatureofHTTP.
Onepossibledisadvantagemightariseonclientswithlowspeedlinks,inthatMGETwouldcommittheclienttofetchallincludeditems.
Aclientleveluser-abortoptionisappropriateforthesesituations.
Spero[1994]discussesotherperformanceproblemsthatariseduetointer-actionsbetweenHTTPandTCP.
PadmanabhanandMogul|994)performedamoredetailedstudythatquantifiedtheperformancecostofmultipleGETs.
TheyshowmarkedreductionsinlatencyforimplementationsofaGETALL(getallAvailabilityandLatencyofWWWInformationServers85imagesassociatedwiththisdocument)andGETLIST(getthefollowinglistofdocuments).
GETLISTissemanticallysimilartoourMGET.
5.
3.
Long-TermAvailabilitYItdoesnotappearthatWebavailabilitychangedoverthedurationoftheexperi-ment.
Asmentionedpreviously,theslightdownwardtrendinWAvail(Figure6)forNorthAmericansitesismostlyduetothedeathormigrationofsixservers.
Ifweremovetheseserversfromthemeasurements,thenWAvailwasstable.
Infact,asimplelinearregressionoflheWAvaildataincludingthedeadserversyieldedalinewhoseslopetranslatedtothelossof5.
9serversoverthe90-dayperiod.
Thefltofthislinewasrelativelypoor(r2:0.
15),butthisresultatleastlendssomecredencetooursuppositionaboutstableWebavailability.
ThecumulativehistogramsofWAvail(Figure6)illustratethegoodnewsandbadnewsaboutWebavailability.
ThegoodnewsisthattheWebisalmostalwaysatleast90percentavailableandisusuallyinthe95percentrange.
ThebadnewsisthatatnotimewastheWebcompletelyavailable.
Whilesomeserversshowed100percentmeasuredavailabilityoverthelifeoftheexperiment,themajorityofserversweredownatleastsomeofthetime'TheWorldWideWebisarecentphenomenonanditisinterestingtopositabouttheeffectofitstenderageonourmeasurements,particularlyavailability.
Inparticular,wewonderhowmucheffecttheskillofserveradministratorshasonavailability.
V/orldWideV/ebserversoftwarecanbeinstalledandmaintainedbyalmostanyone.
WefeelthisisonereasonwesawsuchahighvariationintheSAvaitmeasurements.
Someserversaremeticulouslymaintained,othersarenot.
Inaddition,publicdomainserversoftwareunderwentmajorchangesoverthelifeoftheexperiment.
Wehavenowayofmeasuringtheeffectofthesechangesonavailability.
Aninterestingquestiontoexaminewouldbetoseeiftherewereanydifferenceinavailabilitybetween"new"serversando'old"servers.
5.
4.
Client-SideTime-outIntervalsThedistributionsofsuccessfulandfailedconnectionsdepictedinFigure12pto-videusefulinformationtotheclient.
Itisclearthatthereisverylittlepayofftowaitinglongerfhan20secondstoestablishaconnection.
Iftheentirecumulativedistributionwereavailabletotheclient,thenuserscouldsetthetime-outintervalfortheirsessionsbasedontheirimmediateinformationneeds.
Simplebrowsingandwanderingmightcallforashorttime-outinterval,whileexhaustivesearchingmightrequirelongerintervalstoensurethatnoserversthatwereupweremissed.
UsersmightusethecumulativedistributiondirectlybyspecifyingtheintervaltheyCharlesL.
VilesandJamesC.
French86werewillingtowait,ortheymightusetheinversefunctiontospecifythecover-agetheywanttoachieveandthensettheintervalneededtoachievethatcoverage.
5.
5.
NamingTheobservationthatsomeservershavemovedordiedunderscoresageneralre-sourceanddocumentnamingproblemthattheWebcommunityhasexperiencedinitsinitialgrowingperiodandwhichwilllikelyonlygetworse.
Initsinitialfor-mulation,andasofthewritingofthispaper,resourceswereexpressedintermsofUniformResourceLocators(URLs).
TheURLmechanismcanbeconsideredtheinstantiationoftheURIlBerners-Lee1993.
5]conceptintheexistingnamingschemesandprotocolsontheInternet.
AURLisessentiallyanaddressforthere-source,notitsname.
Wheneveraresourcemoves,itsURLbecomesuseless.
Thetechnicalcommunityiscurrentlyattemptingtoaddressthisissuethroughthein-troductionofUniformResourceNames(URNs)[WeiderandDeutsch1994],artattemptatlocationtransparentnames.
Atthispoint,theexactshapeofthenamingsystemandthemigrationfromURLstoURNsisasubjectofdebate.
5.
6.
StudyMethodologYAswementionedatthebeginningofSection4,anyinterpretationofthelatencyandavailabilitymeasurementsneedstobemadewiththelocationofthemeasure-mentsiteinmind.
TheresultsreportedhereconsistentlyshowhigheravailabilityandlowerlatencyforNorthAmerica.
IfthemeasufementsitewereinEurope,thenwewouldlikelyseetheoppositebehavior,withlongerlatenciesandloweravailabilityinNorthAmerica.
Wecautionagainsttoogeneralaninterpretationoftheresultshere.
Themea-surementsweremadefromasingleendpointintheWebandwouldlikelybedifferentifmadeelsewhere.
However,themeasurementsitehastypicalIntemetconnectivityforaUSuniversity.
Asitewithpoorerconnectivitywouldalmostas-suredlyshowlongerlatencies,andperhapslowerapparentavailabilityduetolocalnetworkcongestionandmoretime-outs.
Asitewithbetterconnectivitymighttakelesstimetogettoahighbandwidthnetwork(e.
g.
NSFNETorDante),butwouldstillbeboundbythenetworkclosetothedestinationserver.
ChoosinganonsensemethodtosendtotheHTTPserverseliminatedsomebutnotallofthearbitrarysystemdelaysontheserverside.
Forexample,someHTTPserversrunoutofinetdandsomerunasastand-alonedaemon.
\üedidnotattempttocontactserveradministratorstogetthisinformation.
Pagingandmachineloadareotheractivitiesthatmightintroducedelays.
AvailbilityandLtencyofWWWInformationServers87Latenciespresentedhereareoptimisticfromtheuser'sperspective,becausetheydonotincludetheretrievalofanydocumentsbythecontactedserver,onlythetimetosetuptheconnection.
Ifweviewavailabilityintermsofobtainingtherequesteddocumentratherthansimplyserversbeingup,thentheavailabilitymea-surementsarealsooptimistic,sincetheydonotincludethepossibilityofservertime-outsafterconnectionestablishment,orofstaleURLs.
WefeeloursplittingofthesiteslistintoEuropeanandNorthAmericanlistsbydomainnamerepresentsafairsplitofserversbygeographicalarea.
Therearelikelysomesitesinthe.
organd.
eduhierarchiesthatarenotNorthAmericanbased,butthevastmajorityofthesesitesarelocatedaccordingtoourassump-tions.
Examinationofnetworkconnectionsusingtraceroutefsun1993.
3l1verifiedtheEuropeanlist.
6.
SummaryMeasurementswehavemadeofalargenumberofWorldWideV/ebserversin-dicateconsistentavailabilityofserversinthe95percentrange.
WeviewthisasanoptimisticestimateofWebavailability.
WefoundnostatisticallysignificantdifferencesinavailabilitybetweenNorthAmericanandEuropeanservers.
Exam-inationoftheEuropeandatashowedacorrelationbetweenserveravailabilityandgeographiclocationoftheserver(asexpressedbythenetworkpathtotheserver).
Latencymeasurementsshowconnectionestablishmenttakinginthe200-500msecrangeforNorthAmericatoNorthAmericaconnections,andinthe400-2500msecmngeforNorthAmericatoEurope.
Fromtheuserperspective,theselatencyestimatesareoptimisticbecausetheydonotincludedocumentretrieval.
Statis-ticaldifferenceswerefoundbetweenthemedianlatenciestothetwogroupsofservers.
Weattributethisdifferencetothelocationofthemeasurementsite.
Vari-abilityintheselatencieswasconsistentlyhigherinthefirsthalfoftheexperimentthaninthelasthalf,anditisunclearwhichhalfconstitutes'normal'behavior.
Themagnitudeofconnectionlatencyisaffectednotonlybythephysicalconnec-tionbetweenclientandsewer,butthelongset-uptimeforTCPconnections.
WeadvocatetheadditionofanMGETmethodtoHTTPtohelpovercomethelargestart-upcostsassociatedwithretrievalof"compound"documents.
Ourdata(endofSection4.
2.
2andFigure12)indicatethatsettingofclient-sidetime-outintervalswoulddrasticallyimproveworst-caseconnectionattemptswithonlyaveryminoreffectonavailability.
Theparticularintervalmayvryovertime,butongoingmonitoringwouldallowup-to-dateestimatesofthisinterval.
MonitoringwouldalsoallowserveradministratorstoseeavailabilityandlatenciesoftheirserversastheyareviewedfromotherpointsontheIntemet.
CharlesL.
VilesandJamesC.
French88AcknowledgmentsThisworkwassupportedbytheNASAGoddardSpaceFlightCenterunderNASAGraduateStudentResearchProgramFellowshipNGT-51018andbyNASA/CESDISGrant5555-25.
WethankJorgLiebeherr,BertDempsey,andtheanonymousreviewersfortheirhelpfulcomments.
AversionofthispaperappearedasUniversityofVirginia,DepafinentofComputerScienceTchnicalReport#CS-94.
36.
AvailabíIìtyandLtencyofWWWInformationServersReferences1.
A.
K.
Agrawala,D.
Sanghi.
NetworkDynamics:AnExperimentalStudyoftheIntemet,Proc.
Globecom,pp.
182-786,1992.
2.
P.
Beebee,TheSG-ScoutHomePage,availableathttp://www-swiss.
ai.
mit.
edu/ptbb/SG-Scout.
html,1994.
3.
T.
Berners-Lee,HypertextTransrProtocol(HTTP),\W'orkingDraftoftheInter-netEngineeringTaskForce,1993.
4.
T.
Bemers-Lee,D.
Connolly,HypertextMarkupLanguage(HTML),tilorkingDraftoftheIntemetEngineeringTaskForce,1993.
5.
T.
Bemers-Lee,(JniversalResourceldentifiersinWWW:AUnifyingSyntaxfortheExpressionofNamesandAddressesofObjectsontheNetworkasusedintheWorld-WideWeb,InfemetRFC1630,19936.
T.
Bemers-Lee,R.
Cailliau,A.
Luotonen,H.
F.
Neilsen,A.
Secret.
Therù/orldWideV/eb,CommunicationsoftheACM.
37(8):76-82,1994.
1.
T.
Bemers-Lee,TheWorldWideWebInitiative:TheProject,Availableathttp://info.
cem.
ch/hypertextflilWwheProject.
html,I994.
8.
J.
C.
Bolot,End-to-EndPacketDelayandLossBehaviorintheIntemet",ACMSIGCOMM,SanFrancisco,California,pp.
289-298,1993.
9.
N.
Borenstein,N.
Freed.
MIME(MultipurposeInternetMailExtensions):Mech-anismsforSpecifyingandDescribingtheFormtofInternetMessageBodies,InternetWC1341,1992.
10.
H.
W.
Braun,K.
Claffy,WebTrafficCharacterizaúon:AnAssessmentoftheIm-pactofCachingDocumentsfromNCSAsWebServer,SecondInternationalConf.
ontheWorldWideWeb,Chicago,IL,October20-23,1994'11.
R.
Caceres,P.
B.
Danzig,S.
Jamin,D.
J.
Mitzel,CharacteristicsofWide-AreaTCP/PConversations,ACMSIGCOMM,Zuticll',Switzerland,pp.
101-I12,1991.
12.
K.
C.
Claffy,G.
C.
Polyzos,H.
Braun,TrafficCharacteristicsoftheT1NSFNETBackbone,ProceedingsofIEEEINFOCOM,pp.
885-892,1993.
13.
PeterB.
Danzig,K.
Obraczka,A.
Kumar.
AnAnalysisofWide-AreaNameServerTraffic,ACMSIGCOMM,Baltimore,Maryland,pp.
28l-292,1992.
14.
DECNetworkSystemsLaboratory,UsenetFlowAnalysis,availablefromUsenetnewsgroupnews.
lists,1994.
15.
D.
Eichmann,TheRBSESpider:BalancingEffectiveSearchAgainstWebLoad,Proc.
FirstIntl.
ConfontheWorldWideWeb,Geneva,Switzerland,May25-27,1994.
16.
R.
T.
Fielding,MaintainingDistributedHypertextInfostructures:'WelcometoMOMspider'sWeb,Proc.
FirstIntl.
ConfontheWorldWideWeb,Geneva,Switzerland,Nf'ay25-27,1994.
17.
J.
Fletcher,TheJumpstation,availableathttp://www.
stir.
ac.
uk/jsbin/js,1994.
18.
M.
T.
Gray,GrowthoftheWorldWideWeb,Availableathttp://www.
mit.
edu:8001/afs/sipb/user/mkgray/ht/web-growth.
html,1994.
90CharlesL.
VilesandJamesC.
French19.
M.
Lottor,InternetGrowth(1981-1991),IntemetRFC1296,1992.
20.
M.
Lottor,InternetDomainSurveTt,availableathttp://www.
nwcom/,1994.
21.
S.
Maffeis,FileAccessPatternsinPublicFTPArchivesandanIndexforLocalityofReference,ACMSIGMETRICSPerformanceEvaluationReview,20(5):22-35,1993.
22.
M.
Mauldin,J.
R.
R.
lavitt,TheLycosHomePage:HuntingWWWInformation,availableathttpllfuzine.
mt.
cs.
cmu.
edu/mlm/lycos-home.
html,1994.
23.
O.
A.
McBryan,GENVLandWWilW:ToolsforTmingtheWeb,Proc.
FirstIntl.
Conf.
ontheWorldWideWeb,Geneva,Switzerland,M:ay25-27,1994.
24.
MeritNetworks,Inc.
,ANSNETT3DelayMatrixReport,availablebyanonymousftpatftp://nic.
merit.
edu/nsfnelstatistics,1994.
25.
D.
L.
Mills,InternetDelayExperiments,IrfiernetRFC889,1983.
26.
V.
N.
Padmanabhan,J.
C.
Mogul.
ImprovingHTTPLatency,SecondInternationalConfontheWorldWideWeb,Chicago,IL,October20-23,1994.
n.
J.
Postel,1981.
TransmissionControlProtocol,InternetRFC793,1981.
28.
D.
Sanghi,A.
K.
Agrawala,O.
Gudmundsson,B.
N.
Jain,ExperimentalAssess-mentofEnd-to-EndBehavioronIntemet,ProcIEEEINFOCOM,pp.
867-874,1993.
29.
S.
Spero,AnalysisofHTTPPerformanceProblems,availableathttp://elanor.
oit.
unc.
eduhttp-probs.
html,1994.
30.
SunMicrosystems,Ping,SunOSUser'sManual,1993.
31.
SunMicrosystems,Traceroute,SznOSUser'sManual,1993.
32.
C.
Tronche,TheWWWMMRobot,availableathttp://www-ihm.
lri.
frl-tronche/W3M2/,1994.
33.
C.
Tronche,PersonalCommunication,1994.
34.
C.
Weider,P.
Deutsch,UnirmResourceNmes,WorkingDraftoftheInternetEngineeringTaskForce,1994.
AvailbilíryndLatencyofWWWInformtionSer-ters9l