equal114dns
114dns 时间:2021-01-10 阅读:()
TrendsinWideAreaIPTrafficPatternsAViewfromAmesInternetExchangeSeanMcCreary(mccreary@caida.
org)kcclaffy(kc@caida.
org)UniversityofCalifornia,SanDiegoCooperativeAssociationforInternetDataAnalysis(CAIDA)U.
S.
A.
Contents1.
Introduction2.
Background1.
MonitorSite2.
MonitoringMethodologyandTools3.
AnalysisMethodologyandTools3.
Results1.
PacketLengths2.
ProtocolMix1.
RawData2.
LongTermTrends3.
ShorterTermTrends4.
ConclusionsandFutureDirections5.
Acknowledgments6.
ReferencesAbstractWereportresultsfromalongitudinalanalysisoftheIPtrafficworkloadseenatasinglemeasurementsiteinsideamajorInternettrafficexchangepoint.
UsingdatacollectedbytheNLANR/MOATNetworkAnalysisInfrastructure(NAI)project[NAI]andanalysissoftwarefromCAIDA'sCoralReefproject[CoralReef],wepresenttrendsinapplicationusageseenattheNASAAmesInternetExchangeover10months,fromMay1999throughMarch2000.
Weshowchangesinthefractionoftrafficfromstreamingmediaandonlinegaming,aswellasanincreaseintrafficfromnewapplicationssuchasNapsterandIPSECtunneling.
WealsoshowthatourdatadoesnotindicateanyoverallchangeintheTCP/UDPtrafficratioattheAmesInternetExchangeduringthisperiod,orsignificantdifferencesfromtheanalysesbyMCIWorldcomandCAIDAin1998.
IntroductionOvertheyearstherehasbeenagreatdealof'commonwisdom'developedaboutthenatureofwideareaInternettraffic.
Unfortunately,theserulesofthumbhavedevelopedintheabsenceofobservationsaboutthecompositionofactualtrafficcarriedintheInternetbackbone.
ManyanalysesofInternettrafficbehaviorrequireaccurateknowledgeofthetrafficcharacteristicsinthebackbonetoday,forpurposesrangingfromtheoptimizationoffuturenetworkingequipmenttomodelingtheeffectsofnewprotocolsontheexistingtrafficmix.
CAIDAaimstofillthisgap,supplyingaccurateinformationaboutbackbonetrafficcharacteristicstobothindustryandtheacademiccommunity.
Otherstudies[Feldmann98]haveusedInternettraffictracedataasinputforevaluationofspecificprotocolperformanceissues,buthavenotconcentratedoncharacterizingtheoverallworkloadcontainedinthetraces.
Therearealsoseveralongoingprojects[Feldman98,AIXstats]thatmonitortheutilizationofnetworklinksatourmeasurementsiteintermsofthenumberofpacketsandbytestransferredovertime.
However,thesemonitoringprojectsdonotattempttoprovideafinergrainedpictureofthelinkutilizationintermsoftheprotocolsinuseonthelinks.
EarlierworkloadanalysisstudiesintheNSFnetbackbone[Heimlich89,Claffy93,Merit95]includedcourse-grainestimatesofthefractionoftrafficduetothemostpopularapplicationsbeforetheadventofwebbrowsers.
ThisincludedbasicstatisticsabouttheprevalenceofFTP,e-mail,netnews,telnet,DNS,andasinglecategoryfor'other'TCPandUDPapplications.
Severalstudiesdetailingtheworkloadpresentedattheborderoflargeorganizations[Caceres89,Caceres91,Paxson94,WISCstats]haveincludedtrafficbreakdownsbyprotocolaswell.
Thesesite-specificstudiesformacomplementtobackbonestudies,astheycanrevealdifferencesinthewaydifferentkindsoforganizationsusetheInternet.
Previousstudiesofbackbonetraffic[Apisdorf97,Thompson97,claffy98]haveanalyzedtheworkloadpresentedattwositesinsidetheInternetMCIbackbone(AS3561),nowownedbyCableandWireless.
Thesestudiesspannedarelativelyshortperiodoftime,rangingfromasingledaytoafullweek.
Theseshort'snapshots'ofbackbonetrafficarenotlongenoughtoaccuratelydeterminetrendsinthetrafficmix,asweseealargeamountofvariabilityonthesetimescales.
InthispaperwepresentananalysisoftheworkloadseenatAIXoveramuchlongerperiod,spanningmorethan10months.
NLANR/MOAT'sDatacubeproject[MOAT99]isaframeworkforstoringandanalyzingtracescollectedfromasetofCoralmonitorsatHPCsites.
Thethreedimensionsofthe'cube'are:dataorigin(measurementlocationwherethedataoriginated);projectname(e.
g.
,analysismethod);anddatacollectiondate.
MOATmakesthepublicdataavailableonitswebserverviathisDataCubestructure,toallowdataexplorationalongeachdimensionofthecube.
Inthenextsection,wepresentbackgroundinformationaboutourdatacollectionsiteandthetoolsweusedtocollectandanalyzethedata.
Wethenpresentsomesampleresultsfromourstudy,ananalysisofthedistributionofpacketlengths,andsometrendsintheprotocolmixseenatoursamplingsite.
WeconcludewithasummaryofthechallengeswefaceincontinuingandextendingourmeasurementprogramatCAIDA.
BackgroundTheMonitorSiteThetracesusedforthisstudywerecollectedfromtheNASAAmesInternetexchange(AIX)inMountainView,CA[AIX]aspartofanNSF/NASAcollaborativeeffortwithNLANR/MOAT.
Theywerecollectedfromoneoffour(nowfive)OC-3ATMlinksthatinterconnectAIXandMAE-WestinSanJose,CA.
Fig1:Adiagramshowingthelocationoftheopticalsplitterusedtocollectthedata.
NotethattherearecurrentlyfivelinksbetweenNASA-AmesandMAE-West.
ThankstoHans-WernerBraunandNLANR/MOATforuseofthisfigure.
ThisgroupoflinksformastripedconnectionbetweentwoDECGigaswitches,withanaggregatebandwidthofapproximatelyasingleOC-12link.
TheGigaswitchesuseaproprietaryschedulingalgorithmforsendingpacketsacrossthislink,buteachpacketissentacrossanindividuallinkinsideanAAL5PDU.
ThismeanstheschedulinginsidetheGigaswitcheshappensatthepacketlevel,sinceallcellsfromaPDUaresentoverthesamelink.
Consequently,thedatawecollectfromthissiteisessentiallysub-sampledfromtheactualdatatraversingthislinkusingtheproprietaryschedulingalgorithminsidetheGigaswitches.
Thisalgorithmisapproximatelyround-robin,butalsodependsoninternalloadcharacteristicsintheGigaswitchswitchingfabric.
However,weknowthatthedistributionofpacketsamongtheOC-3linksisnotentirelyuniform,sincemeasurementsofthelinkutilizationsshowtwoofthemcarryapproximatelytwicethetrafficoftheothertwo(measuredbybytevolume,notpacketvolume)[Feldman98].
However,weassumethattheGigaswitchschedulingalgorithmisindependentoftheencapsulatedprotocol,e.
g.
notdependentuponpacketlength.
Becauseofthiscomplexschedulingalgorithmwearenotabletoaccuratelyestimatethenumberofconversationstraversingthemonitoredlinks,orthelengthoftheseconversationsinpacketsorbytes.
Consequently,weonlycharacterizetheworkloadobservedatAIXintermsofrelativefractionsofpacketsandbytes.
MonitoringMethodologyandToolsThedatacollectionsystemisessentiallysimilartotheoneusedin[Thompson97].
ACoral/OC3monplatformwasconnectedtoonelinkineachdirectionusingopticalsplitters.
ThetraceswestudiedwerecollectedaspartofNLANR/MOAT'sNetworkAnalysisInfrastructure(NAI)project[Braun98].
Foreachpacketthatpassesthemonitor,onlythefirstATMcellfromtheAAL5PDUiscapturedandwrittentodisk.
Thefirstcellcontainsthefirst40bytesofeachpacket,whichisusuallyenoughtoextracttheTCPorUDPportnumbersfromthetransportlayerheaders.
However,themonitordoesnotverifythattheentireAAL5PDUiscarriedbythelink,andsoestimatesofthedataratecarriedbythelinkmaybeinflatedinthepresenceofcellloss.
Sixtoeighttraceswerecollectedeachday,usuallywithadurationof90secondseach.
Thestartingtimeforeachtracewassetatequalintervalsduringthe24hourperiod,andrandomizedoverarangeofanhouratthebeginningofeachinterval.
Aftercollection,thetracesareprocessedtoremoveanyinformationthatmightcompromisetheprivacyoftheindividualsgeneratingthetraffic.
ThisprocessingmasksthesourceanddestinationIPaddresses,anddeletesalldatafromtheIPpayloadexceptforaTCPorUDPheader(ifpresent),ortheICMPorIGMPtypeandcodefields.
IfthepacketcarriesenoughbytesofIPheaderoptions,thentheTCPorUDPportnumbersmaynotbepresentinthefirstcellofthePDU.
Inthiscase,weignorethatpacketinsubsequentapplicationworkloadanalysis.
SincethefractionofpacketswithIPheaderoptionsistypicallylessthan0.
003%,thisdoesn'tseriouslyimpactourmeasurementsofthetrafficfractiongeneratedbythemostpopularTCPandUDPapplications.
AnalysisMethodologyandToolsWeusedCoralReef[CoralReef]toreduceeachrawtracetoasetofsummarytablesthatwearchivedforlateranalysis.
ThetablesincludeaggregatenumberssuchasthenumberofpacketsandbytesinthetraceaswellasdistributionsofpacketlengthsandthenumberofpacketsandbytesseenforeachIP-layerprotocol.
ForTCPandUDP,weanalyzeapplicationusageusingportaddresspairs.
ThepackettracesavailablefromtheNAIarchive[Braun98]onlyincludeIPandtransportlayerheaders,soourmethodologydoesnotuseencapsulateddatatoidentifytheapplicationthatgeneratedthepackets.
TracesintheNAIarchivehavehadallpayloaddataremovedtoprotecttheprivacyofInternetusers.
Inmostcases,wehaveassumedthatpacketssentbetweenanyportnumberhigherthan1023andawell-knownportnumberbelow1023aregeneratedbythesameprotocol(e.
g.
,HTTPonport80).
Thismatchestypicalendhostbehavior,inwhichclientsallocateephemeralportsfromtherange1024to32767[Stevens94].
Forsomeoftheprotocols,wehavecondensedrangesofportnumbersinboththesourceanddestinationfields.
Forexample,theRealAudiocategoryintheUDPtableincludesalltrafficwithdestinationportsbetween6970and7170inclusive[RealNetworks].
Unfortunately,thisrangealsoincludestheportsusedbyAFS,andsowearepotentiallyconfusinganunknownamountofAFStrafficwithRealAudio.
However,themajorityofRealAudiotrafficappearsonUDPports6970,6971,and6972,noneofwhichareusedbyAFS.
ByonlyconsideringtrafficonUDPportsfromthisrangethatarenotusedbyAFS,theamountofRealAudiotrafficcanbeestimatedindependentlyfromtheamountofAFStrafficthatmaybepresentaswell.
WearecurrentlyinvestigatingbettertechniquesfordifferentiatingRealAudioandAFStrafficusingpacketsizedistributionandpacketinterarrivalpatterns,andwehopetobeabletoconclusivelydifferentiatebetweenthetwointhefuture.
ArecentanalysisofthetrafficpatternsexhibitedbyRealAudiotraffic[Mena00]hasshownseveralparametersthatmaybeusedtodifferentiatebetweenRealAudioandotherprotocols.
AfurtherstudycharacterizingAFStrafficpatternsneedstobeundertakentoidentifythebestmetricstousetoseparatethetwo.
ForbothTCPandUDPtraffic,thereisasignificantfractionoftrafficthatcannotbemappedtoapplicationsusingwellknownportnumbers.
Manyprotocolsdonotdependonwell-knownportnumbers,buteitheruseawell-knownservicefornegotiatingtheportnumbersusedbysecondaryconnections,orusearbitrarybutfixedportnumbersthatarenotregisteredwithIANA.
Themostpopularapplicationwithnegotiatedportnumbersispassive-modeFTP,inwhichtheclientsendstheportnumbertouseforadataconnectionoverthecommandchannel.
Therearemanyotherprotocolsthatusesimilarbehavior,suchasNapsterandInternettelephonyapplications.
Mostonlinegamesdonotregisterwell-knownportswithIANA,butusearbitraryportnumbersabove5000.
Wehavecollectedtheportnumbersusedbyseveralofthepopulargamesandusethisinformationtoestimatethefractionoftrafficgeneratedbythem.
OuranalysisofonlinegametrafficincludesgametrafficonthefollowingUDPports:HalfLifeanytoorfrom27005anytoorfrom27015Quake3:Arenaanytoorfrom27960Starcraft6112to6112QuakeIIanytoorfrom27901anytoorfrom27910QuakeWorldanytoorfrom27500anytoorfrom27001Unrealanytoorfrom7777Table1:UDPportsusedbyOnlineGamesAsisthecasewithRealAudioandAFS,therearemanypossibilitiesforconfusionbetweengametrafficandotherapplicationswhenonlyportnumbersareusedtomaketheclassification.
Weassumethattherearenootherprotocolsthatpreferentiallyusethesesameports,andthatapplicationsthatephemerallyusetheseportscontributeequalamountsoftrafficacrossalltrafficcategories.
Thisassumptioncarriessignificantrisks,andneedsfurtheranalysistofullyevaluateitsimpactonourdata.
ResultsPacketlengthsThefollowinggraphsshowthedistributionofIPpacketsizesseenatAIX.
Thesedistributionsarebuiltfromtwoapproximatelyoneweekperiodsnearthebeginningandendofourstudy.
Theycontaincontributionsfromthedifferentworkloadscarriedbythenetworkatdifferenttimesofday,andsotheyshouldrepresentmoreofan'average'pictureofthepacketsizedistributionthananyindividualtrace.
However,noattempthasbeenmadetonormalizethecontributionsofindividualtraces.
Thedistributionspresentedherearesimplythoseoftheconcatenatedtraces.
Additionally,thesedistributionshaveonlybeenplottedforpacketsizeslessthan1600bytes.
Thisallowsthestructureofthedistributionstobepresentedingreaterdetail,buthidestheverysmallfractionoflargerpacketsthatappearinthesetraces(typicallylessthan0.
005%ofpackets).
Fig2:IPpacketlengthdistributionfrom39tracefilescapturedbetweenThursday,May13th1999at19:14:36PDTandWednesday,May19th1999at13:02:20PDT,overanintervalofalmostsixdays.
Statisticsfortheunderlyingpacketlengthdistribution:Mean:413bytes,StandardDeviation:509bytesMedian:93bytes,Percentiles:5th40bytes,25th40bytes,75th576bytes,95th1500bytesNumberofObservations:127millionpackets[127710031packets]Thesenumberscorrespondtotheuppercurveinthefigureabove.
Fig3:IPpacketlengthdistributionfrom43tracefilescapturedbetweenSunday,February20th2000at16:01:51PSTandSunday,February27th2000at13:59:18PST,overanintervalofalmost7days.
Statisticsfortheunderlyingpacketlengthdistribution:Mean:420bytes,StandardDeviation:521bytesMedian:78bytesPercentiles:5th40bytes,25th40bytes,75th576bytes,95th1500bytesNumberofObservations:84millionpackets[84415871]Thesenumberscorrespondtotheuppercurveinthefigureabove.
TheprimaryfeaturesofthisdistributionoriginateinthewaycommonTCPimplementationsdivideadatastreamintopackets.
Approximately85%ofthetrafficinthesetracesisTCP,andalargeproportionofthisTCPtrafficisgeneratedbybulktransferapplicationssuchasHTTPandFTP.
Consequently,themajorityofthepacketsseenareoneofthreesizes:40bytepackets(theminimumpacketsizeforTCP)whichcarryTCPacknowledgmentsbutnopayload;1500bytepackets(themaximumEthernetpayloadsize)fromTCPimplementationsthatusepathMTUdiscovery;and552byteand576bytepacketsfromTCPimplementationsthatdonotusepathMTUdiscovery.
Thesetwodistributionsarestrikinglysimilardespitethefactthatthesecondisbasedondatacollectedmorethan9monthsafterthefirst.
Theseconddistributionhasaslightlylargercontributionfrompacketssmallerthan100bytes,butthedifferenceisquitesmall.
Thefollowinggraphshowshowthemeanandmedianvaluesofthepacketsizedistributionvaryovertheentiredurationofourstudy.
Fig4:MeanandmedianpacketlengthvaluesforeachpackettracecollectedbetweenThursday,May13th1999at19:14:36PDTandSatMar1113:57:38PST2000.
Valueshavebeenbinnedbyweek,andthemedianforeachbinisplottedwithfirstandthirdquartileerrorbars.
Itisnotsurprisingthattheseparametersdonotshowanysignificantlongtermtrend.
AslongasTCPisthedominantprotocolinuseintheInternet,thepacketlengthdistributionisunlikelytochangeverymuchunlesstheTCPprotocolitselfchanges.
ProtocolMixRawDataInthetablesbelow,weprovidesomesampledatafromthemonthofFebruary,2000.
Eachofthe216tracesfromthisperiodwereconcatenated,andthesummaryinformationpresentedbelowisforallthetracescombined.
TotalIPBytes193692014407TotalIPPackets451971619TotalDurationofTraces24345secIPpacketswithDFset329241464FragmentsofIPDatagrams1023206FragmentsofUDPDatagrams597228FragmentsofTCPDatagrams9702IPPacketswithoptions3213Non-IPpackets462085Table2:AggregatetotalsforalltracescollectedinFebruary,2000Table3presentsthetop10IPlayerprotocolsseeninthesetraces.
TCPandUDPtypicallyaccountforalmostallofthetrafficseeninanytrace,withGREandICMPasthenexttwomostpopularprotocolsProtocolNumberPacketsBytesAverageSizeTCP6374801201176706563104471UDP17624567319842511709157GRE4775664155272240819696ICMP159380441350011401227ESP50517353197216792381IPinIP4265103179257606676AH517442343454671583IPIP9414350241707350290SKIP5711705041633952355IGMP2684047729038112Table3:Top10protocolsseenduringFebruary,2000Inthenexttwotables,wehaveconsolidatedtheTCPandUDPportaddresspairsintocategoriesbyapplication.
Asdescribedpreviously,wehaveassumedthattrafficbetweenanyportnumberhigherthan1023andawell-knownportnumberbelow1023isexclusivelygeneratedbythesameprotocol(e.
g.
HTTPonport80).
Wedenotethecondensedsetofportnumbersinthetablewiththelabel'0'.
Forsomeoftheprotocols,wehavecondensedrangesofportnumbersinboththesourceanddestinationfields.
Forexample,theRealAudiocategoryintheUDPtableincludesalltrafficwithdestinationportsbetween6970and7170inclusive,representedwiththelabel'7070'.
NotethatasignificantportionofbothTCPandUDPtrafficfallsintoacategorylabeledwith'0'inboththesourceanddestinationfields.
Thisincludesallthetrafficthatwasnotmappedtoaspecificprotocolusingourmethodology.
Thistrafficiseithergeneratedbyprotocolsthatusenegotiatedportnumbersatbothends(e.
g.
,passive-modeFTP),orbynewapplicationsthatuseunregisteredfixedports.
Wecontinuetoaddportutilizationprofilesfornewprotocolsasweobtainnewandupdatedinformation.
ProtocolSourceDestinationPacketsBytesAverageSizeHTTP800140780543100044030753710004531984217319763013382NNTP01191789548115992967942893HTTP08094578965784416385082FTPData20067280976689611587994SMTP02588789256071084052683NNTP119088572175399672480609HTTP/WebProxy8080023316692327032104998Napster0669933311091838804438552HTTPS443030358091535037132505Napster6699033778281528188686452FTPData02054980971262294037229Napster668801230335935883810760175501182358908626624768POP311001820887798255125438Hotline550106855367871220081148RTSP5540754508616087123816Napster066881149382348845629303SMTP250643867233978802052RealAudio70700445551298598442670Shoutcast80000353545296161291837WebCache31280325447280739543862HTTPS04432635048280418901106NetBIOSSSN139031265826496521284721890294654174140223590Table4:Top25TCPapplicationcategoriesseenduringFebruary,2000ProtocolSourceDestinationPacketsBytesAverageSize00151088222568130721169RealAudio0707046100702029483625440DNS535392908721064980650114DNS5303444558638796849185HalfLife27015270052199098452384485205DNS05352865543325982496206770619334230280312371Starcraft61126112416762521778375552EverQuest90019000908432171755388189HalfLife2700527015275441616080617658RealAudio69700532356154663054290Unreal777701109005141327485127EverQuest90059000777485138637166178Unreal07777189261310761327956Quake3:Arena279602796078425880146922102HalfLife27015042461477756002183QuakeII27901279109466745884258362HalfLife02700531773558062692182022503374599504691328001025508144789873175NetBIOSNS1371375098544399607586QuakeII27910279012328654394243818803714653140574996871CU-SeeMe7648764813684339681727289QuakeII2790106014273732778562HalfLife2700504983002919106258Table5:Top25UDPapplicationcategoriesseenduringFebruary,2000LongTermTrendsIneachofthetimeseriesgraphsbelow,wehavecollectedthedatavaluesintoweek-longbins,andweplotonlythemedianandfirstandthirdquartilesforeachbin.
Forgraphswithmorethanonedataseries,wehaveintroducedaslightoffsettothebinsforeachseriestopreventoverlap.
Wepresentonlythefractionoftrafficcontributedbyeachapplicationorapplicationgroupratherthanabsolutemeasurementsoftrafficvolumeinanycategory.
Shiftsintherelativefractionsamongmultipleprotocolsorapplicationsovertimerepresentchangesinhowthenetworkisused,ratherthanhowbusyourmeasuredlinkisatanygiventime.
However,duringtheperiodofthisstudy,themonitoredlinkhadamedianutilizationofapproximately85Mb/s.
Medianandquartilevalueshavebeencalculatedusingthemethodologydescribedin[RFC2330].
Specifically,thequartilevaluesareactualdatapointsandnotinterpolatedvalues,andthemedianiseitheranactualdatapointortheaverageofthetwomiddlevalues.
Eachweeklybintypicallycontains40-50observations.
ThefirstgraphshowsthefractionofpacketsduetoTCPandUDPseenatAIX.
Thegraphdoesnotshowanyclearlongtermshiftsinthebalancebetweenthesetwodominanttransportlayerprotocols.
DespitetheincreaseinthenumberofUDPapplicationsinrecentyears,growthinthetotalamountofUDPtrafficisoffsetbygrowthinTCPtrafficaswell.
Figure5:FractionofTCPandUDPpacketsseenatAIX.
Tracesarecollectedintoweeklybins,andthemedianandfirstandthirdquartilesforeachbinareplotted.
ThisnextgraphshowsthefractionofIPSECtrafficseenatAIX,includingbothauthenticationheader(AH)andencapsulatingsecuritypayload(ESP)traffic.
Thegraphshowsanincreaseofalmostanorderofmagnitudeduringthefirst5monthsofourstudy,butthenlevelsofforevendeclinesslightly.
Thissuggeststhatafteraninitialperiodofseriousinterest,IPSECtraffichasstoppedgrowingfasterthannon-IPSECtraffic.
Figure6:Fractionofauthenticationheader(AH)andencapsulatingsecuritypayload(ESP)trafficseenatAIX.
Tracesarecollectedintoweeklybins,andthemedianandfirstandthirdquartilesforeachbinareplotted.
Theamountoffragmentedtrafficinwideareanetworkshasbeenatopicofgreatinterestinthepastfewmonths,especiallyasitrelatestoIPtracebacktechniques[Savage00].
Ourdataindicatesthatthefractionoffragmentedtrafficisontherise,andthatthemajorityofthisgrowthisintheformofUDPpackets.
WeneedtoperformfurtheranalysistodeterminewhichUDPprotocolsaregeneratingthesefragments.
Notsurprisingly,TCPtrafficisvirtuallyneverfragmented.
MostlikelythisisduetowidespreaddeploymentofpathMTUdiscoverycombinedwithTCP'srelativelysmalldefaultpacketsize.
Figure7:Fractionoffragmentedtraffic.
Tracesarecollectedintoweeklybins,andthemedianandfirstandthirdquartilesforeachbinareplotted.
FTPisthetraditionalbulktransferprotocolwidelyusedbeforetheadventofHTTPandtheweb.
However,weseeacleardeclineinthecontributionofFTPtotheoveralltrafficmixsinceOctober1999.
ThisdeclinemayactuallybeduetoashiftfromactivetopassivemodeFTPassociatedwithanincreaseinpacketfilteringfirewallsintheInternet,oritmaybeduetoashiftawayfromFTPtoalternativeprotocolsforfiletransfer.
However,thegraphofHTTPtrafficfractionoverthissametimeperioddoesnotshowacorrespondingincrease.
Figure8:ThefractionofactivemodeFTPtrafficisdeclining.
Tracesarecollectedintoweeklybins,andthemedianandfirstandthirdquartilesforeachbinareplotted.
ThefollowinggraphshowsarelativelysurprisingdecreaseinthefractionofRealAudiotrafficseenatAIX.
BothTCPandUDPRealAudiotraffichavedecreasedinpacketvolumecomparedtonon-RealAudiotraffic,althoughthistrendseemstohaveflattenedoutinthelastfewmonthsofourstudy.
AlthoughRealNetworkshasreleasednewerversionsoftheirRealPlayersoftwaresincethisstudybegan,thenewversionsusethesamesetofTCPandUDPportsastheoldversions,withtheadditionofTCPport554forRTSPtraffic[RealNetworks].
Consequently,thetrendweobserveisnotduetoashiftfromtheoldersoftwaretothenewerversions,butrepresentseitheraslowinginthegrowthofRealAudiotrafficoradeclinerelativetothegrowthinnon-RealAudiotraffic.
Figure9:ThefractionofRealAudiotraffichasdeclinedoverthepast10months.
Tracesarecollectedintoweeklybins,andthemedianandfirstandthirdquartilesforeachbinareplotted.
Thisgraphshowsadeclinetheinthefractionoftrafficgeneratedbyseveralpopularonlinegamesoverthefirst8monthsofourstudy.
TheonlinegamesincludedinthisgraphareStarcraft,QuakeII,andQuakeWorld(avariantofQuakeII).
Thesegameswerepopularwhenwestartedcollectingourdata,butasthegraphshows,theirpopularityhasdeclinedfairlysteadilysinceJuly,1999.
Figure10:Fractionofonlinegametraffic,includingQuakeII,QuakeWorld,andStarcraft.
Tracesarecollectedintoweeklybins,andthemedianandfirstandthirdquartilesforeachbinareplotted.
Thissecondgraphofonlinegamingtrafficshowsquiteadifferenttrendfromthepreviousgraph.
Unlikethepreviousgraph,thisoneincludestrafficfromseveralnewergamesinadditiontotheolderones.
ThenewgraphincludestrafficgeneratedbyHalfLife,Quake3:Arena,andUnrealinadditiontoStarcraft,QuakeII,andQuakeWorld.
Althoughthereisnotenoughdatatodetermineatrend,themediantrafficfractionsaremuchhigheronthisgraphthanthepreviousone.
Hence,theoverallfractionofonlinegametrafficseemstobeontherise,butitisamovingtarget.
Theincreaseisprimarilygeneratedbynewgamesastheygainpopularity,whileoldergamesseemtodecreaseinpopularityovertime.
Figure11:Fractionofonlinegametraffic,includingHalfLife,QuakeII,QuakeWorld,Quake3:Arena,Starcraft,andUnreal.
Tracesarecollectedintoweeklybins,andthemedianandfirstandthirdquartilesforeachbinareplotted.
AlthoughNapsterdoesnotuseafixedsetofportsforfiletransfers,weidentifiedthethreemostcommonlyusedTCPportsinuseinlateJanuary:TCPports6688,6697,and6699.
Bulktransfertrafficmaybeeithersenttoorreceivedfromtheseports,sinceNapstersupportsbothactiveandpassivemodetransfers[Napster].
AsUniversitiesandothersitesmovetoblocktrafficontheseports,Napstertrafficwillundoubtedlymigratetoothers.
However,theshorttermtrendclearlyshowsdramaticgrowth,increasingbyover50%inthelasttwomonthsofourstudy.
Figure12:FractionofNapsterbulk-transfertrafficseenatAIX.
Tracesarecollectedintoweeklybins,andthemedianandfirstandthirdquartilesforeachbinareplotted.
ShorterTermTrendsSomeshortertermtrendscanbeeasilyassociatedwithcommonuserbehavior.
Forexample,thefractionofemailtraffic(SMTPpacketstoorfromTCPport25)increasedsignificantlyinNovemberandearlyDecember,onlytodropoffagainduringtheholidaysattheendofDecember.
Perhapsthiswasrelatedtoonlinecommerce,asmanypeopleusedtheInternettopurchasetheirChristmasgiftsFigure13:FractionofmailtrafficseenatAIX,showingapeakinusagejustbeforetheChristmasholidays.
Tracesarecollectedintoweeklybins,andthemedianandfirstandthirdquartilesforeachbinareplotted.
Onlinegamingisclearlymorepopularonweekends,asthefollowinggraphshows.
Thefractionofgametrafficcannearlydoubleonweekendscomparedtothetypicalweekday.
Thischangeinworkloadoverthecourseofaweekhasbeensmoothedoutinourpreviousgraphsbyourchoiceofbinsize.
Figure14:WeeklyvariationsinthefractionofonlinegamingtrafficseenatAIX.
Tracesarecollectedintodailybins,andthemedianandfirstandthirdquartilesforeachbinareplotted.
ConclusionsandFutureDirectionsInourlongtermstudyofthetrafficworkloadattheAmesInternetExchange,wefoundnosignificanttrendsovertimeineithertheoverallpacketsizedistributionorchangesintheratioofTCPtoUDPtraffic.
WedidfindanorderofmagnitudeincreaseinthevolumeofIPSECtraffic,andastrongincreasingtrendinthevolumeoffragmentedUDPtraffic.
Someotherinterestingtrendsincludedecreasesinthevolumeofactive-modeFTPandRealAudiotraffic,andincreasesinthevolumeofonlinegamingandNapstertraffic.
Wealsofoundshorttermtrends,includingatransientincreaseinthevolumeofe-mailtrafficduringtheholidayshoppingseason,andastrongweekday/weekendvariationinthevolumeofonlinegamingtraffic.
Thecollectionofbackbonetrafficstatisticshasbeenseverelyimpactedbythebreakupofthenetworkcoreintoanumberofmutuallycompetitiveorganizations.
DatacollectioneffortsmustnowprotectboththeprivacyofindividualInternetusersandtheproprietaryinterestsofthehostorganizations.
CAIDAisinauniquepositiontomonitorandanalyzetrafficcollectedfrombackbonelinks,asithasongoingrelationshipswithseveralcoreISPsandexchangepointproviders.
Ourworkloadcharacterizationeffortsalsofaceadditionaltechnicalchallenges,bothinextendingourmethodologytoincreasetheaccuracyofourmeasurements,andincopingwiththeintroductionofnewprotocols.
Ourportbasedtrafficclassificationschemehassomeseverelimitations,asillustratedbythevolumeoftrafficitfailstomaptoanyspecificprotocol.
Additionaltechniquesarenecessarytofurtherdifferentiatethistraffic,aswellasreduceconfusionamongprotocolsthatusethesamesetofsourceordestinationports.
Furthermore,inordertoaccuratelyclassifyprotocolsthatusenegotiatedportsonbothendsitwillbenecessarytocorrelatemultipletrafficstreamsfromthesamehosttothesameordifferentdestinations[Plonka00].
However,thesetechniquesmayhavelimitedusabilityintheInternetcorewherethemeasurementsiteisfarawayfromboththesourceanddestinationofthetraffic.
Consequently,wemaybeunabletoeffectivelysortthistrafficbyprotocolwithoutusingtechniquesthatmonitorthealltrafficexchangedontheassociatedcontrolchannels,asthesechannelstypicallyuseawell-knownportatoneendevenifthedatachanneldoesnot.
IPSECrepresentsanotherpotentialproblemforworkloadanalysis,asESPtrafficencryptsthesourceanddestinationportsweusefortrafficclassification.
IfuseofESPbecomeswidespread,wemaylosetheabilitytoestimatehowthenetworkisbeingutilizedaltogether.
[Bellovin99]proposesamodificationtoESPthatsendscleartextportnumbersforencapsulatedtraffic.
Ifthismodificationwasadopted,ESPwouldnotposesuchaproblemforourmeasurements.
TheinterconnectionlinkbetweenAIXandMAE-WesthasrecentlybeenupgradedtoasingleOC-12PacketoverSONET(POS)link.
Thiswillimprovethedatacollectionsite,andallowustoperformpacketflowanalysissinceeverypacketsentbetweenAIXandFIXWestwillbevisibletoourmonitor.
However,itwillalsorequirePOSsupportforCoral/OCXmonhardware.
DevelopmentofadditionalhardwareandsoftwaretosupportPOSlinksatOC-3andOC-12speedsiscurrentlyunderwayatCAIDA,andwehopetocontinuethisanalysisafterthelinkupgradewithminimalinterruptionindatacollection.
CAIDAisalsointerestedinotherbackbonesamplingpointsforcomparisonagainsttheworkloadseenatAIX.
PreviousstudiesdonewithdatacollectedbyMCI[claffy98,Apisdorf97,Thompson97]bearsomesimilaritiestoourresults,butmoredatacollectionmustbeperformedbeforeanyconclusionscanbedrawnabouthowrepresentativeourresultsareofotherwide-areaInternetinfrastructure.
AcknowledgmentsThisanalysiswouldnothavebeenpossiblewithouttheworkofHans-WernerBraunandtheNationalScienceFoundation'sNLANR/MOATprojectforcollectingandarchivingInternettraffictraces.
TheparticularmonitorthatweusedattheAMESInternetExchange(AIX)existsasaresultofacollaborationamongNASA,NLANR,andCAIDA.
WearegratefultoBillJones,LanceTatman,BobbyCates,andJeffOsbourne(allfromNASA-Ames),andtoSteveFeldman(previouslyatMCI/Worldcom)fortheircontinuedsupportandmaintenanceofthisandothertrafficmonitorsattheexchangepoints.
WewouldalsoliketothankKenKeys,DavidMoore,andtherestoftheCoralReefteamfortheanalysislibraryusedthroughoutthisproject.
References1.
[AIX]AmesInterneteXchange,http://aix.
arc.
nasa.
gov/.
2.
[AIXstats]AmesInterneteXchangeLinkUtilizationStatistics,http://aix.
arc.
nasa.
gov/graphlink.
html.
3.
[Apisdorf97]J.
Apisdorf,kclaffy,K.
Thompson,andR.
Wilder.
OC3mon:Flexible,Affordable,High-PerformanceStatisticsCollection,http://www.
isoc.
org/isoc/whatis/conferences/inet/97/proceedings/F1/F1_2.
HTM.
4.
[Bellovin99]S.
Bellovin.
Transport-FriendlyESP,http://www.
research.
att.
com/~smb/talks/tfesp-ndss/index.
htm.
5.
[Braun98]H.
-W.
Braun.
TowardsasystemicunderstandingoftheInternetorganism:aframeworkforthecreationofaNetworkAnalysisInfrastructure,http://moat.
nlanr.
net/NAI.
6.
[Caceres89]R.
Caceres.
MeasurementsofWide-AreaInternetTraffic.
UCB/CSD89/550,UniversityofCalifornia,Berkeley,CA,December1989,http://sunsite.
berkeley.
edu/Dienst/UI/2.
0/Describe/ncstrl.
ucb/CSD-89-550.
7.
[Caceres91]R.
Caceres,P.
Danzig,S.
Jamin,andD.
MitzelCharacteristicsofwide-areaTCP/IPconversationsProceedingsofACMSIGCOMM'91,September1991,http://www.
acm.
org/pubs/citations/proceedings/comm/115992/p101-caceres.
8.
[CAIDA]CooperativeAssociationforInternetDataAnalysis,http://www.
caida.
org.
9.
[claffy93]K.
ClaffyandH.
-W.
BraunandG.
Polyzos.
Long-termtrafficaspectsoftheNSFNET,ProceedingsofINET'93.
http://wwwdev.
caida.
org/outreach/papers/lta.
html.
10.
[claffy95]K.
C.
Claffy,Hans-WernerBraun,GeorgeC.
Polyzos.
AparameterizablemethodologyforInternettrafficflowprofiling,IEEEJSACApril1996,http://wwwdev.
caida.
org/outreach/papers/pmi.
html.
11.
[claffy97]kclaffyandT.
Monk.
What'snextforInternetdataanalysisIEEESpecialIssueonCommunicationsinthe21stCentury85,1563-1571(1997).
.
12.
[claffy98]kclaffy,G.
Miller,andK.
Thompson.
thenatureofthebeast:recenttrafficmeasurementsfromanInternetbackbone,http://www.
isoc.
org/inet98/proceedings/6g/6g_3.
htm.
13.
[claffy99]kcclaffy.
Internetmeasurementanddataanalysis:topology,workload,performanceandroutingstatistics,,NAE99,http://wwwdev.
caida.
org/outreach/papers/Nae/.
14.
[CoralReef]CoralReefhomepage,http://www.
caida.
org/Tools/CoralReef.
15.
[Feldman98]S.
Feldman.
MAE-WestLinkUtilizationStatistics,http://www.
mae.
net/~feldman/gigaswitch/ames.
16.
[Feldmann98]A.
Feldmann,J.
Rexford,andR.
Caceres.
EfficientpoliciesforcarryingWebtrafficoverflow-switchednetworks,,IEEE/ACMTransactionsonNetworking,December1998,pp.
673-685,http://www.
research.
att.
com/~jrex/papers/ton98.
ps.
17.
[Heimlich89]H.
Heimlich.
TrafficCharacterizationoftheNSFNETBackbone,,USENIXConferenceProceedings,Winter1989,http://www.
research.
att.
com/~jrex/papers/ton98.
ps.
18.
[Merit95]NSFNETStatistics,MeritNetwork,Inc.
,1995,http://www.
merit.
edu/merit/archive/nsfnet/statistics.
19.
[Mena00]A.
MenaandJ.
Heidemann.
AnEmpiricalStudyofRealAudioTraffic,,IEEEINFOCOM2000,http://www.
ieee-infocom.
org/2000/papers/84.
ps.
20.
[MOAT]NationalLaboratoryforAppliedNetworkResearch(NLANR)/MeasurementandOperationsAnalysisTeam(MOAT),http://moat.
nlanr.
net/.
21.
[MOAT99]NationalLaboratoryforAppliedNetworkResearch(NLANR)/MeasurementandOperationsAnalysisTeam(MOAT),DataanalysisbasedonCoralpackettraceshttp://moat.
nlanr.
net/Datacube.
22.
[Napster]NapsterProtocolSpecification,http://opennap.
sourceforge.
net/napster.
txt.
23.
[OC3mon]Coral/OC3MonitoringHardware,http://moat.
nlanr.
net/OC3mon-monitors.
24.
[OC3mon-sw]OC3monDataCollectionSoftware,http://moat.
nlanr.
net/Software/OC3mon.
25.
[RFC2330]V.
Paxson.
GrowthTrendsinWide-AreaTCPConnections,IEEENetwork,8(4),pp.
8-17,July/August1994.
ftp://ftp.
ee.
lbl.
gov/papers/WAN-TCP-growth-trends.
ps.
Z.
26.
[Plonka00]D.
Plonka.
UW-MadisonNapsterTrafficMeasurement,http://net.
doit.
wisc.
edu/data/Napster/.
27.
[RealNetworks]RealNetworksRealSystemFirewallSupport,http://service.
real.
com/firewall/adminfw.
html.
28.
[RFC2330]V.
Paxson,G.
Almes,J.
Mahdavi,andM.
Mathis.
FrameworkforIPPerformanceMetricsRFC2330,ftp://ftp.
isi.
edu/in-notes/rfc2330.
txt.
29.
[Savage00]S.
Savage,D.
Wetherall,A.
KarlinandT.
Anderson.
PracticalNetworkSupportforIPTraceback,,insubmission.
CurrentlyavailableasUW-CSE-00-02-01.
.
30.
[Stevens94]W.
RichardStevens.
TCP/IPIllustrated,Volume1:TheProtocolsAddison-Wesley,1994.
31.
[Thompson97]K.
Thompson,G.
Miller,andR.
Wilder.
WideAreaInternetTrafficPatternsandCharacteristicsIEEENetwork,Vol.
11No.
6,pp.
10-23,Nov/Dec1997.
http://www.
vbns.
net/presentations/papers/MCItraffic.
ps.
32.
[WISCstats]D.
Plonka.
UniversityofWisconsinNetworkUsageStatistics,http://wwwstats.
net.
wisc.
edu/.
Formoreinformation:info@caida.
orglastupdate:thispage:
org)kcclaffy(kc@caida.
org)UniversityofCalifornia,SanDiegoCooperativeAssociationforInternetDataAnalysis(CAIDA)U.
S.
A.
Contents1.
Introduction2.
Background1.
MonitorSite2.
MonitoringMethodologyandTools3.
AnalysisMethodologyandTools3.
Results1.
PacketLengths2.
ProtocolMix1.
RawData2.
LongTermTrends3.
ShorterTermTrends4.
ConclusionsandFutureDirections5.
Acknowledgments6.
ReferencesAbstractWereportresultsfromalongitudinalanalysisoftheIPtrafficworkloadseenatasinglemeasurementsiteinsideamajorInternettrafficexchangepoint.
UsingdatacollectedbytheNLANR/MOATNetworkAnalysisInfrastructure(NAI)project[NAI]andanalysissoftwarefromCAIDA'sCoralReefproject[CoralReef],wepresenttrendsinapplicationusageseenattheNASAAmesInternetExchangeover10months,fromMay1999throughMarch2000.
Weshowchangesinthefractionoftrafficfromstreamingmediaandonlinegaming,aswellasanincreaseintrafficfromnewapplicationssuchasNapsterandIPSECtunneling.
WealsoshowthatourdatadoesnotindicateanyoverallchangeintheTCP/UDPtrafficratioattheAmesInternetExchangeduringthisperiod,orsignificantdifferencesfromtheanalysesbyMCIWorldcomandCAIDAin1998.
IntroductionOvertheyearstherehasbeenagreatdealof'commonwisdom'developedaboutthenatureofwideareaInternettraffic.
Unfortunately,theserulesofthumbhavedevelopedintheabsenceofobservationsaboutthecompositionofactualtrafficcarriedintheInternetbackbone.
ManyanalysesofInternettrafficbehaviorrequireaccurateknowledgeofthetrafficcharacteristicsinthebackbonetoday,forpurposesrangingfromtheoptimizationoffuturenetworkingequipmenttomodelingtheeffectsofnewprotocolsontheexistingtrafficmix.
CAIDAaimstofillthisgap,supplyingaccurateinformationaboutbackbonetrafficcharacteristicstobothindustryandtheacademiccommunity.
Otherstudies[Feldmann98]haveusedInternettraffictracedataasinputforevaluationofspecificprotocolperformanceissues,buthavenotconcentratedoncharacterizingtheoverallworkloadcontainedinthetraces.
Therearealsoseveralongoingprojects[Feldman98,AIXstats]thatmonitortheutilizationofnetworklinksatourmeasurementsiteintermsofthenumberofpacketsandbytestransferredovertime.
However,thesemonitoringprojectsdonotattempttoprovideafinergrainedpictureofthelinkutilizationintermsoftheprotocolsinuseonthelinks.
EarlierworkloadanalysisstudiesintheNSFnetbackbone[Heimlich89,Claffy93,Merit95]includedcourse-grainestimatesofthefractionoftrafficduetothemostpopularapplicationsbeforetheadventofwebbrowsers.
ThisincludedbasicstatisticsabouttheprevalenceofFTP,e-mail,netnews,telnet,DNS,andasinglecategoryfor'other'TCPandUDPapplications.
Severalstudiesdetailingtheworkloadpresentedattheborderoflargeorganizations[Caceres89,Caceres91,Paxson94,WISCstats]haveincludedtrafficbreakdownsbyprotocolaswell.
Thesesite-specificstudiesformacomplementtobackbonestudies,astheycanrevealdifferencesinthewaydifferentkindsoforganizationsusetheInternet.
Previousstudiesofbackbonetraffic[Apisdorf97,Thompson97,claffy98]haveanalyzedtheworkloadpresentedattwositesinsidetheInternetMCIbackbone(AS3561),nowownedbyCableandWireless.
Thesestudiesspannedarelativelyshortperiodoftime,rangingfromasingledaytoafullweek.
Theseshort'snapshots'ofbackbonetrafficarenotlongenoughtoaccuratelydeterminetrendsinthetrafficmix,asweseealargeamountofvariabilityonthesetimescales.
InthispaperwepresentananalysisoftheworkloadseenatAIXoveramuchlongerperiod,spanningmorethan10months.
NLANR/MOAT'sDatacubeproject[MOAT99]isaframeworkforstoringandanalyzingtracescollectedfromasetofCoralmonitorsatHPCsites.
Thethreedimensionsofthe'cube'are:dataorigin(measurementlocationwherethedataoriginated);projectname(e.
g.
,analysismethod);anddatacollectiondate.
MOATmakesthepublicdataavailableonitswebserverviathisDataCubestructure,toallowdataexplorationalongeachdimensionofthecube.
Inthenextsection,wepresentbackgroundinformationaboutourdatacollectionsiteandthetoolsweusedtocollectandanalyzethedata.
Wethenpresentsomesampleresultsfromourstudy,ananalysisofthedistributionofpacketlengths,andsometrendsintheprotocolmixseenatoursamplingsite.
WeconcludewithasummaryofthechallengeswefaceincontinuingandextendingourmeasurementprogramatCAIDA.
BackgroundTheMonitorSiteThetracesusedforthisstudywerecollectedfromtheNASAAmesInternetexchange(AIX)inMountainView,CA[AIX]aspartofanNSF/NASAcollaborativeeffortwithNLANR/MOAT.
Theywerecollectedfromoneoffour(nowfive)OC-3ATMlinksthatinterconnectAIXandMAE-WestinSanJose,CA.
Fig1:Adiagramshowingthelocationoftheopticalsplitterusedtocollectthedata.
NotethattherearecurrentlyfivelinksbetweenNASA-AmesandMAE-West.
ThankstoHans-WernerBraunandNLANR/MOATforuseofthisfigure.
ThisgroupoflinksformastripedconnectionbetweentwoDECGigaswitches,withanaggregatebandwidthofapproximatelyasingleOC-12link.
TheGigaswitchesuseaproprietaryschedulingalgorithmforsendingpacketsacrossthislink,buteachpacketissentacrossanindividuallinkinsideanAAL5PDU.
ThismeanstheschedulinginsidetheGigaswitcheshappensatthepacketlevel,sinceallcellsfromaPDUaresentoverthesamelink.
Consequently,thedatawecollectfromthissiteisessentiallysub-sampledfromtheactualdatatraversingthislinkusingtheproprietaryschedulingalgorithminsidetheGigaswitches.
Thisalgorithmisapproximatelyround-robin,butalsodependsoninternalloadcharacteristicsintheGigaswitchswitchingfabric.
However,weknowthatthedistributionofpacketsamongtheOC-3linksisnotentirelyuniform,sincemeasurementsofthelinkutilizationsshowtwoofthemcarryapproximatelytwicethetrafficoftheothertwo(measuredbybytevolume,notpacketvolume)[Feldman98].
However,weassumethattheGigaswitchschedulingalgorithmisindependentoftheencapsulatedprotocol,e.
g.
notdependentuponpacketlength.
Becauseofthiscomplexschedulingalgorithmwearenotabletoaccuratelyestimatethenumberofconversationstraversingthemonitoredlinks,orthelengthoftheseconversationsinpacketsorbytes.
Consequently,weonlycharacterizetheworkloadobservedatAIXintermsofrelativefractionsofpacketsandbytes.
MonitoringMethodologyandToolsThedatacollectionsystemisessentiallysimilartotheoneusedin[Thompson97].
ACoral/OC3monplatformwasconnectedtoonelinkineachdirectionusingopticalsplitters.
ThetraceswestudiedwerecollectedaspartofNLANR/MOAT'sNetworkAnalysisInfrastructure(NAI)project[Braun98].
Foreachpacketthatpassesthemonitor,onlythefirstATMcellfromtheAAL5PDUiscapturedandwrittentodisk.
Thefirstcellcontainsthefirst40bytesofeachpacket,whichisusuallyenoughtoextracttheTCPorUDPportnumbersfromthetransportlayerheaders.
However,themonitordoesnotverifythattheentireAAL5PDUiscarriedbythelink,andsoestimatesofthedataratecarriedbythelinkmaybeinflatedinthepresenceofcellloss.
Sixtoeighttraceswerecollectedeachday,usuallywithadurationof90secondseach.
Thestartingtimeforeachtracewassetatequalintervalsduringthe24hourperiod,andrandomizedoverarangeofanhouratthebeginningofeachinterval.
Aftercollection,thetracesareprocessedtoremoveanyinformationthatmightcompromisetheprivacyoftheindividualsgeneratingthetraffic.
ThisprocessingmasksthesourceanddestinationIPaddresses,anddeletesalldatafromtheIPpayloadexceptforaTCPorUDPheader(ifpresent),ortheICMPorIGMPtypeandcodefields.
IfthepacketcarriesenoughbytesofIPheaderoptions,thentheTCPorUDPportnumbersmaynotbepresentinthefirstcellofthePDU.
Inthiscase,weignorethatpacketinsubsequentapplicationworkloadanalysis.
SincethefractionofpacketswithIPheaderoptionsistypicallylessthan0.
003%,thisdoesn'tseriouslyimpactourmeasurementsofthetrafficfractiongeneratedbythemostpopularTCPandUDPapplications.
AnalysisMethodologyandToolsWeusedCoralReef[CoralReef]toreduceeachrawtracetoasetofsummarytablesthatwearchivedforlateranalysis.
ThetablesincludeaggregatenumberssuchasthenumberofpacketsandbytesinthetraceaswellasdistributionsofpacketlengthsandthenumberofpacketsandbytesseenforeachIP-layerprotocol.
ForTCPandUDP,weanalyzeapplicationusageusingportaddresspairs.
ThepackettracesavailablefromtheNAIarchive[Braun98]onlyincludeIPandtransportlayerheaders,soourmethodologydoesnotuseencapsulateddatatoidentifytheapplicationthatgeneratedthepackets.
TracesintheNAIarchivehavehadallpayloaddataremovedtoprotecttheprivacyofInternetusers.
Inmostcases,wehaveassumedthatpacketssentbetweenanyportnumberhigherthan1023andawell-knownportnumberbelow1023aregeneratedbythesameprotocol(e.
g.
,HTTPonport80).
Thismatchestypicalendhostbehavior,inwhichclientsallocateephemeralportsfromtherange1024to32767[Stevens94].
Forsomeoftheprotocols,wehavecondensedrangesofportnumbersinboththesourceanddestinationfields.
Forexample,theRealAudiocategoryintheUDPtableincludesalltrafficwithdestinationportsbetween6970and7170inclusive[RealNetworks].
Unfortunately,thisrangealsoincludestheportsusedbyAFS,andsowearepotentiallyconfusinganunknownamountofAFStrafficwithRealAudio.
However,themajorityofRealAudiotrafficappearsonUDPports6970,6971,and6972,noneofwhichareusedbyAFS.
ByonlyconsideringtrafficonUDPportsfromthisrangethatarenotusedbyAFS,theamountofRealAudiotrafficcanbeestimatedindependentlyfromtheamountofAFStrafficthatmaybepresentaswell.
WearecurrentlyinvestigatingbettertechniquesfordifferentiatingRealAudioandAFStrafficusingpacketsizedistributionandpacketinterarrivalpatterns,andwehopetobeabletoconclusivelydifferentiatebetweenthetwointhefuture.
ArecentanalysisofthetrafficpatternsexhibitedbyRealAudiotraffic[Mena00]hasshownseveralparametersthatmaybeusedtodifferentiatebetweenRealAudioandotherprotocols.
AfurtherstudycharacterizingAFStrafficpatternsneedstobeundertakentoidentifythebestmetricstousetoseparatethetwo.
ForbothTCPandUDPtraffic,thereisasignificantfractionoftrafficthatcannotbemappedtoapplicationsusingwellknownportnumbers.
Manyprotocolsdonotdependonwell-knownportnumbers,buteitheruseawell-knownservicefornegotiatingtheportnumbersusedbysecondaryconnections,orusearbitrarybutfixedportnumbersthatarenotregisteredwithIANA.
Themostpopularapplicationwithnegotiatedportnumbersispassive-modeFTP,inwhichtheclientsendstheportnumbertouseforadataconnectionoverthecommandchannel.
Therearemanyotherprotocolsthatusesimilarbehavior,suchasNapsterandInternettelephonyapplications.
Mostonlinegamesdonotregisterwell-knownportswithIANA,butusearbitraryportnumbersabove5000.
Wehavecollectedtheportnumbersusedbyseveralofthepopulargamesandusethisinformationtoestimatethefractionoftrafficgeneratedbythem.
OuranalysisofonlinegametrafficincludesgametrafficonthefollowingUDPports:HalfLifeanytoorfrom27005anytoorfrom27015Quake3:Arenaanytoorfrom27960Starcraft6112to6112QuakeIIanytoorfrom27901anytoorfrom27910QuakeWorldanytoorfrom27500anytoorfrom27001Unrealanytoorfrom7777Table1:UDPportsusedbyOnlineGamesAsisthecasewithRealAudioandAFS,therearemanypossibilitiesforconfusionbetweengametrafficandotherapplicationswhenonlyportnumbersareusedtomaketheclassification.
Weassumethattherearenootherprotocolsthatpreferentiallyusethesesameports,andthatapplicationsthatephemerallyusetheseportscontributeequalamountsoftrafficacrossalltrafficcategories.
Thisassumptioncarriessignificantrisks,andneedsfurtheranalysistofullyevaluateitsimpactonourdata.
ResultsPacketlengthsThefollowinggraphsshowthedistributionofIPpacketsizesseenatAIX.
Thesedistributionsarebuiltfromtwoapproximatelyoneweekperiodsnearthebeginningandendofourstudy.
Theycontaincontributionsfromthedifferentworkloadscarriedbythenetworkatdifferenttimesofday,andsotheyshouldrepresentmoreofan'average'pictureofthepacketsizedistributionthananyindividualtrace.
However,noattempthasbeenmadetonormalizethecontributionsofindividualtraces.
Thedistributionspresentedherearesimplythoseoftheconcatenatedtraces.
Additionally,thesedistributionshaveonlybeenplottedforpacketsizeslessthan1600bytes.
Thisallowsthestructureofthedistributionstobepresentedingreaterdetail,buthidestheverysmallfractionoflargerpacketsthatappearinthesetraces(typicallylessthan0.
005%ofpackets).
Fig2:IPpacketlengthdistributionfrom39tracefilescapturedbetweenThursday,May13th1999at19:14:36PDTandWednesday,May19th1999at13:02:20PDT,overanintervalofalmostsixdays.
Statisticsfortheunderlyingpacketlengthdistribution:Mean:413bytes,StandardDeviation:509bytesMedian:93bytes,Percentiles:5th40bytes,25th40bytes,75th576bytes,95th1500bytesNumberofObservations:127millionpackets[127710031packets]Thesenumberscorrespondtotheuppercurveinthefigureabove.
Fig3:IPpacketlengthdistributionfrom43tracefilescapturedbetweenSunday,February20th2000at16:01:51PSTandSunday,February27th2000at13:59:18PST,overanintervalofalmost7days.
Statisticsfortheunderlyingpacketlengthdistribution:Mean:420bytes,StandardDeviation:521bytesMedian:78bytesPercentiles:5th40bytes,25th40bytes,75th576bytes,95th1500bytesNumberofObservations:84millionpackets[84415871]Thesenumberscorrespondtotheuppercurveinthefigureabove.
TheprimaryfeaturesofthisdistributionoriginateinthewaycommonTCPimplementationsdivideadatastreamintopackets.
Approximately85%ofthetrafficinthesetracesisTCP,andalargeproportionofthisTCPtrafficisgeneratedbybulktransferapplicationssuchasHTTPandFTP.
Consequently,themajorityofthepacketsseenareoneofthreesizes:40bytepackets(theminimumpacketsizeforTCP)whichcarryTCPacknowledgmentsbutnopayload;1500bytepackets(themaximumEthernetpayloadsize)fromTCPimplementationsthatusepathMTUdiscovery;and552byteand576bytepacketsfromTCPimplementationsthatdonotusepathMTUdiscovery.
Thesetwodistributionsarestrikinglysimilardespitethefactthatthesecondisbasedondatacollectedmorethan9monthsafterthefirst.
Theseconddistributionhasaslightlylargercontributionfrompacketssmallerthan100bytes,butthedifferenceisquitesmall.
Thefollowinggraphshowshowthemeanandmedianvaluesofthepacketsizedistributionvaryovertheentiredurationofourstudy.
Fig4:MeanandmedianpacketlengthvaluesforeachpackettracecollectedbetweenThursday,May13th1999at19:14:36PDTandSatMar1113:57:38PST2000.
Valueshavebeenbinnedbyweek,andthemedianforeachbinisplottedwithfirstandthirdquartileerrorbars.
Itisnotsurprisingthattheseparametersdonotshowanysignificantlongtermtrend.
AslongasTCPisthedominantprotocolinuseintheInternet,thepacketlengthdistributionisunlikelytochangeverymuchunlesstheTCPprotocolitselfchanges.
ProtocolMixRawDataInthetablesbelow,weprovidesomesampledatafromthemonthofFebruary,2000.
Eachofthe216tracesfromthisperiodwereconcatenated,andthesummaryinformationpresentedbelowisforallthetracescombined.
TotalIPBytes193692014407TotalIPPackets451971619TotalDurationofTraces24345secIPpacketswithDFset329241464FragmentsofIPDatagrams1023206FragmentsofUDPDatagrams597228FragmentsofTCPDatagrams9702IPPacketswithoptions3213Non-IPpackets462085Table2:AggregatetotalsforalltracescollectedinFebruary,2000Table3presentsthetop10IPlayerprotocolsseeninthesetraces.
TCPandUDPtypicallyaccountforalmostallofthetrafficseeninanytrace,withGREandICMPasthenexttwomostpopularprotocolsProtocolNumberPacketsBytesAverageSizeTCP6374801201176706563104471UDP17624567319842511709157GRE4775664155272240819696ICMP159380441350011401227ESP50517353197216792381IPinIP4265103179257606676AH517442343454671583IPIP9414350241707350290SKIP5711705041633952355IGMP2684047729038112Table3:Top10protocolsseenduringFebruary,2000Inthenexttwotables,wehaveconsolidatedtheTCPandUDPportaddresspairsintocategoriesbyapplication.
Asdescribedpreviously,wehaveassumedthattrafficbetweenanyportnumberhigherthan1023andawell-knownportnumberbelow1023isexclusivelygeneratedbythesameprotocol(e.
g.
HTTPonport80).
Wedenotethecondensedsetofportnumbersinthetablewiththelabel'0'.
Forsomeoftheprotocols,wehavecondensedrangesofportnumbersinboththesourceanddestinationfields.
Forexample,theRealAudiocategoryintheUDPtableincludesalltrafficwithdestinationportsbetween6970and7170inclusive,representedwiththelabel'7070'.
NotethatasignificantportionofbothTCPandUDPtrafficfallsintoacategorylabeledwith'0'inboththesourceanddestinationfields.
Thisincludesallthetrafficthatwasnotmappedtoaspecificprotocolusingourmethodology.
Thistrafficiseithergeneratedbyprotocolsthatusenegotiatedportnumbersatbothends(e.
g.
,passive-modeFTP),orbynewapplicationsthatuseunregisteredfixedports.
Wecontinuetoaddportutilizationprofilesfornewprotocolsasweobtainnewandupdatedinformation.
ProtocolSourceDestinationPacketsBytesAverageSizeHTTP800140780543100044030753710004531984217319763013382NNTP01191789548115992967942893HTTP08094578965784416385082FTPData20067280976689611587994SMTP02588789256071084052683NNTP119088572175399672480609HTTP/WebProxy8080023316692327032104998Napster0669933311091838804438552HTTPS443030358091535037132505Napster6699033778281528188686452FTPData02054980971262294037229Napster668801230335935883810760175501182358908626624768POP311001820887798255125438Hotline550106855367871220081148RTSP5540754508616087123816Napster066881149382348845629303SMTP250643867233978802052RealAudio70700445551298598442670Shoutcast80000353545296161291837WebCache31280325447280739543862HTTPS04432635048280418901106NetBIOSSSN139031265826496521284721890294654174140223590Table4:Top25TCPapplicationcategoriesseenduringFebruary,2000ProtocolSourceDestinationPacketsBytesAverageSize00151088222568130721169RealAudio0707046100702029483625440DNS535392908721064980650114DNS5303444558638796849185HalfLife27015270052199098452384485205DNS05352865543325982496206770619334230280312371Starcraft61126112416762521778375552EverQuest90019000908432171755388189HalfLife2700527015275441616080617658RealAudio69700532356154663054290Unreal777701109005141327485127EverQuest90059000777485138637166178Unreal07777189261310761327956Quake3:Arena279602796078425880146922102HalfLife27015042461477756002183QuakeII27901279109466745884258362HalfLife02700531773558062692182022503374599504691328001025508144789873175NetBIOSNS1371375098544399607586QuakeII27910279012328654394243818803714653140574996871CU-SeeMe7648764813684339681727289QuakeII2790106014273732778562HalfLife2700504983002919106258Table5:Top25UDPapplicationcategoriesseenduringFebruary,2000LongTermTrendsIneachofthetimeseriesgraphsbelow,wehavecollectedthedatavaluesintoweek-longbins,andweplotonlythemedianandfirstandthirdquartilesforeachbin.
Forgraphswithmorethanonedataseries,wehaveintroducedaslightoffsettothebinsforeachseriestopreventoverlap.
Wepresentonlythefractionoftrafficcontributedbyeachapplicationorapplicationgroupratherthanabsolutemeasurementsoftrafficvolumeinanycategory.
Shiftsintherelativefractionsamongmultipleprotocolsorapplicationsovertimerepresentchangesinhowthenetworkisused,ratherthanhowbusyourmeasuredlinkisatanygiventime.
However,duringtheperiodofthisstudy,themonitoredlinkhadamedianutilizationofapproximately85Mb/s.
Medianandquartilevalueshavebeencalculatedusingthemethodologydescribedin[RFC2330].
Specifically,thequartilevaluesareactualdatapointsandnotinterpolatedvalues,andthemedianiseitheranactualdatapointortheaverageofthetwomiddlevalues.
Eachweeklybintypicallycontains40-50observations.
ThefirstgraphshowsthefractionofpacketsduetoTCPandUDPseenatAIX.
Thegraphdoesnotshowanyclearlongtermshiftsinthebalancebetweenthesetwodominanttransportlayerprotocols.
DespitetheincreaseinthenumberofUDPapplicationsinrecentyears,growthinthetotalamountofUDPtrafficisoffsetbygrowthinTCPtrafficaswell.
Figure5:FractionofTCPandUDPpacketsseenatAIX.
Tracesarecollectedintoweeklybins,andthemedianandfirstandthirdquartilesforeachbinareplotted.
ThisnextgraphshowsthefractionofIPSECtrafficseenatAIX,includingbothauthenticationheader(AH)andencapsulatingsecuritypayload(ESP)traffic.
Thegraphshowsanincreaseofalmostanorderofmagnitudeduringthefirst5monthsofourstudy,butthenlevelsofforevendeclinesslightly.
Thissuggeststhatafteraninitialperiodofseriousinterest,IPSECtraffichasstoppedgrowingfasterthannon-IPSECtraffic.
Figure6:Fractionofauthenticationheader(AH)andencapsulatingsecuritypayload(ESP)trafficseenatAIX.
Tracesarecollectedintoweeklybins,andthemedianandfirstandthirdquartilesforeachbinareplotted.
Theamountoffragmentedtrafficinwideareanetworkshasbeenatopicofgreatinterestinthepastfewmonths,especiallyasitrelatestoIPtracebacktechniques[Savage00].
Ourdataindicatesthatthefractionoffragmentedtrafficisontherise,andthatthemajorityofthisgrowthisintheformofUDPpackets.
WeneedtoperformfurtheranalysistodeterminewhichUDPprotocolsaregeneratingthesefragments.
Notsurprisingly,TCPtrafficisvirtuallyneverfragmented.
MostlikelythisisduetowidespreaddeploymentofpathMTUdiscoverycombinedwithTCP'srelativelysmalldefaultpacketsize.
Figure7:Fractionoffragmentedtraffic.
Tracesarecollectedintoweeklybins,andthemedianandfirstandthirdquartilesforeachbinareplotted.
FTPisthetraditionalbulktransferprotocolwidelyusedbeforetheadventofHTTPandtheweb.
However,weseeacleardeclineinthecontributionofFTPtotheoveralltrafficmixsinceOctober1999.
ThisdeclinemayactuallybeduetoashiftfromactivetopassivemodeFTPassociatedwithanincreaseinpacketfilteringfirewallsintheInternet,oritmaybeduetoashiftawayfromFTPtoalternativeprotocolsforfiletransfer.
However,thegraphofHTTPtrafficfractionoverthissametimeperioddoesnotshowacorrespondingincrease.
Figure8:ThefractionofactivemodeFTPtrafficisdeclining.
Tracesarecollectedintoweeklybins,andthemedianandfirstandthirdquartilesforeachbinareplotted.
ThefollowinggraphshowsarelativelysurprisingdecreaseinthefractionofRealAudiotrafficseenatAIX.
BothTCPandUDPRealAudiotraffichavedecreasedinpacketvolumecomparedtonon-RealAudiotraffic,althoughthistrendseemstohaveflattenedoutinthelastfewmonthsofourstudy.
AlthoughRealNetworkshasreleasednewerversionsoftheirRealPlayersoftwaresincethisstudybegan,thenewversionsusethesamesetofTCPandUDPportsastheoldversions,withtheadditionofTCPport554forRTSPtraffic[RealNetworks].
Consequently,thetrendweobserveisnotduetoashiftfromtheoldersoftwaretothenewerversions,butrepresentseitheraslowinginthegrowthofRealAudiotrafficoradeclinerelativetothegrowthinnon-RealAudiotraffic.
Figure9:ThefractionofRealAudiotraffichasdeclinedoverthepast10months.
Tracesarecollectedintoweeklybins,andthemedianandfirstandthirdquartilesforeachbinareplotted.
Thisgraphshowsadeclinetheinthefractionoftrafficgeneratedbyseveralpopularonlinegamesoverthefirst8monthsofourstudy.
TheonlinegamesincludedinthisgraphareStarcraft,QuakeII,andQuakeWorld(avariantofQuakeII).
Thesegameswerepopularwhenwestartedcollectingourdata,butasthegraphshows,theirpopularityhasdeclinedfairlysteadilysinceJuly,1999.
Figure10:Fractionofonlinegametraffic,includingQuakeII,QuakeWorld,andStarcraft.
Tracesarecollectedintoweeklybins,andthemedianandfirstandthirdquartilesforeachbinareplotted.
Thissecondgraphofonlinegamingtrafficshowsquiteadifferenttrendfromthepreviousgraph.
Unlikethepreviousgraph,thisoneincludestrafficfromseveralnewergamesinadditiontotheolderones.
ThenewgraphincludestrafficgeneratedbyHalfLife,Quake3:Arena,andUnrealinadditiontoStarcraft,QuakeII,andQuakeWorld.
Althoughthereisnotenoughdatatodetermineatrend,themediantrafficfractionsaremuchhigheronthisgraphthanthepreviousone.
Hence,theoverallfractionofonlinegametrafficseemstobeontherise,butitisamovingtarget.
Theincreaseisprimarilygeneratedbynewgamesastheygainpopularity,whileoldergamesseemtodecreaseinpopularityovertime.
Figure11:Fractionofonlinegametraffic,includingHalfLife,QuakeII,QuakeWorld,Quake3:Arena,Starcraft,andUnreal.
Tracesarecollectedintoweeklybins,andthemedianandfirstandthirdquartilesforeachbinareplotted.
AlthoughNapsterdoesnotuseafixedsetofportsforfiletransfers,weidentifiedthethreemostcommonlyusedTCPportsinuseinlateJanuary:TCPports6688,6697,and6699.
Bulktransfertrafficmaybeeithersenttoorreceivedfromtheseports,sinceNapstersupportsbothactiveandpassivemodetransfers[Napster].
AsUniversitiesandothersitesmovetoblocktrafficontheseports,Napstertrafficwillundoubtedlymigratetoothers.
However,theshorttermtrendclearlyshowsdramaticgrowth,increasingbyover50%inthelasttwomonthsofourstudy.
Figure12:FractionofNapsterbulk-transfertrafficseenatAIX.
Tracesarecollectedintoweeklybins,andthemedianandfirstandthirdquartilesforeachbinareplotted.
ShorterTermTrendsSomeshortertermtrendscanbeeasilyassociatedwithcommonuserbehavior.
Forexample,thefractionofemailtraffic(SMTPpacketstoorfromTCPport25)increasedsignificantlyinNovemberandearlyDecember,onlytodropoffagainduringtheholidaysattheendofDecember.
Perhapsthiswasrelatedtoonlinecommerce,asmanypeopleusedtheInternettopurchasetheirChristmasgiftsFigure13:FractionofmailtrafficseenatAIX,showingapeakinusagejustbeforetheChristmasholidays.
Tracesarecollectedintoweeklybins,andthemedianandfirstandthirdquartilesforeachbinareplotted.
Onlinegamingisclearlymorepopularonweekends,asthefollowinggraphshows.
Thefractionofgametrafficcannearlydoubleonweekendscomparedtothetypicalweekday.
Thischangeinworkloadoverthecourseofaweekhasbeensmoothedoutinourpreviousgraphsbyourchoiceofbinsize.
Figure14:WeeklyvariationsinthefractionofonlinegamingtrafficseenatAIX.
Tracesarecollectedintodailybins,andthemedianandfirstandthirdquartilesforeachbinareplotted.
ConclusionsandFutureDirectionsInourlongtermstudyofthetrafficworkloadattheAmesInternetExchange,wefoundnosignificanttrendsovertimeineithertheoverallpacketsizedistributionorchangesintheratioofTCPtoUDPtraffic.
WedidfindanorderofmagnitudeincreaseinthevolumeofIPSECtraffic,andastrongincreasingtrendinthevolumeoffragmentedUDPtraffic.
Someotherinterestingtrendsincludedecreasesinthevolumeofactive-modeFTPandRealAudiotraffic,andincreasesinthevolumeofonlinegamingandNapstertraffic.
Wealsofoundshorttermtrends,includingatransientincreaseinthevolumeofe-mailtrafficduringtheholidayshoppingseason,andastrongweekday/weekendvariationinthevolumeofonlinegamingtraffic.
Thecollectionofbackbonetrafficstatisticshasbeenseverelyimpactedbythebreakupofthenetworkcoreintoanumberofmutuallycompetitiveorganizations.
DatacollectioneffortsmustnowprotectboththeprivacyofindividualInternetusersandtheproprietaryinterestsofthehostorganizations.
CAIDAisinauniquepositiontomonitorandanalyzetrafficcollectedfrombackbonelinks,asithasongoingrelationshipswithseveralcoreISPsandexchangepointproviders.
Ourworkloadcharacterizationeffortsalsofaceadditionaltechnicalchallenges,bothinextendingourmethodologytoincreasetheaccuracyofourmeasurements,andincopingwiththeintroductionofnewprotocols.
Ourportbasedtrafficclassificationschemehassomeseverelimitations,asillustratedbythevolumeoftrafficitfailstomaptoanyspecificprotocol.
Additionaltechniquesarenecessarytofurtherdifferentiatethistraffic,aswellasreduceconfusionamongprotocolsthatusethesamesetofsourceordestinationports.
Furthermore,inordertoaccuratelyclassifyprotocolsthatusenegotiatedportsonbothendsitwillbenecessarytocorrelatemultipletrafficstreamsfromthesamehosttothesameordifferentdestinations[Plonka00].
However,thesetechniquesmayhavelimitedusabilityintheInternetcorewherethemeasurementsiteisfarawayfromboththesourceanddestinationofthetraffic.
Consequently,wemaybeunabletoeffectivelysortthistrafficbyprotocolwithoutusingtechniquesthatmonitorthealltrafficexchangedontheassociatedcontrolchannels,asthesechannelstypicallyuseawell-knownportatoneendevenifthedatachanneldoesnot.
IPSECrepresentsanotherpotentialproblemforworkloadanalysis,asESPtrafficencryptsthesourceanddestinationportsweusefortrafficclassification.
IfuseofESPbecomeswidespread,wemaylosetheabilitytoestimatehowthenetworkisbeingutilizedaltogether.
[Bellovin99]proposesamodificationtoESPthatsendscleartextportnumbersforencapsulatedtraffic.
Ifthismodificationwasadopted,ESPwouldnotposesuchaproblemforourmeasurements.
TheinterconnectionlinkbetweenAIXandMAE-WesthasrecentlybeenupgradedtoasingleOC-12PacketoverSONET(POS)link.
Thiswillimprovethedatacollectionsite,andallowustoperformpacketflowanalysissinceeverypacketsentbetweenAIXandFIXWestwillbevisibletoourmonitor.
However,itwillalsorequirePOSsupportforCoral/OCXmonhardware.
DevelopmentofadditionalhardwareandsoftwaretosupportPOSlinksatOC-3andOC-12speedsiscurrentlyunderwayatCAIDA,andwehopetocontinuethisanalysisafterthelinkupgradewithminimalinterruptionindatacollection.
CAIDAisalsointerestedinotherbackbonesamplingpointsforcomparisonagainsttheworkloadseenatAIX.
PreviousstudiesdonewithdatacollectedbyMCI[claffy98,Apisdorf97,Thompson97]bearsomesimilaritiestoourresults,butmoredatacollectionmustbeperformedbeforeanyconclusionscanbedrawnabouthowrepresentativeourresultsareofotherwide-areaInternetinfrastructure.
AcknowledgmentsThisanalysiswouldnothavebeenpossiblewithouttheworkofHans-WernerBraunandtheNationalScienceFoundation'sNLANR/MOATprojectforcollectingandarchivingInternettraffictraces.
TheparticularmonitorthatweusedattheAMESInternetExchange(AIX)existsasaresultofacollaborationamongNASA,NLANR,andCAIDA.
WearegratefultoBillJones,LanceTatman,BobbyCates,andJeffOsbourne(allfromNASA-Ames),andtoSteveFeldman(previouslyatMCI/Worldcom)fortheircontinuedsupportandmaintenanceofthisandothertrafficmonitorsattheexchangepoints.
WewouldalsoliketothankKenKeys,DavidMoore,andtherestoftheCoralReefteamfortheanalysislibraryusedthroughoutthisproject.
References1.
[AIX]AmesInterneteXchange,http://aix.
arc.
nasa.
gov/.
2.
[AIXstats]AmesInterneteXchangeLinkUtilizationStatistics,http://aix.
arc.
nasa.
gov/graphlink.
html.
3.
[Apisdorf97]J.
Apisdorf,kclaffy,K.
Thompson,andR.
Wilder.
OC3mon:Flexible,Affordable,High-PerformanceStatisticsCollection,http://www.
isoc.
org/isoc/whatis/conferences/inet/97/proceedings/F1/F1_2.
HTM.
4.
[Bellovin99]S.
Bellovin.
Transport-FriendlyESP,http://www.
research.
att.
com/~smb/talks/tfesp-ndss/index.
htm.
5.
[Braun98]H.
-W.
Braun.
TowardsasystemicunderstandingoftheInternetorganism:aframeworkforthecreationofaNetworkAnalysisInfrastructure,http://moat.
nlanr.
net/NAI.
6.
[Caceres89]R.
Caceres.
MeasurementsofWide-AreaInternetTraffic.
UCB/CSD89/550,UniversityofCalifornia,Berkeley,CA,December1989,http://sunsite.
berkeley.
edu/Dienst/UI/2.
0/Describe/ncstrl.
ucb/CSD-89-550.
7.
[Caceres91]R.
Caceres,P.
Danzig,S.
Jamin,andD.
MitzelCharacteristicsofwide-areaTCP/IPconversationsProceedingsofACMSIGCOMM'91,September1991,http://www.
acm.
org/pubs/citations/proceedings/comm/115992/p101-caceres.
8.
[CAIDA]CooperativeAssociationforInternetDataAnalysis,http://www.
caida.
org.
9.
[claffy93]K.
ClaffyandH.
-W.
BraunandG.
Polyzos.
Long-termtrafficaspectsoftheNSFNET,ProceedingsofINET'93.
http://wwwdev.
caida.
org/outreach/papers/lta.
html.
10.
[claffy95]K.
C.
Claffy,Hans-WernerBraun,GeorgeC.
Polyzos.
AparameterizablemethodologyforInternettrafficflowprofiling,IEEEJSACApril1996,http://wwwdev.
caida.
org/outreach/papers/pmi.
html.
11.
[claffy97]kclaffyandT.
Monk.
What'snextforInternetdataanalysisIEEESpecialIssueonCommunicationsinthe21stCentury85,1563-1571(1997).
.
12.
[claffy98]kclaffy,G.
Miller,andK.
Thompson.
thenatureofthebeast:recenttrafficmeasurementsfromanInternetbackbone,http://www.
isoc.
org/inet98/proceedings/6g/6g_3.
htm.
13.
[claffy99]kcclaffy.
Internetmeasurementanddataanalysis:topology,workload,performanceandroutingstatistics,,NAE99,http://wwwdev.
caida.
org/outreach/papers/Nae/.
14.
[CoralReef]CoralReefhomepage,http://www.
caida.
org/Tools/CoralReef.
15.
[Feldman98]S.
Feldman.
MAE-WestLinkUtilizationStatistics,http://www.
mae.
net/~feldman/gigaswitch/ames.
16.
[Feldmann98]A.
Feldmann,J.
Rexford,andR.
Caceres.
EfficientpoliciesforcarryingWebtrafficoverflow-switchednetworks,,IEEE/ACMTransactionsonNetworking,December1998,pp.
673-685,http://www.
research.
att.
com/~jrex/papers/ton98.
ps.
17.
[Heimlich89]H.
Heimlich.
TrafficCharacterizationoftheNSFNETBackbone,,USENIXConferenceProceedings,Winter1989,http://www.
research.
att.
com/~jrex/papers/ton98.
ps.
18.
[Merit95]NSFNETStatistics,MeritNetwork,Inc.
,1995,http://www.
merit.
edu/merit/archive/nsfnet/statistics.
19.
[Mena00]A.
MenaandJ.
Heidemann.
AnEmpiricalStudyofRealAudioTraffic,,IEEEINFOCOM2000,http://www.
ieee-infocom.
org/2000/papers/84.
ps.
20.
[MOAT]NationalLaboratoryforAppliedNetworkResearch(NLANR)/MeasurementandOperationsAnalysisTeam(MOAT),http://moat.
nlanr.
net/.
21.
[MOAT99]NationalLaboratoryforAppliedNetworkResearch(NLANR)/MeasurementandOperationsAnalysisTeam(MOAT),DataanalysisbasedonCoralpackettraceshttp://moat.
nlanr.
net/Datacube.
22.
[Napster]NapsterProtocolSpecification,http://opennap.
sourceforge.
net/napster.
txt.
23.
[OC3mon]Coral/OC3MonitoringHardware,http://moat.
nlanr.
net/OC3mon-monitors.
24.
[OC3mon-sw]OC3monDataCollectionSoftware,http://moat.
nlanr.
net/Software/OC3mon.
25.
[RFC2330]V.
Paxson.
GrowthTrendsinWide-AreaTCPConnections,IEEENetwork,8(4),pp.
8-17,July/August1994.
ftp://ftp.
ee.
lbl.
gov/papers/WAN-TCP-growth-trends.
ps.
Z.
26.
[Plonka00]D.
Plonka.
UW-MadisonNapsterTrafficMeasurement,http://net.
doit.
wisc.
edu/data/Napster/.
27.
[RealNetworks]RealNetworksRealSystemFirewallSupport,http://service.
real.
com/firewall/adminfw.
html.
28.
[RFC2330]V.
Paxson,G.
Almes,J.
Mahdavi,andM.
Mathis.
FrameworkforIPPerformanceMetricsRFC2330,ftp://ftp.
isi.
edu/in-notes/rfc2330.
txt.
29.
[Savage00]S.
Savage,D.
Wetherall,A.
KarlinandT.
Anderson.
PracticalNetworkSupportforIPTraceback,,insubmission.
CurrentlyavailableasUW-CSE-00-02-01.
.
30.
[Stevens94]W.
RichardStevens.
TCP/IPIllustrated,Volume1:TheProtocolsAddison-Wesley,1994.
31.
[Thompson97]K.
Thompson,G.
Miller,andR.
Wilder.
WideAreaInternetTrafficPatternsandCharacteristicsIEEENetwork,Vol.
11No.
6,pp.
10-23,Nov/Dec1997.
http://www.
vbns.
net/presentations/papers/MCItraffic.
ps.
32.
[WISCstats]D.
Plonka.
UniversityofWisconsinNetworkUsageStatistics,http://wwwstats.
net.
wisc.
edu/.
Formoreinformation:info@caida.
orglastupdate:thispage:
阿里云香港 16核32G 20M 999元/月
阿里云香港配置图提速啦是成立于2012年的十分老牌的一个商家这次给大家评测的是 阿里云香港 16核32G 20M 这款产品，单单说价格上就是十分的离谱原价8631元/月的现价只要 999元 而且还有个8折循环优惠。废话不多说直接进入正题。优惠时间 2021年8月20日-2021年9月20日 优惠码 wn789 8折优惠阿里云香港BGP专线 16核32G 10M带宽 优惠购买 399元购买链接阿里云...
萤光云（13.25元）香港CN2 新购首月6.5折
萤光云怎么样？萤光云是一家国人云厂商，总部位于福建福州。其成立于2002年，主打高防云服务器产品，主要提供福州、北京、上海BGP和香港CN2节点。萤光云的高防云服务器自带50G防御，适合高防建站、游戏高防等业务。目前萤光云推出北京云服务器优惠活动，机房为北京BGP机房，购买北京云服务器可享受6.5折优惠+51元代金券（折扣和代金券可叠加使用）。活动期间还支持申请免费试用，需提交工单开通免费试用体验...
企鹅小屋6折年付240元起，美国CN2 GIA VPS促销,独享CPU,三网回程CN2 GIA
企鹅小屋怎么样？企鹅小屋最近针对自己的美国cn2 gia套餐推出了2个优惠码：月付7折和年付6折，独享CPU，100%性能，三网回程CN2 GIA网络，100Mbps峰值带宽，用完优惠码1G内存套餐是年付240元，线路方面三网回程CN2 GIA。如果新购IP不能正常使用，请在开通时间60分钟内工单VPS技术部门更换正常IP;特价主机不支持退款。点击进入：企鹅小屋官网地址企鹅小屋优惠码：年付6折优惠...
114dns为你推荐
-
国际域名注册现在注册一个WWW的国际域名要多少钱？是什么步骤？linux主机linux主机有什么区别？求答案海外主机美国主机与国内主机有哪些区别网站空间商网站空间商的选择？？虚拟主机评测浅谈建站新手如何挑选虚拟主机深圳虚拟主机深圳鼎峰网络科技 虚拟主机空间怎么样河南虚拟主机谁那有好的虚拟主机？虚拟主机试用哪儿的虚拟主机可以试用？？安徽虚拟主机合肥蜀山区哪家网络公司做网站最好虚拟主机提供商哪个虚拟主机的服务商比较好？