consensusacceptencoding

ELSEVIERKeydifferencesbetweenHTTP=1.
0andHTTP=1.
1BalachanderKrishnamurthya,,JeffreyC.
Mogulb,DavidM.
KristolcaAT&TLabs-Research,180ParkAvenue,FlorhamPark,NJ07932,USAbWesternResearchLab,CompaqComputerCorp.
,250UniversityAvenue,PaloAlto,CA94301,USAcBellLaboratories,600MountainAvenue,MurrayHill,NJ07974,USAAbstractTheHTTP=1.
1protocolistheresultoffouryearsofdiscussionanddebateamongabroadgroupofWebresearchersanddevelopers.
Itimprovesuponitsphenomenallysuccessfulpredecessor,HTTP=1.
0,innumerousways.
WediscussthedifferencesbetweenHTTP=1.
0andHTTP=1.
1,aswellassomeoftherationalebehindthesechanges.
1999PublishedbyElsevierScienceB.
V.
Allrightsreserved.
Keywords:HTTP=1.
0;HTTP=1.
11.
IntroductionByanyreasonablestandard,theHTTP=1.
0pro-tocolhasbeenstunninglysuccessful.
Asameasureofitspopularity,HTTPaccountedforabout75%ofInternetbackbonetrafcinarecentstudy[35].
Inspiteofitssuccess,however,HTTP=1.
0iswidelyunderstoodtohavenumerousaws.
HTTP=1.
0evolvedfromtheoriginal'0.
9'versionofHTTP(whichisstillinrareuse).
TheprocessleadingtoHTTP=1.
0involvedsignicantdebateandexperimentation,butneverproducedaformalspec-ication.
TheHTTPWorkingGroup(HTTP-WG)oftheInternetEngineeringTaskForce(IETF)pro-ducedadocument(RFC1945)[2]thatdescribedthe'commonusage'ofHTTP=1.
0,butdidnotattempttocreateaformalstandardoutofthemanyvariantimplementations.
Instead,overaperiodofroughlyfouryears,theHTTP-WGdevelopedanimprovedprotocol,knownasHTTP=1.
1.
TheHTTP=1.
1spec-Correspondingauthor.
E-mail:bala@research.
att.
comication[9]issoontobecomeanIETFDraftStan-dard.
Recentversionsofsomepopularagents(MSIE,Apache)claimHTTP=1.
1complianceintheirre-questsorresponses,andmanyimplementationshavebeentestedforinteroperablecompliancewiththespecication[24,30].
TheHTTP=1.
1specicationstatesthevari-ousrequirementsforclients,proxies,andservers.
However,additionalcontextandrationalesforthechangedornewfeaturescanhelpdevelopersunder-standthemotivationbehindthechanges,andprovidethemwitharicherunderstandingoftheprotocol.
Additionally,theserationalescangiveimplementorsabroaderfeelfortheprosandconsofindividualfeatures.
Inthispaperwedescribethemajorchangesbe-tweentheHTTP=1.
0andHTTP=1.
1protocols.
TheHTTP=1.
1specicationisalmostthreetimesaslongasRFC1945,reectinganincreaseincomplexity,clarity,andspecicity.
Evenso,numerousrulesareimpliedbytheHTTP=1.
1specication,ratherthanbeingexplicitlystated.
Whilesomeattemptshave1999PublishedbyElsevierScienceB.
V.
Allrightsreserved.
660beenmadetodocumentthedifferencesbetweenHTTP=1.
0andHTTP=1.
1([23,36],section19.
6.
1of[9])weknowofnopublishedanalysisthatcoversmajordifferencesandtherationalebehindthem,andthatreectsthemostrecent(andprobablynear-nal)revisionoftheHTTP=1.
1specication.
BecausetheHTTP-WG,alargeandinternationalgroupofre-searchersanddevelopers,conductedmostofitsdis-cussionsviaitsmailinglist,thearchiveofthatlist[5]documentsthehistoryoftheHTTP=1.
1effort.
Butthatarchivecontainsover8500messages,renderingitopaquetoallbutthemostdeterminedprotocolhistorian.
Westructureourdiscussionby(somewhatarbi-trarily)dividingtheprotocolchangesintoninemajorareas:(1)Extensibility(2)Caching(3)Bandwidthoptimization(4)Networkconnectionmanagement(5)Messagetransmission(6)Internetaddressconservation(7)Errornotication(8)Security,integrity,andauthentication(9)ContentnegotiationWedevoteasectiontoeacharea,includingthemotivationforchangesandadescriptionofthecorrespondingnewfeatures.
2.
ExtensibilityTheHTTP=1.
1effortassumed,fromtheoutset,thatcompatibilitywiththeinstalledbaseofHTTPimplementations(includingmanythatdidnotcon-formwith[2])wasmandatory.
ItseemedunlikelythatmostsoftwarevendorsorWebsiteoperatorswoulddeploysystemsthatfailedtointeroperatewiththemillionsofexistingclients,servers,andproxies.
BecausetheHTTP=1.
1efforttookoverfouryears,andgeneratednumerousinterimdraftdoc-uments,manyimplementorsdeployedsystemsusingthe'HTTP=1.
1'protocolversionbeforethenalversionofthespecicationwasnished.
Thiscre-atedanothercompatibilityproblem:thenalver-sionhadtobesubstantiallycompatiblewiththesepseudo-HTTP=1.
1versions,eveniftheinterimdraftsturnedouttohaveerrorsinthem.
Theseabsoluterequirementsforcompatibilitywithpoorlyspeciedpriorversionsledtoanumberofidiosyncrasiesandnon-uniformitiesinthenaldesign.
ItisnotpossibletounderstandtherationaleforalloftheHTTP=1.
1featureswithoutrecognizingthispoint.
Thecompatibilityissuealsounderlinedtheneedtoinclude,inHTTP=1.
1,asmuchsupportaspossibleforfutureextensibility.
Thatis,ifafutureversionofHTTPweretobedesigned,itshouldnotbehamstrungbyanyadditionalcompatibilityproblems.
NotethatHTTPhasalwaysspeciedthatifanimplementationreceivesaheaderthatitdoesnotunderstand,itmustignoretheheader.
Thisruleallowsamultitudeofextensionswithoutanychangetotheprotocolversion,althoughitdoesnotbyitselfsupportallpossibleextensions.
2.
1.
VersionnumbersInspiteoftheconfusionoverthemeaningofthe'HTTP=1.
1'protocolversiontoken(doesitimplycompatibilitywithoneoftheinterimdrafts,orwiththenalstandard),inmanycasestheversionnum-berinanHTTPmessagecanbeusedtodeducethecapabilitiesofthesender.
AcompaniondocumenttotheHTTPspecication[26]clearlyspeciedthegroundrulesfortheuseandinterpretationofHTTPversionnumbers.
TheversionnumberinanHTTPmessagereferstothehop-by-hopsenderofthemessage,nottheend-to-endsender.
Thusthemessage'sversionnum-berisdirectlyusefulindetermininghop-by-hopmessage-levelcapabilities,butnotveryusefulindeterminingend-to-endcapabilities.
Forexample,ifanHTTP=1.
1originserverreceivesamessageforwardedbyanHTTP=1.
1proxy,itcannottellfromthatmessagewhethertheultimateclientusesHTTP=1.
0orHTTP=1.
1.
Forthisreason,aswellastosupportdebugging,HTTP=1.
1denesaViaheaderthatdescribesthepathfollowedbyaforwardedmessage.
ThepathinformationincludestheHTTPversionnumbersofallsendersalongthepathandisrecordedbyeachsuccessiverecipient.
(Onlythelastofmultiplecon-secutiveHTTP=1.
0senderswillbelisted,becauseHTTP=1.
0proxieswillnotaddinformationtotheViaheader.
)6612.
2.
TheOPTIONSmethodHTTP=1.
1introducestheOPTIONSmethod,awayforaclienttolearnaboutthecapabilitiesofaserverwithoutactuallyrequestingaresource.
Forexample,aproxycanverifythattheservercomplieswithaspecicversionoftheprotocol.
Unfortunately,theprecisesemanticsoftheOPTIONSmethodwerethesubjectofanintenseandunresolveddebate,andwebelievethatthemechanismisnotyetfullyspecied.
2.
3.
UpgradingtootherprotocolsInordertoeasethedeploymentofincompat-iblefutureprotocols,HTTP=1.
1includesthenewUpgraderequest-header.
BysendingtheUpgradeheader,aclientcaninformaserverofthesetofpro-tocolsitsupportsasanalternatemeansofcommu-nication.
Theservermaychoosetoswitchprotocols,butthisisnotmandatory.
3.
CachingWebdevelopersrecognizedearlyonthatthecachingofresponseswasbothpossibleandhighlydesirable.
Cachingiseffectivebecauseafewre-sourcesarerequestedoftenbymanyusers,orrepeat-edlybyagivenuser.
CachesareemployedinmostWebbrowsersandinmanyproxyservers;occa-sionallytheyarealsoemployedinconjunctionwithcertainoriginservers.
Webcachingproducts,suchasCisco'scacheengine[4]andInktomi'sTrafcServer[18](tonametwo),arenowamajorbusiness.
ManyresearchershavestudiedtheeffectivenessofHTTPcaching[20,6,1,17].
Cachingimprovesuser-perceivedlatencybyeliminatingthenetworkcommunicationwiththeoriginserver.
Cachingalsoreducesbandwidthconsumption,byavoidingthetransmissionofunnecessarynetworkpackets.
Re-ducedbandwidthconsumptionalsoindirectlyre-duceslatencyforuncachedinteractions,byreducingnetworkcongestion.
Finally,cachingcanreducetheloadonoriginservers(andonintermediateproxies),furtherimprovinglatencyforuncachedinteractions.
Oneriskwithcachingisthatthecachingmech-anismmightnotbe'semanticallytransparent':thatis,itmightreturnaresponsedifferentfromwhatwouldbereturnedbydirectcommunicationwiththeoriginserver.
Whilesomeapplicationscantoleratenon-transparentresponses,manyWebapplications(electroniccommerce,forexample)cannot.
3.
1.
CachinginHTTP=1.
0HTTP=1.
0providedasimplecachingmechanism.
Anoriginservermaymarkaresponse,usingtheExpiresheader,withatimeuntilwhichacachecouldreturntheresponsewithoutviolatingseman-tictransparency.
Further,acachemaycheckthecur-rentvalidityofaresponseusingwhatisknownasaconditionalrequest:itmayincludeanIf-Modi-fied-Sinceheaderinarequestfortheresource,specifyingthevaluegiveninthecachedresponse'sLast-Modifiedheader.
Theservermaytheneitherrespondwitha304(NotModied)statuscode,im-plyingthatthecacheentryisvalid,oritmaysendanormal200(OK)responsetoreplacethecacheentry.
HTTP=1.
0alsoincludedamechanism,thePragma:no-cacheheader,fortheclienttoindicatethatarequestshouldnotbesatisedfromacache.
TheHTTP=1.
0cachingmechanismworkedmod-eratelywell,butithadmanyconceptualshortcom-ings.
Itdidnotalloweitheroriginserversorclientstogivefullandexplicitinstructionstocaches;there-fore,itdependedonabodyofheuristicsthatwerenotwell-specied.
Thisledtotwoproblems:incor-rectcachingofsomeresponsesthatshouldnothavebeencached,andfailuretocachesomeresponsesthatcouldhavebeencached.
Theformercausessemanticproblems;thelattercausesperformanceproblems.
3.
2.
CachinginHTTP=1.
1HTTP=1.
1attemptstoclarifytheconceptsbehindcaching,andtoprovideexplicitandextensiblepro-tocolmechanismsforcaching.
WhileitretainsthebasicHTTP=1.
0design,itaugmentsthatdesignbothwithnewfeatures,andwithmorecarefulspecica-tionsoftheexistingfeatures.
InHTTP=1.
1terminology,acacheentryisfreshuntilitreachesitsexpirationtime,atwhichpointitbecomesstale.
Acacheneednotdiscardastaleentry,butitnormallymustrevalidateitwiththe662originserverbeforereturningitinresponsetoasubsequentrequest.
However,theprotocolallowsbothoriginserversandend-userclientstooverridethisbasicrule.
InHTTP=1.
0,acacherevalidatedanentryus-ingtheIf-Modified-Sinceheader.
Thisheaderusesabsolutetimestampswithone-secondresolu-tion,whichcouldleadtocachingerrorseitherbe-causeofclocksynchronizationerrors,orbecauseoflackofresolution.
Therefore,HTTP=1.
1introducesthemoregeneralconceptofanopaquecachevalida-torstring,knownasanentitytag.
Iftworesponsesforthesameresourcehavethesameentitytag,thentheymust(byspecication)beidentical.
Becauseanentitytagisopaque,theoriginservermayuseanyinformationitdeemsnecessarytoconstructit(suchasane-grainedtimestamporaninternaldatabasepointer),aslongasitmeetstheuniquenessrequire-ment.
Clientsmaycompareentitytagsforequality,butcannototherwisemanipulatethem.
HTTP=1.
1serversattachentitytagstoresponsesusingtheETagheader.
HTTP=1.
1includesanumberofnewcondi-tionalrequest-headers,inadditiontoIf-Modified-Since.
ThemostbasicisIf-None-Match,whichallowsaclienttopresentoneormoreentitytagsfromitscacheentriesforaresource.
Ifnoneofthesematchestheresource'scurrententitytagvalue,theserverreturnsanormalresponse;otherwise,itmayreturna304(NotModied)responsewithanETagheaderthatindicateswhichcacheentryiscurrentlyvalid.
Notethatthismechanismallowstheservertocyclethroughasetofpossibleresponses,whiletheIf-Modified-Sincemechanismonlygeneratesacachehitifthemostrecentresponseisvalid.
HTTP=1.
1alsoaddsnewconditionalheaderscalledIf-Unmodified-SinceandIf-Match,cre-atingotherformsofpreconditionsonrequests.
Thesepreconditionsareusefulinmorecomplexsituations;inparticular,seethediscussioninSection4.
1ofrangerequests.
3.
3.
TheCache-ControlheaderInordertomakecachingrequirementsmoreex-plicit,HTTP=1.
1addsthenewCache-Controlheader,allowinganextensiblesetofcache-controldirectivestobetransmittedinbothrequestsandre-sponses.
ThesetdenedbyHTTP=1.
1isquitelarge,soweconcentrateonseveralnotablemembers.
BecausetheabsolutetimestampsintheHTTP=1.
0Expiresheadercanleadtofailuresinthepresenceofclockskew(andobservationssuggestthatseriousclockskewiscommon),HTTP=1.
1canuserelativeexpirationtimes,viathemax-agedirective.
(ItalsointroducesanAgeheader,sothatcachescanindicatehowlongaresponsehasbeensittingincachesalongtheway.
)Becausesomeusershaveprivacyrequirementsthatlimitcachingbeyondtheneedforsemantictransparency,theprivateandno-storedirectivesallowserversandclientstopreventthestorageofsomeorallofaresponse.
However,thisdoesnotguaranteeprivacy;onlycryptographicmechanismscanprovidetrueprivacy.
Someproxiestransformresponses(forexample,toreduceimagecomplexitybeforetransmissionoveraslowlink[8]),butbecausesomeresponsescannotbeblindlytransformedwithoutlosinginformation,theno-transformdirectivemaybeusedtopreventtransformations.
3.
4.
TheVaryheaderAcachendsacacheentrybyusingakeyvalueinalookupalgorithm.
ThesimplisticcachingmodelinHTTP=1.
0usesjusttherequestedURLasthecachekey.
However,thecontentnegotiationmechanism(describedinSection10)breaksthismodel,becausetheresponsemayvarynotonlybasedontheURL,butalsobasedononeormorerequest-headers(suchasAccept-LanguageandAccept-Charset).
Tosupportcachingofnegotiatedresponses,andforfutureextensibility,HTTP=1.
1includestheVaryresponse-header.
Thisheadereldcarriesalistoftherelevantselectingrequest-headereldsthatpar-ticipatedintheselectionoftheresponsevariant.
Inordertousetheparticularvariantofthecachedresponseinreplyingtoasubsequentrequest,theselectingrequest-headersofthenewrequestmustexactlymatchthoseoftheoriginalrequest.
Thissimpleandelegantextensionmechanismworksformanycasesofnegotiation,butitdoesnotallowformuchintelligenceatthecache.
Forexample,asmartcachecould,inprinciple,realize663thatonerequestheadervalueiscompatiblewithan-other,withoutbeingequal.
TheHTTP=1.
1develop-menteffortincludedanattempttoprovideso-called'transparentcontentnegotiation'thatwouldallowcachessomeactiveparticipation,butultimatelynoconsensusdeveloped,andthisattempt[16,15]wasseparatedfromtheHTTP=1.
1specication.
4.
BandwidthoptimizationNetworkbandwidthisalmostalwayslimited.
Bothbyintrinsicallydelayingthetransmissionofdata,andthroughtheaddedqueuingdelaycausedbycongestion,wastingbandwidthincreaseslatency.
HTTP=1.
0wastesbandwidthinseveralwaysthatHTTP=1.
1addresses.
Atypicalexampleisaserver'ssendinganentire(large)resourcewhentheclientonlyneedsasmallpartofit.
TherewasnowayinHTTP=1.
0torequestpartialobjects.
Also,itispossibleforbandwidthtobewastedintheforwarddirection:ifaHTTP=1.
0servercouldnotacceptlargerequests,itwouldreturnanerrorcodeafterbandwidthhadalreadybeenconsumed.
Whatwasmissingwastheabilitytonegotiatewithaserverandtoensureitsabilitytohandlesuchrequestsbeforesendingthem.
4.
1.
RangerequestsAclientmayneedonlypartofaresource.
Forexample,itmaywanttodisplayjustthebeginningofalongdocument,oritmaywanttocontinuedownloadingaleafteratransferwasterminatedinmid-stream.
HTTP=1.
1rangerequestsallowaclienttorequestportionsofaresource.
Whiletherangemechanismisextensibletootherunits(suchaschap-tersofadocument,orframesofamovie),HTTP=1.
1supportsonlyrangesofbytes.
AclientmakesarangerequestbyincludingtheRangeheaderinitsrequest,specifyingoneormorecontiguousrangesofbytes.
TheservercaneitherignoretheRangeheader,oritcanreturnoneormorerangesintheresponse.
Ifaresponsecontainsarange,ratherthantheentireresource,itcarriesthe206(PartialContent)statuscode.
ThiscodepreventsHTTP=1.
0proxycachesfromaccidentallytreatingtheresponseasafullone,andthenusingitasacachedre-sponsetoasubsequentrequest.
Inarangere-sponse,theContent-Rangeheaderindicatestheoffsetandlengthofthereturnedrange,andthenewmultipart=byterangesMIMEtypeallowsthetransmissionofmultiplerangesinonemessage.
Rangerequestscanbeusedinavarietyofways,suchas:(1)Toreadtheinitialpartofanimage,tode-termineitsgeometryandthereforedopagelayoutwithoutloadingtheentireimage.
(2)Tocompletearesponsetransferthatwasin-terrupted(eitherbytheuserorbyanetworkfailure);inotherwords,toconvertapartialcacheentryintoacompleteresponse.
(3)Toreadthetailofagrowingobject.
Someoftheseformsofrangerequestinvolvecacheconditionals.
Thatis,theproperresponsemaydependonthevalidityoftheclient'scacheentry(ifany).
Forexample,therstkind(gettingaprexoftheresource)mightbedoneunconditionally,oritmightbedonewithanIf-None-Matchheader;thelatterimpliesthattheclientonlywantstherangeiftheunderlyingobjecthaschanged,andotherwisewilluseitscacheentry.
Thesecondkindofrequest,ontheotherhand,ismadewhentheclientdoesnothaveacacheentrythatincludesthedesiredrange.
Therefore,theclientwantstherangeonlyiftheunderlyingobjecthasnotchanged;otherwise,itwantsthefullresponse.
ThiscouldbeaccomplishedbyrstsendingarangerequestwithanIf-Matchheader,andthenrepeat-ingtherequestwithouteitherheaderiftherstrequestfails.
However,sincethisisanimportantop-timization,HTTP=1.
1includesanIf-Rangeheader,whicheffectivelyperformsthatsequenceinasingleinteraction.
RangerequestswereoriginallyproposedbyAriLuotonenandJohnFranks[13],usinganextensiontotheURLsyntaxinsteadofaseparateheadereld.
However,thisapproachprovedlessgeneralthantheapproachultimatelyusedinHTTP=1.
1,especiallywithrespecttoconditionalrequests.
4.
2.
Expectand100(Continue)SomeHTTPrequests(forexample,thePUTorPOSTmethods)carryrequestbodies,whichmaybe664arbitrarilylong.
If,theserverisnotwillingtoaccepttherequest,perhapsbecauseofanauthenticationfailure,itwouldbeawasteofbandwidthtotransmitsuchalargerequestbody.
HTTP=1.
1includesanewstatuscode,100(Con-tinue),toinformtheclientthattherequestbodyshouldbetransmitted.
Whenthismechanismisused,theclientrstsendsitsrequestheaders,thenwaitsforaresponse.
Iftheresponseisanerrorcode,suchas401(Unauthorized),indicatingthattheserverdoesnotneedtoreadtherequestbody,therequestisterminated.
Iftheresponseis100(Continue),theclientcanthensendtherequestbody,knowingthattheserverwillacceptit.
However,HTTP=1.
0clientsdonotunderstandthe100(Continue)response.
Therefore,inordertotriggertheuseofthismechanism,theclientsendsthenewExpectheader,withavalueof100-continue.
(TheExpectheadercouldbeusedforother,futurepurposesnotdenedinHTTP=1.
1.
)Becausenotallserversusethismechanism(theExpectheaderisarelativelylateadditiontoHTTP=1.
1,andearly'HTTP=1.
1'serversdidnotimplementit),theclientmustnotwaitindenitelyfora100(Continue)responsebeforesendingitsrequestbody.
HTTP=1.
1speciesanumberofsome-whatcomplexrulestoavoideitherinnitewaitsorwastedbandwidth.
Welacksufcientexperiencebasedondeployedimplementationstoknowifthisdesignwillworkefciently.
4.
3.
CompressionOnewell-knownwaytoconservebandwidthisthroughtheuseofdatacompression.
Whilemostim-ageformats(GIF,JPEG,MPEG)areprecompressed,manyotherdatatypesusedintheWebarenot.
Onestudyshowedthataggressiveuseofadditionalcom-pressioncouldsavealmost40%ofthebytessentviaHTTP[25].
WhileHTTP=1.
0includedsomesupportforcompression,itdidnotprovideadequatemecha-nismsfornegotiatingtheuseofcompression,orfordistinguishingbetweenend-to-endandhop-by-hopcompression.
HTTP=1.
1makesadistinctionbetweencontent-codings,whichareend-to-endencodingsthatmightbeinherentinthenativeformatofaresource,andtransfer-codings,whicharealwayshop-by-hop.
Compressioncanbedoneeitherasacontent-cod-ingorasatransfer-coding.
Tosupportthischoice,andthechoicebetweenvariousexistingandfuturecompressioncodings,withoutbreakingcompatibilitywiththeinstalledbase,HTTP=1.
1hadtocarefullyreviseandextendthemechanismsfornegotiatingtheuseofcodings.
HTTP=1.
0includestheContent-Encodingheader,whichindicatestheend-to-endcontent-cod-ing(s)usedforamessage;HTTP=1.
1addstheTransfer-Encodingheader,whichindicatesthehop-by-hoptransfer-coding(s)usedforamessage.
HTTP=1.
1(unlikeHTTP=1.
0)carefullyspeciestheAccept-Encodingheader,usedbyaclienttoindicatewhatcontent-codingsitcanhandle,andwhichonesitprefers.
Onetrickyissueistheneedtosupport'robot'clientsthatareattemptingtocre-atemirrorsoftheoriginserver'sresources;anotherproblemistheneedtointeroperatewithHTTP=1.
0implementations,forwhichAccept-Encodingwaspoorlyspecied.
HTTP=1.
1alsoincludestheTEheader,whichallowstheclienttoindicatewhichtransfer-codingsareacceptable,andwhicharepreferred.
Notethatoneimportanttransfer-coding,Chunked,hasaspe-cialfunction(notrelatedtocompression),andisdiscussedfurtherinSection6.
1.
5.
NetworkconnectionmanagementHTTPalmostalwaysusesTCPasitstransportprotocol.
TCPworksbestforlong-livedconnections,buttheoriginalHTTPdesignusedanewTCPcon-nectionforeachrequest,soeachrequestincurredthecostofsettingupanewTCPconnection(atleastoneround-triptimeacrossthenetwork,plusseveraloverheadpackets).
SincemostWebinteractionsareshort(themedianresponsemessagesizeisabout4Kbytes[25]),theTCPconnectionsseldomgetpastthe'slow-start'region[19]andthereforefailtomaximizetheiruseoftheavailablebandwidth.
Webpagesfrequentlyhaveembeddedimages,sometimesmanyofthem,andeachimageisre-trievedviaaseparateHTTPrequest.
TheuseofanewTCPconnectionforeachimageretrievalseri-alizesthedisplayoftheentirepageontheconnec-tion-setuplatenciesforalloftherequests.
Netscape665introducedtheuseofparallelTCPconnectionstocompensateforthisserialization,butthepossibil-ityofincreasedcongestionlimitstheutilityofthisapproach.
Toresolvetheseproblems,PadmanabhanandMogul[33]recommendedtheuseofpersistentcon-nectionsandthepipeliningofrequestsonapersistentconnection.
5.
1.
TheConnectionheaderBeforediscussingpersistentconnections,wead-dressamorebasicissue.
Giventheuseofinter-mediateproxies,HTTPmakesadistinctionbetweentheend-to-endpathtakenbyamessage,andtheactualhop-by-hopconnectionbetweentwoHTTPimplementations.
HTTP=1.
1introducestheconceptofhop-by-hopheaders:messageheadersthatapplyonlytoagivenconnection,andnottotheentirepath.
(Forex-ample,wehavealreadydescribedthehop-by-hopTransfer-EncodingandTEheaders.
)Theuseofhop-by-hopheaderscreatesapotentialproblem:ifsuchaheaderweretobeforwardedbyanaiveproxy,itmightmisleadtherecipient.
Therefore,HTTP=1.
1includestheConnectionheader.
Thisheaderlistsallofthehop-by-hophead-ersinamessage,tellingtherecipientthattheseheadersmustberemovedfromthatmessagebeforeitisforwarded.
Thisextensiblemechanismallowsthefutureintroductionofnewhop-by-hophead-ers;thesenderneednotknowwhethertherecipientunderstandsanewheaderinordertopreventtherecipientfromforwardingtheheader.
BecauseHTTP=1.
0proxiesdonotunderstandtheConnectionheader,however,HTTP=1.
1imposesanadditionalrule.
IfaConnectionheaderisre-ceivedinanHTTP=1.
0message,thenitmusthavebeenincorrectlyforwardedbyanHTTP=1.
0proxy.
Therefore,alloftheheadersitlistswerealsoincor-rectlyforwarded,andmustbeignored.
TheConnectionheadermayalsolistconnec-tion-tokens,whicharenotheadersbutratherper-connectionBooleanags.
Forexample,HTTP=1.
1denesthetokenclosetopermitthepeertoindi-catethatitdoesnotwanttouseapersistentcon-nection.
Again,theConnectionheadermechanismpreventsthesetokensfrombeingforwarded.
5.
2.
PersistentconnectionsHTTP=1.
0,initsdocumentedform,madenopro-visionforpersistentconnections.
SomeHTTP=1.
0implementations,however,useaKeep-Aliveheader(describedin[12])torequestthatacon-nectionpersist.
Thisdesigndidnotinteroperatewithintermediateproxies(seesection19.
6.
2of[9]);HTTP=1.
1speciesamoregeneralsolution.
Inrecognitionoftheirdesirableproperties,HTTP=1.
1makespersistentconnectionsthedefault.
HTTP=1.
1clients,servers,andproxiesassumethataconnectionwillbekeptopenafterthetransmissionofarequestanditsresponse.
Theprotocoldoesal-lowanimplementationtocloseaconnectionatanytime,inordertomanageitsresources,althoughitisbesttodosoonlyaftertheendofaresponse.
Becauseanimplementationmayprefernottousepersistentconnectionsifitcannotefcientlyscaletolargenumbersofconnectionsormaywanttocleanlyterminateoneforresource-managementreasons,theprotocolpermitsittosendaConnection:closeheadertoinformtherecipientthattheconnectionwillnotbereused.
5.
3.
PipeliningAlthoughHTTP=1.
1encouragesthetransmissionofmultiplerequestsoverasingleTCPconnection,eachrequestmuststillbesentinonecontiguousmessage,andaservermustsendresponses(onagivenconnection)intheorderthatitreceivedthecorrespondingrequests.
However,aclientneednotwaittoreceivetheresponseforonerequestbeforesendinganotherrequestonthesameconnection.
Infact,aclientcouldsendanarbitrarilylargenumberofrequestsoveraTCPconnectionbeforereceiv-inganyoftheresponses.
Thispractice,knownaspipelining,cangreatlyimproveperformance[31].
Itavoidstheneedtowaitfornetworkround-trips,anditmakesthebestpossibleuseoftheTCPpro-tocol.
6.
MessagetransmissionHTTPmessagesmaycarryabodyofarbitrarylength.
Therecipientofamessageneedstoknow666wherethemessageends.
ThesendercanusetheContent-Lengthheader,whichgivesthelengthofthebody.
However,manyresponsesaregenerateddynamically,byCGI[3]processesandsimilarmech-anisms.
Withoutbufferingtheentireresponse(whichwouldaddlatency),theservercannotknowhowlongitwillbeandcannotsendaContent-Lengthheader.
Whennotusingpersistentconnections,theso-lutionissimple:theserverclosestheconnection.
ThisoptionisavailableinHTTP=1.
1,butitdefeatstheperformanceadvantagesofpersistentconnec-tions.
6.
1.
TheChunkedtransfer-codingHTTP=1.
1resolvestheproblemofdelimitingmessagebodiesbyintroducingtheChunkedtrans-fer-coding.
Thesenderbreaksthemessagebodyintochunksofarbitrarylength,andeachchunkissentwithitslengthprepended;itmarkstheendofthemessagewithazero-lengthchunk.
ThesenderusestheTransfer-Encoding:chunkedheadertosig-naltheuseofchunking.
Thismechanismallowsthesendertobuffersmallpiecesofthemessage,insteadoftheentiremessage,withoutaddingmuchcomplexityoroverhead.
AllHTTP=1.
1implementationsmustbeabletoreceivechunkedmessages.
TheChunkedtransfer-codingsolvesanotherproblem,notrelatedtoperformance.
InHTTP=1.
0,ifthesenderdoesnotincludeaContent-Lengthheader,therecipientcannottellifthemessagehasbeentruncatedduetotransmissionproblems.
Thisambiguityleadstoerrors,especiallywhentruncatedresponsesarestoredincaches.
6.
2.
TrailersChunkingsolvesanotherproblemrelatedtosender-sidemessagebuffering.
Someheaderelds,suchasContent-MD5(acryptographicchecksumoverthemessagebody),cannotbecomputeduntilafterthemessagebodyisgenerated.
InHTTP=1.
0,theuseofsuchheadereldsrequiredthesendertobuffertheentiremessage.
InHTTP=1.
1,achunkedmessagemayincludeatrailerafterthenalchunk.
Atrailerissimplyasetofoneormoreheaderelds.
Byplacingthemattheendofthemessage,thesenderallowsitselftocomputethemaftergeneratingthemessagebody.
ThesenderalertstherecipienttothepresenceofmessagetrailersbyincludingaTrailerheader,whichliststhesetofheadersdeferreduntilthetrailer.
Thisalert,forexample,allowsabrowsertoavoiddisplayingaprexoftheresponsebeforeithasreceivedauthenticationinformationcarriedinatrailer.
HTTP=1.
1imposescertainconditionsontheuseoftrailers,topreventcertainkindsofinteroperabil-ityfailure.
Forexample,ifaserversendsalengthymessagewithatrailertoanHTTP=1.
1proxythatisforwardingtheresponsetoanHTTP=1.
0client,theproxymusteitherbuffertheentiremessageordropthetrailer.
Ratherthaninsistthatproxiesbufferarbi-trarilylongmessages,whichwouldbeinfeasible,theprotocolsetsrulesthatshouldpreventanycriticalinformationinthetrailer(suchasauthenticationin-formation)frombeinglostbecauseofthisproblem.
Specically,aservercannotsendatrailerunlessei-thertheinformationitcontainsispurelyoptional,ortheclienthassentaTE:trailersheader,indicatingthatitiswillingtoreceivetrailers(and,implicitly,tobuffertheentireresponseifitisforwardingthemessagetoanHTTP=1.
0client).
6.
3.
Transfer-lengthissuesSeveralHTTP=1.
1mechanisms,suchasDigestAccessAuthentication(seeSection9.
1),requireend-to-endagreementonthelengthofthemessagebody;thisisknownastheentity-length.
Hop-by-hoptransfer-codings,suchascompressionorchunk-ing,cantemporarilychangethetransfer-lengthofamessage.
Beforethisdistinctionwasclaried,someearlierimplementationsusedtheContent-Lengthheaderindiscriminately.
Therefore,HTTP=1.
1givesalengthysetofrulesforindicatingandinferringtheentity-lengthofamessage.
Forexample,ifanon-identitytransfer-cod-ingisused(sothetransfer-lengthandentity-lengthdiffer),thesenderisnotallowedtousetheCon-tent-Lengthheaderatall.
Whenaresponsecon-tainsmultiplebyteranges,usingContent-Type:multipart=byteranges,thenthisself-delimitingformatdenesthetransfer-length.
6677.
InternetaddressconservationCompaniesandorganizationsuseURLstoad-vertisethemselvesandtheirproductsandservices.
WhenaURLappearsinamediumotherthantheWebitself,peopleseemtoprefer'purehostname'URLs;i.
e.
,URLswithoutanypathsyntaxfollow-ingthehostname.
Theseareoftenknownas'vanityURLs',butinspiteoftheimplieddisparagement,it'sunlikelythatnon-puristuserswillabandonthispractice,whichhasledtothecontinuingcreationofhugenumbersofhostnames.
IPaddressesarewidelyperceivedasascarcere-source(pendingtheuncertaintransitiontoIPv6[7]).
TheDomainNameSystem(DNS)allowsmultiplehostnamestobeboundtothesameIPaddress.
Un-fortunately,becausetheoriginaldesignersofHTTPdidnotanticipatethe'successdisaster'theywereenabling,HTTP=1.
0requestsdonotpassthehost-namepartoftherequestURL.
Forexample,ifausermakesarequestfortheresourceatURLhttp://example1.
org/home.
html,thebrowsersendsamessagewiththeRequest-Line:GET/home.
htmlHTTP/1.
0totheserveratexample1.
org.
ThispreventsthebindingofanotherHTTPserverhostname,suchasexampleB.
orgtothesameIPaddress,becausetheserverreceivingsuchamessagecannottellwhichserverthemessageismeantfor.
Thus,theprolif-erationofvanityURLscausesaproliferationofIPaddressallocations.
TheInternetEngineeringSteeringGroup(IESG),whichmanagestheIETFprocess,insistedthatHTTP=1.
1takestepstoimproveconservationofIPaddresses.
SinceHTTP=1.
1hadtointeroperatewithHTTP=1.
0,itcouldnotchangetheformatoftheRequest-Linetoincludetheserverhostname.
In-stead,HTTP=1.
1requiresrequeststoincludeaHostheader,rstproposedbyJohnFranks[14],thatcar-riesthehostname.
Thisconvertstheexampleaboveto:GET/home.
htmlHTTP/1.
1Host:example1.
orgIftheURLreferencesaportotherthanthedefault(TCPport80),thisisalsogivenintheHostheader.
Clearly,sinceHTTP=1.
0clientswillnotsendHostheaders,HTTP=1.
1serverscannotsimplyrejectallmessageswithoutthem.
However,theHTTP=1.
1specicationrequiresthatanHTTP=1.
1servermustrejectanyHTTP=1.
1messagethatdoesnotcontainaHostheader.
TheintentoftheHostheadermechanism,andinparticulartherequirementthatenforcesitspresenceinHTTP=1.
1requests,istospeedthetransitionawayfromassigninganewIPaddressforeveryvanityURL.
However,aslongasasubstantialfractionoftheusersontheInternetusebrowsersthatdonotsendHost,noWebsiteoperator(suchasanelectroniccommercebusiness)thatdependsontheseuserswillgiveupavanity-URLIPaddress.
Thetransition,therefore,maytakemanyyears.
ItmaybeobviatedbyanearliertransitiontoIPv6,orbytheuseofmarketmechanismstodiscouragetheunnecessaryconsumptionofIPv4addresses.
8.
ErrornoticationHTTP=1.
0denedarelativelysmallsetofsixteenstatuscodes,includingthenormal200(OK)code.
Experiencerevealedtheneedfornerresolutioninerrorreporting.
8.
1.
TheWarningheaderHTTPstatuscodesindicatethesuccessorfailureofarequest.
Forasuccessfulresponse,thestatuscodecannotprovideadditionaladvisoryinformation,inpartbecausetheplacementofthestatuscodeintheStatus-Line,insteadofinaheadereld,preventstheuseofmultiplestatuscodes.
HTTP=1.
1introducesaWarningheader,whichmaycarryanynumberofsubsidiarystatusindica-tions.
Theintentistoallowasendertoadvisetherecipientthatsomethingmaybeunsatisfactoryaboutanostensiblysuccessfulresponse.
HTTP=1.
1denesaninitialsetofWarningcodes,mostlyrelatedtotheactionsofcachesalongtheresponsepath.
Forexample,aWarningcanmarkaresponseashavingbeenreturnedbyacacheduringdisconnectedoperation,whenitisnotpossibletovalidatethecacheentrywiththeoriginserver.
TheWarningcodesaredividedintotwoclasses,668basedontherstdigitofthe3-digitcode.
Oneclassofwarningsmustbedeletedafterasuccessfulrevalidationofacacheentry;theotherclassmustberetainedwitharevalidatedcacheentry.
Becausethisdistinctionismadebasedontherstdigitofthecode,ratherthanthroughanexhaustivelistingofthecodes,itisextensibletoWarningcodesdenedinthefuture.
8.
2.
OthernewstatuscodesThereare24newstatuscodesinHTTP=1.
1;wehavediscussed100(Continue),206(PartialCon-tent),and300(MultipleChoices)elsewhereinthispaper.
Afewofthemorenotableadditionsinclude:409(Conict),returnedwhenarequestwouldconictwiththecurrentstateoftheresource.
Forexample,aPUTrequestmightviolateaversion-ingpolicy.
410(Gone),usedwhenaresourcehasbeenre-movedpermanentlyfromaserver,andtoaidinthedeletionofanylinkstotheresource.
Mostoftheothernewstatuscodesareminorexten-sions.
9.
Security,integrity,andauthenticationInrecentyears,theIETFhasheighteneditssen-sitivitytoissuesofprivacyandsecurity.
Onespecialconcernhasbeentheeliminationofpasswordstrans-mitted'intheclear'.
ThisincreasedemphasishasmanifesteditselfintheHTTP=1.
1specication(andothercloselyrelatedspecications).
9.
1.
DigestaccessauthenticationHTTP=1.
0providesachallenge-responseaccesscontrolmechanism,Basicauthentication.
TheoriginserverrespondstoarequestforwhichitneedsauthenticationwithaWWW-Authenticateheaderthatidentiestheauthenticationscheme(inthiscase,'Basic')andrealm.
(Therealmvalueallowsaservertopartitionsetsofresourcesinto'protectionspaces',eachwithitsownauthorizationdatabase.
)Theclient(useragent)typicallyqueriestheuserforausernameandpasswordfortherealm,thenrepeatstheoriginalrequest,thistimeincludinganAuthorizationheaderthatcontainstheusernameandpassword.
Assumingthesecredentialsareac-ceptabletoit,theoriginserverrespondsbysendingtheexpectedcontent.
Aclientmaycontinuetosendthesamecredentialsforotherresourcesinthesamerealmonthesameserver,thuseliminatingtheextraoverheadofthechallengeandresponse.
AseriousawinBasicauthenticationisthattheusernameandpasswordinthecredentialsareunen-cryptedandthereforevulnerabletonetworksnoop-ing.
Thecredentialsalsohavenotimedependency,sotheycouldbecollectedatleisureandusedlongaftertheywerecollected.
Digestaccessauthentica-tion[10,11]providesasimplemechanismthatusesthesameframeworkasBasicauthenticationwhileeliminatingmanyofitsaws.
(Digestaccessauthen-tication,beinglargelyseparablefromtheHTTP=1.
1specication,hasdevelopedinparallelwithit.
)ThemessageowinDigestaccessauthenticationmirrorsthatofBasicandusesthesameheaders,butwithaschemeof'Digest'.
Theserver'schallengeinDigestaccessauthenticationusesanonce(one-time)value,amongotherinformation.
Torespondsuccess-fully,aclientmustcomputeachecksum(MD5,bydefault)oftheusername,password,nonce,HTTPmethodoftherequest,andtherequestedURI.
Notonlyisthepasswordnolongerunencrypted,butthegivenresponseiscorrectonlyforasingleresourceandmethod.
Thus,anattackerthatcansnooponthenetworkcouldonlyreplaytherequest,theresponseforwhichhehasalreadyseen.
UnlikewithBasicauthentication,obtainingthesecredentialsdoesnotprovideaccesstootherresources.
AswithBasicauthentication,theclientmaymakefurtherrequeststothesamerealmandincludeDi-gestcredentials,computedwiththeappropriatere-questmethodandrequest-URI.
However,theoriginserver'snoncevaluemaybetime-dependent.
Theservercanrejectthecredentialsbysayingthere-sponseusedastalenonceandbyprovidinganewone.
Theclientcanthenrecomputeitscredentialswithoutneedingtoasktheuserforusernameandpasswordagain.
Inadditiontothestraightforwardauthenticationcapability,Digestaccessauthenticationofferstwootherfeatures:supportforthird-partyauthentica-tionservers,andalimitedmessageintegrityfeature(throughtheAuthentication-Infoheader).
6699.
2.
ProxyauthenticationSomeproxyserversprovideserviceonlytoprop-erlyauthenticatedclients.
Thisprevents,forexam-ple,otherclientsfromstealingbandwidthfromanunsuspectingproxy.
Tosupportproxyauthentication,HTTP=1.
1in-troducestheProxy-AuthenticateandProxy-Authorizationheaders.
TheyplaythesameroleastheWWW-AuthenticateandAuthorizationheadersinHTTP=1.
0,exceptthatthenewheadersarehop-by-hop,ratherthanend-to-end.
Proxyau-thenticationmayuseeitheroftheDigestorBasicauthenticationschemes,buttheformerispreferred.
AproxyserversendstheclientaProxy-Authenticateheader,containingachallenge,ina407(ProxyAuthenticationRequired)response.
Theclientthenrepeatstheinitialrequest,butaddsaProxy-Authorizationheaderthatcontainscre-dentialsappropriatetothechallenge.
Aftersuccess-fulproxyauthentication,aclienttypicallysendsthesameProxy-Authorizationheadertotheproxywitheachsubsequentrequest,ratherthanwaittobechallengedagain.
9.
3.
ProtectingtheprivacyofURIsTheURIofaresourceoftenrepresentsinforma-tionthatsomeusersmayviewasprivate.
Usersmayprefernottohaveitwidelyknownthattheyhavevisitedcertainsites.
TheReferer[sic]headerinarequestprovidestheserverwiththeURIoftheresourcefromwhichtherequest-URIwasobtained.
Thisgivestheserverinformationabouttheuser'spreviouspage-view.
Toprotectagainstunexpectedprivacyviolations,theHTTP=1.
1specicationtakespainstodiscouragesendingtheRefererheaderinappropriately;forex-ample,whenauserentersaURLfromthekeyboard,theapplicationshouldnotsendaRefererheaderdescribingthecurrently-visiblepage,norshouldaclientsendtheRefererheaderinaninsecurere-questifthereferringpagehadbeentransferredse-curely.
TheuseofaGET-basedHTMLformcausestheencodingofformparametersintherequest-URI.
Mostproxyserverslogtheserequest-URIs.
Topro-tectagainstrevealingsensitiveinformation,suchaspasswordsorcredit-cardnumbers,inaURI,theHTTP=1.
1specicationstronglydiscouragestheuseofGET-basedformsforsubmittingsuchdata.
TheuseofPOST-basedformspreventstheformparame-tersfromappearinginarequest-URI,andthereforefrombeingloggedinappropriately.
9.
4.
Content-MD5MIME'sContent-MD5headercontainstheMD5digestoftheentitybeingsent[28].
TheHTTP=1.
1specicationprovidesspecicrulesfortheuseofthisheaderintheWeb,whichdifferslightlyfromitsuseinMIME(electronicmail).
ThesendermaychoosetosendContent-MD5sotherecipientcandetectaccidentalchangestotheentityduringitstransmission.
Content-MD5isagoodexampleofaheaderthataservermightusefullysendasatrailer.
ClearlytheContent-MD5valuecouldeasilybespoofedandcannotserveasameansofsecurity.
Also,becauseContent-MD5coversjusttheentityinonemessage,itcannotbeusedtodetermineifafullresponsehasbeensuccessfullyreassembledfromanumberofpartial(range)responses,orwhetherresponseheadershavebeenaltered.
9.
5.
StatemanagementHTTPrequestsarestateless.
Thatis,fromaserver'sperspective,eachrequestcanordinarilybetreatedasindependentofanyother.
ForWebap-plications,however,statecansometimesbeuseful.
Forexample,ashoppingapplicationwouldliketokeeptrackofwhatisinauser's'shoppingbasket',asthebasket'scontentschangeoverthecourseofasequenceofHTTPrequests.
Netscapeintroduced'cookies'[29]inversion1.
1oftheirbrowserasastatemanagementmecha-nism.
TheIETFsubsequentlystandardizedcookiesinRFC2109[21].
(Thecookiespecicationisan-otherexampleofhowHTTPcanbeextendedbyaseparatespecicationwithoutaffectingthecorepro-tocol.
Cookiesupportisoptionalinserversanduseragents,althoughsomeWeb-basedserviceswillnotworkintheirabsence.
)Thebasiccookiemechanismissimple.
Anoriginserversendsanarbitrarypieceof(state)informationtotheclientinitsresponse.
Theclientisresponsi-670bleforsavingtheinformationandreturningitwithitsnextrequesttotheoriginserver.
RFC2109andNetscape'soriginalspecicationrelaxthismodelsothatacookiecanbereturnedtoanyofacollectionofrelatedservers,ratherthanjusttoone.
Thespecica-tionsalsorestrictsforwhichURIsonagivenserverthecookiemaybereturned.
Aserversmayassignalifetimetoacookie,afterwhichitisnolongerused.
Cookieshavebothprivacyandsecurityimplica-tions.
Becausetheircontentisarbitrary,cookiesmaycontainsensitiveapplication-dependentinformation.
Forexample,theycouldcontaincreditcardnum-bers,usernamesandpasswords,orotherpersonalinformation.
Applicationsthatsendsuchinformationoverunencryptedconnectionsleaveitvulnerabletosnooping,andcookiesstoredataclientsystemmightrevealsensitiveinformationtoanotheruserof(orin-truderinto)thatclient.
RFC2109provedtobecontroversial,primarilybecauseofrestrictionsthatwereintroducedtopro-tectprivacy.
Probablythemostcontroversialofthesehastodowith'unveriabletransactions'and'third-partycookies'.
Considerthisscenario.
(1)Theuservisitshttp://www.
example1.
com/home.
html.
(2)ThereturnedpagecontainsanIMG(image)tagwithareferencetohttp://ad.
example.
com/adv1.
gif,anadvertisement.
(3)Theuser'sbrowserautomaticallyrequeststheimage.
Theresponseincludesacookiefromad.
example.
com.
(4)Theuservisitshttp://www.
exampleB.
com/home.
html.
(5)ThereturnedpagecontainsanIMGtagwithareferencetohttp://ad.
example.
com/adv2.
gif.
(6)Theuser'sbrowserautomaticallyrequeststheimage,sendingthepreviouslyreceivedcookietoad.
example.
comintheprocess.
Theresponseincludesanewcookiefromad.
example.
com.
Privacyadvocates,andothers,worriedthat:theuserreceives,instep3,a('third-party')cookiefromad.
example.
com,asiteshedidn'tevenknowshewasgoingtovisit(an'unveriabletransac-tion');andthatrstcookiegetsreturnedtoad.
example.
cominthesecondimagerequest(step6).
IfaRefererheaderissentwitheachoftheim-agerequeststoad.
example.
com,thenthatsitecanbegintoaccumulateaproleoftheuser'sinterestsfromthesitesshevisited,herehttp://www.
example1.
com/home.
htmlandhttp://www.
exampleB.
com/home.
html.
Suchanadvertisingsitecouldpotentiallyselectadvertisementsthatarelikelytobeinterestingtoher.
Whilethatprolingprocessisrelativelybenigninisolation,itcouldbecomemorepersonaliftheprolecanalsobetiedtoaspecicrealperson,notjustapersona.
Forexample,thismighthappeniftheusergoesthroughsomekindofregistrationatwww.
example1.
com.
RFC2109soughttolimitthepossibleperniciouseffectsofcookiesbyrequiringuseragentstorejectcookiesthatarrivefromtheresponsestounveri-abletransactions.
RFC2109furtherstatedthatuseragentscouldbeconguredtoacceptsuchcook-ies,providedthatthedefaultwasnottoacceptthem.
Thisdefaultsettingwasasourceofcon-cernforadvertisingnetworks(companiesthatrunsiteslikead.
example.
comintheexample)whosebusinessmodeldependedoncookies,andwhosebusinessblossomedintheintervalbetweenwhenthespecicationwasessentiallycomplete(July,1996)andthetimeitappearedasanRFC(February,1997).
RFC2109hasundergonefurtherrenement[22]inresponsetocomments,bothpoliticalandtechnical.
10.
ContentnegotiationWebusersspeakmanylanguagesandusemanycharactersets.
SomeWebresourcesareavail-ableinseveralvariantstosatisfythismultiplicity.
HTTP=1.
0includedthenotionofcontentnegotia-tion,amechanismbywhichaclientcaninformtheserverwhichlanguage(s)and=orcharacterset(s)areacceptabletotheuser.
Contentnegotiationhasprovedtobeacon-tentiousandconfusingarea.
Someaspectsthatap-pearedsimpleatrstturnedouttobequitedifculttoresolve.
Forexample,althoughcurrentIETFprac-ticeistoinsistonexplicitcharactersetlabelinginallrelevantcontexts,theexistingHTTPpracticehasbeentouseadefaultcharactersetinmostcontexts,butnotallimplementationschosethesamedefault.
671TheuseofunlabeleddefaultsgreatlycomplicatestheproblemofinternationalizingtheWeb.
HTTP=1.
0providedafewfeaturestosupportcontentnegotiation,butRFC1945[2]neverusesthattermanddevoteslessthanapagetotherelevantpro-tocolfeatures.
TheHTTP=1.
1specicationspeciesthesefeatureswithfargreatercare,andintroducesanumberofnewconcepts.
Thegoalofthecontentnegotiationmechanismistochoosethebestavailablerepresentationofaresource.
HTTP=1.
1providestwoorthogonalformsofcontentnegotiation,differinginwherethechoiceismade:(1)Inserver-drivennegotiation,themorematureform,theclientsendshintsabouttheuser'sprefer-encestotheserver,usingheaderssuchasAccept-Language,Accept-Charset,etc.
Theserverthenchoosestherepresentationthatbestmatchesthepref-erencesexpressedintheseheaders.
(2)Inagent-drivennegotiation,whentheclientrequestsavaryingresource,theserverreplieswitha300(MultipleChoices)responsethatcontainsalistoftheavailablerepresentationsandadescriptionofeachrepresentation'sproperties(suchasitslanguageandcharacterset).
Theclient(agent)thenchoosesonerepresentation,eitherautomaticallyorwithuserintervention,andresubmitstherequest,specifyingthechosenvariant.
AlthoughtheHTTP=1.
1specicationreservestheAlternates[16]headernameforuseinagent-drivennegotiation,theHTTPworkinggroupnevercompletedaspecicationofthisheader,andserver-drivennegotiationremainstheonlyusableform.
Someusersmayspeakmultiplelanguages,butwithvaryingdegreesofuency.
Similarly,aWebresourcemightbeavailableinitsoriginallanguage,andinseveraltranslationsofvaryingfaithfulness.
HTTPintroducestheuseofqualityvaluestoexpresstheimportanceordegreeofacceptabilityofvariousnegotiableparameters.
Aqualityvalue(orqvalue)isaxed-pointnumberbetween0.
0and1.
0.
Forexam-ple,anativespeakerofEnglishwithsomeuencyinFrench,andwhocanimposeonaDanish-speakingofce-mate,mightcongureabrowsertogeneraterequestsincludingAccept-Language:en,fr;q=0.
5,da;q=0.
1Becausethecontent-negotiationmechanismal-lowsqvaluesandwildcards,andexpressesvaria-tionacrossmanydimensions(language,character-set,content-type,andcontent-encoding)theauto-matedchoiceofthe'bestavailable'variantcanbecomplexandmightgenerateunexpectedoutcomes.
Thesechoicescaninteractwithcachinginsubtleways;seethediscussioninSection3.
4.
Contentnegotiationpromisestobeafertileareaforadditionalprotocolevolution.
Forexample,theHTTPworkinggrouprecognizedtheutilityofau-tomaticnegotiationregardingclientimplementationfeatures,suchasscreensize,resolution,andcolordepth.
TheIETFhascreatedtheContentNegotiationworkinggrouptocarryforwardwithworkinthearea.
11.
PendingproposalsAlthoughtheHTTPworkinggroupwilldisbandafterthepublicationoftheHTTP=1.
1specication,therearenumerouspendingproposalsforfurtherimprovements.
Herewesketchafewofthemoresignicantones.
TheHTTP=1.
1specicationplacedastrongem-phasisonextensibilitybutwasnotabletoresolvethisissueentirely.
Althoughithasnotyetachievedthestatusofaworkinggroup,oneefforthasbeentryingtodeneageneralextensibilitymechanism[32].
AswenotedinSection10,theContentNegotia-tionworkinggroupisworkingonproposalstobetterdenecontentnegotiation[16,15]andfeaturetags.
SeveralresearchershaveobservedthatwhenWebresourceschange(thusinvalidatingcacheentries),theyusuallydonotchangeverymuch[6,25].
Thissuggeststhattransmittingonlythedifferences(ordelta)betweenthecurrentresourcevalueandacachedresponse,ratherthananentirenewresponse,couldsavebandwidthandtime.
Twoofus,incon-junctionwithseveralotherpeople,haveproposedasimpleextensiontoHTTP=1.
1tosupportdeltaencoding[27].
Intoday'sWeb,contentiswidelyshared,butitmostlyowsinonedirection,fromserverstoclients.
TheWebcouldbecomeamediumfordistributedupdatestosharedcontent.
TheIETF'sWorldWideWebDistributedAuthoringandVersioning(WEB-DAV)workinggroupisintheprocessofdeningHTTPextensionstoenablethisvision[34].
67212.
ObservationsHTTP=1.
1differsfromHTTP=1.
0innumer-ousways,bothlargeandsmall.
Whilemanyofthesechangesareclearlyforthebetter,thepro-tocoldescriptionhastripledinlength,andmanyofthenewfeatureswereintroducedwithoutanyrealexperimentalevaluationtobackthemup.
TheHTTP=1.
1specicationalsoincludesnumerousir-regularitiesforcompatibilitywiththeinstalledbaseofHTTP=1.
0implementations.
Thisincreaseincomplexitycomplicatesthejobofclient,server,andespeciallyproxycacheimple-mentors.
Ithasalreadyledtounexpectedinteractionsbetweenfeatures,andwillprobablyleadtoothers.
WedonotexpecttheadoptionofHTTP=1.
1togoentirelywithoutglitches.
Fortunately,thenumerousprovisionsinHTTP=1.
1forextensibilityshouldsim-plifytheintroductionoffuturemodications.
AcknowledgementsWewouldliketothanktheanonymousrefereesfortheirreviews.
References[1]M.
Abrams,C.
R.
Standridge,G.
Abdulla,S.
WilliamsandE.
A.
Fox,Cachingproxies:Limitationsandpotentials,in:Proc.
oftheInternationalWorldWideWebConference,December1995,http://ei.
cs.
vt.
edu/succeed/WWW4/WWW4.
html.
[2]T.
Berners-Lee,R.
FieldingandH.
Frystyck,HypertextTransferProtocol-HTTP=1.
0,RFC1945,HTTPWorkingGroup,May1998.
[3]CGI:CommonGatewayInterface,March1998,http://www.
w3.
org/CGI/.
[4]Ciscocacheengine,http://www.
cisco.
com/warp/public/751/cache.
[5]HTTP-WGMailinglistarchives,Sept–Nov1994–1998.
http://www.
ics.
uci.
edu/pub/ietf/http/hypermail/.
[6]F.
Douglis,A.
Feldmann,B.
KrishnamurthyandJ.
Mogul,Rateofchangeandothermetrics:AlivestudyoftheWorldWideWeb,in:Proc.
USENIXSymp.
onInternetTechnologiesandSystems,December1997,pp.
147–158.
http://www.
usenix.
org/events/usits97.
[7]S.
DeeringandR.
Hinden,InternetProtocolVersion6,RFC1883,IETF,December1995.
[8]A.
Fox,S.
D.
Gribble,E.
A.
BrewerandE.
Amir,Adapt-ingtonetworkandclientvariationviaon-demanddynamictranscoding,in:Proc.
ASPLOSVII,Cambridge,MA,Oc-tober1996,pp.
160–170.
[9]R.
Fielding,J.
Gettys,J.
C.
Mogul,H.
Frystyk,L.
Mas-inter,P.
LeachandT.
Berners-Lee,HypertextTrans-ferProtocol-HTTP=1.
1,InternetDraftdraft-ietf-http-v11-spec-rev-06.
txt,HTTPWorkingGroup,November18,1998,ftp://ftp.
ietf.
org/internet-drafts/draft-ietf-http-v11-spec-rev-06.
txt;willbeissuedasanRFC.
[10]J.
Franks,P.
Hallam-Baker,J.
Hostetler,P.
Leach,A.
Lu-otonen,E.
SinkandL.
Stewart,AnExtensiontoHTTP:DigestAccessAuthentication,RFC2069,IETF,January1997,ftp://ftp.
ietf.
org/internet-drafts/draft-ietf-http-authentication-03.
txt;thisisaworkinprogress.
[11]J.
Franks,P.
Hallam-Baker,J.
Hostetler,S.
Lawrence,P.
Leach,A.
LuotonenandL.
Stewart,HTTPAuthenti-cation:BasicandDigestAccessAuthentication,InternetDraftdraft-ietf-http-authentication-03.
txt,IETF,September1998,ftp://ftp.
ietf.
org/internet-drafts/draft-ietf-http-authentication-03.
txt,thisisaworkinprogress.
[12]R.
T.
Fielding,Keep-alivenotes,October10,1995.
http://www.
ics.
uci.
edu/pub/ietf/http/hypermail/1995q4/0063.
html.
[13]J.
FranksandA.
Luotonen,Byteranges—formalspecproposal,May17,1995,http://www.
ics.
uci.
edu/pub/ietf/http/hypermail/1995q2/0122.
html.
[14]J.
Franks,TwoproposalsforHTTP/2.
0,November,161994.
http://www.
ics.
uci.
edu/pub/ietf/http/hypermail/1994q4/0019.
html.
[15]K.
HoltmanandA.
H.
Mutz,HTTPRemoteVariantSe-lectionAlgorithm-RVSA=1.
0,RFC2296,HTTPWorkingGroup,March1998.
[16]K.
HoltmanandA.
H.
Mutz,TransparentContentNegoti-ationinHTTP,RFC2295,HTTPWorkingGroup,March1998.
[17]A.
IyengarandJ.
Challenger,ImprovingWebserverperfor-mancebycachingdynamicdata,in:Proc.
USENIXSymp.
onInternetTechnologiesandSystems,December1997,http://www.
usenix.
org/events/usits97.
[18]Inktomitrafcserver,http://www.
inktomi.
com/products/trafc/.
[19]V.
Jacobson,Congestionavoidanceandcontrol,in:Proc.
1988SIGCOMM'88SymposiumonCommunicationsAr-chitecturesandProtocols,Stanford,CA,1988,pp.
314–329.
[20]T.
M.
Kroeger,D.
D.
E.
LongandJ.
C.
Mogul,Exploringtheboundsofweblatencyreductionfromcachingandprefetch-ing,in:SymposiumonInternetTechnologyandSystems,USENIXAssociation,December1997,http://www.
usenix.
org/publications/library/proceedings/usits97/kroeger.
html.
[21]D.
M.
KristolandL.
Montulli,HTTPStateManagement,RFC2109,HTTPWorkingGroup,February1997.
[22]D.
M.
KristolandL.
Montulli,HTTPStateManagement,InternetDraftdraft-ietf-http-state-man-mec-10.
txt,HTTPWorkingGroup,July1998,http://portal.
research.
bell-labs.
com/dmk/cookie-3.
6.
txt;thisisaworkinprogress.
[23]J.
Marshall,HTTPmadereallyeasy,http://www.
jmarshall.
com/easy/http/,1997.
[24]L.
Masinter,ImplementationreportforHTTP=1.
1toDraft673Standard,September30,1998,http://www.
ietf.
org/IESG/http1.
1-implementations.
txt.
[25]J.
C.
Mogul,F.
Douglis,A.
FeldmannandB.
Krishna-murthy,Potentialbenetsofdeltaencodinganddatacom-pressionforHTTP,in:Proc.
SIGCOMM'97Conference,Cannes,France,September1997,pp.
181–194.
[26]J.
C.
Mogul,R.
T.
Fielding,J.
GettysandH.
FrystyckNielsen,UseandInterpretationofHTTPVersionNumbers,RFC2145,HTTPWorkingGroup,May1997.
[27]J.
Mogul,B.
Krishnamurthy,F.
Douglis,A.
Feldmann,Y.
GolandandA.
vanHoff,DeltaEncodinginHTTP,InternetDraftdraft-mogul-http-delta-01.
txt,IETF,February1999.
ftp://ftp.
ietf.
org/internet-drafts/draft-mogul-http-delta-01.
txt;thisisaworkinprogress.
[28]J.
MyersandM.
Rose,TheContent-MD5HeaderField,RFC1864,IETF,October1995.
[29]Netscape,PersistentclientstateHTTPcookies,http://www.
netscape.
com/newsref/std/cookie_spec.
html.
[30]H.
FrystyckNielsenandJ.
Gettys,HTTP=1.
1FeatureListReportSummary,July1998,http://www.
w3.
org/Protocols/HTTP/Forum/Reports/.
[31]H.
FrystyckNielsen,J.
Gettys,A.
Baird-Smith,E.
Prud'hommeaux,H.
W.
LieandC.
Lilley,Networkper-formanceeffectsofHTTP=1.
1,CSS1,andPNG,in:Proc.
SIGCOMM'97,Cannes,France,September1997.
[32]H.
FrystyckNielsen,P.
LeachandS.
Lawrence,HTTPExtensionFrameworkforMandatoryandOptionalEx-tensions,Internet-Draftdraft-frystyk-http-extensions-01.
txt,IETF,November1998,ftp://ftp.
ietf.
org/internet-drafts/draft-frystyk-http-extensions-01.
txt;thisisaworkinprogress.
[33]V.
N.
PadmanabhanandJ.
C.
Mogul,ImprovingHTTPla-tency,Comp.
NetworksISDNSyst.
28(1/2)(1995)25–35.
[34]J.
Slein,F.
Vitali,E.
WhiteheadandD.
Durand,Require-mentsforaDistributedAuthoringandVersioningProtocolfortheWorldWideWeb,RFC2291,WEBDAVWorkingGroup,February1998.
[35]K.
Thompson,G.
J.
MillerandR.
Wilder,Wide-areainternettrafcpatternsandcharacteristics,IEEENetwork11(6)(1997)19–23.
[36]K.
Yap,AtechnicaloverviewofthenewHTTP=1.
1spec-ication,in:Proc.
AustralianWorldWideWebTechnicalConference,Brisbane,Queensland,Australia,May1997,http://www.
dstc.
edu.
au/aw3tc/papers/yap/.
BalachanderKrishnamurthyisaresearcheratAT&TLabs–ResearchinFlorhamPark,NJ,andcanbereachedatbala@research.
att.
com.
JeffreyC.
MogulreceivedanS.
B.
fromtheMassachusettsInstituteofTechnologyin1979,anM.
S.
fromStanfordUniversityin1980,andhisPhDfromtheStanfordUniver-sityComputerScienceDepartmentin1986.
Dr.
Mogulhasbeenanac-tiveparticipantintheInternetcom-munity,andistheauthororco-authorofseveralInternetStandards;mostrecently,hehascontributedex-tensivelytotheHTTP=1.
1speci-cation.
Since1986,hehasbeenaresearcherattheCompaq(formerlyDigital)WesternResearchLaboratory,workingonnet-workandoperatingsystemsissuesforhigh-performancecom-putersystems,andonimprovingperformanceoftheInternetandtheWorldWideWeb.
HeisamemberofACM,SigmaXi,andCPSR,andwasProgramCommitteeChairfortheWinter1994USENIXTechnicalConference,andfortheIEEETCOSSixthWorkshoponHotTopicsinOperatingSystems.
Addressforcorrespondence:CompaqComputerCorporationWesternResearchLaboratory,250UniversityAvenue,PaloAlto,California,94301(mogul@pa.
dec.
com)DavidM.
KristoliscurrentlyaMemberofTechnicalStaffatBellLaboratories,LucentTechnologies,intheInformationSciencesRe-searchCenter,wherehenowdoesresearchinsecurityandelectroniccommerceontheInternet.
Previ-ously,heworkedonformalspec-icationsforcommunicationspro-tocols.
Forsixyearshewasre-sponsiblefortheCcompilersforUNIX(r)SystemVasamemberofBellLabs'sUnixsupportorganization,andhewastheprincipaldeveloperoftheSystemVANSICcompiler.
HejoinedBellLaboratoriesin1981.
Earlier,KristolworkedatMassachusettsComputerAssociates,wherehedevelopedamassspectrometrydatasystem,andatGenRad,Inc.
,wherehedevelopedautomatictestequipmentsystems.
HereceivedBAandBSEEdegreesfromtheUniversityofPennsylvania,Philadelphia,andMSandMEdegrees(inAppliedMathematics)fromHarvardUniversity.
Hisemailaddressis:dmk@bell-labs.
com.