portabilitylinuxcp
linuxcp 时间:2021-04-10 阅读:(
)
ArstlookatscalableI/OinLinuxcommandsKenMatney1,ShaneCanon1,andSarpOral1CenterforComputationalSciencesOakRidgeNationalLaboratoryOakRidge,TN,37831Abstract.
Datacreatedfromandusedbyterascaleandpetascaleapplicationscontinuestoincrease,butourabilitytohandleandmanagetheselesisstilllimitedbythecapabilitiesofthestandardserializedLinuxcommandset.
ThispaperintroducestheCenterforComputationalSciences(NCCS)atOakRidgeNationalLaboratory(ORNL)eortstowardsprovidingparallelizedandmoreecientversionsofthecommonlyusedLinuxcommands.
Thedesignandimplementationdetailsaswellasperformanceanalysisofanin-housedevelopeddistributedparallelizedversionofthecptool,spdcpispresented.
Testsshowthatourspdcputilitycanachieve73timesmoreperformancethanitsserializedcounterpart.
Inaddition,weintroducecurrentworktoextendthisapproachtoothertools.
1IntroductionUsersofHPCsystemswithparallellesystemsstillrelyonlegacyserialtoolstoperformmanyday-to-dayoperations.
ParallellesystemssuchasLustreandGPFSaretodaycapableofdeliveringhundredsofGigabytespersecond(GB/s)inaggregatebandwidth,butstandardserial-basedLinuxutilitiescannotharnessthiscapability.
Forexample,makingabackupcopyofcheckpointles,compressingoutput,orcreatingatarleofresultstypicallyiscarriedoutwithstandardLinuxtools.
Consequently,usersarelimitedtotheperformancethatcanbesustainedbyasinglenodeforthesetasks.
Thus,theuserisnotabletotakeadvantageoftheextensivecapabilitiesoftheparallellesystem.
TheCenterforComputationalSciencesatOakRidgeNationalLaboratoryhasbegunworkingontoolstoaddressthisissues.
Inthispaperwewilldescribetheapproachusedindevelopingthesetoolsandpresentsomeearlyperformanceresults.
Wewillalsodiscussworkinprogressandfutureplans.
2MotivationTheNationalCenterforComputationalSciences(NCCS)atOakRidgeNationalLaboratoryoperatesanumberofthemostpowerfulcomputersystemsusedforopenresearch[1][2].
Theagshipsystem,Jaguar,isaCrayXT4withover20,000coresand40TBofmemory.
Itisconguredwithaparallellesystemwithnearly1PBofdiskcapacityandover40GB/soflesystembandwidth.
ThesystemusestheLustrelesystem[3].
TheLustrelesystemaggregatesdistributedstorageunitsintoonelogicallesystem.
Filesarestripedtransparentlybythelesystemacrossmultiplestoragetargetstoaggregatebothcapacityandbandwidth.
Asaresult,userscanachievehighthroughputtostorageforcriticalI/Ooperationssuchaswritingorreadingacheckpointle.
ApplicationssuchastheGryokineticTokamakCode(GTC)havedemonstratedover10GB/sofaggregatebandwidth.
However,manydaytodayoperationsfailtoachieveevenasmallfractionofthiscapabilitybecausetheunderlyingutilitiessuchascp,bzip2,andtarmustbeconnedtoasinglenode.
Afullystripedle(asinglelestripedacrossallstoragetargets)canbewrittenatover20GB/sonaJaguarlesystem.
However,usingcptocopythislebetweentwolocalLustrelesystemsmightonlysustain200MB/s.
Asaresult,whileitmighthavetakenaround50secondstocreatea1TBcheckpointle,itwouldtakemorethan80minutestomakeacopyofthele.
Theuserwouldlikelyencountersimilarproblemswhencompressinganduncompressingles,creatingatarle,orotheroperationsthatrelyonserial-basedtools.
Fromdiscussionswithourusers,ithasbecomeevidentthatthesebottlenecksinday-to-dayoperationsarethesourceofsomeveryrealbarrierstoproductivityandthattherewasaclearandgrowingneedforparallelversionsofthesecommontools.
Furthermore,ifageneralizedframeworkcouldbecreatedforparallelizingmanyofthesecommontasks,itcouldbeextendedtootherusecases.
Fortunately,manyofthesetoolslendthemselvestoparallelizationwithveryclearwaystodecomposethetheinputdomain.
Wechosetofocusonthoseutilitiesthatwouldquicklyprovidethemostbenettoourusercommunity.
23ApproachTherearesomelimitingfactorsinparallelizingLinuxcommands.
First,thesourcedatamustberandomlyaccessible.
Datafromacheckpointleinalesystemisanexample,whiledatafromasocketorpipeisnot.
Second,thedatasetmustresideonmultipleindependentphysicaldevices.
SinceperformanceimprovementisbasedonparallelI/O,accessingmultipleindependentphysicaldevicesconcurrentlyincreasestheachievableaggregatebandwidth.
Therearetwotypesofparallelizationthatcanbeexploited.
First,thereistheparallelismassociatedwithprocessingmultiplelessimultaneously.
Second,thereistheparallelismassociatedwithusingmultipleprocessorstomapcooperativelythedataofasinglele.
Obviously,thegainfromtheuseofthelatterisdependentonhowwellthelehasbeendistributedacrossmultipleserversandiftheworkcanbeeasilydecomposed.
Anothercriticalfactortoperformanceisthesizeofthedatabuersthatareemployed.
Likemostlesystems,parallellesystemspreferlargebuers.
Forexample,Lustrelesystemachievesbestperformancewith1MBbuers.
Parallellesystemsaretypicallymoresensitivetobuersizessincetheselesystemsrelyonnetworkstotransportdatafromthestorageserverstotheclients.
Furthermore,byterangelockingistypicallyusedtoinsureconsistency.
Largerbuersrequirelessoverheadinmanagingtheselocks,resultinginbetterperformance.
Sincethedetailsofhowtodecomposetheworkdependsonthespeciccommandtargeted,eachcommandhastobeexaminedindividually.
However,thebasisofalgorithmsforperformingI/Oinparallelremainsthesame.
Inaddition,amethodforcommunicatingbetweenthevariousparticipatingprocessorsmustbeestablished.
Whilesystemspeciclow-levelprotocolssuchasPortalsonaCrayXTorVerbsonanInniBandclustermightprovidethebestperformance,theylackportability.
Therefore,MPIisusedtoensureportabilitywhilesacricingsomedegreeofperformance.
Ourparallelizedutilitiescaneasilybeportedandcompiledformostparallelsystems.
WhileaLustrelesystemwasusedinthedevelopmentandtestingoftheinitialimplementation,thesetechniquescanbeappliedtootherparallellesystems.
Incertaincases,Lustre-speciccallstoquerythelayoutofthedataareusedtoimproveeciency.
However,goodperformanceandeciencycanstillbeachievedwithouttheseLustrespeciccalls.
LustreisaPOSIXcompliant,object-basedlesystemcomposedofthreecomponents:MetaDataServerAsingleMetaDataServer(MDS)perlesystemthatstoresandmanagesLustrelemetadata,suchaslenames,directories,permissions,stripingpattern,andlelayout.
ObjectStorageTargetOneormoreObjectStorageTargets(OSTs)areblockdevicesthatactuallystoretheledata.
OSTsaremanagedbytheObjectStorageServers(OSSs).
AtanygivencongurationtherecanbeoneormoreOSTscontrolledbyagivenOSS.
ClientClient(s)accessandusethedata.
LustreprovidesallclientswithstandardPOSIXsemanticsandconcurrentreadandwriteaccesstothelesinthelesystem.
Currently,Lustreusesanenhanchedversionofext3lesystemonMDSandOSTstostoreLustreledata.
LustreachieveshighreadandwriteperformancebydistributingtheledataovermultipleOSTs.
Thisisknownasstriping.
ThenumberofOSTsthataleisstripedacrossisknownasstripecount.
Withstriping,themaximumlesizeisnotlimitedbythesizeofasingleblockdevice,andtheaggregateI/ObandwidthscaleswiththenumberofOSSs.
AmoredetaileddescriptionofLustrelesystemisbeyondthescopeofthisdocument.
Interestedreadersareencouragedtoread[3].
TheLinuxcputilitywasselectedasthersttoolforparallelization,asitisacommonlyusedfunction,andthedecompositionissimplesincethemappingofinputdatatooutputdataisdirect.
Consequently,therearealmostnodependenciesbetweentheindividualthreadscarryingoutthecopy.
Theparallelversionofcpistermedspdcpforstreamingparalleldistributedcp.
Currently,spdcponlyworksonLustrelesystem,butourfutureplansinvolveextendingittootherlesystems,suchasGPFS.
Weareintheprocessofpubliclyreleasingthespdcpsourcecodeunderanopensourcelicense.
34PrototypeforaParallelDistributedCopyInpreparingtheprototype,therearetwopossiblewaysinwhichtoproceed.
TherstistotakethesourceforGNUcpandmodifyit.
Thesecondistowritethefunctionfromscratch.
Itisunlikelythatapatchtoreworkcpcouldmakeitintothemainstreamgiventheamountofchangesthatareneededtoparallelizeit.
Thereforewechosetoimplementanewcopycommandstartingfromscratch.
However,wetriedtopreservemanyofthecommand-lineoptionsandgeneralbehaviourofcp.
Theoveralldesignconsistsofseveralcomponents.
AdiagramofthecomponentsisshowninFig.
1.
Thebasecomponentisthe"launchprocess"whichinvokestheMPI-basedcomponents.
InadditiontolaunchingtheMPIjob,italsoperformsanumberofotheroperations,asdescribedbelow.
The"rank0process"intheMPIjobisdesignatedasamaster.
Itisresponsibleformanagingthework.
Anumberofslaveprocessesareresponsibleforcopyingtheledatafromsourcetotarget.
Howthisworkisdistributedacrosstheslavenodesisdescribedbelow.
Thereareanumberofdesignconsiderationstobemade.
First,theprototypeneedstobeawareoftheparallelcharacteristicsofsourcele(s).
Itneedstobeabletoacquiretheseattributesforsourcele(s)andsettheseontargetle(s).
Next,itneedstobeawareoftheavailableresources.
Thatistosay,iftheLinuxcommandisnotrunwithinthecontextofabatchjob,itneedstospawnabatchjobandrequestappropriateresources.
Anotherdesignchoicewastodecidehowmeta-dataoperationswouldbedecomposed.
Currently,LustreemploysasingleMetadataServer(MDS)foralesystem.
Consequently,havingmultipleclientsinteractwiththeMDSmaynotimproveperformanceandmayevenreduceit.
Therefore,theprototypeperformsmanyofthemeta-dataspecictasksinthelaunchprocess.
Forexample,theLinuxcommandthatlaunchestheMPIjob,performsthesearchforsourcele(s),acquiresbothLinuxmeta-dataandLustremeta-dataforthese,andsendsallofthisinformationtoMPImasterviaapipe.
Furthermore,thisprocesscreatesthetargetdirectoryhierarchybeforesendingthelistoflestotheMPIbasedcomponents.
Thisavoidsduplicationofeortandraceconditions,e.
g.
,multipleprocessesrequestingcreationofthesametargetdirectory.
Finally,thelaunchprocesshandlescorrectlysettingtimestampsontargetdirectorieswhenneeded.
Theadvantagetothisstrategymaynotbeobvious.
Sincethelaunchprocesshasalreadyhastraversedthesourcehierarchy,itonlyneedstoretainalistofthedirectoriesandtheirmeta-data.
ThelaunchprocessmustallowtheMPIjobtocompletesothatitcanensureanyupdatestotheaccesstimearenotoverwrittenbyanyoftheslaveprocesses.
Theprototypeemploysavariablestrategyfordecomposingworktodeterminethenumberofclientstoemployincopyingeachle.
Itmakesthisdeterminationbasedonaperformancepredictionmodelofthedataset.
ForsmalllesorleswithonlyasingleLustrestripe,theentireoperationiscarriedoutbyasingleslavenode.
Forlesthataredistributedovermultiplestripes,theworkisdistributedacrossasubsetofprocesses.
Themasterprocesswaitsuntiltheappropriatenumberofslaveprocessesareavailableandthenschedulesthecopyoperationacrossthesubset.
A"teamleader"isselectedwithinthesubset.
TheteamleaderensuresthatthetargetlehasbeencreatedwiththeappropriateLustremeta-dataparameters,suchasthestripecountandstripewidth.
Ifthetypicallemeta-data(modicationdate,etc.
)istobeanexactcopyoftheoriginal,thenalloftheteammembersreporttotheteamleaderthattheyhavecompletedalloftheirI/Orequests.
Otherwise,theteammembersreportdirectlybacktothemasternodefortheirnextassignment.
Likewise,aftertheteammembersreportbacktotheirteamleaderforcompletionnotication,theyawaitfurtherinstructionsfromthemasternode.
Theteamleaderreportstothemasternodetoindicatethatthecopyhascompletedandtheteammembersarearereadyforthenextassignment.
ThetechniquesdescribedaboveallowtheloadonthetargetOSTstobemanaged.
Byinstructingtheprototypecommandtouseonlyaspeciednumberofprocessorsfortheparallelpart,inconjunctionwithspecifyingthebuerwidth,wecanensurethattheidealnumberofclientsareparticipatinginthecopyoperationforagivenle.
ContentioncanstillarisefromothercopythreadshavingstripesthatoverlaponthesameOST.
However,preventingthiswouldincreasethecomplexityandlikelyprovideonlymarginalimprovementsinperformance.
Theprototypeimplementationofspdcpstrivestomimicthestandardcpcommandthatusersarefamiliarwith.
Theintentistocreateadropinreplacementforcpthatuserscaneasilyemployintheirexistingscripts.
4Fig.
1.
Diagramofthecomponentsusedintheparalleldistributedcopy.
Allcomputenodesaccessthelesystem.
ThenumberofteammembersusedforasourceinputledependsonthesourceleLustrestripepattern.
However,someadditionalcommand-lineoptionshavebeenaddedtocontrolaspectsoftheparallelexecutionoftheutility.
Forexample,thereareoptionstocontrolthenumberoftasksandbuersizes.
Furthermore,sinceourenvironmentrequiressubmittingabatchjobtorunaparalleljob,theutilitycantransparentlysubmititselftothebatchqueue.
Consequentlythereareoptionsrelatedtothebatchsubmissionaswell.
AsampleexecutionisshowninFig.
2.
5PerformanceAseriesofperformancemeasurementswerecarriedoutonspdcptool.
Threereferencedatasetswerecreatedinordertomeasuretheperformanceofthespdcptool.
Therstdataset(workload1)consistedof2400les,eachofsize100MB.
Thisisrepresentativeoflestypicallycreatedbyamodelingapplicationwhichlaterareanalyzedorvisualized.
Theseconddataset(workload2)consistedof10les,eachofsize24000MB.
Thisisrepresentativeofacheckpointwhichisdonetoasharedle.
Thethirddataset(workload3)consistedof1200lesofsize100MBand5lesofsize24,000MB.
Thiswasdonetodemonstratetheabilitytoecientlycopyanon-uniformdataset.
TheLinuxcpcommandwasusedtoestablishbaselineperformance.
Then,weevaluatedtheperformanceatvariousscalesinordertounderstandthescalingbehaviorfortheprototype.
Thesemeasurementswereperformedona3500socketCrayXT3systemagainstitslocalLustrelesystem.
TheLustrelesystemconsistedof80OSTsservedby20ObjectStorageServers(OSSs).
Thebackendstoragewasprovidedby10coupletsofDDN8500[10].
ThislesystemhasbeenmeasuredusingtheIOR[11]benchmarktosustainover10GB/sonale-per-processrun.
5spdcp-s16-r/source/directory//target/directory/spdcp-hUsage:spdcp[options]SRCDESTorspdcp[options]SRC.
.
.
DIRECTORYCopyfileSRCtofileDESTorlistoffilesSRC.
.
.
todirectoryDIRECTORY,replicatingLustrestripeinformationwherepossible.
CopyisperformedinparallelbydistributedclientsusingMPImessagepassingforsynchronizationandcontrol.
Whencomputenoderesourcesareaccessibleonlyinbatchmode,commandwillstagejobandretaincontroluntiljobfinishes.
Thefollowingoptionsoffercontrolovercommand:-hPrintthismessage(disablescopy)-VPrintcommandandagentversions(disablescopy)-dUsedummyform(disablescopy,printstargets)-vIncreaseverbositylevel(maximum2)-pPreservemode,ownership,andtimestamps-r,-RCopyrecursively-cReduceOSTcountatdestinationtosourceusage-nDonotoffsetinitialOSTatdestination-b{F}IncreaseI/OrequestsizebyafactorofF-s{M}EmployMparallelclients-A{P}Ifspawningbatchjob,chargeruntoproject,P-w{T}Ifspawningbatchjob,limitwalltimetoTseconds-q{Q}Ifspawningbatchjob,directtobatchqueue,QFig.
2.
Sampleexecutionofspdcp(Top).
Thetotalnumberofclientsrequestedisidentiedbythe-sswitch.
Notethat,thisnumberalsoincludesthe"master(orrank0)node.
"Thespdcphelpmenu(Bottom).
AscanbeseeninFig.
3,spdcpachievesgoodparallelspeedup.
ThedataexhibitacertainamountofvariationbecausetheywereobtainedduringthecourseofnormalproductionoperationoftheCrayXT3.
ItshouldbenotedthatthestockLinuxcputilityachieved324MB/s,126MB/s,and177MB/sforworkload1,workload2,andworkload3,respectively.
Intermsofpeakperformance,ascanbeseeninFig.
3,theworkload2achievesthebestperformancewithspdcp,ataround9300MB/s.
Thisisa73xperformanceincreasecomparedtotheLinuxcputility.
Thepeakperformanceis7300MB/sforworkload1.
Thisis22xspeedupcomparedtotheLinuxcputility.
Forworkload3thepeakisatapproximately9100MB/s;a51xspeedupovertheLinuxcputility.
Also,ascanbeseeninFigure3,thepeakperformanceisobtainedat160to256clients.
However,fromapracticalpointofview,thescalingofperformancelevelsoataround100clients.
ThismakessensegiventhatthenumberofclientsandOSTsareroughlyequivalent.
Consequently,theOSTshavenearlyreachedtheirpeakbandwidth.
Thisisfurtherdemonstratedbythefactthattheaggregatebandwidthis73%to93%ofthepeakbandwidthasmeasuredbyIOR.
6On-goingworkTheparallelimplementationofthecopyutilityisjusttherststepinabroaderinitiativetocreateasuiteofparallelizedtools.
Towardsthisend,wehavestartedtocreateaframeworktogeneralizetheapproachesused61001000100001101001000AggregateBandwidth(MB/s)NumberofClientsWorkload1Workload2Workload3Fig.
3.
spdcpperformanceforclientsupto512.
Theworkload1iscomposedoflargeles,workload2iscomposedofsmallles,andworkload3isamixoflargeandsmallles.
ThestockLinuxcputilityachieved324MB/s,126MB/s,and177MB/sforworkload1,workload2,andworkload3,respectively(notshownonthegure).
inspdcpsothattheycaneasilybeappliedtoothercommonutilities.
Thespdcputilitydoesnotcurrentlyusetheframework,butmaybere-implementedusingtheframeworkinthenearfuture.
Thisframework,whichiscalledspdframe,hasalreadybeenusedforcompressionanddecompressionofbzip2les[12].
Thispresentsslightlymoredicultythanthecopytool,asthedecompositionfordecompressionismoredicult.
Preliminarytestsshowthatourbzip2implementationispromisingandunderrightcongurations(e.
g.
64processorswitha20MBle)itcanachieve15timesmoreperformanceforcompressioncomparedtoitsserializedversionon.
Futureworkwillfocusonapplyingtheframeworktotarandothercommonlebasedutilities.
Whilewearefocusingonapplyingtheframeworktocommontools,theframeworklendsitselftootherusesaswell.
Theframeworkprovidesaneasywayforuserstoapplyafunctionovermultiplelesinparallel.
So,forexample,ausercouldeasilyapplytheframeworktoperformaparallelgreponasetofles.
7RelatedworkIncreasingtheperformanceofcommonLinuxutilitiesgatheredsomeattentionfromtheresearchcommunityovertheyears.
WilliamGroppandEwingLusk[4]haverstrealizedthelimitationsoflegacyserialUNIXutilitiesinparallelenvironments.
TheyintroducedseveralparallelversionsofcommonlyusedUNIXutilitieswithparallelrshastheunderlyingparallelsynchronizationandcommunicationmechanism.
Asafollowuptotheirwork,EmilOng,EwingLusk,andWilliamGroppdevelopedtheMPI-basedversionoftheirparallelizedUNIXutilities[5].
However,thereisacleardistinctionbetweenourgoalandtheirs.
ThetargetforGroppandLuskwasincreaseeciencybyexecutingthesamecommandwiththesameargumentlistandparametersinparallelovermultipleindependentnodeswithindependentoperatingsystemsandlesystems.
Inmanyaspects,theyhaveimplementedSIMD-likeversionsofthecommonUNIXtools.
However,ourapproachdepartsfromtheirsasourgoalwastoincreasetheeciencyofasingleexecutionagivenLinuxutilitybyparallelizinganddistributingitsworkloadovermultipleworker/computenodes,allsharingacommonlesystem,butindependentOSes.
JeGilchristandAysegulCuhadar[7]introducedtwoparallelizedversionsofBWT-basedbzip2nblock-sortinglecompressor,namelypbzip2andmpibzip2.
Thepbzip2isathread-parallelversionofbzip2foruse7onsharedmemorymachines.
Itproducescompatiblebutlargerarchivescomparedtotheoriginalbzip2.
Thempibzip2isanMPI-basedparallelimplementationofthebzip2block-sortinglecompressorforclusters.
Thebzip2smpprogramisanotherparallelizedversionofthebzip2compressor[8].
ItisspecicallytargetedforSMPsystems.
Itisverycache-dependantanddoesnotperformwellwithhyperthreadedsystems.
Itissimilartopbzip2innature,butunlikepbzip2,bzip2smpsupportscompressionfromstdin.
ConclusionIncreasingparallelisminlesystemspavethewayforprocessinglargerdatasetsinshortertimes.
However,whilecapabilitiesforgeneratinglargerdatasetsareconstantlyincreasing,ourtoolsforhandlingandmanagingsuchles,stillremainserialandlimitedinperformance.
TheCenterforComputationalSciences(NCCS)atOakRidgeNationalLaboratory(ORNL)hasstartedaninitiativeforprovidinghigh-performance,parallelversionsofcommonlyusedLinuxcommands.
Thecpcommandwasourstartingpoint.
WehavedevelopedandimplementedaMPI-basedbatch-processingcapableparallelversionofthestandardcpcommand.
Testsshowthat,ourversioncanachieve73timesmoreperformanceoveritsstandardserializedcounterpart.
Also,thispaperintroducesoureortstowardsdevelopingaparallelizeddistributedversionofthebzip2command.
Theimplementationfollowsaframework,whichifsuccessful,willbeusedfordevelopingandparallelizingotherLinuxcommands.
AcknowledgmentsTheauthorswouldliketothankthestaandcolleagueswhohavecontributedmaterialtothispaper.
ResearchsponsoredbytheMathematical,Information,andComputationalSciencesDivision,OceofAd-vancedScienticComputingResearch,U.
S.
DepartmentofEnergy,underContractNo.
DE-AC05-00OR22725withUT-Battelle,LLC.
AbouttheAuthorsKenMatneyisaresearcherintheTechnologyIntegrationGroupwhichispartoftheNationalCenterforCom-putationalSciencesatOakRidgeNationalLab.
HecanbereachedbyE-Mail:matneykdsr@ornl.
gov.
ShaneCanonistheGroupLeaderforTechnologyIntegrationTeam.
HecanbereachedbyE-Mail:canonrs@ornl.
gov.
SarpOralisaresearcherintheTechnologyIntegrationGroupwhichispartoftheNationalCenterforCom-putationalSciencesatOakRidgeNationalLab.
HecanbereachedbyE-Mail:oralhs@ornl.
gov.
References1.
NationalCenterforComputationalSciences.
WebPagehttp://nccs.
gov.
2.
Top500Supercomputersites-November2007list.
WebPagehttp://www.
top500.
org/list/2007/11.
3.
ClusterFileSystems,Inc.
Lustremanual.
Webpage.
http://www.
lustre.
org/manual.
html.
4.
WilliamGroppandEwingL.
Lusk.
ScalableUnixtoolsonparallelprocessorsInProceedingsoftheScalableHigh-PerformanceComputingConference,pp.
56-62,1994.
5.
EmilOng,EwingL.
Lusk,andWilliamGropp.
ScalableUnixCommandsforParallelProcessors:AHigh-PerformanceImplementationInProceedingsofthe8thEuropeanPVM/MPIUsers'GroupMeetingonRecentAdvancesinParallelVirtualMachineandMessagePassing,pp.
410-418,2001.
6.
M.
BurrowsandD.
J.
Wheeler.
Ablock-sortinglosslessdatacompressionalgorithmTechnicalReport124,DigitalSystemsResearchCenter,1994.
7.
JeGilchristandAysegulCuhadar.
ParallelLosslessDataCompressionBasedontheBurrows-WheelerTransformIn21stInternationalConferenceonAdvancedNetworkingandApplications(AINA'07),pp.
877-884,2007.
8.
WebPagehttp://bzip2smp.
sourceforge.
net/9.
R.
S.
CanonandH.
SarpOral.
ACenter-wideFileSystemusingLustre.
InCUGProceedings,2006.
10.
DataDirectNetworks.
WebPagehttp://datadirectnetworks.
com/11.
HedgesetalParallellesystemtestingforthelunaticfringe:thecareandfeedingofrestlessI/OpowerusersInIEEEMassStorageSystemsandTechnologiesProceedings,200512.
JulianSeward.
Thebzip2andlibbzip2ocialhomepage.
WebPagehttp://sources.
redhat.
com/bzip2
无忧云怎么样?无忧云是一家成立于2017年的老牌商家旗下的服务器销售品牌,现由深圳市云上无忧网络科技有限公司运营,是正规持证IDC/ISP/IRCS商家,主要销售国内、中国香港、国外服务器产品,线路有腾讯云国外线路、自营香港CN2线路等,都是中国大陆直连线路,非常适合免备案建站业务需求和各种负载较高的项目,同时国内服务器也有多个BGP以及高防节点,目前商家开启了夏日清凉补贴活动,商家的机器还是非常...
云基yunbase怎么样?云基成立于2020年,目前主要提供高防海内外独立服务器,欢迎各类追求稳定和高防优质线路的用户。业务可选:洛杉矶CN2-GIA+高防(默认500G高防)、洛杉矶CN2-GIA(默认带50Gbps防御)、香港CN2-GIA高防(双向CN2GIA专线,突发带宽支持,15G-20G DDoS防御,无视CC)。目前,美国洛杉矶CN2-GIA高防独立服务器,8核16G,最高500G ...
虎跃科技怎么样?虎跃科技(虎跃云)是一家成立于2017年的国内专业服务商,专业主营云服务器和独立服务器(物理机)高防机房有着高端华为T级清洗能力,目前产品地区有:山东,江苏,浙江等多地区云服务器和独立服务器,今天虎跃云给大家带来了优惠活动,为了更好的促销,枣庄高防BGP服务器最高配置16核32G仅需550元/月,有需要的小伙伴可以来看看哦!产品可以支持24H无条件退款(活动产品退款请以活动规则为准...
linuxcp为你推荐
哈利波特罗恩升级当爸哈利波特的爸爸妈妈身份蓝色骨头手机宠物的骨头分别代表几级?巨星prince去世Whitney Houston因什么去世的?甲骨文不满赔偿工作不满半年被辞退,请问赔偿金是怎么算的?m88.comm88.com现在的官方网址是哪个啊 ?m88.com分析软件?www.175qq.com最炫的qq分组www.147.qqq.comWWW147EEE.COM这个网站现在改哪个网址了www.xvideos.com请问www.****.com.hk 和www.****.com.cn一样吗?本冈一郎只想问本冈一郎的效果真的和说的一样吗?大概多长时间可以管用呢?用过的进!红玉头冠急求,wow中红玉头冠的第一个任务是什么呀。。
双线服务器租用 mediafire下载 免费cdn加速 12306抢票攻略 镇江联通宽带 刀片式服务器 100m独享 绍兴电信 环聊 web应用服务器 广州服务器托管 双十二促销 电信主机托管 脚本大全 香港博客 magento主机 机柜尺寸 alexa搜 asp简介 asp.net虚拟主机 更多