block阵列卡

ResearchonmassivedatastorageinvirtualroamingsystembasedonRAID5RANFeipeng,DAIHuayang,XINGWujie,WANGXiang,LiXuesongCollegeofGeoscienceandSurveyingEngineering,ChinaUniversityofMining&Technology,Beijing100083,ChinaKeywords:virtualroamingsystem,RAID5,PDSS,panoramicdataAbstract.
AmassivedatamanagementmethodwasproposedbasedonRAID5againstlimitationsofdataexpansion,andalackofautomaticdatarecoverycapabilityincurrentvirtualroamingsystem:thepanoramicdataisseparatedfromtheroamingsystemasanindependentpanoramicdataservicesystem(PDSS),anddistributedtomultiplenodestoperformanceparallelservicetakingadvantageofdiskparallelismofRAID5.
Inthisway,wecanenhancethedatastoragecapacityandfaulttoleranceoftheserverinalargeextend.
Finally,aprototypevirtualroamingsystemwasdesignedbasedonPDSS.
Theresultsshowedthatthismethodhadagoodapplicationprospect.
IntroductionVirtualroamingtechnologybasedonthePanoramaandcombinedwiththeadvantageofGISisanimportantapproachtorealizethevisualizationofGIS[1]-[2].
Withthecontinuousarealexpansioncoveredbythevirtualroamingandgreaterdetailedpanoramaspatialinformation,thestoragenodesindatacenterunderthemassivedatastorageenvironmentareverylarge,itsefficiencyandfaulttoleranceinfluencetheefficiencyandstabilityofvirtualroamingsystem.
Itisdifficulttomeetthepracticaldemands[3],thoughtherearesomesolutionssuchasdatacompressionortheimprovementofthebroadbandtoreleasethepressureoftheserver.
Thesemethodsarenotthebestwaytosolvetheseproblems,becausetheP2Pstructurehasalargenumberofnodesandcomplexencryptionalgorithm[4]-[5].
RedundantArraysofIndependentDisks(RAID)[6]putsapluralityofdiskstogethertoformaunifiedlogicalstoragedevice,thecommonmethodsofwhichwerediskimaging,stripinganderrorcorrection[7].
Forexample,RAID5scattersdatastoredindifferentdisksarraysforthepurposeofmassivestorageandhightransmissionrate,datasecurityandstoragecost,soithasbeenwidelyused[8]-[9].
Inthispaper,anewmethodwasproposedtosolvethemassivedataprobleminvirtualroamingsystembasedonRAID5.
Datastoragestructureandfault-toleranceMechanismTakingonesectionoftheroadforexample,supposingthatthereareipanoramicdataandnisomorphiccorrespondingstorageunits,thethroughputisk.
Theloadingtimeisdifferunderthesamenetworkduetothedifferentsizeofthepanoramicdata,named12,,.
.
.
nttt,thewholeloadingtimeis12,,.
.
.
nTttt=;theaveragenumberoftherequestingaccessisλ,theoretically,therejectionrateis:1(1)rrNTkTrλ1)Nrrepresentsthenumberofdatarejections,Trthetotalaccesstime.
Therefore,toreducetherejectionrateρ,weneedtoreduceT.
Ifthepanoramicdataarestoredonndifferentstorageunits,theservicetimeofeachunitswouldreducetoT/n,theacceptancesuccessfulrequestratewouldrisefromk/(1+λ*T)uptok/(1+λ*T/n),whichmeansthattheactualparallelismoftheapplicationserverisnearlyntimescomparedtosingledisk.
Hence,inordertoimprovedataservicesperformance,aserverstructurewasadoptedwhichissimilartothefeaturesofRAID5[6].
AsisshowninFig1,thepanoramicdataisseparatedfromtheroamingsystemanddistributedtomultiplenodestoestablishadataserverconnectedbyhighspeednetworkbetweenservicenodesandcontrolnodes.
Theservicenodesareusedtodynamicallystorepanoramicdata,whilethecontrolnodesusedtoresponsetotherequestfromtheroamingsystemtoassigntheservicenode.
Datanodesstoreandsendthedatatoservicenodesviathehighspeednetworkaccordingtotheinstructionfromtheservicenodes.
Fig.
1StructureofclusteredparallelpanoramicdataservicesystemTheFig.
2showsthepanoramadataunitandtheparityunitblock.
Ineachrow,Prepresentsparitysectioninformation,andthedataineachnodearestoredintheremainingmodules.
ThepanoramadataaredividedintoseveralStripeUnit,andarowdataunitandverificationunitsarecombinedintoonestripe.
Thelocalpanoramadataarestoredindataunit,andtheparityunitwhichisusefulindatarecoveryisusedtostoreinformationonothernodes.
Wetakeadvantageofrandomallocationalgorithmtoassignthepanoramadata,inotherwords,thestorageunitaredepositedtoeachnodethroughapseudorandomfunctiontosolvetheproblemofnodeoverload.
Fig.
2StorageconfigurationforparallelpanoramicdataThedataandthesystemareoftencombinedtogetherintraditionalvirtualroamingsystemwhichwillreducethestabilityandfaulttolerance,thatistosaythewholesystemwillbeparalyzedoncefatalerrorordiskdamageoccursonthelocalserver.
EVENODDcode[10]isadualfault-tolerancecodingalgorithmwhichiscommonlyusedinarraystoragesystem.
Iftheerroroccursonanytwodisks,theredundantinformationoftwodisksinthesystemwillbeincreasedtorecoverthelostdata.
EVENODDcodeisbasedonthearrayof(p-1)*(p+2),andthefirstPcolumnsisusedtostorerawdata,andthelasttwocolumnswhichisthecheckingcolumnisusedtostoretheredundancycheckingdata.
di,j(0≤i≤p-2,0≤j≤p+1)representsthelineidataofdiskj,ThecolumnPiscalledforrow-bestcheckingcolumn.
Thecheckingblockdi,pwasgainedbyexclusiveORoperationwithalltheoriginaldatainthelinei;Thep+1columnisdiagonal-bestcheckinglist,wecangetthecheckingblockthroughtheexclusiveorbetweentheregulatorsandthecorrespondingrawdatablocks.
Thestructuralformulaisasfollows:1,,0pijijjdd==(2))(,101,jjipjpipdsd>Similarly,therow-bestcheckingcolumnisusedforrecoveringthedataindamageddisk.
Iferroroccurredoncheckingdisk,alldataindisksshouldbereadtorecoveritaccordingtotheformula(2)-(4).
ThevirtualroamingsystembasedonPDSSThelogicalstructuredesign.
AsisshowninFig3,thesystemconsistsofimageprocessingsubsystem,parallelledstoragesubsystemandvirtualroamingsubsystem.
ThevirtualroamingsystemconsistsofIFRAMmoduleandGISmodulewhichisresponsibleforspatialinformationdisplay,query,analysisandoutput.
Thispaperusethemethodinreference[11],whichtransmitsthepanoramicdatatovirtualroamingsystemasthesourceofIFRAM,meanwhilethepanoramicdataservicesystemwillbelinkedtogetherwiththevirtualroamingsystem.
Thispaperonlydiscussthestoragemethodofpanoramicdata,sothestoragemethodaboutgeographicdatawillnotbediscussed.
Fig.
3FrameworkofthesystemfunctionsDesignofSystemFunctionModule.
Fromthepractical,easyoperabilityandotherprinciples,thesystemisdividedintoninemodulesincludingdocumentmanagement,mapoperation,layersettings,tools,geographicanalysis,datamanagement,imageediting,visualfieldlabelsandoutput,asshowninFig4.
Thesystemwillloadthecorrespondingpanoramicviewwhenauserchoosethestartpointandendpointalongtheroadintheelectronicroamingmap,atthesametime,panoramicviewcanbesetindependentlyintheimportantregion.
Thesysteminterfaceincludesdisplaywindow,electronicnavigationmapandviewcontroltoolbar.
Fig.
4FrameworkofthesystemfunctionsSystemperformanceanalysisTheperformanceofPDSShasadirecteffectonwholeroamingsystem.
Wearetoevaluatethesystemperformanceseparatelyfromthroughputrate,reliabilityandresponsetime.
WeSelected80harddiskswiththecapacityis80GB(theharddrivespeedis10000RPM,channelseekingtimeVirtualSpaceSystemGISbasicfunctionsSpatialDatabaseManagementSpatialinformationqueryDataOutputIFRAMEComponentsImageprocessingsubsystemRealdataservicesystemURLURLImageacquisitionImageprocessingGeneratingpanoramaDataconversionFileMapLayersToolsGeographiclDataVisionlblOutputTheimage-basedvirtualspacesystemLoadCloseSaveSaveasPrintExitZoominZoomoutPanFullViewPointlayerrenderingLinelayerrenderingArealayerrenderingRasterlayerrenderingVisionadjustmentScenerotationWalkingsimulationDirectionalradarPauseShortestpathanalysisBufferanalysisTimingAnalysisAddrealdataDeleterealdataAddimagearrayDeleteimagearrayStationsHospitalBusStationHotelShoppingFitnessExportingSWFExportImageExportingPDFMapOutputPrintis5ms,therateofdatatransmissionis200MB/sec),SCSI/FCinterface,MTBF1200000h,and16paritygroups.
Datathroughput.
Throughputrepresentsthespeedofdatatransmissioninstoragesystem,whichusedtobeexpressedintwoways:I/Orateanddatatransferrate.
ThispaperadoptedI/Orate,becauseeachvolumeofrequestissmallinvirtualroamingsystem.
450networkservicesstartedoneachnodeinclientterminalatonetime.
Thereisasimulationtestinwhichdataisrequestedbyseveralusersatonetime.
Experimentsshowthattheserverapplicationcaneasilyhandle10000datarequestandsupport4200concurrentrequests,whichbrokenthroughthelimitsoftraditionalstoragesystem.
Theresponsetimeofdataacquisition.
Theresponsetimereferstothetotaltimeofobtainingthedata.
Usersandreconstructionprocesshaveahigherpriorityincontentionforthediskbandwidth,onthecontrary,thewriteoperationwillhavetowaitfortheendofthereconstructionprocess.
Theresponsetimeisrelativelyfaster.
SimultaneouslytheoperationofdatareconstructionhasalimitonthenumberofreconstructedStripsubmittedbydisks,whichcanensurethattherecoveryworkflowcanevenlydistributedamongthedisks.
Theworkingstateofthesystemisrelativelybalanced.
Reliabilityanalysis.
Reliabilityisoneofthemostimportantindexes[12]totheperformanceofthesystem.
TheindexesofreliabilityconsistofMTTF(averagetimeoffailure),MTTR(meantimetorepair)andMTBF(meantimebetweenfailures),amongthem:MTBFMTTFMTTR=+(5)Foreachdisk,MTBF=1,200,000h,MTTR=1h.
FromMTTR<9999928.
Thatis,theMTTFhasincreasedto1000timesoftraditionalcopyingbackup.
Thefailuretimeperyearis3.
8min,whichismeettherequirements.
ConclusionsTakingadvantageoftheRAID5suchaseasinessinstorageextension,storageefficiency,cheapness,andautomaticdatarecovery,asolutionsofthemassivedatastorageandfault-toleranceinvirtualroamingsystemmethodwaspresented.
Thepanoramicdatawasseparatedfromtheroamingsystemanddistributedtomultiplenodes.
Inthisway,theexpansioncapabilityofthevirtualroamingspatialwouldbeenhancedinlargeextend.
Inaddition,thesystemhadautomaticdatarecoverycapability.
Finally,thePDSSthroughput,reliabilityandresponsetimewastested,theresultsshowedthatthismethodprovidedanewwayofsolvingthedatalimitationofvirtualroamingtechnology.
References[1]JSurong,WSong,FGang,TechnologyofPanoramicViewBasedonImages,ComputerApplications.
16(2002)85-87.
[2]DEREKBRADLEY,ALANBRUNTON,MARKFIALA,GERHARDROTH.
Image-basedNavigationinRealEnvironmentsUsingPanoramas[C],IEEEInternationalWorkshoponHapticAudioVisualEnvironmentsandtheirApplications,Ontario,Canada,2005.
[3]ZHUB,LIK,PATTERSONH.
AvoidingtheDiskBottleneckintheDataDomainReduplicationFileSystem[C].
Proceedingsofthe6thUSENIXConferenceonFileandStorageTechnologies(FAST'08),SanJose,CA,USA,2008.
[4]JibfTian,YaFeiDai.
studyondurablepeer-to-peerstoragetechniques.
[J]Journalofsoftware2007,18(6):1379-1399.
[5]E.
Pinheiro,W.
D.
Weber,L.
A.
Barroso.
Failuretrendsinalargediskdrivepopulation[C].
Proceedingsofthe5thUSENIXconferenceonFileandStorageTechnologies,SanJose,2007,17-28.
[6]HAYESB.
CloudComputing[J].
CommunicationsoftheACM,2008,51(7):9-11[7]HuanQingDong,ZhanHuaiLi,WeiLin.
RAIDVCR:ANewRAIDArchitectureforToleratingTripleDiskFailures[J].
ChineseJournalofcomputers2006,29(5):792-800[8]WenWuNa,JianKe,XuDongKe,etal,ANetworkRAIDSystemwithBackendCentralizedRedundancyManagement[J],ChineseJournalofcomputers,2011,34(5):912-923[9]SUNGHOONBAEK,KYUHOPARK.
Prefetchingwithadaptivecachecullingforstripeddiskarrays[C].
USENIXAnnualTechnicalConference,USA,2008[10]BlaumM,BradyJ,BruckJ.
EVENODD:anefficientschemefortoleratingdoublediskfailuresinRAIDarchitectures[J].
IEEETransactionsonComputers,1995,44(2):192-202[11]FeipengRan,TaoJiang,HuayangDai,etal,Keytechnologyofdigitalcampusestablishmentbasedonflexframework,JournalofGeo-InformationScience,15(2013)123-127(InChinese).
[12]LiangZhao,KeytechnologyofRedundantArraysofIndependentDisks(RAID)[D].
NationalUniversityofDefenseTechnology.
2002.