occursubuntu11.10

ExploitingNeigborhoodSimilarityforVirtualMachineMigrationoverWide-AreaNetworkHsu-FangLai,Yu-SungWu*,andYu-JuiChengDepartmentofComputerScienceNationalChiaoTungUniversity,Taiwanblackxwhite@gmail.
com,hankwu@g2.
nctu.
edu.
tw,chengyj@cs.
nctu.
edu.
twAbstract—Conventionalvirtualmachine(VM)migrationfocusesontransferringaVM'smemoryandCPUstatesacrosshostmachines.
TheVM'sdiskimagehastoremainaccessibletoboththesourceanddestinationhostmachinesthroughsharedstorageduringthemigration.
Asaresult,conventionalvirtualmachinemigrationislimitedtohostmachinesonthesamelocalareanetwork(LAN)sincesharingstorageacrosswide-areanetwork(WAN)isinefficient.
Asdatacentersarebeingconstructedaroundtheglobe,weenvisiontheneedforVMmigrationacrossdatacenterboundaries.
WethusproposeasystemaimingtoachieveefficientVMmigrationoverwideareanetwork.
ThesystemexploitssimilarityinthestoragedataofneighboringVMsbyfirstindexingtheVMstorageimagesandthenusingtheindextolocatestoragedatablocksfromneighboringVMs,asopposedtopullingalldatafromtheremotesourceVMacrossWAN.
Theexperimentresultshowsthatthesystemcanachieveanaverage66%reductionintheamountofdatatransmissionandanaverage59%reductioninthetotalmigrationtime.
Keywords—Livemigration,Storagede-duplication,Wide-areanetwork,Virtualization,DatacenterI.
INTRODUCTIONVirtualizationhasbeenwidelyadoptedinrecentdatacenterconstructionstoallowformultiplevirtualmachines(VMs)runningonasinglehostmachineandachievecost-effectiveresourceutilization.
Virtualizationalsoenablesdynamicresourceallocationthroughthemigrationofvirtualmachinesamonghostmachines.
Forinstance,VMswithheavyworkloadcanbespreadontodifferenthostmachinesforloadshedding.
Conversely,VMswithlightworkloadcanbeaggregatedtogetherontoafewhostmachinessotheotherhostmachinescanbepoweredoffforenergysaving.
VMmigrationalsoprovidesnewpossibilitiesforfaulttoleranceinthesensethatVMscanbemigratedawayfromafailinghostmachine.
ConventionalvirtualmachinemigrationtransfersthememoryandCPUstatesofaVMfromasourcehostmachinetoadestinationhostmachine.
TheVM'sdiskstoragehastobeplacedonasharedstorageserver,whichisattachedtobothofthehostmachines.
Asthesharedstoragecannotbeefficientlyimplementedacrosswide-area-network(WAN),conventionalvirtualmachinemigrationisprimarilyusedwithinlocal-areanetwork(LAN)environment.
ThewidespreadconstructionofdatacentersaroundtheglobehasprovidedanewopportunityforfurtherimprovingresourceutilizationandfaulttoleranceincloudcomputingthroughVMmigration.
Forinstance,differentgeographicregionshavedifferenttimesforpeakworkloads.
Wecanthusimproveresourceutilizationthroughloadbalancingacrossdatacentersindifferentgeographicregions.
And,forthepurposeoffaulttolerance,theabilityofVMmigrationacrossgeographicregionscanimproveresilienceagainstgeographicregionrelatedfailures.
Forinstance,ifaregionisexpectingahurricane,wecanmigratetheVMsawayfromtheregiontoadatacenterthatisnotonthepathofthehurricane.
However,itisnotfeasibletoapplyexistingVMmigrationmechanismsinawide-areanetworkenvironment.
AsharedstorageacrossWANwouldberatherinefficientduetothelimitedbandwidthandthelongtransmissionlatencyofWAN.
Withoutasharedstorage,VMmigrationintheWANenvironmentwillhavetocopytheVMstorageimageovertothedestinationseverinadditiontocopyingtheCPUandthememorystatesoftheVM.
Achallengeisthatinmostcases,thesizeofaVMstorageimageistoolargetobeefficientlytransmittedacrosstheWAN.
Forinstance,AsmallinstanceofAmazonEC2VMisequippedwitha160GBstorageimage[1].
ItwilltakeconsiderableamountoftimetomigratejustonesingleVMacrosstheWAN,andtheapproachiscertainlynotscalableformigratingalargenumberofVMsaroundthesametime.
Inthiswork,weproposeanovelapproachforVMmigrationinWANenvironment.
Theapproachisbasedontheinsightthatalarge-scaledatacenterisadefactowarehouseofdata.
ItislikelythatsomeofdatainaVMstoragetobemigratedmaybepresentinthedestinationdatacenter.
Therearemanyreasonstothisphenomenonbeyondpurecoincidence.
Forexample,mostVMsusestocksystemsoftwaresuchasstandardLinuxdistributionsorWindows.
TheapplicationsoftwarerunningontheVMsalsopossesstypicalcompositions.
Somemayberunningadatabaseserver,somemayberunningawebserver,andetc.
Asidefromthesoftware,theapplicationdataarelikelytohavesimilarityaswell.
Forinstance,someofthedatamaybecollectedfromacommonsource,oranearlierversionoftheVMmighthavehadbeenmigratedtothedatacenter.
Basedontheinsight,theproposedapproachemploysanindexingmechanismtoidentifydatasimilarityintheVMstorageimageskeptonastorageserver.
Duringmigration,theportionsofVMstoragedatathatarefoundintheindexwillbepulledlocallyfromtheneighboringVMinsteadofbeingpulledfromthesourceVMacrossWAN.
TheapproachreducesasignificantamountofnetworkdatatransmissionandmakesitfeasibletomigrateaVMacrossWAN.
AprototypesystemisbuiltonaLinuxhostbasedon20137thInternationalConferenceonSoftwareSecurityandReliability978-0-7695-5021-3/13$26.
002013IEEEDOI10.
1109/SERE.
2013.
2114920137thInternationalConferenceonSoftwareSecurityandReliability978-0-7695-5021-3/13$26.
002013IEEEDOI10.
1109/SERE.
2013.
21149Xenhypervisor[2].
TheprototypeusesaniSCSI-based[3]storageserver.
Therestoftheworkisorganizedasfollows.
SectionIIgivesabriefintroductionofconventionalvirtualmachinemigrationandasurveyofrelatedwork.
SectionIVdescribestheproposedapproachforVMmigrationinWANenvironment.
SectionVdescribestheprototypeimplementation.
SectionVIpresentstheexperimentresults.
SectionVIIconcludesthisworkwithdiscussiononpotentialfuturework.
II.
BACKGROUNDTheconceptofmigrationcanbetracedbacktoclustercomputingsystems,whereprocessesonabusyservercanbemovedtoalessbusyserverforloadbalancing[4].
However,migratingaprocessbetweenserversiscomplicatedbytheinter-dependenciesbetweentheprocessandtheunderlyingOSkernelstates.
Ingeneral,themigrationisnon-transparenttotheupperlayerapplication,and,asaresult,processmigrationonlyfindslimiteduseinreal-worldsystems.
Platformvirtualization(i.
e.
theuseofvirtualmachines)enablesthemigrationofafullsystemstackencapsulatedinavirtualmachine(VM)betweenhostmachines.
Thisiscommonlyreferredtoasavirtualmachinemigration[5].
ConventionalVMmigrationisdesignedtooperateinLANenvironment,whereashareddiskstorageisassumedtobeattachedtobothhostmachinesinvolvedinamigration.
Thereisnoneedtomigratethestoragedata,soaVMmigrationtypicallybeginswithcopyingtheCPUandthememorystatesofaVMrunningonasourcehostmachinetoanewVMonadestinationhostmachine.
Atasuitabletimepoint,thesourceVMwillbesuspended,andthenewVMwilltakeovertheexecutionandstartrunning.
ThetimepointfortheexecutiontransferdifferentiatestwoapproachestoVMmigration:pre-copyvs.
post-copy.
Underthepre-copyapproach,theexecutiontransferisinitiatedafteralargeportionoftheVMstateshavebeencopiedtothedestinationhostmachine.
TheoriginalVMwillkeeprunningonthesourcehostmachineduringthecopyingofVMstates,sosomeofthestatesthathadbeencopiedcanbecomedirty(updatedbytherunningVM)andwillhavetobecopiedagain.
Ifthegenerationofdirtystatesistoofrequent,therewillbealotofre-copying.
Whenthisoccurs,theoriginalVMwillbesuspendedtopreventthegenerationofdirtystates.
ThemigrationprocesswillthencompletethetransferofVMstates,andstartthenewVM.
ThetimeperiodduringwhichneitherVMisrunningisreferredasthemigrationdowntime.
Underthepost-copyapproach,thetransferofexecutiontakesplacerightaftertheCPUstates(andaminimumamountofmemorystates)aretransferredtothedestinationhostmachine.
ThenewVMonthehostmachinewillbeginexecutionwithincompletememorystates.
ThememorystateswillbecopiedondemandfromtheoriginalVM,whichhadbeensuspendedatthemomentoftheexecutiontransfer.
Bothapproacheshavetheirprosandcons.
post-copytendstohaveashortermigrationdowntimethanpre-copyasthedowntimecorrespondstothecopyoftheCPUandaminimumamountofthememorystatesoftheoriginalMV.
Ontheotherhand,pre-copyhastheadvantagethatthemigrationprocesscanbecancelledandrolledbackatanymomentbeforethetransferofexecution.
Cancellationofmigrationprocessismuchmoredifficultwithpost-copy,asneithertheoriginalVMnorthenewVMisguaranteedtohaveconsistentstatesattimeofacancellation.
Underpost-copyapproach,thenewVMmayrunslowlyuntilthefullstateshavebeencompletelycopiedfromthesource.
Mosthypervisorsadoptpre-copyastheirdefaultmechanismforVMmigration[5-7].
StoragemigrationmovesthestorageimageofaVMfromthesourcestorageservertothedestinationstorageserver.
ItalsoconsistsoftwostagessimilartoconventionalVMmigrationwithasharedstorage,whicharethecopyingofstoragestatesandtheswitchofactivestoragetarget(i.
e.
thetransferofexecution).
Similarly,dependingonthetimepointoftheswitchofstoragetargetwithrespecttothecopyingofstoragestates,thereisalsoadistinctionofpre-copyvs.
post-copystoragemigrationmechanisms.
ThediscussiononVMmigrationaboveiscenteredonmovingaVMfromonehostmachinetoanotherhostmachine.
ConventionalVMmigrationassumesasharedstorageisattachedtobothhostmachinesandmovesonlytheCPUandmemorystatesofaVM.
MovingthestoragestatesofaVMacrosswide-area-networkisverytimeconsumingandnotusedinpractice.
AnotherissuewehavenotaddressedisthatwhenmigratingaproductionVM,theVMmayhaveactivenetworkconnections.
Inordertokeeptheconnectionsfrombeingdisruptedbythemigration,thenetworkroutesusedbytheVMwillhavetodynamicallyreprogrammed.
SolutionssuchasmobileIP[8]andnetworkvirtualizationtechniques[9]canbeusedtodealwiththis.
III.
RELATEDWORKConventionalVMmigrationinvolvesthetransferofVMCPUandmemorystates.
Techniquessuchascompression[10]andde-duplication[11]havebeenusedtoexploitdatasimilarityinthememorystatesofaVMtoreducenetworkdatatransmissionandmigrationtime.
Therehasalsotheattempt[12]thatprioritizesthetransferofcoldmemorypagestofurtherreducethemigrationdowntime.
ForVMstoragemigration,thebarrierimposedbystorageareanetworkwasaddressedinthesystem[13]byK.
Haselhorstetal.
,whereDRBD[14]wasusedtosynchronizethedestinationstoragewiththesourcestorage.
ThexNBDsystem[15]byT.
Hirofuchietal.
supportspost-copystoragemigrationbyextendingtheLinuxnetworkblockdevice(NBD).
VMwarealsosupportsstoragemigrationwiththeirvMotion[16]technology.
WhiletheabovestoragemigrationsystemscanbeusedinWANenvironment,thehighamountofnetworkdatatransmissionasrequiredformigratingthestoragedataisstillnotaddressed.
Thesystem[17]byAkoushetal.
delaysthetransmissionofhotsectorsandachievedaconsiderablebandwidthreduction.
Thesystem[18]byTravostinoetal.
involvestheuseoffiber-opticnetworktospeedupthetransmissionofVMstorageandmemorystates.
TheCloudNet[19]appliesde-duplicationandcompressiononsingleVM'sstorageimage(andmemory)toreducetheamountofdatatransmissionduringWANmigration.
150150OurworkisdistinctfromexistingVMstoragemigrationsystemsinthatwearethefirsttoexploitdataredundancyacrossmultipleVMstorageimages.
Theproposedapproachiscomplementarytoexistingtechniquesthatarecenteredonthede-duplication,compression,andadaptivetransmissionofdatawithinasingleVMstorageimage.
IV.
EXPLOITINGNEIGOBORHOODSIMILARITYFORVIRTUALMACHINEMIGRATIONOVERWIDE-AREANETWORKWeproposeasystemtosupportefficientVMmigrationinWANenvironment.
Ahigh-leveloverviewofthesystemarchitectureisshowninFig.
1.
OursystemtransferstheCPUandmemorystatesofaVMdirectlytothedestinationhostmachine.
Weadoptthepre-copyapproach,sotheVMwillkeptrunningonthesourcehostmachinetillasignificantamountofmemorypageshavebeencopiedovertothedestination.
TherunningVMmaymodifysomeofthememorypagesthathadbeencopiedinapreviousround.
Thesepageswillbemarkedasdirtyandwillneedtobecopiedagain.
Ifthedirtypagegenerationrateistoohigh,theVMwillhavetobepausedtopreventthegenerationofdirtypages.
Remainingdirtymemorypageswillbecopiedtothedestinationhostmachineinalastround.
Finally,theVMwillresumeexecutiononthedestinationhostmachine.
MigratingaVMoverWANrequiresnotonlymovingthememorystatesoftheVMbutalsomovingitsdiskstoragestates.
ThesizeofaVMstorageimageistypicallyintherangeofafewhundredgigabytes,whichmakesmigratingthestorageoverwide-areanetworkaveryexpensiveprocess.
However,wenoticedthatinalarge-scaledatacenter,itisverylikelytofindVMswithsoftwaresetupsimilartotheVMbeingmigrated.
Forinstance,VMsrunningweb-basedservicestypicallyemployoneofthepopularwebstacks(e.
g.
J2EE,.
NET,orLAMP),andmostVMswillusewell-establishedserveroperatingsystems(e.
g.
Linux,BSD,Windows,Solaris,etc.
).
Besides,datakeptontheVMsmayalsopossesssomedegreeofsimilarity.
Forinstance,thedatamayoriginallycomefromthesamesources(e.
g.
musicfilesfromthesamealbums).
ItisalsopossiblethatanoldversionoftheVMbeingmigratedpre-existsinthedestinationdatacenter,possiblyduetoapreviousmigration.
IfwecanidentifythecommondataamongneighboringVMsinadatacenterandusethedatatohelpreconstructthestorageoftheVMbeingmigrated,wemaysignificantlyreducetheamountofdatatransmissionacrossthebandwidth-limitedwide-areanetworkandmakeitfeasibletouseVMmigrationinWANenvironment.
Fig.
1.
OverviewofmigrationoverWANWebuiltanindexmechanismtoindexthedataintheVMstorageimages(Sec.
IV.
A).
ThemigrationprocesswillleveragetheindextoidentifyneighboringVMsatthedestinationdatacenterfromwhichcommondataareavailable.
Wesupportthepre-copyapproachofVMmigrationbyaddingthebitmapmechanismonthestorageserverfortrackingdirtyblocks(Sec.
IV.
C).
Fortheswitchofactivestoragetargets,aspartoftheVMexecutiontransfer,weuseanindirectionlayerimplementedbythecombinationofavirtualblockdeviceandtheblockdeviceremappingmechanism(Sec.
V.
B).
A.
IndexingVirtualMachineDiskStorageStatesDiskstoragesizesarerelativelylargecomparedtomemorysizes.
ItisquitetypicalforaVMstoragetohaveatleastafewhundredgigabytesofdataonit.
AstraightforwardapproachforVMstoragemigrationistotransferallthestorageblocksovertothedestination.
TheapproachwouldprobablyworkwellontheLANbutwilldefinitelynotworkontheWAN.
Onthebandwidth-limitedWAN,themigrationwilltakeaverylongtimetofinish,andthehugeamountofdatatransmissionwilllikelyincursomeheftynetworkusagefees.
ToimprovetheefficiencyofVMstoragemigration,weexploitdatasimilaritiesamongVMdiskstoragesbybuildinganindexofthestorageblocksofneighboringVMs.
Theindexisahashtablethatstoresthehashvalues(fingerprints)ofstorageblocks.
Eachentryintheindexpointstoablockindex.
Ablockindexrecordsthefingerprint(fp)andthesignature(sig)ofthecorrespondingstorageblock.
Thesignatureisasmallpieceofdatasampledfromthestorageblock.
Inourimplementation,astorageblockhasasizeof512bytes,andthesignatureisasampleofthe9ththroughthe16thbytesofthestorageblock.
Thesignatureisusedforresolvinghashvalue(fingerprint)collisions.
Ablockreference(br)thatcanbeusedtolocatethecorrespondingstorageblockonthestorageserverisalsokeptintheblockindex.
Throughtheindexandtheblockindexes,wecanquicklycheckifthestorageblocksoftheVMbeingmigratedexistinneighboringVMsatthedestination.
WecanthentransferthestorageblocksfromneighboringVMsinsteadfromthesourceacrosstheWAN.
Fig.
2givesanexampleoftheindexforadiskstorageoffiveblocks.
Whenweindexadiskstorage,wefirstallocateablockreferencearraywithlengthequaltothenumberofblocksinthediskstorage.
Theblockreferencearraystoresthelinktotheblockreferenceforeachofthestorageblocks.
Storageblockswithidenticaldatacontentwillbelinkedtothesameblockreference.
InFig.
2,wecanseethatblock1,2,and5haveidenticaldata,astheyarealllinkedtothesameblockreference.
Giventhehashvalue(fingerprint)ofastorageblock,wecanchecktheindexatthedestination.
Ifablockindexismatched,wecanfollowtheblockreferencetoretrievetheblockdatafromaneighboringVMatthedestinationandavoidthecostlytransmissionoftheblockdataacrosstheWAN.
VMHostStorageServerVMHostStorageServerVMHostStorageServerDatacenterADatacenterBStorageServerMemoryStorageWAN151151Fig.
2.
ExampleofindexdatastructuresTable1.
Sourcestorageservermigrationprocessflow1//Blockinterval(I,N):=theblocksstartfromtheIthblockandendatthe(I+N-1)thblock2//Bitinterval(I,N):=thebitsstartfromtheIthbitandendatthe(I+N-1)thbit3//D:=theblockdevicethatistobemigrated4//D[I]:=theIthblockofD5//D.
block_nr:=thenumberofblocksofD6//D.
bitmap:=thebitmapofD7//D.
bitmap[I]:=theIthbitofthebitmapofD8//D.
br:=theblockreferencesofD9//D.
br[I]:=theblockreferenceoftheIthblockofD10//D.
br[I].
index:=theblockindexoftheblockreferenceD.
br[I]11//BI:=blockindex12//BI.
fingerprint:=thefingerprintoftheblockindex13//BI.
signature:=thesignatureoftheblockindex14//ZBI:=theblockindexofzeroblock1516//ClearbitmapofblockdeviceD17For(I=0;IdirtythresholdandIUbuntu11.
102.
09%3.
28%4.
97%4.
83%3.
91%40.
23%n/a23.
04%0.
62%1.
02%9.
33%Ubuntu12.
041.
79%3.
05%3.
90%4.
86%5.
35%15.
72%22.
70%n/a0.
54%0.
80%6.
52%Sense41.
75%6.
26%4.
66%4.
40%3.
71%3.
95%4.
26%3.
86%n/a1.
22%8.
23%Better5.
79%14.
20%24.
16%19.
08%13.
43%13.
65%15.
97%11.
33%2.
15%n/a13.
31%Fig.
6.
Storageimagesimilarityfordifferentblocksizes155155thesimilaritybetweenanytwosystemstorageimagesareonaverage10%~20%accordingtoTable3,soanaverage80~90%ofnewdatawillhavetobeaddedtotheindexforeachsystemimage.
Inpractice,ifthesizeoftheindexisaconcern,onecanlimitthesizeoftheindexattheexpenseofsacrificingsimilarityandapotentiallylongermigrationtime.
Fig.
7.
IndexingtimeforVMstorageimagesFig.
8.
IndexsizewithrespecttotheadditionofstorageimagesFig.
9.
MigrationtimewithandwithoutindexFig.
10.
AmountofdatasentbythesourcestorageserverD.
MigrationtimeandamountofdatatransmissionInthisexperiment,welookatthemigrationtimeandtheamountofdatatransmission.
Theresultsarecomparedagainstabaselinesystem,whichtransmitsallthestorageblocksdirectlyfromthesourceservertothedestinationserver.
WeusethetwoproductionsystemsSenseandBetterfortheexperiment.
WesetuptwoiSCSIstorageserversandusea100MbpsnetworktoemulatetheWAN.
Theindexispopulatedwithtwofreshlyinstalledsystems,whichrunthesameoperatingsystemsasSenseandBetter.
Thefreshlyinstalledsystemshaveneithertheapplicationprograms(i.
e.
thewebserverandtheonlinejudgesystem)northeapplicationdataasSenseandBetter.
WemigratethetwosystemsystemsrespectivelyovertheemulatedWANandmeasurethemigrationtimeandtheamountofnetworkdatatransmission.
Fig.
9showstheresultonthemigrationtime,andFig.
10showstheamountofdatasentbythesourcestorageserver.
Oursystemreducesabout59%ofthemigrationtimeforSenseandabout82%ofthemigrationtimeforBetter.
Intermsofnetworkdatasentbythesourcestorageserver,oursystemreducesabout66%ofdatatransmissionforSenseandabout86%ofdatatransmissionforBetter.
Thepercentageofreductionisroughlythesameforthemigrationtimeandforthenetworkdatatransmission.
Thisindicatesthatmostofthemigrationtimeisduetodatatransmissionduringmigration.
Fig.
11.
MigrationtimewhenmigratingoverWANWealsoconductanexperimentinreal-worldWANenvironment.
WemigrateSensefromtheNCTUcampusnetworktoaremotesite,whichisconnectedthrough50MbpsADSLtotheInternet.
Thenetworkrouteconsistedof14hopsandhadanend-to-endlatencyof13.
7ms.
Fig.
11showstheresultoftheexperiment.
Oursystemreducedabout69%ofthemigrationtimeandtookonlyabouthalfanhourtocompletethemigration.
Fromtheexperiment,wecanclearlyseethatoursystemispracticalforsupportingVMmigrationinWANenvironment.
E.
DowntimeevaluationTheWANmigrationsystememploysthepre-copyapproachforVMmigrationandsupportslivemigration.
However,theVMwouldstillneedtobepausedbrieflyduringthetransferofexecution.
Remainingdirtymemorypagesanddirtystorageblockswillallhavetobetransferred115120125130135140145150155Times(sec)010002000300040005000600070008000900010000Memoryusage(MB)0.
797.
130.
321.
26012345678SenseBetterTotalmigrationtime(hours)WithoutindexWithindex050000100000150000200000250000300000350000SenseBetterAmountofdatasentbysourcestorageserver(MBs)WithoutindexWithindex99.
7330.
42020406080100120Sense(overWAN)Totalmigrationtime(minutes)WithoutindexWithindex156156tothedestinationhostmachineandstorageserver.
TheapplicationsrunningontheVMwillbecometemporarilyunavailable.
Thetimeperiodisreferredtoasthemigrationdowntime.
ThisdowntimecanvarydependingontheloadingoftheVM(i.
e.
aheavily-loadedVMislikelytocreatemoredirtypages/blocks)andalsodependingonthebandwidthofthenetwork.
Inthisexperiment,weevaluatethedowntimeoftheWANmigrationsystembyrunningI/Ointensivebenchmarksincludingdbench[31]andkcbench[32]ontheVMtobemigrated.
Fig.
12.
ComparisonofmigrationtimeandnetworktransmissionwithandwithoutindexingmechanismFig.
12showstheresultfromtheexperiment.
TheaveragedowntimeformigratinganidleVMisabout539milliseconds.
It'sshortenoughformostservicestooperatecontinuouslywithoutinterruption.
ThedowntimeincreaseswhenthereisworkloadontheVM.
Forinstance,thedowntimewithdbenchrunningontheVMis2.
627secondsandthedowntimewithkcbenchrunningontheVMis1.
048seconds.
Overall,theaveragedowntimeislessthan3secondsevenunderheavyI/Oworkload.
Thisisgoodenoughfornon-realtimeapplicationstobemigratedwithoutmuchimpactontheuserexperience.
Overall,oursystemperformswellwithrespecttothedowntimeevaluation.
VII.
CONCLUSIONANDFUTUREWORKConventionalvirtualmachinemigrationislimitedtoLANenvironment,becauseboththesharingandthemigrationofVMstorageacrosswide-areanetwork(WAN)areexpensiveduetotheamountofdataintheVMstorageandthelimitedbandwidthofWAN.
Ontheotherhand,theadoptionofcloudcomputinghascausedactiveconstructionofdatacentersaroundtheglobe.
BeingabletocarryoutVMmigrationacrossdatacenterboundariesandacrossWANenvironmentwouldopenupnewpossibilitiesformorepowerfulresourceutilizationandfaulttoleranceincloudcomputing.
WeproposeasystemtofacilitateVMstoragemigrationinWANenvironment,therebyenablingVMmigrationacrossdatacenterboundaries.
ThekeytechniqueistoexploitdatasimilarityinthestorageimagesofneighboringVMsonastorageserver.
ThesystembuildsanindexoftheVMstorageimagesoneachstorageserverandusestheindextoassistthereconstructionofthestorageimageoftheVMtobemigrated.
ThetechniquereducestheamountofdatatransmissioninvolvedinVMmigrationsignificantlyandbringstheoverallWANmigrationtimedowntoanlevelthatisacceptableforpracticaluse.
Thesystemadoptsthepre-copyapproachandsupportslivemigration.
TheevaluationoftheprototypesystemconfirmsthatneighboringVMsdopresentconsiderableamountofduplicatedata.
Throughtheproposedsystem,themigrationtimeofaproductionVMacrossreal-worldWANenvironmentwasshowntobereducedby70%.
Withrespecttolivemigration,thesystemwasabletokeepthedowntimebelow3secsforallthebenchmarksusedintheevaluation.
Theevaluationalsoidentifiedsomedeficienciesoftheprototypesystem.
Onedeficiencyisthememoryusagebytheindexingmechanismisstillalittlebittoohigh.
Theindexingmechanismmaintainsanindexdataentryforeach512bytesstorageblock.
Whilewecanreducethenumberofindexdataentriesbyusingalargeblocksize,itwillcausethesimilarityofstorageblockstodrop(Sec.
VI.
B).
Forfuturework,wecanleverageupper-layerfilesysteminformationtoaddresstheissue.
Theotherdeficiencyofthecurrentprototypeisthatthedowntimemaynotbeshortenoughforlivemigrationofreal-timeapplications.
Forfuturework,weareconsideringtointegratepost-copymechanism[15]toreducethedowntime.
ACKNOWLEDGMENTTheworkwassupportedinpartbyTaiwanInformationSecurityCenterandNationalScienceCouncilundergrant101-2221-E-009-076.
REFERENCES[1]Amazon.
com.
AmazonEC2InstanceTypes.
Available:http://aws.
amazon.
com/ec2/instance-types/[2]Xen.
org.
XenHypervisor.
Available:http://www.
xen.
org/[3]Open-iSCSIproject:Open-iSCSI–RFC3720architectureandimplementation.
Available:http://www.
open-iscsi.
org/[4]D.
S.
Milojii,F.
Douglis,Y.
Paindaveine,R.
Wheeler,andS.
Zhou,"Processmigration,"ACMComputingSurvey,vol.
32,pp.
241-299,2000/09//2000.
[5]C.
Clark,K.
Fraser,S.
Hand,J.
G.
Hansen,E.
Jul,C.
Limpach,etal.
,"Livemigrationofvirtualmachines,"2005,pp.
273-286.
[6]Migration-KVM.
Available:http://www.
linux-kvm.
org/page/Migration[7]VMware-VMotion-DS-EN.
pdf.
Available:http://www.
vmware.
com/files/pdf/VMware-VMotion-DS-EN.
pdf[8]Q.
Li,J.
Huai,J.
Li,T.
Wo,andM.
Wen,"HyperMIP:HypervisorControlledMobileIPforVirtualMachineLiveMigrationacrossNetworks,"in11thIEEEHighAssuranceSystemsEngineeringSymposium,2008.
HASE2008,2008,pp.
80-88.
[9]M.
Tsugawa,P.
Riteau,A.
Matsunaga,andJ.
Fortes,"User-levelvirtualnetworkingmechanismstosupportvirtualmachinemigrationovermultipleclouds,"in2010IEEEGLOBECOMWorkshops(GCWkshps),2010,pp.
568-572.
[10]H.
Jin,L.
Deng,S.
Wu,X.
Shi,andX.
Pan,"Livevirtualmachinemigrationwithadaptive,memorycompression,"2009,pp.
1-10.
[11]X.
Zhang,Z.
Huo,J.
Ma,andD.
Meng,"ExploitingDataDeduplicationtoAccelerateLiveVirtualMachineMigration,"in2010IEEEInternationalConferenceonClusterComputing(CLUSTER),2010,pp.
88-96.
[12]F.
F.
MoghaddamandM.
Cheriet,"Decreasinglivevirtualmachinemigrationdown-timeusingamemorypageselection0.
5392.
6271.
04800.
511.
522.
53IdledbenchkcbenchAveragedowntime(sec)157157basedonmemorychange,"inInternationalConferenceonNetworking,SensingandControl,2010,pp.
355-359.
[13]K.
Haselhorst,M.
Schmidt,R.
Schwarzkopf,N.
Fallenbeck,andB.
Freisleben,"EfficientStorageSynchronizationforLiveMigrationinCloudInfrastructures,"2011,pp.
511-518.
[14]DRBD.
Org.
(2012/12/31).
DRBD:SoftwareDevelopmentforHighAvailabilityClusters.
Available:http://www.
drbd.
org/[15]T.
Hirofuchi,H.
Ogawa,H.
Nakada,S.
Itoh,andS.
Sekiguchi,"ALiveStorageMigrationMechanismoverWANforRelocatableVirtualMachineServicesonClouds,"2009,pp.
460-465.
[16]VMware-Storage-VMotion-DS-EN.
pdf.
Available:http://www.
vmware.
com/files/pdf/VMware-Storage-VMotion-DS-EN.
pdf[17]S.
Akoush,R.
Sohan,B.
Roman,A.
Rice,andA.
Hopper,"ActivityBasedSectorSynchronisation:EfficientTransferofDisk-StateforWANLiveMigration,"in2011IEEE19thInternationalSymposiumonModeling,AnalysisSimulationofComputerandTelecommunicationSystems(MASCOTS),2011,pp.
22-31.
[18]F.
Travostino,P.
Daspit,L.
Gommans,C.
Jog,C.
d.
Laat,J.
Mambretti,etal.
,"SeamlesslivemigrationofvirtualmachinesovertheMAN/WAN,"FutureGener.
Comput.
Syst.
,vol.
22,pp.
901-907,2006.
[19]T.
Wood,K.
Ramakrishnan,P.
Shenoy,andJ.
VanderMerwe,"CloudNet:dynamicpoolingofcloudresourcesbyliveWANmigrationofvirtualmachines,"inACMInternationalConferenceonVirtualExecutionEnvironments(VEE),2011,pp.
121-132.
[20]LinuxUnifiedTarget-MainPage.
Available:http://linux-iscsi.
org/wiki/Main_Page[21]LinuxKernelDocumentation::filesystems:configfs.
Available:http://www.
mjmwired.
net/kernel/Documentation/filesystems/configfs/[22]proc(5):processinfopseudo-filesystem-Linuxmanpage.
Available:http://linux.
die.
net/man/5/proc[23]procfs-Wikipedia,thefreeencyclopedia.
Available:http://en.
wikipedia.
org/wiki/Procfs[24]mmap(2)-Linuxmanualpage.
Available:http://www.
kernel.
org/doc/man-pages/online/pages/man2/mmap.
2.
html[25]ioctl-Wikipedia,thefreeencyclopedia.
Available:http://en.
wikipedia.
org/wiki/Ioctl[26]ioctl(2)-Linuxmanualpage.
Available:http://www.
kernel.
org/doc/man-pages/online/pages/man2/ioctl.
2.
html[27]Device-mapperResourcePage.
Available:http://sources.
redhat.
com/dm/[28]LVM2ResourcePage.
Available:http://sourceware.
org/lvm2/[29]multipath-tools:Home.
Available:http://christophe.
varoqui.
free.
fr/[30]LAMP(softwarebundle)-Wikipedia,thefreeencyclopedia.
Available:http://en.
wikipedia.
org/wiki/LAMP_(software_bundle)[31]DBENCH.
Available:http://dbench.
samba.
org/[32]kcbench(1):Kernelcompilebenchmark-Linuxmanpage.
Available:http://linux.
die.
net/man/1/kcbench158158