integratedclusterware

POWERSOLUTIONSNovember200392BuildingaHighlyScalableandAvailableDataEnvironmentforOracle9iRACToprovideahighlyscalableandavailabledatabaseenvironmentforOracle9iRealApplicationClusters(RAC),administratorsmustestablishahighlyavailablestorageinfrastructure.
Astorageareanetwork(SAN)canprovideredundantpathstostorage,andrunningEMCPowerPathcanleveragethisredundancybyprovidingamechanismforpathfailoverinanOracle9iRACinfrastructure.
Intoday'sbusinessenvironment,highavailabilityisrequiredformission-criticalapplications.
TheOracle9irelationaldatabasemanagementsystem(RDBMS)ishighlyavailableandscalable.
TheOracle9iRealApplicationClusters(RAC)optionenablestheOracle9iRDBMStobeconfiguredinaclusterdatabasearchitecturewheremul-tiplenodessharethesamestorage.
Oracle9iRACprovideshighavailability;ifonenodefails,theotherstakeoverandprovideuninterruptedaccesstothedatabase.
However,ifonlyoneI/Opathexistsfromeachnodetothesharedstor-ageinsuchanenvironment,thisI/Opathpotentiallybecomesasinglepointoffailure.
AtypicalOracle9iconfigurationincludesastorageareanetwork(SAN),whichcanhelpprovideahighlyavailabledatainfrastructurebyusingredundantcompo-nentstoensurethatnocomponentbecomesasinglepointoffailure.
AFibreChannel–basedSANfabricsupportsmultipathroutingbetweenSANswitches.
Inatypicaltopology,anodehasmultipleFibreChannelhostbusadapters(HBAs),eachofwhichareconnectedtothesameSAN,resultinginmultiplepathstothesamedevice(seeFigure1).
SANstoragedevicescanalsoacceptmultipleFibreChannelconnections.
RedundantpathsinaSANprovidefailovercapabilitywhenanycomponentinthedatapathfails.
Multiplepathsalsoenhanceefficiency,allowingadministratorstoloadbalanceSANtrafficbyconsideringI/Oacrossallavailablepaths.
Indoingso,theSANtakesadvantageoftheaddi-tionalbandwidthprovidedbyeachphysicalconnection.
AlthoughredundantI/Opathsarebeneficialforloadbalancingandlinkfailover,theycancreatecomplications.
BecauseeachdeviceonaSANappearsasaSCSIIDoneachHBAthatisconnectedtotheSAN,asystemwithmultipleHBAsconnectedtotheSANwillbehaveasifeachdeviceoneachpathisaseparateSCSIdevice.
Thus,theoperatingsystembehavesasthoughmultipleENTERPRISECLUSTERENVIRONMENTBYZAFARMAHMOOD,PAULRAD,ANDROBERTNADONstorageresourcesexistwheninfactthereareonlymultiplepathstothesameresource.
Pointingtothesamestoragedevicealongdifferentpathscouldpotentiallycausedatacorruptionandsystemcrashes.
Topreventsuchproblems,administratorscaninstallpathmanagementsoftware,suchasEMCPowerPath,oneachnodeintheSAN.
PowerPathenablesmultipleI/Opathstothesharedstoragebymaskingthepaths,andpresentstheoperatingsystemwiththeappearanceofasingleSCSIconnection.
ThismaskingensuresthatthenodereceivesasingleviewofthestoragedevicesacrossmultipleHBAs.
PowerPathalsoautomaticallydetectsavailablepaths,restoresfailedpaths,andloadbalancesI/Oacrossallpaths.
WhenintegratedinanOracle9iRACenvironment,PowerPathprovideshighlyavailable,scalable,andfault-tolerantsharedstorage.
UsingPowerPathforpathfailuredetectionEMCPowerPathhelpsprovidehighavailabilitybyautomaticallydetectingandrestoringfailedpathswhilestoragearrays,nodes,andapplicationsremainavailable.
ShouldanHBA,astorageproces-sor,oracablefail,PowerPathcompletesanI/Orequestthroughanotheravailablechannel,helpingpreventtheinterruptionofdatatoanapplication.
PowerPathalsoprovidesautomaticonlinepathrecoveryafterthepathisrepaired,whichcanreduceplannedout-agestorestoreservices.
PowerPathvirtualdevicesmappathstostoragePowerPathresidesonanodeasasoftwarecomponentbetweenOracle9iRAC,theOracleclusterfilesystem(OCFS),andtheHBAdevicedriverlayer(seeFigure2).
PowerPathoperatesindependentlyofapplications,theRDBMS,managementutilities,andfilesystems,allowingadministratorstoinstallandconfigurePowerPathwithoutmodifyingexistingsoftware.
ThePowerPathdriverresidesonthenode,abovetheHBAdriver.
ThenodehasmultipleHBAssothatitcanprovidepathfailoverthroughthePowerPathdriver.
ThePowerPathdriverenablesvirtualdevices,whichprovidefailure-resistantandload-balancedpathstotheDell|EMCstoragearray.
AnapplicationreferencesaPowerPathvirtualdevice;inturn,thePowerPathdrivermanagespathallocationtothestoragearray.
Inthefollowingexample,fourlogicalunitnumbers(LUNs)areconfiguredonaDell|EMCstoragearray,whichusestwostorageprocessorstoconnect—throughtwoseparateFibreChannelswitches—toanodecontainingtwoHBAs.
TheresultingpathsandtheirmappingstothePowerPathvirtualdevicesprovideatotalof16pathsfromanodetostorage.
Having16pathstostoragewouldordinarilyresultin16logicaldevicesbeingvisibletothenode.
However,thePowerPathdrivercreatesfourPowerPathvirtualdevices,andeachofthesevirtualdevicesmapsfourpathstoalogicaldeviceonthestoragearray,asindicatedinFigure3.
PowermtmanagementutilityfacilitatespathmanagementForautomaticfailuredetectionandrecovery,PowerPathprovidesanadministrationutilitycalledPowermt,whichprovidesacommandENTERPRISECLUSTERENVIRONMENTwww.
dell.
com/powersolutionsPOWERSOLUTIONS93HBAHBAHBAHBAHBAHBAHBAHBAFibreChannelswitchFibreChannelswitchLogicallayerPhysicallayerStorageprocessor1Storageprocessor2ServerServerServerServerDiskDiskDiskDiskDiskDiskDiskDiskDiskDiskLUNLUNLUNFigure1.
AhighlyavailablestorageinfrastructurePowerPathOracleclusterfilesystemOracle9iRACDatabasemanagementutilitiesHBAdriverHBAcontrollerHBAdriverHBAcontrollerFigure2.
PowerPathinOracle9iRACsoftwarelayerPowerPathvirtualdevices(4)Paths(16)/dev/emcpowera/dev/sdb/dev/sdf/dev/sdj/dev/sdn/dev/emcpowerb/dev/sdc/dev/sdg/dev/sdk/dev/sdo/dev/emcpowerc/dev/sdd/dev/sdh/dev/sdl/dev/sdp/dev/emcpowerd/dev/sde/dev/sdi/dev/sdm/dev/sdqFigure3.
PathstologicaldevicesforPowerPathvirtualdeviceslineinterfacetothePowerPathenvironment.
PowerPathperiodi-callyteststhepathsforfailuredetectionaccordingtoabuilt-inalgorithm.
UsingthePowermtutility,administratorscansetupahostnodetoperformautorecoveryonfailedpathsbyusingthefollowingcommand-lineinstructiontothePowerPathdriver:powermtsetperiodic_autorestore=on|offForconvenientmanagementofaUNIXorLinuxnodeconfiguredwithPowerPath,thePowermtutilityprovidesseveralfeatures,including:CheckingaPowerPathconfiguration:Thepowermtcheckcommandchecksthespecifiedpathsand,ifdesired,removesfromthePowerPathconfigurationanypathsmarkeddead.
Configuringpathstologicaldevices:ThepowermtconfigcommandconfiguresalldetectedlogicaldevicesasPowerPathdevicesandaddsthesedevicestothePowerPathconfigura-tion,creatingdevicesasrequired.
RemovingpathsfromPowerPathmanagement:Thepowermtremovecommanddeletesthespecifiedpath(orpaths)fromPowerPath'slistofconfiguredpaths.
Itdoesnotdeletethelogicaldevicetowhichthepathsrefer.
IntegratingPowerPathinanOracle9iRACenvironmentToperformI/O,Oracle9iRACusesPowerPathvirtualdevices(suchas/dev/emcpoweraand/dev/emcpowerb,showninFigure3)insteadofLUNs,orlogicaldevices(suchas/dev/sdband/dev/sdc).
Ifanactualdevicepathfails,thePowerPathdriverroutesI/OtoanalternativepathwithoutcausinganyinterruptiontotheRDBMSfunctionality.
IntegrationofPowerPathwithOracle9iRACisasimpleprocess.
PowerPathmaybedeployedineitheraneworexistingOracle9iRACimplementation,asexplainedinthefollowingsections.
IntegratingPowerPathwithanewOracle9iRACimplementationOracle9iRACusessharedstorageontheSANforitsdatabase,redolog,andcontrolfiles.
TheRDBMSenginefromeachnodeintheclustermusthavedirectaccesstothisstoragetocreateorupdateanyofthesefiles.
WithoutPowerPath,theRDBMSenginewoulddirectlyaccesstheLUNsonthesharedstoragetocreatetherequiredfiles.
WithPowerPathintegrated,theprocedureisthesame,withoneimportantexception.
ThepartitionsandOCFSarecreatedonthePowerPathvirtualdevicesratherthanonthepartitionedLUNs:1.
PartitionthePowerPathvirtualdevicesonthesharedstoragearrayaccordingtothedatabasesizingrequirements:fdisk/dev/emcpowerafdisk/dev/emcpowerbfdisk/dev/emcpowercfdisk/dev/emcpowerd2.
CreatetheOCFSonthenewpartitionsandmountthefilesystem:mkfs.
ocfs-F-b128-Lu01-m/u01-u200-g300-p0775/dev/emcpowera1mkfs.
ocfs-F-b128-Lu02-m/u02-u200-g300-p0775/dev/emcpowerb1mkfs.
ocfs-F-b128-Lu03-m/u03-u200-g300-p0775/dev/emcpowerc1mkfs.
ocfs-F-b128-Lu04-m/u04-u200-g300-p0775dev/emcpowerd13.
SetuptheclusterwareontheOCFS,andcreatetheOracle9iRACdatabaseonthesharedSANstorage.
TheintegrationofPowerPathistransparenttotheOracle9iRACdatabaseengine,asisthefactthateachPowerPathvirtualdevicepointstomultiplephysicalI/Opaths.
IntegratingPowerPathwithanexistingOracle9iRACimplementationInanexistingOracle9iRACdatabasethatissetupwithoutPowerPath,thedatabaseengineusestheLUNsonthesharedSANstoragetocreateandupdatethedatabasefiles.
IntegratingPowerPathisastraight-forwardprocessusingthefollowingsteps:1.
ShutdownallOracleservicesinthecluster,includingdatabaselisteners,OracleIntelligentAgents,andOracleClusterManager.
2.
Shutdownthedatabaseonallclusternodes.
3.
UsetheumountcommandtounmountallOCFSvolumes.
4.
ConfiguretheDell|EMCstoragearraytosupportPowerPath,andinstallthePowerPathsoftwareonallclusternodes.
Oneachnode,thePowerPathsoftwarewillautomaticallycreatePowerPathdevicesthatpointtotheexistingLUNs.
5.
Modifythe/etc/fstabconfigurationfileandreplacethedevicenamesofeachLUN(forexample,/dev/sdb,/dev/sdf,dev/sdj,/dev/sdn)onthestoragearraywiththePowerPathvirtualdevicename(forexample,/dev/emcpowera,/dev/emcpowerb,/dev/emcpowerc,/dev/emcpowerd).
www.
dell.
com/powersolutionsSANsprovidecentralizeddatastorage,andEMCPowerPathcomplementsSANarchitecturebyhelp-ingmanageredundantSANpathstoprovideahigh-availabilityenviron-mentforOracle9iRAC.
ENTERPRISECLUSTERENVIRONMENTPOWERSOLUTIONSNovember2003966.
RestarttheOCFSandusethemountcommandtomountthefilesystem.
7.
StartupOracleservices,clusterware,andOracle9iRACdatabaseinstancesonallclusternodes.
UsingPowerPathtomaintainavailabilityAsreliableaccesstoinformationbecomesacriticalmissionfortoday'sdatacenters,ITarchitectsandadministratorsmustusecentralized,scalablestoragetohelpcreateahighlyavailabledatainfrastructureacrosstheenterprise.
SANsprovidecentralizeddatastorage,andEMCPowerPathcomplementsSANarchitecturebyhelpingmanageredundantSANpathstoprovideahigh-availabilityenvironmentforOracle9iRAC.
FactorssuchasI/Oload,effectofdowntime,andavailabilityofadministratormaintenancetimecanhelpdeterminewhetheranorganization'snetworkedDell|EMCstoragewouldbenefitfromPowerPath.
PowerPathoffersfaulttolerancetohelpeliminatedown-time,loadbalancingtoenablemoreefficientI/Otraffic,andauto-maticdetectionandrepairofpathsthathavefailed,therebyhelpingtoreduceadministrativeoverhead.
ByintegratingPowerPathwithOracle9iRAC,administratorscanimproveSANuptimeandprovideastableinfrastructureformission-criticaldata.
ZafarMahmood(zafar_mahmood@dell.
com)isasoftwareengineerintheDellDatabaseandApplicationEngineeringDeploymentDepartmentoftheDellProductGroup.
Hehasbeeninvolvedindatabaseperformanceoptimization,databasesystems,anddatabaseclusteringformorethansixyears.
HeiscurrentlyworkingonOracle9iRACimplementationsatDell.
ZafarhasanM.
S.
inElectricalEngineeringwithaspecializationinComputerCommunica-tionsfromtheCityUniversityofNewYork.
PaulRad(paul_rad@dell.
com)isaseniorsoftwareengineerintheDellDatabaseandApplicationEngineeringDepartmentoftheDellProductGroup.
Paulhasmaster'sdegreesinbothComputerScienceandComputerEngineeringfromtheUniversityofTexasatSanAntonio.
RobertNadon(robert_nadon@dell.
com)isasoftwareengineeringconsultantintheFactoryInstallationDevelopmentGroupoftheDellProductGroup.
HefocusesonfactoryinstallationforUNIX-based(LinuxandBSDi)operatingsystemsandfortheOracle9idatabase.
AnotherareaofinterestisDellsoftwareRAIDofferings.
RoberthasaB.
S.
inComputerEngineeringfromTexasA&MUniversity.
HeisaRedHatCertifiedEngineer(RHCE)forRedHatLinux6.
2and7.
0.
FORMOREINFORMATIONDell|EMCstorage:http://www.
dell.
com/emcDellandOracle:http://www.
dell.
com/oracleOracledatabase:http://www.
oracle.
com/oracle9i