generateclusterware
clusterware 时间:2021-04-01 阅读:()
Solution-in-a-box:BestpracticesfordeployingOracleE-BusinessSuiteRelease12.
2.
4onOracleDatabaseAppliance12.
1.
2.
7ORACLEWHITEPAPER|JULY20161|ENTERTITLEOFDOCUMENTHERETableofContentsExecutiveOverview2Scope2OracleDatabaseApplianceVirtualizedPlatformDeploymentArchitecture2Softwareoneachphysicalserver4InstallingOracleE-BusinessSuite12.
2.
4onODAVirtualizedPlatform4STEP1:DeployOracleDatabaseApplianceVirtualizedPlatform4STEP2:PreparetheTargetORACLE_HOME5STEP3:SetuptheOracleE-BusinessSuiteDatabaseVM6STEP4:MigratetheDatabasetoODA_BASE9STEP5:SetuptheOracleE-BusinessSuiteApplicationsTierVM12STEP6:EnableHighAvailability16STEP7:Cleanup16STEP8:ValidateEnvironment17BestPractices18ConsiderResizingtheResourceConfigurationforODA_BASE18ConsiderResizingtheResourceConfigurationfortheApplicationsVMs18ConsiderImplementingWorkloadIsolationusingCPUPools18PatchingOracleDatabaseAppliance19SystemSizing19Conclusion21AppendixA:References22AppendixB:ConfiguringVMsafterFirstBoot232|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCEExecutiveOverviewOracleE-BusinessSuiteisthemostcomprehensivesuiteofintegrated,globalbusinessapplicationsthatenableorganizationstomakebetterdecisions,reducecosts,andincreaseperformance.
OracleDatabaseApplianceisanOracleengineeredsystemconsistingofhardwareandsoftwaredesignedtosimplifythedeployment,maintenanceandsupportofhighavailabilityOracleDatabaseandOracleE-BusinessSuiteApplicationsTiersolutions.
OracleDatabaseApplianceofferstheoptiontodeployhighlyavailableOracleDatabasesinabaremetalenvironmentortodeploybothOracleDatabasesandtheApplicationsTierinvirtualizedenvironmentswithintheappliance.
ThispaperdescribeshowtorapidlydeployOracleE-BusinessSuite12.
2.
4databaseandApplicationstiersonOracleDatabaseApplianceVirtualizedPlatform12.
1.
2.
7(andlater)usingthepreconfiguredOracleE-BusinessSuiteVMtemplates.
EachOracleE-BusinessSuitedeploymentconsistsoftwoOracleVMtemplates:adatabaseimagetemplateandaprimaryApplicationstiertemplate.
Thesetemplatesdeliverthecompletesoftwarestackandarepre-configuredforaVisiondemowithsampledataorafreshinstalldatabaseforproductionenvironment.
Theprocedureinthispaperprovidesasimple,reliableandcosteffectivesolution-in-a-boxwhichisfullysupportedforproductionuse.
ScopeThispaperdescribeshowtoinstallOracleE-BusinessSuiteRelease12.
2.
4databaseandApplicationstiersontheOracleDatabaseApplianceVirtualizedPlatform12.
1.
2.
7(andlater)usingOracleE-BusinessSuitetemplates.
ThisdocumentisanupdatetoincludeOracleACFSsupportwithlatestversionsofOracleDatabaseAppliancewithsupportforDriverDomains.
YouneedtobefamiliarwiththeOracleDatabaseAppliancedocumentationand,inparticular,theOracleDatabaseApplianceVirtualizedPlatformdocumentation.
FurtherinformationabouttheOracleDatabaseApplianceisavailableat:http://docs.
oracle.
com/cd/E22693_01/index.
htmUnderstandingOracleE-BusinessSuite12.
2architectureisessential.
Thedocumentationlibraryisavailableat:http://docs.
oracle.
com/cd/V39571_01/current/html/docset.
htmlOracleDatabaseApplianceVirtualizedPlatformDeploymentArchitectureThissectionsummarizessomeofthekeyarchitecturalpointsthatwillmakeiteasiertounderstandtheinstallationprocedureinthispaper.
DeployingOracleDatabaseApplianceVirtualizedPlatformcreatesanODA_BASEprivilegeduserdomain.
TheApplianceManagerprovisionsandmanagestheOracleDatabasewithintheODA_BASEdomain.
Thisdomainisoptimizedtohostdatabaseinstances,andalltheshareddisksaredirectlyattachedtothisdomainminimizingvirtualizationoverhead.
OracleDatabaseAppliance12.
1.
2.
0introducesafundamentalchangeinthestoragelayer–theintroductionofOracleACFS.
OracleACFSisdesignedtodeliverthebestperformanceforOracledatabases.
OracleACFSsupportsthecreationoffilesystemsnapshots–afeaturethatvastlysimplifiesthecreationofspaceefficientdatabaseclones.
Goingforward,newdatabasesshouldonlybecreatedonACFSfilesystemsinanewODAdeploymentscenarios.
3|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCEAlthoughOracleE-BusinessSuitedatabasecanbeinstalledinanyoftheuserdomains,itisbestpracticetodeploytheOracleE-BusinessSuitedatabaseinODA_BASEasdescribedinthispapersinceitsignificantlyimprovesperformanceandmanageability.
TosupporthighavailabilityattheApplicationstier,theOracleE-BusinessSuiteApplicationstierVMtemplatecanbeinstalledinmultipleVMs.
Aftercompletingtheprocedureoutlinedinthispaper,OracleDatabaseAppliancewillbeconfiguredwithatwonodeOracleRACdatabaseandtheOracleE-BusinessSuiteApplicationstierrunningintwoVMs,oneoneachnode.
HalfthephysicalCPUandmemoryresourceswillbeallocatedtoODA_BASEfordatabase,andtheotherhalfwillbeallocatedtotheApplicationstierVMs.
Dom0usesveryfewresourcesanddoesn'tneedaseparateallocation.
Note:ThispaperdescribesdeployingthesolutiononanOracleDatabaseApplianceX5-2runningODAsoftwarerelease12.
1.
2.
7.
Thestepsareapplicableforlatersoftwareandhardwarereleases,butsomeofthecomponentversionsmaybedifferent.
Thefollowingliststhemaincomponentsoftheconfiguration:Node1Node2DOM0VMStorageRepositoryODABaseApplianceMgrGridInfrastructureASMInstanceACFSFilesystemRACDBInstanceEBSApplicationVMEBSApplicationVMDOM0VMStorageRepositoryODABaseApplianceMgrGridInfrastructureASMInstanceACFSFilesystemRACDBInstance4|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCESoftwareoneachphysicalserverDom0ManagementDomainOracleVMServer3.
2.
9ODA_BASEVirtualMachineSpecialprivilegeduserdomainfordatabaseinstances.
OracleLinux6.
7.
OracleDatabase11gR2and12c.
OracleRealApplicationClusters(RAC)12c.
OracleACFS12c.
AutomaticStorageManagement(ASM)12c.
OracleClusterware12c.
128GBmainmemoryand16coresallocated.
Thisallocationcaneasilybechangedlaterbasedonperformancerequirementwithinthelimitsavailablesoftwarelicenses.
OracleE-BusinessSuiteVirtualMachineUserdomain.
OracleLinux6.
5.
OracleE-BusinessSuite12.
2.
4.
96GBmainmemoryand16coresallocated.
Thisallocationcanbechangedlater.
InstallingOracleE-BusinessSuite12.
2.
4onODAVirtualizedPlatformThissectionliststhestepsnecessarytoinstallOracleE-BusinessSuiteRelease12.
2.
4withOracleDatabase11gR2onOracleDatabaseAppliance12.
1.
2.
7(orlater).
TheTable1providesaquickreferencetotheenvironmentdetailsreferredtointhisdocument:Table1:ExampleEnvironmentDetailsHostServer1Server2Dom0node1-d0node2-d0ODA_BASEnode1node2ApplicationsVMapp1app2Inthisdocument,allpasswordsareassumedtobewelcome1.
Thesepasswordsshouldalwaysbechanged.
ThisdocumentrequiresaccesstobothnodesoftheOracleDatabaseApplianceasroot,gridandoracleusers.
STEP1:DeployOracleDatabaseApplianceVirtualizedPlatform1.
TheOracleDatabaseAppliancehastobedeployedasaVirtualizedPlatforminordertosupportOracleE-BusinessSuiteApplicationsVMs.
OracleDatabaseApplianceVirtualizedPlatformSetupinstructionsareprovidedontheSetupPoster,whichisavailableat:http://docs.
oracle.
com/cd/E22693_01/.
FollowtheinstructionstosetuptheOracleDatabaseAppliance.
Itisnotnecessarytodeployadatabaseduringtheappliancesetup.
Deselect"CreateInitialDatabase"checkboxinthe"DatabaseInformation"pageoftheApplianceManager.
2.
OracleE-BusinessSuiterequiresthedatabaseVM(ODA_BASE)runscertainUNIXpackagesandspecifiedminimumversions,whicharelistedinMyOracleSupport(MOS)KnowledgeDocument5|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCE1330701.
1OracleE-BusinessSuiteInstallationandUpgradeNotesRelease12(12.
2)forLinuxx86-64.
EnsurethattheODA_BASEonbothnodesmeettheserequirements.
3.
ThespacerequirementforthetemplatesandVMimagesarelistedinTable2:Table2:SpaceRequirementEBSVMTemplatesDiskSpaceRequiredOracleE-BusinessSuiteDatabaseVMtemplate300GBOracleE-BusinessSuiteDatabaseVM300GBOracleE-BusinessSuiteApplicationsVMtemplate300GBOracleE-BusinessSuiteApplicationsVM(2VMs)300GBx2=600GBTotalSpacerequired1500GB4.
Loginastherootuseronnode1andcreatea1600GBsharedrepositorywhichwillbeusedtohosttheVMsusingthefollowingcommand:$oakclicreaterepoappsrepo–size1600G–dgRECO$oakclishowrepo5.
TheOracleDatabaseApplianceteamreleasesRDBMSCloneBundlesforeachOracleDatabaseApplianceversion.
OracleE-BusinessSuite12.
2.
4VMsrequireOracleDatabase11.
2.
0.
4.
3whichisavailableinpatch:17770873–OracleDatabaseAppliance11.
2.
0.
4.
3End-User(RDBMSClone).
DownloadtheRDBMSclone.
Astherootuserstagethesoftwareusingoakcliutilityasfollows:$oakcliunpack–package\/p17770873_121200_Linux-x86-64.
zipSTEP2:PreparetheTargetORACLE_HOME1.
CreateanewORACLE_HOMEtohostthedatabaseusingoakcli.
$oakclicreatedbhome–version11.
2.
0.
4.
3$oakclishowdbhomes2.
ThenewORACLE_HOMEcreatedbytheoakcliutilitydoesnotincludetheOracleDatabase11gR2Examplescomponent,whichisrequiredbyOracleE-BusinessSuite.
ThiscanbedownloadedfromtheOracleTechnologyNetworkat:http://www.
oracle.
com/technetwork/database/enterprise-edition/downloads/index.
html.
Unziptheinstallationmediaintoastagingdirectory.
Astheoracleuser,runtheOUIfromthedownloadedinstallationmediadirectoryusingthefollowingcommandandfollowtheon-screeninstructions:$.
/runInstaller3.
OracleE-BusinessSuiteneedsadditionalRDBMSpatchestobeinstalled.
Requiredpatchesfor11.
2.
0.
4.
3RDBMSPSUareenumeratedintheMOSKnowledgeDocument1147107.
1DatabasePatchSetUpdateOverlayPatchesRequiredforUsewithPSUsandOracleE-BusinessSuite.
6|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCE4.
WhenOracleDatabaseApplianceisdeployed,RoleSeparationusesdifferentownersfortheGridInfrastructureandOracleDatabase.
Applyingtheone-offRDBMSpatcheswillre-generatetheOraclebinaryin$ORACLE_HOME/binanditspermissionswillberesettothedefaultownership.
YouneedtoresettheOraclebinarypermissionsforthegriduseronbothnode1andnode2,usingthefollowingcommand:$/u01/app/12.
1.
0.
2/grid/bin/setasmgidwrap\o=/u01/app/oracle/product/11.
2.
0.
4/dbhome_1/bin/oracle5.
OracleE-BusinessSuite12.
2usesOracle9iNLSinformation.
TherequiredsupportfilesarecreatedbyrunningthefollowingscriptundertheRDBMShomeastheoracleuseronbothnode1andnode2:$perl$ORACLE_HOME/nls/data/old/cr9idata.
plSTEP3:SetuptheOracleE-BusinessSuiteDatabaseVMTheOracleE-BusinessSuiteVMsincludeadatabaseVMwiththerequiredmetadataforanOracleE-BusinessSuitedeployment.
ThisdatabasewillbemigratedtoruninODA_BASEwhichprovidesoptimalperformance.
1.
OracleE-BusinessSuiteRelease12.
2.
4x86(64bit)VMtemplatesareavailableat:https://edelivery.
oracle.
com/linux.
Downloadthefollowingfilestoastagingareaunderthe/cloudfsdirectoryonnode1:V56219-01_1of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ProductionDatabaseTierInstallX86(64bit)(Part1of2)V56219-01_2of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ProductionDatabaseTierInstallX86(64bit)(Part2of2)V56220-01_1of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ProductionDatabaseTierInstallX86(64bit)(Part1of2)V56220-01_2of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ProductionDatabaseTierInstallX86(64bit)(Part2of2)V56218-01_1of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ProductionDatabaseTierInstallX86(64bit)(Part1of2)V56218-01_2of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ProductionDatabaseTierInstallX86(64bit)(Part2of2)2.
Unzipthefilesusingthefollowingcommands:$unzipV56218-01_1of2.
zip$unzipV56218-01_2of2.
zip$unzipV56219-01_1of2.
zip$unzipV56219-01_2of2.
zip$unzipV56220-01_1of2.
zip$unzipV56220-01_2of2.
zip3.
Combinethesixfilesintoasinglecompressedfileusingthefollowingcommand:$cat\Oracle-E-Business-Suite-PROD-DB-12.
2.
4.
ova.
0\Oracle-E-Business-Suite-PROD-DB-12.
2.
4.
ova.
1\Oracle-E-Business-Suite-PROD-DB-12.
2.
4.
ova.
2\Oracle-E-Business-Suite-PROD-DB-12.
2.
4.
ova.
3\Oracle-E-Business-Suite-PROD-DB-12.
2.
4.
ova.
4\Oracle-E-Business-Suite-PROD-DB-12.
2.
4.
ova.
5\>Oracle-E-Business-Suite-PROD-DB-12.
2.
4.
ova4.
StartPythonbasedHTTPserverfromthesamedirectorywhichcontainstheassemblyfile.
LeavethesessionrunninguntilVMtemplateimportcompletes.
7|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCE$python–mSimpleHTTPServer5.
Loginasrootuseronnode1andcreateaVNCsessionthatwillbeusedforGUIcomponents,suchastheVMconsole,usingthefollowingcommand:$vncserver6.
BeforethedownloadedOracleDatabaseVMtemplatecanbeusedtostartaVM,itneedstobestagedwithintheODA.
Astherootuseronnode1,importtheVMtemplateusingthefollowingcommand:$oakcliimportvmtemplatedbtempl–assembly\http://node1:8000/Oracle-E-Business-Suite-PROD-DB-12.
2.
4.
ova\–repoappsrepo–node07.
Asrootuseronnode1,usetheoakcliclonecommandtocreatetheVMasfollows:$oakcliclonevmdbvm-vmtemplatedbtempl-repoappsrepo–node08.
ConfirmtheVMhasbeencreatedusingthefollowingcommand:$oakclishowvmNAMENODENUMMEMORYVCPUSTATEREPOSITORYDbvm040964OFFLINEappsrepo$oakclishowvmdbvmResource:appdbAutoStart:restoreCPUPriority:100Disks:|file:/OVS/Repositories/appsrepo/.
ACFS/snaps/dbvm/VirtualMachines/dbvm/e6794c5f000c48d3a6da48544c1318bd.
img,xvda,w||,xvdb:/OVS/Repositories/appsrepo/.
ACFS/snaps/dbvm/VirtualMachines/dbvm/cdrom,r|Domain:XEN_PVMExpectedState:offlineFailOver:falseIsSharedRepo:trueKeyboard:en-usMaxMemory:2048MaxVcpu:2Memory:2048Mouse:OS_DEFAULTName:dbvmNetworks:NodeNumStart:0OS:OL_5PrefNodeNum:0PrivateIP:NoneProcessorCap:100RepoName:appsrepoSource:0State:OfflineTemplateName:otml_dbtemplVDisks:|0|Vcpu:1cpupool:default-unpinned-poolvncport:None9.
ReconfiguretheVMtousenet1asthepublicinterface:8|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCE$oakclimodifyvmdbvm-addnetworknet1$oakclishowvmdbvmResource:appdbAutoStart:restoreCPUPriority:100Disks:|file:/OVS/Repositories/appsrepo/.
ACFS/snaps/dbvm/VirtualMachines/dbvm/e6794c5f000c48d3a6da48544c1318bd.
img,xvda,w||,xvdb:/OVS/Repositories/appsrepo/.
ACFS/snaps/dbvm/VirtualMachines/dbvm/cdrom,r|Domain:XEN_PVMExpectedState:offlineFailOver:falseIsSharedRepo:trueKeyboard:en-usMaxMemory:2048MaxVcpu:2Memory:2048Mouse:OS_DEFAULTName:dbvmNetworks:['bridge=net1']NodeNumStart:0OS:OL_5PrefNodeNum:0PrivateIP:NoneProcessorCap:100RepoName:appsrepoSource:0State:OfflineTemplateName:otml_dbtemplVDisks:|0|Vcpu:1cpupool:default-unpinned-poolvncport:None10.
Duetobug:21101080,theVMconfigurationhasanincorrectentryforthecdrominthevm.
cfgfile.
Thishastobecorrectedmanually.
Editthevm.
cfgandchangefollowing:xvdb:/OVS/Repositories/appsrepo/.
ACFS/snaps/dbvm/VirtualMachines/dbvm/cdrom,rto:xvdb:cdrom,r11.
StartthedatabaseVMusingthefollowingcommand:$oakclistartvmdbvm12.
ThedatabaseVMcanbeconfiguredfromtheVMconsole.
LogintotheVNCsessioncreatedearlierandstarttheVMconsoleusingthefollowingcommand:$oakclishowvmconsoledbvm13.
ThefirstbootprocessstartstheconfigurationoftheVM.
Thepasswordsforroot,oracleandapplmgruserswillhavetobechangedwhenprompted.
TheconfigurationprocesswillthencontinuetosetupthenetworkrequiringinputsforIPaddress,netmask,gateway,DNSserveraddressandhostname.
Finally,theE-BusinessSuitedatabaseconfigurationstartsbypromptingforthedatabaseSID.
Changingpasswordforuserroot.
Newpassword:Retypenewpassword:passwd:allauthenticationtokensupdatedsuccessfully.
9|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCEChangingpasswordforuseroracle.
Newpassword:Retypenewpassword:passwd:allauthenticationtokensupdatedsuccessfully.
Changingpasswordforuserapplmgr.
Newpassword:Retypenewpassword:passwd:allauthenticationtokensupdatedsuccessfully.
ConfigureaStaticIPConfiguringnetworkinterface.
Networkdevice:eth0Hardwareaddress:00:16:3E:53:49:0CEnterstaticIPaddress:1.
2.
3.
4Enternetmask:[255.
0.
0.
0]255.
255.
255.
0Entergateway:1.
2.
3.
1EnterDNSserver:1.
2.
3.
1Shuttingdowninterfaceeth0:[OK]Shuttingdownloopbackinterface:[OK]Configuringnetworksettings.
IPconfiguration:StaticIPaddressBringinguploopbackinterface:[OK]Bringingupinterfaceeth0:Determiningifipaddress1.
2.
3.
4isalreadyinusefordeviceeth0.
.
.
[OK]Enterhostname(e.
g,host.
example.
com):[dbhost.
us.
oracle.
com]Networkconfigurationchangedsuccessfully.
IPconfiguration:StaticIPaddressIPaddress:1.
2.
3.
4Netmask:255.
255.
255.
0Gateway:1.
2.
3.
1DNSserver:1.
2.
3.
1Hostname:dbhost.
us.
oracle.
comSTEP4:MigratetheDatabasetoODA_BASE1.
LogintothesourcedatabaseVMastheoracleuser.
Sourcetheenvironmentusingthefollowingcommand:$.
/u01/install/PROD/11.
2.
0/PROD_dbhost.
env2.
CreateapasswordfilefortheOracledatabaseinordertofacilitateremotesysdbalogin(thisisrequiredforsubsequentdatamigrationoperations):$$ORACLE_HOME/bin/orapwdfile=$ORACLE_HOME/dbs/orapwPRODpassword=welcome13.
OracleDatabaseduplicationproceduresaresimplerwhenusinganspfile.
Createanspfileforthedatabaseasfollows:$sqlplus/assysdbaSQL>createspfilefrompfile;10|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCE4.
Preparethesourcedatabasetierforcloning:$cd$ORACLE_HOME/appsutil/scripts/PROD_dbhost$perladpreclone.
pldbTier5.
TheAutoConfigfileswillneedtobeinstalledinthetargetORACLE_HOME.
Asafirststep,createanewzipfileoftheappsutildirectoryinsourceORACLE_HOMEandcopyittoODA_BASE:$cd$ORACLE_HOME$zip–rappsutil.
zipappsutil$scpappsutil.
zipnode1:/u01/app/oracle/product/11.
2.
0.
4/dbhome_1/6.
Shutdownthesourcedatabaseandstartitinmountstateusingthefollowingcommand:$sqlplus/assysdbaSQL>shutdownimmediateSQL>startupmount7.
LoginastheOracleusertoODA_BASEonnode1andinstalltheAutoConfigfilesasfollows:$cd$ORACLE_HOME$unzipappsutil.
zip8.
OracleDatabaseAppliancehasanoptimalconfigurationofHDD&SSDdisks.
Thedatabasehastobelocatedsuchthatunderlyingstoragesubsystemisutilizedoptimally.
FilesystemsmustbecreatedandmountedtohostdatafilesonHDDsandredologsonSSDs.
ThefollowingcommandcreatesthestoragestructureforadatabasenamedPROD:$oakclicreatedbstorage–dbPROD9.
Preparethetargetdatabasetier.
WhileconfiguringtheOracleApplicationsenvironmentchoose"/u02/app/oracle/oradata/datastore/.
ACFS/snaps/PROD"ACFSfilesystemtobetheDATA_TOP.
WhenpromptedforORACLERACdatabase,respondwithNoasthedatabasewillbeconvertedtoORACLERACatalaterstage.
Also,chooseanon-zeroportpoolsothatitdoesnotclashwiththeexistinglistenersettingsontheOracleDatabaseAppliance.
ThisprocedureusesPRODasthedatabaseSID:$cd$ORACLE_HOME/appsutil/clone/bin$perladcfgclone.
pldbTechStack10.
WhenOracleDatabaseApplianceisdeployed,RoleSeparationusesdifferentownersfortheGridInfrastructureandOracleDatabase.
RunnngAutoconfigwillre-generatetheOraclebinaryin$ORACLE_HOME/binanditspermissionswillberesettothedefaultownership.
YouneedtoresettheOraclebinarypermissionsforthegriduseronbothnode1andnode2,usingthefollowingcommand:$/u01/app/12.
1.
0.
2/grid/bin/setasmgidwrap\o=/u01/app/oracle/product/11.
2.
0.
4/dbhome_1/bin/oracle11.
Createthepasswordfileonthetargetdatabasehomeusingthefollowingcommandastheoracleuseronnode1:$$ORACLE_HOME/bin/orapwdfile=$ORACLE_HOME/dbs/orapwPRODpassword=welcome112.
SetuptheApplicationsenvironmentandstartthetargetdatabaseinstanceinanomountstate:$.
$ORACLE_HOME/PROD_node1.
env$$ORACLE_HOME/bin/sqlplus/assysdbaSQL>startupnomount13.
AddtheTNSaliasforbothsourceandtargetdatabasesinthetnsnames.
oraofboththesource&targetORACLE_HOMEs.
Forexample,PROD_DBHOST=(DESCRIPTION=(ADDRESS=11|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCE(PROTOCOL=tcp)(HOST=dbhost)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=PROD)))PROD_ODABASE=(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=node1)(PORT=1529))(CONNECT_DATA=(SERVICE_NAME=PROD)))14.
ConnecttoboththesourceanddestinationdatabaseinstancesusingRMAN.
ThesourcedatabaseisthetargetdatabaseandthedestinationdatabaseistheauxiliarydatabaseinRMANterminology.
$$ORACLE_HOME/bin/rmantargetsys/welcome1@prod_dbhost\auxiliarysys/welcome1@prod_odabase15.
DuplicatethedatabaseusingthefollowingRMANscript.
EnsurethattheparametersLOCAL_LISTENER,AUDIT_FILE_DEST,DIAGNOSTIC_DEST,CONTROL_FILESandDB_UNIQUE_NAMEcorrespondtothevaluesforyourenvironment.
TheremainderoftheparametershavethesamevaluesacrossalltheOracleDatabaseApplianceenvironments.
RMAN>run{allocatechanneltgt1devicetypedisk;allocateauxiliarychannelaux1devicetypedisk;allocatechanneltgt2devicetypedisk;allocateauxiliarychannelaux2devicetypedisk;allocatechanneltgt3devicetypedisk;allocateauxiliarychannelaux3devicetypedisk;allocatechanneltgt4devicetypedisk;allocateauxiliarychannelaux4devicetypedisk;DUPLICATETARGETDATABASETOPRODFROMACTIVEDATABASESPFILESETDB_UNIQUE_NAME='PROD'SETCLUSTER_DATABASE='false'SETDB_CREATE_FILE_DEST="/u02/app/oracle/oradata/datastore/.
ACFS/snaps/PROD"SETDB_RECOVERY_FILE_DEST="/u01/app/oracle/fast_recovery_area/datastore"SETDB_RECOVERY_FILE_DEST_SIZE="500G"SETDB_CREATE_ONLINE_LOG_DEST_1="/u01/app/oracle/oradata/datastore"SETCONTROL_FILES="/u01/app/oracle/oradata/datastore/PROD/control.
ctl"SETDB_BLOCK_CHECKSUM='FULL'SETDB_BLOCK_CHECKING='FULL'SETDB_LOST_WRITE_PROTECT='TYPICAL'SETFILESYSTEMIO_OPTIONS='SETALL'SETUSE_LARGE_PAGES='ONLY'SETCOMPATIBLE='11.
2.
0.
4'SETPARALLEL_ADAPTIVE_MULTI_USER='FALSE'SETPARALLEL_EXECUTION_MESSAGE_SIZE='16384'SETPARALLEL_THREADS_PER_CPU='2'SETPARALLEL_MIN_SERVERS='0'SETDIAGNOSTIC_DEST='/u01/app/oracle/product/11.
2.
0.
4/dbhome_1/admin/PROD_node1'SETAUDIT_FILE_DEST='/u01/app/oracle/product/11.
2.
0.
4/dbhome_1/rdbms/audit'SETLOG_ARCHIVE_DEST_1=''SETLOCAL_LISTENER='PROD_LOCAL'12|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCENOFILENAMECHECK;}16.
Runthelibraryupdatescriptonthetargetdatabase.
$cd$ORACLE_HOME/appsutil/install/PROD_node1$sqlplus/assysdba@adupdlib.
sqlso17.
Configurethetargetdatabasetier.
$cd$ORACLE_HOME/appsutil/clone/bin$perladcfgclone.
pldbconfig$ORACLE_HOME/appsutil/PROD_node1.
xml18.
TheAutoconfigtooldoesnotgeneratethecorrectconfig.
cin$ORACLE_HOME/rdbms/lib/directory.
Workaroundistomakethefollowingchangestoconfig.
c:a.
Addanewline:#defineSS_ASM_GRP""b.
Changethefollowingline:char*ss_dba_grp[]={SS_DBA_GRP,SS_OPER_GRP};to:char*ss_dba_grp[]={SS_DBA_GRP,SS_OPER_GRP,SS_ASM_GRP};c.
Generateconfig.
o:$make-f$ORACLE_HOME/rdbms/lib/ins_rdbms.
mk$ORACLE_HOME/rdbms/lib/config.
od.
Generateosdbagrpbinaryagain:$cd$ORACLE_HOME/srvm/lib$make-fins_srvm.
mkiosdbagrp19.
RegisterdatabasewithCRS.
$srvctladddatabase-dPROD-o/u01/app/oracle/product/11.
2.
0.
4/dbhome_1–p/u02/app/oracle/oradata/datastore/PROD/spfilePROD.
ora-csingle-xnode1$srvctlsetenvdatabase–dPROD–TORA_NLS10=$ORACLE_HOME/nls/data/9idata$srvctlsetenvdatabase–dPROD–TTNS_ADMIN=$ORACLE_HOME/network/admin/PROD_node1$srvctlstartdatabase–dPROD20.
Updatethefollowinginit.
oraparametersinaSQL*Plussession:SQL>altersystemset"_disable_interface_checking"=truescope=spfilesid='*';SQL>altersystemset"_gc_undo_affinity"=falsescope=spfilesid='*';SQL>altersystemset"_gc_policy_time"=0scope=spfilesid='*';SQL>altersystemset"_enable_numa_support"=falsescope=spfilesid='*';SQL>altersystemset"_file_size_increase_increment"=2143289344scope=spfilesid='*';21.
Restartthedatabasefortheparametertotakeeffect:$srvctlstopdatabase–dPROD$srvctlstartdatabase–dPRODSTEP5:SetuptheOracleE-BusinessSuiteApplicationsTierVM13|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCE1.
OracleVMTemplatesforOracleE-BusinessSuiteRelease12.
2.
4MediaPackv1forx86(64bit)areavailableat:https://edelivery.
oracle.
com/linux.
Thestagingdirectory/cloudfswouldrequire75GBfreespace.
Downloadthefollowingfilestoastagingareaunderthe/cloudfsdirectoryonnode1:V59638-01_1of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ApplicationTierInstallX86(64bit)(Part1of2)V59638-01_2of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ApplicationTierInstallX86(64bit)(Part2of2)V59637-01_1of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ApplicationTierInstallX86(64bit)(Part1of2)V59637-01_2of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ApplicationTierInstallX86(64bit)(Part2of2)V59636-01_1of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ApplicationTierInstallX86(64bit)(Part1of2)V59636-01_2of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ApplicationTierInstallX86(64bit)(Part2of2)V59635-01_1of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ApplicationTierInstallX86(64bit)(Part1of2)V59635-01_2of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ApplicationTierInstallX86(64bit)(Part2of2)V59639-01_1of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ApplicationTierInstallX86(64bit)(Part1of2)V59639-01_2of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ApplicationTierInstallX86(64bit)(Part2of2)2.
Unzipthefilesusingthefollowingcommands:$unzipV59638-01_1of2.
zip$unzipV59638-01_2of2.
zip$unzipV59637-01_1of2.
zip$unzipV59637-01_2of2.
zip$unzipV59636-01_1of2.
zip$unzipV59636-01_2of2.
zip$unzipV59635-01_1of2.
zip$unzipV59635-01_2of2.
zip$unzipV59639-01_1of2.
zip$unzipV59639-01_2of2.
zip3.
Combinethesplitfilesintoasinglecompressedfile.
$cat\Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova.
0\Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova.
1\Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova.
2\Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova.
3\Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova.
4\Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova.
5\Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova.
6\Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova.
7\Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova.
8\Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova.
9\>Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova4.
BeforethedownloadedApplicationsVMtemplatecanbeusedtostartaVM,itneedstobestagedwithintheODA.
Itrequires285GBofrepositoryspace.
Astherootuser,importtheVMtemplateusingthefollowingcommand:$oakcliimportvmtemplateapptempl–assembly\https:/node1:8000/Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova\–repoappsrepo–node05.
Astherootuser,usetheoakcliclonecommandtocreatetheVMasfollows:$oakcliclonevmappvm-vmtemplateapptempl-repoappsrepo-node1\-driverdomain6.
AstherootuserconfirmtheVMhasbeencreatedusingthefollowingcommand:$oakclishowvm14|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCE$oakclishowvmappvmResource:appvmAutoStart:restoreCPUPriority:100Disks:|file:/OVS/Repositories/appsrepo/.
ACFS/snaps/appvm/VirtualMachines/appvm/d84a641d07a340dfa550bf3518244896.
img,xvda,w||,xvdb:/OVS/Repositories/appsrepo/.
ACFS/snaps/appvm/VirtualMachines/appvm/cdrom,r|Domain:XEN_PVMExpectedState:onlineFailOver:falseIsSharedRepo:trueKeyboard:en-usMaxMemory:16384MaxVcpu:8Memory:16384Mouse:OS_DEFAULTName:appvmNetworks:NodeNumStart:0OS:OL_5PrefNodeNum:0PrivateIP:NoneProcessorCap:100RepoName:appsrepoSource:0State:OnlineTemplateName:otml_apptemplVDisks:|0|Vcpu:8cpupool:default-unpinned-poolvncport:None7.
ReconfiguretheVMtousenet1asthepublicinterface.
Astherootuser,runthefollowingcommand:$oakclimodifyvmappvm-addnetworknet1$oakclishowvmappvmResource:appvmAutoStart:restoreCPUPriority:100Disks:|file:/OVS/Repositories/appsrepo/.
ACFS/snaps/appvm/VirtualMachines/appvm/d84a641d07a340dfa550bf3518244896.
img,xvda,w||,xvdb:/OVS/Repositories/appsrepo/.
ACFS/snaps/appvm/VirtualMachines/appvm/cdrom,r|Domain:XEN_PVMExpectedState:onlineFailOver:falseIsSharedRepo:trueKeyboard:en-usMaxMemory:16384MaxVcpu:8Memory:16384Mouse:OS_DEFAULTName:appvmNetworks:['bridge=net1']NodeNumStart:0OS:OL_5PrefNodeNum:015|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCEPrivateIP:NoneProcessorCap:100RepoName:appsrepoSource:0State:OnlineTemplateName:otml_apptemplVDisks:|0|Vcpu:8cpupool:default-unpinned-poolvncport:None8.
Duetobug:21101080,theVMconfigurationhasanincorrectentryforcdrominthevm.
cfgfile.
Thishastobecorrectedmanually.
Editthevm.
cfgandchangefollowing:xvdb:/OVS/Repositories/appsrepo/.
ACFS/snaps/appvm/VirtualMachines/appvm/cdrom,rto:xvdb:cdrom,r9.
TheApplicationsVMcannowbestartedusingthefollowingcommandastherootuser:$oakclistartvmappvm10.
TheApplicationsVMrequiresthattheApplicationsenvironmentnetworkisconfiguredinthesamewayasforthedatabaseVM.
Inordertodothis,youneedtouseTheVMconsole,whichisaccessedastherootuserwithintheVNCsessionusingthefollowingcommand:$oakclishowvmconsoleappvm11.
ThefirstbootprocessstartstheconfigurationoftheVM.
Thepasswordsforroot,oracleandapplmgruserswillhavetobechangedwhenprompted.
TheconfigurationprocesswillthencontinuetosetupthenetworkrequiringinputsforIPaddress,netmask,gateway,DNSserveraddressandhostname.
WhentheOracleE-BusinessSuiteApplicationstierisstartedyouwillneedtosupplytheDatabaseHostIP,DatabaseHostname,DatabaseHostdomain,DatabaseSIDandListenerPort.
Changingpasswordforuserroot.
Newpassword:Retypenewpassword:passwd:allauthenticationtokensupdatedsuccessfully.
Changingpasswordforuseroracle.
Newpassword:Retypenewpassword:passwd:allauthenticationtokensupdatedsuccessfully.
Changingpasswordforuserapplmgr.
Newpassword:Retypenewpassword:passwd:allauthenticationtokensupdatedsuccessfully.
ConfigureaStaticIPConfiguringnetworkinterface.
Networkdevice:eth0Hardwareaddress:00:16:3E:53:49:0CEnterstaticIPaddress:1.
2.
3.
5Enternetmask:[255.
0.
0.
0]255.
255.
255.
016|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCEEntergateway:1.
2.
3.
1EnterDNSserver:1.
2.
3.
1Shuttingdowninterfaceeth0:[OK]Shuttingdownloopbackinterface:[OK]Configuringnetworksettings.
IPconfiguration:StaticIPaddressBringinguploopbackinterface:[OK]Bringingupinterfaceeth0:Determiningifipaddress1.
2.
3.
5isalreadyinusefordeviceeth0.
.
.
[OK]Enterhostname(e.
g,host.
example.
com):[apphost.
us.
oracle.
com]Networkconfigurationchangedsuccessfully.
IPconfiguration:StaticIPaddressIPaddress:1.
2.
3.
5Netmask:255.
255.
255.
0Gateway:1.
2.
3.
1DNSserver:1.
2.
3.
1Hostname:apphost.
us.
oracle.
comEntertheIPAddressoftheDBTierHost:1.
2.
3.
4EntertheDatabaseTierHostName(withoutthedomain):dbhostEntertheDatabaseTierDomainName:us.
oracle.
comEntertheOracleDatabaseSID:PRODEntertheTNSListenerPortNumber:1521STEP6:EnableHighAvailability1.
ToprovideHighAvailability(HA)attheApplicationslayer,anadditionalApplicationsVMneedstobedeployedasdescribedinSTEP5.
6:SetupOracleE-BusinessSuiteApplicationsVM.
Repeatsteps5.
6to5.
11onthesecondnodeintheappliance.
2.
ConvertingasingledatabaseonOracleDatabaseAppliancetoOracleRACusesstandardOracleE-BusinessSuiteproceduresasprovidedinSection4oftheMOSKnowledgeDocument1453213.
1,UsingOracle11gRelease2RealApplicationClustersandAutomaticstoragemanagementwithOracleE-BusinessSuiteRelease12.
2.
EnsuretherequirementslistedinSection4.
1Pre-RequisitesaremetpriortomigratingtoOracleRACdatabase.
Astheclusterwarehasalreadybeeninstalled,theprocedureisfurthersimplified.
FollowthestepsfromSection4.
4ListenerConfigurationin11gR2toSection4.
7EstablishApplicationsEnvironmentforOracleRACtoconvertthedatabasetoORACLERAC.
3.
Loadbalancingtheapplicationserversprovidesbothprotectionagainstfailureaswellasincreasedscalability.
OracleE-BusinessSuitesupportsloadbalancingatmultiplelayers,including,forexample,DNSandWebServices.
OracleDatabaseApplianceputsnospecificrestrictionintheconfigurationoftheLoadBalancersforOracleE-BusinessSuite.
MOSKnowledgeDocument380489.
1:UsingLoad-BalancerswithOracleE-BusinessSuiteRelease12providesdetailedinformationonconfiguringloadbalancing.
STEP7:Cleanup17|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCE1.
ThepythonHTTPserverprocesscanbekilled.
$kill-92.
Filesthatweredownloaded,uncompressedandcombinedintheprocessofcreatingthedatabaseandApplicationsVMscannowbedeletedfromthe/cloudfsdirectory.
3.
SincethedatabasehasbeenmigratedtoODA_BASE,thedatabaseVMcanbedeletedusingthefollowingcommand:$oakclistopvmdbvm$oakclideletevmdbvm4.
ThedatabaseVMtemplatecanalsoberemoved.
Toremovethetemplate,usethefollowingcommand:$oakclideletevmtemplatedbvmSTEP8:ValidateEnvironment5.
ThepresenceofApplicationsVMs–runthefollowingcommandasrootuseronODA_BASEonnode1:$oakclishowvm6.
Thepresenceofaclusterdatabase–runthefollowingcommandasoracleuseronODA_BASEonnode1:$srvctlconfigdatabase–dPROD–a7.
TheapplicationavailabilitycanbevalidatedbyloggingintotheapplicationusingtheURLhttp://app1.
company.
com:8008/18|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCEBestPracticesThissectionliststhebestpractices,whichshouldbereviewedoncethedeploymentiscomplete.
ConsiderResizingtheResourceConfigurationforODA_BASEAreasonablestartingpointistoallocate50%oftheavailableCPUandmemoryresourcestoODA_BASE.
Thesecanbemodifiedtoreflectcapacityrequirementswhileremainingwithinthesoftwarelicensingrestrictions.
ThenumberofCPUcoresandmemoryallocationscanbechangedusingthefollowingcommandastherootuserondom0(onanX4-2):$oakcliconfigureoda_baseCoreLicensingOptions:1.
2CPUCores2.
4CPUCores3.
6CPUCores4.
8CPUCores5.
10CPUCores6.
12CPUCores7.
16CPUCores8.
24CPUCores9.
36CPUCoresCurrentCPUCores:16Selection[1.
.
9](default36CPUCores):7ODAbasedomainmemoryinGB(min16,max244)(CurrentMemory128G)[default192]:Thecorecountsabovereflectphysicalcores.
TheOSinsideODA_BASEwillreportvirtualcores(vcpu),whichequatestoahalfphysicalcore.
ThechangesaremadetoODA_BASEonbothnodes;theresourceallocationshouldbethesameacrossboth.
ThechangestakeeffectwhentheODA_BASEisrestarted.
TorestarttheODABase,astherootuserexecutethefollowingcommandoneachnode:$oakclirestartoda_baseConsiderResizingtheResourceConfigurationfortheApplicationsVMsAswiththeOracleDatabase,theCPUandmemoryallocationfortheApplicationsVMscanalsobereconfiguredaftertheinitialdeployment.
AsVMsaremanagedfromODA_BASE,toreconfiguretheresourcesforanApplicationsVM,thecommandisexecutedastherootuserfromODA_BASE.
UsethefollowingexamplecommandtochangetheCPUandmemoryallocationforanApplicationsVM:$oakcliconfigurevmappvm–vcpu32–maxcpu32$oakcliconfigurevmappvm–memory96G–maxmemory96GThiscommandreferencesvirtualcpus(vcpu).
ThechangesarestaticinnatureandrequiretheVMtoberestarted.
TorestarttheVMs,usethefollowingcommands:$oakclistopvmappvm$oakclistartvmappvmConsiderImplementingWorkloadIsolationusingCPUPoolsOracleDatabaseAppliancehastheabilitytoisolateworkloadswithinCPUpools.
Onceassigned,aVMisconstrainedtoonlyusetheCPUsavailablewithinthatpool.
TwopoolsarecreatedbydefaultwhenOracleDatabaseApplianceisdeployed:odaBaseCpuPoolanddefault-unpinned-pool.
TheODA_BASEdomainisallocatedtotheodaBaseCpuPool,andthenumberofthevCPUsintheodaBaseCpuPoolisequaltothevCPUsallocatedtoODA_BASEduringdeploymentorre-configuration.
19|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCETheApplicationsVMscanalsobenefitfromCPUisolationbycreatingadditionalCPUPools.
OncethesepoolsarecreatedandtheApplicationsVMsareassignedtothesepools,nootherVMcanusethesevCPUrescources.
AdditionalCPUpoolshavetobecreatedmanuallyoneachODA_BASEastheCPUpoolisnodespecific.
Astherootuser,usethefollowingcommandtocreateaCPUpool:$oakclicreatecpupoolappVmPool–numcpu4–node0TovalidatethecreationoftheCPUpool:$oakclishowcpupool–node0TheApplicationsVMcanthenbeassignedtothenewCPUpoolasfollows:$oakcliconfigurevmappvm-cpupoolappVmPoolPatchingOracleDatabaseApplianceOracleDatabaseApplianceprovidesaninfrastructureforsimplifiedpatchingastherequisitepatchesforallcomponentsoftheOracleDatabaseAppliancestack(excludingApplicationsVMpatches)areincludedinasinglebundlepatch.
Thisvastlysimplifiestheend-to-endpatchingofbothhardwareandsoftwaresinceasinglecommandisusedtopatchallthecomponents.
Patchesfordifferentcomponentspresentwithinabundlepatchhavebeenvalidatedtoworkwitheachother.
ThiseliminatesanyadversepatchconflictsforeachofthecomponentsonOracleDatabaseApplianceandensuresend-to-endcompatibilityafterpatching.
TheOracleDatabaseAppliancepatchbundlesarereleasedregularlytokeepup-to-datewiththeOracleDatabasePatchesSetUpdatereleases.
ItisrecommendedthatyounotapplyanyGridInfrastructure/Databaseone-offpatchesorinstallanyadditionalOSpackagesoutsideofthestandardOracleDatabaseAppliancepatchingmechanism.
However,OracleE-BusinessSuitefunctionalpatchesareexcludedfromthispolicy.
Intherareeventofaconflictbetweenaone-offRDBMSpatchandtheOracleDatabaseAppliancepatchbundle,therecommendedapproachtoaddressthisistorollbacktheproblempatchpriortoapplyingtheOracleDatabaseApplianceBundlePatch.
IfthepatchhasnotbeenincludedintheOracleDatabaseAppliancebundlepatch,thenitcanbereapplied.
TheOracleE-BusinessSuiteApplicationstiercanbepatchedusingstandardpatchingutilities.
OracleDatabaseApplianceputsnorestrictiononpatchingtheApplicationstiers.
Forfurtherdetailsonpatching,pleaserefertotheMaintenanceGuideintheOracleE-BusinessSuitedocumentationat:http://docs.
oracle.
com/cd/V39571_01/current/html/docset.
htmlSystemSizingGeneralguidelinesonsystemsizingandcapacitycanbefoundintheOracleDatabaseAppliancedocumentation,availableat:http://docs.
oracle.
com/cd/E22693_01/index.
htmOracleE-BusinessSuiteworkloadscanvarywidelydependingonmanyfactors,suchasmodulesdeployed,userspermoduleandbusinesscycles.
WhenmigratinglegacysystemstoOracleDatabaseAppliance,theexistingpeakCPU,memory,IOPSandstoragespaceutilizationcanbeusedasguidancetoproperlysizethenewenvironment.
YoushouldworkwithyourlocalOraclerepresentativeonthesizingexercise.
EnableDriverDomainDriverDomainisanewfeatureavailableinOAKrelease12.
1.
2.
7.
0andhigher.
Adriverdomainisauserdomainthathasbeengivenspecialprivilegesandresponsibilitiesforaparticularpieceofhardwaretherebyeliminatingdom0asabottleneck.
TheDriverDomainfunctionalityallowsfor20|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCEimplementationofguestvirtualmachinestouseODA_BASEasthedriverdomainforaccesstovirtualdisks.
OneofthelimitationsofthecurrentimplementationofDriverDomainsonODAislackofsupportforbootpartitioninadiskimageaccessedviadriverdomain.
TheE-BusinessSuiteVMsprovideasinglediskimagewhichincludesbothbootpartitionandtheapplicationpartitiononthesamedisk.
Thisconfigurationeffectivelyexcludesout-of-the-boximplementationDriverDomainsfortheE-BusinessSuiteapplicationsVMs.
DriverDomainscanbeimplementedonE-BusinessSuiteapplicationVMsbymovingoutthebootpartitiontoadifferentdiskimage.
Thefollowingstepsdetailtheprocess:1.
Createanewbootdiskofsize500Minthesamelocationastheoriginal.
ddif=/dev/zeroof=/u01/app/sharedrepo/ebsrepo/.
ACFS/snaps/appvm/VirtualMachines/appvm/boot.
imgoflag=directcount=0bs=1Mseek=500M2.
Attachbootdisktothevmbyupdatingdiskparameterinthevm.
cfgdisk=[u'file:/u01/app/sharedrepo/ebsrepo/.
ACFS/snaps/ebsapp/VirtualMachines/appvm/99a674656e49406382ac741f07a7a582.
img,xvda,w',u'file:/OVS/Repositories/ebsrepo/.
ACFS/snaps/appvm/VirtualMachines/appvm/boot.
img,xvdb,w']3.
Restartapplicationvmoakclistopvmappvmoakclistartvmappvm4.
Partitionthenewdiskandmakeitbootableecho-e"n\np\n1\n\n\nt\n83\na\n1\nw"|fdisk/dev/xvdc5.
Formatthenewpartitionmkfs.
ext4/dev/xvdb16.
Mountthenewfilesystemmount/dev/xvdb1/mnt7.
Copyeverytingunder/boottonewfilesystemcp–dpR/boot/*/mnt8.
Mountthenewfilesystemas/bootumount/mntumount/bootmount/dev/xvb1/boot9.
Installgrubinthenewbootdisk$grubgrub>device(hd0)/dev/xvdbgrub>root(hd0,0)grub>setup(hd0)10.
GettheUUIDofthenewbootpartitionblkid/dev/xvdb11.
Updatethe/bootentryin/etc/fstabwithUUIDofthenewpartitionUUID=cc3635e4-4030-4881-91b0-a8279ddb1ef0/bootext4defaults1221|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCE12.
Editthevm.
cfgtoupdatethedriverdomaininformationforeachdisk.
Thebootpartitionshouldrefertothe/OVS/Repositorieslocationandtherootpartitionshouldrefertooda_baselocationasfollows:disk=[u'file:/u01/app/sharedrepo/ebsrepo/.
ACFS/snaps/appvm/VirtualMachines/ebsdb/471489424d0e46a6a3cd6fc53f4370ee.
img,xvdb,w,oakDom1',u'file:/OVS/Repositories/ebsrepo/.
ACFS/snaps/appvm/VirtualMachines/appvm/boot.
img,xvdb,w']13.
RestarttheapplicationVMoakclistopvmappvmoakclistartvmappvmConclusionOracleDatabaseApplianceisidealforsmalltomediumsizedeploymentsofOracleE-BusinessSuitedeployments.
ItenablestheOracleDatabaseandOracleE-BusinessSuiteApplicationstiertobedeployedinasinglecosteffectiveandflexiblesystemthatisengineeredforsimplicityandreliability.
22|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCEAppendixA:References1.
OracleDatabaseAppliancedocumentation:http://docs.
oracle.
com/cd/E22693_01/index.
htm2.
OracleE-BusinessSuitedocumentation:http://docs.
oracle.
com/cd/V39571_01/current/html/docset.
html3.
MOSKnowledgeDocument1330701.
1:OracleE-BusinessSuiteInstallationandUpgradeNotesRelease12(12.
2)forLinuxx86-644.
MOSKnowledgeDocument1349240.
1:DatabasePreparationGuidelinesforanE-BusinessSuiteRelease12.
2Upgrade5.
MOSKnowledgeDocument1453213.
1:UsingOracle11gRelease2RealApplicationClustersandAutomaticstoragemanagementwithOracleE-BusinessSuiteRelease12.
26.
MOSKnowledgeDocument1566935.
1:ImplementingOracleE-BusinessSuite12.
1DatabasesonOracleDatabaseAppliance7.
MOSKnowledgeDocument380489.
1:UsingLoad-BalancerswithOracleE-BusinessSuiteRelease128.
MOSKnowledgeDocument1147107.
1:DatabasePatchSetUpdateOverlayPatchesRequiredforUsewithPSUsandOracleE-BusinessSuite23|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCEAppendixB:ConfiguringVMsafterFirstBootTheVirtualMachinesareconfiguredduringthefirstboot.
TheE-BusinessSuiteVMshavescriptsthatcanbeusedtoreconfiguretheVMsafterthefirstbootorafterafailureofconfigurationduringfirstboot.
Thesescriptsarepresentinthefollowinglocations:OnbothDataabaseandApplicationsVM:SCRIPTSBASE_DIR:/u01/install/scriptsCONFIGUREDHCP:/u01/install/scripts/configdhcp.
shCONFIGURESTATICIP:/u01/install/scripts/configstatic.
shSCRIPTTOCLEANUPVM:/u01/install/scripts/cleanup.
shOnDatabaseVM:SCRIPTSBASE_DIR:/u01/install/PROD/scripts/STARTSCRIPT:/u01/install/PROD/scripts/startproddb.
shSTOPSCRIPT:/u01/install/PROD/scripts/stopproddb.
shDBVMRECONFIGSCRIPT:/u01/install/PROD/scripts/prodbconfig.
shDBVMCLEANUPSCRIPT:/u01/install/PROD/scripts/proddbcleanup.
shOnApplicationsVM:SCRIPTSBASE_DIR:/u01/install/APPS/scripts/STARTSCRIPT:/u01/install/APPS/scripts/startapps.
shSTOPSCRIPT:/u01/install/APPS/scripts/stopapps.
shAPPSVMRECONFIGSCRIPT:/u01/install/APPS/scripts/appsconfig.
shAPPSVMCLEANUPSCRIPT:/u01/install/APPS/scripts/appscleanup.
shCONFIGUREANEWWEBENTRYPOINT:/u01/install/scripts/configwebentry.
sh2|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCEOracleCorporation,WorldHeadquarters500OracleParkwayRedwoodShores,CA94065,USAWorldwideInquiriesPhone:+1.
650.
506.
7000Fax:+1.
650.
506.
7200Copyright2015,Oracleand/oritsaffiliates.
Allrightsreserved.
Thisdocumentisprovidedforinformationpurposesonly,andthecontentshereofaresubjecttochangewithoutnotice.
Thisdocumentisnotwarrantedtobeerror-free,norsubjecttoanyotherwarrantiesorconditions,whetherexpressedorallyorimpliedinlaw,includingimpliedwarrantiesandconditionsofmerchantabilityorfitnessforaparticularpurpose.
Wespecificallydisclaimanyliabilitywithrespecttothisdocument,andnocontractualobligationsareformedeitherdirectlyorindirectlybythisdocument.
Thisdocumentmaynotbereproducedortransmittedinanyformorbyanymeans,electronicormechanical,foranypurpose,withoutourpriorwrittenpermission.
OracleandJavaareregisteredtrademarksofOracleand/oritsaffiliates.
Othernamesmaybetrademarksoftheirrespectiveowners.
IntelandIntelXeonaretrademarksorregisteredtrademarksofIntelCorporation.
AllSPARCtrademarksareusedunderlicenseandaretrademarksorregisteredtrademarksofSPARCInternational,Inc.
AMD,Opteron,theAMDlogo,andtheAMDOpteronlogoaretrademarksorregisteredtrademarksofAdvancedMicroDevices.
UNIXisaregisteredtrademarkofTheOpenGroup.
0115Solution-in-a-box:BestpracticesfordeployingOracleE-BusinessSuiteRelease12.
2.
4onOracleDatabaseAppliance12.
1.
2.
7July2016Author:RamasubramanianAthmanathanContributingAuthors:AndyTremayne,RACPack,E-BusinessSuiteProductTeamCONNECTWITHUSblogs.
oracle.
com/oraclefacebook.
com/oracletwitter.
com/oracleoracle.
com
2.
4onOracleDatabaseAppliance12.
1.
2.
7ORACLEWHITEPAPER|JULY20161|ENTERTITLEOFDOCUMENTHERETableofContentsExecutiveOverview2Scope2OracleDatabaseApplianceVirtualizedPlatformDeploymentArchitecture2Softwareoneachphysicalserver4InstallingOracleE-BusinessSuite12.
2.
4onODAVirtualizedPlatform4STEP1:DeployOracleDatabaseApplianceVirtualizedPlatform4STEP2:PreparetheTargetORACLE_HOME5STEP3:SetuptheOracleE-BusinessSuiteDatabaseVM6STEP4:MigratetheDatabasetoODA_BASE9STEP5:SetuptheOracleE-BusinessSuiteApplicationsTierVM12STEP6:EnableHighAvailability16STEP7:Cleanup16STEP8:ValidateEnvironment17BestPractices18ConsiderResizingtheResourceConfigurationforODA_BASE18ConsiderResizingtheResourceConfigurationfortheApplicationsVMs18ConsiderImplementingWorkloadIsolationusingCPUPools18PatchingOracleDatabaseAppliance19SystemSizing19Conclusion21AppendixA:References22AppendixB:ConfiguringVMsafterFirstBoot232|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCEExecutiveOverviewOracleE-BusinessSuiteisthemostcomprehensivesuiteofintegrated,globalbusinessapplicationsthatenableorganizationstomakebetterdecisions,reducecosts,andincreaseperformance.
OracleDatabaseApplianceisanOracleengineeredsystemconsistingofhardwareandsoftwaredesignedtosimplifythedeployment,maintenanceandsupportofhighavailabilityOracleDatabaseandOracleE-BusinessSuiteApplicationsTiersolutions.
OracleDatabaseApplianceofferstheoptiontodeployhighlyavailableOracleDatabasesinabaremetalenvironmentortodeploybothOracleDatabasesandtheApplicationsTierinvirtualizedenvironmentswithintheappliance.
ThispaperdescribeshowtorapidlydeployOracleE-BusinessSuite12.
2.
4databaseandApplicationstiersonOracleDatabaseApplianceVirtualizedPlatform12.
1.
2.
7(andlater)usingthepreconfiguredOracleE-BusinessSuiteVMtemplates.
EachOracleE-BusinessSuitedeploymentconsistsoftwoOracleVMtemplates:adatabaseimagetemplateandaprimaryApplicationstiertemplate.
Thesetemplatesdeliverthecompletesoftwarestackandarepre-configuredforaVisiondemowithsampledataorafreshinstalldatabaseforproductionenvironment.
Theprocedureinthispaperprovidesasimple,reliableandcosteffectivesolution-in-a-boxwhichisfullysupportedforproductionuse.
ScopeThispaperdescribeshowtoinstallOracleE-BusinessSuiteRelease12.
2.
4databaseandApplicationstiersontheOracleDatabaseApplianceVirtualizedPlatform12.
1.
2.
7(andlater)usingOracleE-BusinessSuitetemplates.
ThisdocumentisanupdatetoincludeOracleACFSsupportwithlatestversionsofOracleDatabaseAppliancewithsupportforDriverDomains.
YouneedtobefamiliarwiththeOracleDatabaseAppliancedocumentationand,inparticular,theOracleDatabaseApplianceVirtualizedPlatformdocumentation.
FurtherinformationabouttheOracleDatabaseApplianceisavailableat:http://docs.
oracle.
com/cd/E22693_01/index.
htmUnderstandingOracleE-BusinessSuite12.
2architectureisessential.
Thedocumentationlibraryisavailableat:http://docs.
oracle.
com/cd/V39571_01/current/html/docset.
htmlOracleDatabaseApplianceVirtualizedPlatformDeploymentArchitectureThissectionsummarizessomeofthekeyarchitecturalpointsthatwillmakeiteasiertounderstandtheinstallationprocedureinthispaper.
DeployingOracleDatabaseApplianceVirtualizedPlatformcreatesanODA_BASEprivilegeduserdomain.
TheApplianceManagerprovisionsandmanagestheOracleDatabasewithintheODA_BASEdomain.
Thisdomainisoptimizedtohostdatabaseinstances,andalltheshareddisksaredirectlyattachedtothisdomainminimizingvirtualizationoverhead.
OracleDatabaseAppliance12.
1.
2.
0introducesafundamentalchangeinthestoragelayer–theintroductionofOracleACFS.
OracleACFSisdesignedtodeliverthebestperformanceforOracledatabases.
OracleACFSsupportsthecreationoffilesystemsnapshots–afeaturethatvastlysimplifiesthecreationofspaceefficientdatabaseclones.
Goingforward,newdatabasesshouldonlybecreatedonACFSfilesystemsinanewODAdeploymentscenarios.
3|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCEAlthoughOracleE-BusinessSuitedatabasecanbeinstalledinanyoftheuserdomains,itisbestpracticetodeploytheOracleE-BusinessSuitedatabaseinODA_BASEasdescribedinthispapersinceitsignificantlyimprovesperformanceandmanageability.
TosupporthighavailabilityattheApplicationstier,theOracleE-BusinessSuiteApplicationstierVMtemplatecanbeinstalledinmultipleVMs.
Aftercompletingtheprocedureoutlinedinthispaper,OracleDatabaseAppliancewillbeconfiguredwithatwonodeOracleRACdatabaseandtheOracleE-BusinessSuiteApplicationstierrunningintwoVMs,oneoneachnode.
HalfthephysicalCPUandmemoryresourceswillbeallocatedtoODA_BASEfordatabase,andtheotherhalfwillbeallocatedtotheApplicationstierVMs.
Dom0usesveryfewresourcesanddoesn'tneedaseparateallocation.
Note:ThispaperdescribesdeployingthesolutiononanOracleDatabaseApplianceX5-2runningODAsoftwarerelease12.
1.
2.
7.
Thestepsareapplicableforlatersoftwareandhardwarereleases,butsomeofthecomponentversionsmaybedifferent.
Thefollowingliststhemaincomponentsoftheconfiguration:Node1Node2DOM0VMStorageRepositoryODABaseApplianceMgrGridInfrastructureASMInstanceACFSFilesystemRACDBInstanceEBSApplicationVMEBSApplicationVMDOM0VMStorageRepositoryODABaseApplianceMgrGridInfrastructureASMInstanceACFSFilesystemRACDBInstance4|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCESoftwareoneachphysicalserverDom0ManagementDomainOracleVMServer3.
2.
9ODA_BASEVirtualMachineSpecialprivilegeduserdomainfordatabaseinstances.
OracleLinux6.
7.
OracleDatabase11gR2and12c.
OracleRealApplicationClusters(RAC)12c.
OracleACFS12c.
AutomaticStorageManagement(ASM)12c.
OracleClusterware12c.
128GBmainmemoryand16coresallocated.
Thisallocationcaneasilybechangedlaterbasedonperformancerequirementwithinthelimitsavailablesoftwarelicenses.
OracleE-BusinessSuiteVirtualMachineUserdomain.
OracleLinux6.
5.
OracleE-BusinessSuite12.
2.
4.
96GBmainmemoryand16coresallocated.
Thisallocationcanbechangedlater.
InstallingOracleE-BusinessSuite12.
2.
4onODAVirtualizedPlatformThissectionliststhestepsnecessarytoinstallOracleE-BusinessSuiteRelease12.
2.
4withOracleDatabase11gR2onOracleDatabaseAppliance12.
1.
2.
7(orlater).
TheTable1providesaquickreferencetotheenvironmentdetailsreferredtointhisdocument:Table1:ExampleEnvironmentDetailsHostServer1Server2Dom0node1-d0node2-d0ODA_BASEnode1node2ApplicationsVMapp1app2Inthisdocument,allpasswordsareassumedtobewelcome1.
Thesepasswordsshouldalwaysbechanged.
ThisdocumentrequiresaccesstobothnodesoftheOracleDatabaseApplianceasroot,gridandoracleusers.
STEP1:DeployOracleDatabaseApplianceVirtualizedPlatform1.
TheOracleDatabaseAppliancehastobedeployedasaVirtualizedPlatforminordertosupportOracleE-BusinessSuiteApplicationsVMs.
OracleDatabaseApplianceVirtualizedPlatformSetupinstructionsareprovidedontheSetupPoster,whichisavailableat:http://docs.
oracle.
com/cd/E22693_01/.
FollowtheinstructionstosetuptheOracleDatabaseAppliance.
Itisnotnecessarytodeployadatabaseduringtheappliancesetup.
Deselect"CreateInitialDatabase"checkboxinthe"DatabaseInformation"pageoftheApplianceManager.
2.
OracleE-BusinessSuiterequiresthedatabaseVM(ODA_BASE)runscertainUNIXpackagesandspecifiedminimumversions,whicharelistedinMyOracleSupport(MOS)KnowledgeDocument5|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCE1330701.
1OracleE-BusinessSuiteInstallationandUpgradeNotesRelease12(12.
2)forLinuxx86-64.
EnsurethattheODA_BASEonbothnodesmeettheserequirements.
3.
ThespacerequirementforthetemplatesandVMimagesarelistedinTable2:Table2:SpaceRequirementEBSVMTemplatesDiskSpaceRequiredOracleE-BusinessSuiteDatabaseVMtemplate300GBOracleE-BusinessSuiteDatabaseVM300GBOracleE-BusinessSuiteApplicationsVMtemplate300GBOracleE-BusinessSuiteApplicationsVM(2VMs)300GBx2=600GBTotalSpacerequired1500GB4.
Loginastherootuseronnode1andcreatea1600GBsharedrepositorywhichwillbeusedtohosttheVMsusingthefollowingcommand:$oakclicreaterepoappsrepo–size1600G–dgRECO$oakclishowrepo5.
TheOracleDatabaseApplianceteamreleasesRDBMSCloneBundlesforeachOracleDatabaseApplianceversion.
OracleE-BusinessSuite12.
2.
4VMsrequireOracleDatabase11.
2.
0.
4.
3whichisavailableinpatch:17770873–OracleDatabaseAppliance11.
2.
0.
4.
3End-User(RDBMSClone).
DownloadtheRDBMSclone.
Astherootuserstagethesoftwareusingoakcliutilityasfollows:$oakcliunpack–package\/p17770873_121200_Linux-x86-64.
zipSTEP2:PreparetheTargetORACLE_HOME1.
CreateanewORACLE_HOMEtohostthedatabaseusingoakcli.
$oakclicreatedbhome–version11.
2.
0.
4.
3$oakclishowdbhomes2.
ThenewORACLE_HOMEcreatedbytheoakcliutilitydoesnotincludetheOracleDatabase11gR2Examplescomponent,whichisrequiredbyOracleE-BusinessSuite.
ThiscanbedownloadedfromtheOracleTechnologyNetworkat:http://www.
oracle.
com/technetwork/database/enterprise-edition/downloads/index.
html.
Unziptheinstallationmediaintoastagingdirectory.
Astheoracleuser,runtheOUIfromthedownloadedinstallationmediadirectoryusingthefollowingcommandandfollowtheon-screeninstructions:$.
/runInstaller3.
OracleE-BusinessSuiteneedsadditionalRDBMSpatchestobeinstalled.
Requiredpatchesfor11.
2.
0.
4.
3RDBMSPSUareenumeratedintheMOSKnowledgeDocument1147107.
1DatabasePatchSetUpdateOverlayPatchesRequiredforUsewithPSUsandOracleE-BusinessSuite.
6|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCE4.
WhenOracleDatabaseApplianceisdeployed,RoleSeparationusesdifferentownersfortheGridInfrastructureandOracleDatabase.
Applyingtheone-offRDBMSpatcheswillre-generatetheOraclebinaryin$ORACLE_HOME/binanditspermissionswillberesettothedefaultownership.
YouneedtoresettheOraclebinarypermissionsforthegriduseronbothnode1andnode2,usingthefollowingcommand:$/u01/app/12.
1.
0.
2/grid/bin/setasmgidwrap\o=/u01/app/oracle/product/11.
2.
0.
4/dbhome_1/bin/oracle5.
OracleE-BusinessSuite12.
2usesOracle9iNLSinformation.
TherequiredsupportfilesarecreatedbyrunningthefollowingscriptundertheRDBMShomeastheoracleuseronbothnode1andnode2:$perl$ORACLE_HOME/nls/data/old/cr9idata.
plSTEP3:SetuptheOracleE-BusinessSuiteDatabaseVMTheOracleE-BusinessSuiteVMsincludeadatabaseVMwiththerequiredmetadataforanOracleE-BusinessSuitedeployment.
ThisdatabasewillbemigratedtoruninODA_BASEwhichprovidesoptimalperformance.
1.
OracleE-BusinessSuiteRelease12.
2.
4x86(64bit)VMtemplatesareavailableat:https://edelivery.
oracle.
com/linux.
Downloadthefollowingfilestoastagingareaunderthe/cloudfsdirectoryonnode1:V56219-01_1of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ProductionDatabaseTierInstallX86(64bit)(Part1of2)V56219-01_2of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ProductionDatabaseTierInstallX86(64bit)(Part2of2)V56220-01_1of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ProductionDatabaseTierInstallX86(64bit)(Part1of2)V56220-01_2of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ProductionDatabaseTierInstallX86(64bit)(Part2of2)V56218-01_1of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ProductionDatabaseTierInstallX86(64bit)(Part1of2)V56218-01_2of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ProductionDatabaseTierInstallX86(64bit)(Part2of2)2.
Unzipthefilesusingthefollowingcommands:$unzipV56218-01_1of2.
zip$unzipV56218-01_2of2.
zip$unzipV56219-01_1of2.
zip$unzipV56219-01_2of2.
zip$unzipV56220-01_1of2.
zip$unzipV56220-01_2of2.
zip3.
Combinethesixfilesintoasinglecompressedfileusingthefollowingcommand:$cat\Oracle-E-Business-Suite-PROD-DB-12.
2.
4.
ova.
0\Oracle-E-Business-Suite-PROD-DB-12.
2.
4.
ova.
1\Oracle-E-Business-Suite-PROD-DB-12.
2.
4.
ova.
2\Oracle-E-Business-Suite-PROD-DB-12.
2.
4.
ova.
3\Oracle-E-Business-Suite-PROD-DB-12.
2.
4.
ova.
4\Oracle-E-Business-Suite-PROD-DB-12.
2.
4.
ova.
5\>Oracle-E-Business-Suite-PROD-DB-12.
2.
4.
ova4.
StartPythonbasedHTTPserverfromthesamedirectorywhichcontainstheassemblyfile.
LeavethesessionrunninguntilVMtemplateimportcompletes.
7|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCE$python–mSimpleHTTPServer5.
Loginasrootuseronnode1andcreateaVNCsessionthatwillbeusedforGUIcomponents,suchastheVMconsole,usingthefollowingcommand:$vncserver6.
BeforethedownloadedOracleDatabaseVMtemplatecanbeusedtostartaVM,itneedstobestagedwithintheODA.
Astherootuseronnode1,importtheVMtemplateusingthefollowingcommand:$oakcliimportvmtemplatedbtempl–assembly\http://node1:8000/Oracle-E-Business-Suite-PROD-DB-12.
2.
4.
ova\–repoappsrepo–node07.
Asrootuseronnode1,usetheoakcliclonecommandtocreatetheVMasfollows:$oakcliclonevmdbvm-vmtemplatedbtempl-repoappsrepo–node08.
ConfirmtheVMhasbeencreatedusingthefollowingcommand:$oakclishowvmNAMENODENUMMEMORYVCPUSTATEREPOSITORYDbvm040964OFFLINEappsrepo$oakclishowvmdbvmResource:appdbAutoStart:restoreCPUPriority:100Disks:|file:/OVS/Repositories/appsrepo/.
ACFS/snaps/dbvm/VirtualMachines/dbvm/e6794c5f000c48d3a6da48544c1318bd.
img,xvda,w||,xvdb:/OVS/Repositories/appsrepo/.
ACFS/snaps/dbvm/VirtualMachines/dbvm/cdrom,r|Domain:XEN_PVMExpectedState:offlineFailOver:falseIsSharedRepo:trueKeyboard:en-usMaxMemory:2048MaxVcpu:2Memory:2048Mouse:OS_DEFAULTName:dbvmNetworks:NodeNumStart:0OS:OL_5PrefNodeNum:0PrivateIP:NoneProcessorCap:100RepoName:appsrepoSource:0State:OfflineTemplateName:otml_dbtemplVDisks:|0|Vcpu:1cpupool:default-unpinned-poolvncport:None9.
ReconfiguretheVMtousenet1asthepublicinterface:8|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCE$oakclimodifyvmdbvm-addnetworknet1$oakclishowvmdbvmResource:appdbAutoStart:restoreCPUPriority:100Disks:|file:/OVS/Repositories/appsrepo/.
ACFS/snaps/dbvm/VirtualMachines/dbvm/e6794c5f000c48d3a6da48544c1318bd.
img,xvda,w||,xvdb:/OVS/Repositories/appsrepo/.
ACFS/snaps/dbvm/VirtualMachines/dbvm/cdrom,r|Domain:XEN_PVMExpectedState:offlineFailOver:falseIsSharedRepo:trueKeyboard:en-usMaxMemory:2048MaxVcpu:2Memory:2048Mouse:OS_DEFAULTName:dbvmNetworks:['bridge=net1']NodeNumStart:0OS:OL_5PrefNodeNum:0PrivateIP:NoneProcessorCap:100RepoName:appsrepoSource:0State:OfflineTemplateName:otml_dbtemplVDisks:|0|Vcpu:1cpupool:default-unpinned-poolvncport:None10.
Duetobug:21101080,theVMconfigurationhasanincorrectentryforthecdrominthevm.
cfgfile.
Thishastobecorrectedmanually.
Editthevm.
cfgandchangefollowing:xvdb:/OVS/Repositories/appsrepo/.
ACFS/snaps/dbvm/VirtualMachines/dbvm/cdrom,rto:xvdb:cdrom,r11.
StartthedatabaseVMusingthefollowingcommand:$oakclistartvmdbvm12.
ThedatabaseVMcanbeconfiguredfromtheVMconsole.
LogintotheVNCsessioncreatedearlierandstarttheVMconsoleusingthefollowingcommand:$oakclishowvmconsoledbvm13.
ThefirstbootprocessstartstheconfigurationoftheVM.
Thepasswordsforroot,oracleandapplmgruserswillhavetobechangedwhenprompted.
TheconfigurationprocesswillthencontinuetosetupthenetworkrequiringinputsforIPaddress,netmask,gateway,DNSserveraddressandhostname.
Finally,theE-BusinessSuitedatabaseconfigurationstartsbypromptingforthedatabaseSID.
Changingpasswordforuserroot.
Newpassword:Retypenewpassword:passwd:allauthenticationtokensupdatedsuccessfully.
9|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCEChangingpasswordforuseroracle.
Newpassword:Retypenewpassword:passwd:allauthenticationtokensupdatedsuccessfully.
Changingpasswordforuserapplmgr.
Newpassword:Retypenewpassword:passwd:allauthenticationtokensupdatedsuccessfully.
ConfigureaStaticIPConfiguringnetworkinterface.
Networkdevice:eth0Hardwareaddress:00:16:3E:53:49:0CEnterstaticIPaddress:1.
2.
3.
4Enternetmask:[255.
0.
0.
0]255.
255.
255.
0Entergateway:1.
2.
3.
1EnterDNSserver:1.
2.
3.
1Shuttingdowninterfaceeth0:[OK]Shuttingdownloopbackinterface:[OK]Configuringnetworksettings.
IPconfiguration:StaticIPaddressBringinguploopbackinterface:[OK]Bringingupinterfaceeth0:Determiningifipaddress1.
2.
3.
4isalreadyinusefordeviceeth0.
.
.
[OK]Enterhostname(e.
g,host.
example.
com):[dbhost.
us.
oracle.
com]Networkconfigurationchangedsuccessfully.
IPconfiguration:StaticIPaddressIPaddress:1.
2.
3.
4Netmask:255.
255.
255.
0Gateway:1.
2.
3.
1DNSserver:1.
2.
3.
1Hostname:dbhost.
us.
oracle.
comSTEP4:MigratetheDatabasetoODA_BASE1.
LogintothesourcedatabaseVMastheoracleuser.
Sourcetheenvironmentusingthefollowingcommand:$.
/u01/install/PROD/11.
2.
0/PROD_dbhost.
env2.
CreateapasswordfilefortheOracledatabaseinordertofacilitateremotesysdbalogin(thisisrequiredforsubsequentdatamigrationoperations):$$ORACLE_HOME/bin/orapwdfile=$ORACLE_HOME/dbs/orapwPRODpassword=welcome13.
OracleDatabaseduplicationproceduresaresimplerwhenusinganspfile.
Createanspfileforthedatabaseasfollows:$sqlplus/assysdbaSQL>createspfilefrompfile;10|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCE4.
Preparethesourcedatabasetierforcloning:$cd$ORACLE_HOME/appsutil/scripts/PROD_dbhost$perladpreclone.
pldbTier5.
TheAutoConfigfileswillneedtobeinstalledinthetargetORACLE_HOME.
Asafirststep,createanewzipfileoftheappsutildirectoryinsourceORACLE_HOMEandcopyittoODA_BASE:$cd$ORACLE_HOME$zip–rappsutil.
zipappsutil$scpappsutil.
zipnode1:/u01/app/oracle/product/11.
2.
0.
4/dbhome_1/6.
Shutdownthesourcedatabaseandstartitinmountstateusingthefollowingcommand:$sqlplus/assysdbaSQL>shutdownimmediateSQL>startupmount7.
LoginastheOracleusertoODA_BASEonnode1andinstalltheAutoConfigfilesasfollows:$cd$ORACLE_HOME$unzipappsutil.
zip8.
OracleDatabaseAppliancehasanoptimalconfigurationofHDD&SSDdisks.
Thedatabasehastobelocatedsuchthatunderlyingstoragesubsystemisutilizedoptimally.
FilesystemsmustbecreatedandmountedtohostdatafilesonHDDsandredologsonSSDs.
ThefollowingcommandcreatesthestoragestructureforadatabasenamedPROD:$oakclicreatedbstorage–dbPROD9.
Preparethetargetdatabasetier.
WhileconfiguringtheOracleApplicationsenvironmentchoose"/u02/app/oracle/oradata/datastore/.
ACFS/snaps/PROD"ACFSfilesystemtobetheDATA_TOP.
WhenpromptedforORACLERACdatabase,respondwithNoasthedatabasewillbeconvertedtoORACLERACatalaterstage.
Also,chooseanon-zeroportpoolsothatitdoesnotclashwiththeexistinglistenersettingsontheOracleDatabaseAppliance.
ThisprocedureusesPRODasthedatabaseSID:$cd$ORACLE_HOME/appsutil/clone/bin$perladcfgclone.
pldbTechStack10.
WhenOracleDatabaseApplianceisdeployed,RoleSeparationusesdifferentownersfortheGridInfrastructureandOracleDatabase.
RunnngAutoconfigwillre-generatetheOraclebinaryin$ORACLE_HOME/binanditspermissionswillberesettothedefaultownership.
YouneedtoresettheOraclebinarypermissionsforthegriduseronbothnode1andnode2,usingthefollowingcommand:$/u01/app/12.
1.
0.
2/grid/bin/setasmgidwrap\o=/u01/app/oracle/product/11.
2.
0.
4/dbhome_1/bin/oracle11.
Createthepasswordfileonthetargetdatabasehomeusingthefollowingcommandastheoracleuseronnode1:$$ORACLE_HOME/bin/orapwdfile=$ORACLE_HOME/dbs/orapwPRODpassword=welcome112.
SetuptheApplicationsenvironmentandstartthetargetdatabaseinstanceinanomountstate:$.
$ORACLE_HOME/PROD_node1.
env$$ORACLE_HOME/bin/sqlplus/assysdbaSQL>startupnomount13.
AddtheTNSaliasforbothsourceandtargetdatabasesinthetnsnames.
oraofboththesource&targetORACLE_HOMEs.
Forexample,PROD_DBHOST=(DESCRIPTION=(ADDRESS=11|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCE(PROTOCOL=tcp)(HOST=dbhost)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=PROD)))PROD_ODABASE=(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=node1)(PORT=1529))(CONNECT_DATA=(SERVICE_NAME=PROD)))14.
ConnecttoboththesourceanddestinationdatabaseinstancesusingRMAN.
ThesourcedatabaseisthetargetdatabaseandthedestinationdatabaseistheauxiliarydatabaseinRMANterminology.
$$ORACLE_HOME/bin/rmantargetsys/welcome1@prod_dbhost\auxiliarysys/welcome1@prod_odabase15.
DuplicatethedatabaseusingthefollowingRMANscript.
EnsurethattheparametersLOCAL_LISTENER,AUDIT_FILE_DEST,DIAGNOSTIC_DEST,CONTROL_FILESandDB_UNIQUE_NAMEcorrespondtothevaluesforyourenvironment.
TheremainderoftheparametershavethesamevaluesacrossalltheOracleDatabaseApplianceenvironments.
RMAN>run{allocatechanneltgt1devicetypedisk;allocateauxiliarychannelaux1devicetypedisk;allocatechanneltgt2devicetypedisk;allocateauxiliarychannelaux2devicetypedisk;allocatechanneltgt3devicetypedisk;allocateauxiliarychannelaux3devicetypedisk;allocatechanneltgt4devicetypedisk;allocateauxiliarychannelaux4devicetypedisk;DUPLICATETARGETDATABASETOPRODFROMACTIVEDATABASESPFILESETDB_UNIQUE_NAME='PROD'SETCLUSTER_DATABASE='false'SETDB_CREATE_FILE_DEST="/u02/app/oracle/oradata/datastore/.
ACFS/snaps/PROD"SETDB_RECOVERY_FILE_DEST="/u01/app/oracle/fast_recovery_area/datastore"SETDB_RECOVERY_FILE_DEST_SIZE="500G"SETDB_CREATE_ONLINE_LOG_DEST_1="/u01/app/oracle/oradata/datastore"SETCONTROL_FILES="/u01/app/oracle/oradata/datastore/PROD/control.
ctl"SETDB_BLOCK_CHECKSUM='FULL'SETDB_BLOCK_CHECKING='FULL'SETDB_LOST_WRITE_PROTECT='TYPICAL'SETFILESYSTEMIO_OPTIONS='SETALL'SETUSE_LARGE_PAGES='ONLY'SETCOMPATIBLE='11.
2.
0.
4'SETPARALLEL_ADAPTIVE_MULTI_USER='FALSE'SETPARALLEL_EXECUTION_MESSAGE_SIZE='16384'SETPARALLEL_THREADS_PER_CPU='2'SETPARALLEL_MIN_SERVERS='0'SETDIAGNOSTIC_DEST='/u01/app/oracle/product/11.
2.
0.
4/dbhome_1/admin/PROD_node1'SETAUDIT_FILE_DEST='/u01/app/oracle/product/11.
2.
0.
4/dbhome_1/rdbms/audit'SETLOG_ARCHIVE_DEST_1=''SETLOCAL_LISTENER='PROD_LOCAL'12|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCENOFILENAMECHECK;}16.
Runthelibraryupdatescriptonthetargetdatabase.
$cd$ORACLE_HOME/appsutil/install/PROD_node1$sqlplus/assysdba@adupdlib.
sqlso17.
Configurethetargetdatabasetier.
$cd$ORACLE_HOME/appsutil/clone/bin$perladcfgclone.
pldbconfig$ORACLE_HOME/appsutil/PROD_node1.
xml18.
TheAutoconfigtooldoesnotgeneratethecorrectconfig.
cin$ORACLE_HOME/rdbms/lib/directory.
Workaroundistomakethefollowingchangestoconfig.
c:a.
Addanewline:#defineSS_ASM_GRP""b.
Changethefollowingline:char*ss_dba_grp[]={SS_DBA_GRP,SS_OPER_GRP};to:char*ss_dba_grp[]={SS_DBA_GRP,SS_OPER_GRP,SS_ASM_GRP};c.
Generateconfig.
o:$make-f$ORACLE_HOME/rdbms/lib/ins_rdbms.
mk$ORACLE_HOME/rdbms/lib/config.
od.
Generateosdbagrpbinaryagain:$cd$ORACLE_HOME/srvm/lib$make-fins_srvm.
mkiosdbagrp19.
RegisterdatabasewithCRS.
$srvctladddatabase-dPROD-o/u01/app/oracle/product/11.
2.
0.
4/dbhome_1–p/u02/app/oracle/oradata/datastore/PROD/spfilePROD.
ora-csingle-xnode1$srvctlsetenvdatabase–dPROD–TORA_NLS10=$ORACLE_HOME/nls/data/9idata$srvctlsetenvdatabase–dPROD–TTNS_ADMIN=$ORACLE_HOME/network/admin/PROD_node1$srvctlstartdatabase–dPROD20.
Updatethefollowinginit.
oraparametersinaSQL*Plussession:SQL>altersystemset"_disable_interface_checking"=truescope=spfilesid='*';SQL>altersystemset"_gc_undo_affinity"=falsescope=spfilesid='*';SQL>altersystemset"_gc_policy_time"=0scope=spfilesid='*';SQL>altersystemset"_enable_numa_support"=falsescope=spfilesid='*';SQL>altersystemset"_file_size_increase_increment"=2143289344scope=spfilesid='*';21.
Restartthedatabasefortheparametertotakeeffect:$srvctlstopdatabase–dPROD$srvctlstartdatabase–dPRODSTEP5:SetuptheOracleE-BusinessSuiteApplicationsTierVM13|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCE1.
OracleVMTemplatesforOracleE-BusinessSuiteRelease12.
2.
4MediaPackv1forx86(64bit)areavailableat:https://edelivery.
oracle.
com/linux.
Thestagingdirectory/cloudfswouldrequire75GBfreespace.
Downloadthefollowingfilestoastagingareaunderthe/cloudfsdirectoryonnode1:V59638-01_1of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ApplicationTierInstallX86(64bit)(Part1of2)V59638-01_2of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ApplicationTierInstallX86(64bit)(Part2of2)V59637-01_1of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ApplicationTierInstallX86(64bit)(Part1of2)V59637-01_2of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ApplicationTierInstallX86(64bit)(Part2of2)V59636-01_1of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ApplicationTierInstallX86(64bit)(Part1of2)V59636-01_2of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ApplicationTierInstallX86(64bit)(Part2of2)V59635-01_1of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ApplicationTierInstallX86(64bit)(Part1of2)V59635-01_2of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ApplicationTierInstallX86(64bit)(Part2of2)V59639-01_1of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ApplicationTierInstallX86(64bit)(Part1of2)V59639-01_2of2.
zipOracleE-BusinessSuiteRelease12.
2.
4ApplicationTierInstallX86(64bit)(Part2of2)2.
Unzipthefilesusingthefollowingcommands:$unzipV59638-01_1of2.
zip$unzipV59638-01_2of2.
zip$unzipV59637-01_1of2.
zip$unzipV59637-01_2of2.
zip$unzipV59636-01_1of2.
zip$unzipV59636-01_2of2.
zip$unzipV59635-01_1of2.
zip$unzipV59635-01_2of2.
zip$unzipV59639-01_1of2.
zip$unzipV59639-01_2of2.
zip3.
Combinethesplitfilesintoasinglecompressedfile.
$cat\Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova.
0\Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova.
1\Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova.
2\Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova.
3\Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova.
4\Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova.
5\Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova.
6\Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova.
7\Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova.
8\Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova.
9\>Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova4.
BeforethedownloadedApplicationsVMtemplatecanbeusedtostartaVM,itneedstobestagedwithintheODA.
Itrequires285GBofrepositoryspace.
Astherootuser,importtheVMtemplateusingthefollowingcommand:$oakcliimportvmtemplateapptempl–assembly\https:/node1:8000/Oracle-E-Business-Suite-APPS-TIER-12.
2.
4.
ova\–repoappsrepo–node05.
Astherootuser,usetheoakcliclonecommandtocreatetheVMasfollows:$oakcliclonevmappvm-vmtemplateapptempl-repoappsrepo-node1\-driverdomain6.
AstherootuserconfirmtheVMhasbeencreatedusingthefollowingcommand:$oakclishowvm14|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCE$oakclishowvmappvmResource:appvmAutoStart:restoreCPUPriority:100Disks:|file:/OVS/Repositories/appsrepo/.
ACFS/snaps/appvm/VirtualMachines/appvm/d84a641d07a340dfa550bf3518244896.
img,xvda,w||,xvdb:/OVS/Repositories/appsrepo/.
ACFS/snaps/appvm/VirtualMachines/appvm/cdrom,r|Domain:XEN_PVMExpectedState:onlineFailOver:falseIsSharedRepo:trueKeyboard:en-usMaxMemory:16384MaxVcpu:8Memory:16384Mouse:OS_DEFAULTName:appvmNetworks:NodeNumStart:0OS:OL_5PrefNodeNum:0PrivateIP:NoneProcessorCap:100RepoName:appsrepoSource:0State:OnlineTemplateName:otml_apptemplVDisks:|0|Vcpu:8cpupool:default-unpinned-poolvncport:None7.
ReconfiguretheVMtousenet1asthepublicinterface.
Astherootuser,runthefollowingcommand:$oakclimodifyvmappvm-addnetworknet1$oakclishowvmappvmResource:appvmAutoStart:restoreCPUPriority:100Disks:|file:/OVS/Repositories/appsrepo/.
ACFS/snaps/appvm/VirtualMachines/appvm/d84a641d07a340dfa550bf3518244896.
img,xvda,w||,xvdb:/OVS/Repositories/appsrepo/.
ACFS/snaps/appvm/VirtualMachines/appvm/cdrom,r|Domain:XEN_PVMExpectedState:onlineFailOver:falseIsSharedRepo:trueKeyboard:en-usMaxMemory:16384MaxVcpu:8Memory:16384Mouse:OS_DEFAULTName:appvmNetworks:['bridge=net1']NodeNumStart:0OS:OL_5PrefNodeNum:015|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCEPrivateIP:NoneProcessorCap:100RepoName:appsrepoSource:0State:OnlineTemplateName:otml_apptemplVDisks:|0|Vcpu:8cpupool:default-unpinned-poolvncport:None8.
Duetobug:21101080,theVMconfigurationhasanincorrectentryforcdrominthevm.
cfgfile.
Thishastobecorrectedmanually.
Editthevm.
cfgandchangefollowing:xvdb:/OVS/Repositories/appsrepo/.
ACFS/snaps/appvm/VirtualMachines/appvm/cdrom,rto:xvdb:cdrom,r9.
TheApplicationsVMcannowbestartedusingthefollowingcommandastherootuser:$oakclistartvmappvm10.
TheApplicationsVMrequiresthattheApplicationsenvironmentnetworkisconfiguredinthesamewayasforthedatabaseVM.
Inordertodothis,youneedtouseTheVMconsole,whichisaccessedastherootuserwithintheVNCsessionusingthefollowingcommand:$oakclishowvmconsoleappvm11.
ThefirstbootprocessstartstheconfigurationoftheVM.
Thepasswordsforroot,oracleandapplmgruserswillhavetobechangedwhenprompted.
TheconfigurationprocesswillthencontinuetosetupthenetworkrequiringinputsforIPaddress,netmask,gateway,DNSserveraddressandhostname.
WhentheOracleE-BusinessSuiteApplicationstierisstartedyouwillneedtosupplytheDatabaseHostIP,DatabaseHostname,DatabaseHostdomain,DatabaseSIDandListenerPort.
Changingpasswordforuserroot.
Newpassword:Retypenewpassword:passwd:allauthenticationtokensupdatedsuccessfully.
Changingpasswordforuseroracle.
Newpassword:Retypenewpassword:passwd:allauthenticationtokensupdatedsuccessfully.
Changingpasswordforuserapplmgr.
Newpassword:Retypenewpassword:passwd:allauthenticationtokensupdatedsuccessfully.
ConfigureaStaticIPConfiguringnetworkinterface.
Networkdevice:eth0Hardwareaddress:00:16:3E:53:49:0CEnterstaticIPaddress:1.
2.
3.
5Enternetmask:[255.
0.
0.
0]255.
255.
255.
016|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCEEntergateway:1.
2.
3.
1EnterDNSserver:1.
2.
3.
1Shuttingdowninterfaceeth0:[OK]Shuttingdownloopbackinterface:[OK]Configuringnetworksettings.
IPconfiguration:StaticIPaddressBringinguploopbackinterface:[OK]Bringingupinterfaceeth0:Determiningifipaddress1.
2.
3.
5isalreadyinusefordeviceeth0.
.
.
[OK]Enterhostname(e.
g,host.
example.
com):[apphost.
us.
oracle.
com]Networkconfigurationchangedsuccessfully.
IPconfiguration:StaticIPaddressIPaddress:1.
2.
3.
5Netmask:255.
255.
255.
0Gateway:1.
2.
3.
1DNSserver:1.
2.
3.
1Hostname:apphost.
us.
oracle.
comEntertheIPAddressoftheDBTierHost:1.
2.
3.
4EntertheDatabaseTierHostName(withoutthedomain):dbhostEntertheDatabaseTierDomainName:us.
oracle.
comEntertheOracleDatabaseSID:PRODEntertheTNSListenerPortNumber:1521STEP6:EnableHighAvailability1.
ToprovideHighAvailability(HA)attheApplicationslayer,anadditionalApplicationsVMneedstobedeployedasdescribedinSTEP5.
6:SetupOracleE-BusinessSuiteApplicationsVM.
Repeatsteps5.
6to5.
11onthesecondnodeintheappliance.
2.
ConvertingasingledatabaseonOracleDatabaseAppliancetoOracleRACusesstandardOracleE-BusinessSuiteproceduresasprovidedinSection4oftheMOSKnowledgeDocument1453213.
1,UsingOracle11gRelease2RealApplicationClustersandAutomaticstoragemanagementwithOracleE-BusinessSuiteRelease12.
2.
EnsuretherequirementslistedinSection4.
1Pre-RequisitesaremetpriortomigratingtoOracleRACdatabase.
Astheclusterwarehasalreadybeeninstalled,theprocedureisfurthersimplified.
FollowthestepsfromSection4.
4ListenerConfigurationin11gR2toSection4.
7EstablishApplicationsEnvironmentforOracleRACtoconvertthedatabasetoORACLERAC.
3.
Loadbalancingtheapplicationserversprovidesbothprotectionagainstfailureaswellasincreasedscalability.
OracleE-BusinessSuitesupportsloadbalancingatmultiplelayers,including,forexample,DNSandWebServices.
OracleDatabaseApplianceputsnospecificrestrictionintheconfigurationoftheLoadBalancersforOracleE-BusinessSuite.
MOSKnowledgeDocument380489.
1:UsingLoad-BalancerswithOracleE-BusinessSuiteRelease12providesdetailedinformationonconfiguringloadbalancing.
STEP7:Cleanup17|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCE1.
ThepythonHTTPserverprocesscanbekilled.
$kill-92.
Filesthatweredownloaded,uncompressedandcombinedintheprocessofcreatingthedatabaseandApplicationsVMscannowbedeletedfromthe/cloudfsdirectory.
3.
SincethedatabasehasbeenmigratedtoODA_BASE,thedatabaseVMcanbedeletedusingthefollowingcommand:$oakclistopvmdbvm$oakclideletevmdbvm4.
ThedatabaseVMtemplatecanalsoberemoved.
Toremovethetemplate,usethefollowingcommand:$oakclideletevmtemplatedbvmSTEP8:ValidateEnvironment5.
ThepresenceofApplicationsVMs–runthefollowingcommandasrootuseronODA_BASEonnode1:$oakclishowvm6.
Thepresenceofaclusterdatabase–runthefollowingcommandasoracleuseronODA_BASEonnode1:$srvctlconfigdatabase–dPROD–a7.
TheapplicationavailabilitycanbevalidatedbyloggingintotheapplicationusingtheURLhttp://app1.
company.
com:8008/18|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCEBestPracticesThissectionliststhebestpractices,whichshouldbereviewedoncethedeploymentiscomplete.
ConsiderResizingtheResourceConfigurationforODA_BASEAreasonablestartingpointistoallocate50%oftheavailableCPUandmemoryresourcestoODA_BASE.
Thesecanbemodifiedtoreflectcapacityrequirementswhileremainingwithinthesoftwarelicensingrestrictions.
ThenumberofCPUcoresandmemoryallocationscanbechangedusingthefollowingcommandastherootuserondom0(onanX4-2):$oakcliconfigureoda_baseCoreLicensingOptions:1.
2CPUCores2.
4CPUCores3.
6CPUCores4.
8CPUCores5.
10CPUCores6.
12CPUCores7.
16CPUCores8.
24CPUCores9.
36CPUCoresCurrentCPUCores:16Selection[1.
.
9](default36CPUCores):7ODAbasedomainmemoryinGB(min16,max244)(CurrentMemory128G)[default192]:Thecorecountsabovereflectphysicalcores.
TheOSinsideODA_BASEwillreportvirtualcores(vcpu),whichequatestoahalfphysicalcore.
ThechangesaremadetoODA_BASEonbothnodes;theresourceallocationshouldbethesameacrossboth.
ThechangestakeeffectwhentheODA_BASEisrestarted.
TorestarttheODABase,astherootuserexecutethefollowingcommandoneachnode:$oakclirestartoda_baseConsiderResizingtheResourceConfigurationfortheApplicationsVMsAswiththeOracleDatabase,theCPUandmemoryallocationfortheApplicationsVMscanalsobereconfiguredaftertheinitialdeployment.
AsVMsaremanagedfromODA_BASE,toreconfiguretheresourcesforanApplicationsVM,thecommandisexecutedastherootuserfromODA_BASE.
UsethefollowingexamplecommandtochangetheCPUandmemoryallocationforanApplicationsVM:$oakcliconfigurevmappvm–vcpu32–maxcpu32$oakcliconfigurevmappvm–memory96G–maxmemory96GThiscommandreferencesvirtualcpus(vcpu).
ThechangesarestaticinnatureandrequiretheVMtoberestarted.
TorestarttheVMs,usethefollowingcommands:$oakclistopvmappvm$oakclistartvmappvmConsiderImplementingWorkloadIsolationusingCPUPoolsOracleDatabaseAppliancehastheabilitytoisolateworkloadswithinCPUpools.
Onceassigned,aVMisconstrainedtoonlyusetheCPUsavailablewithinthatpool.
TwopoolsarecreatedbydefaultwhenOracleDatabaseApplianceisdeployed:odaBaseCpuPoolanddefault-unpinned-pool.
TheODA_BASEdomainisallocatedtotheodaBaseCpuPool,andthenumberofthevCPUsintheodaBaseCpuPoolisequaltothevCPUsallocatedtoODA_BASEduringdeploymentorre-configuration.
19|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCETheApplicationsVMscanalsobenefitfromCPUisolationbycreatingadditionalCPUPools.
OncethesepoolsarecreatedandtheApplicationsVMsareassignedtothesepools,nootherVMcanusethesevCPUrescources.
AdditionalCPUpoolshavetobecreatedmanuallyoneachODA_BASEastheCPUpoolisnodespecific.
Astherootuser,usethefollowingcommandtocreateaCPUpool:$oakclicreatecpupoolappVmPool–numcpu4–node0TovalidatethecreationoftheCPUpool:$oakclishowcpupool–node0TheApplicationsVMcanthenbeassignedtothenewCPUpoolasfollows:$oakcliconfigurevmappvm-cpupoolappVmPoolPatchingOracleDatabaseApplianceOracleDatabaseApplianceprovidesaninfrastructureforsimplifiedpatchingastherequisitepatchesforallcomponentsoftheOracleDatabaseAppliancestack(excludingApplicationsVMpatches)areincludedinasinglebundlepatch.
Thisvastlysimplifiestheend-to-endpatchingofbothhardwareandsoftwaresinceasinglecommandisusedtopatchallthecomponents.
Patchesfordifferentcomponentspresentwithinabundlepatchhavebeenvalidatedtoworkwitheachother.
ThiseliminatesanyadversepatchconflictsforeachofthecomponentsonOracleDatabaseApplianceandensuresend-to-endcompatibilityafterpatching.
TheOracleDatabaseAppliancepatchbundlesarereleasedregularlytokeepup-to-datewiththeOracleDatabasePatchesSetUpdatereleases.
ItisrecommendedthatyounotapplyanyGridInfrastructure/Databaseone-offpatchesorinstallanyadditionalOSpackagesoutsideofthestandardOracleDatabaseAppliancepatchingmechanism.
However,OracleE-BusinessSuitefunctionalpatchesareexcludedfromthispolicy.
Intherareeventofaconflictbetweenaone-offRDBMSpatchandtheOracleDatabaseAppliancepatchbundle,therecommendedapproachtoaddressthisistorollbacktheproblempatchpriortoapplyingtheOracleDatabaseApplianceBundlePatch.
IfthepatchhasnotbeenincludedintheOracleDatabaseAppliancebundlepatch,thenitcanbereapplied.
TheOracleE-BusinessSuiteApplicationstiercanbepatchedusingstandardpatchingutilities.
OracleDatabaseApplianceputsnorestrictiononpatchingtheApplicationstiers.
Forfurtherdetailsonpatching,pleaserefertotheMaintenanceGuideintheOracleE-BusinessSuitedocumentationat:http://docs.
oracle.
com/cd/V39571_01/current/html/docset.
htmlSystemSizingGeneralguidelinesonsystemsizingandcapacitycanbefoundintheOracleDatabaseAppliancedocumentation,availableat:http://docs.
oracle.
com/cd/E22693_01/index.
htmOracleE-BusinessSuiteworkloadscanvarywidelydependingonmanyfactors,suchasmodulesdeployed,userspermoduleandbusinesscycles.
WhenmigratinglegacysystemstoOracleDatabaseAppliance,theexistingpeakCPU,memory,IOPSandstoragespaceutilizationcanbeusedasguidancetoproperlysizethenewenvironment.
YoushouldworkwithyourlocalOraclerepresentativeonthesizingexercise.
EnableDriverDomainDriverDomainisanewfeatureavailableinOAKrelease12.
1.
2.
7.
0andhigher.
Adriverdomainisauserdomainthathasbeengivenspecialprivilegesandresponsibilitiesforaparticularpieceofhardwaretherebyeliminatingdom0asabottleneck.
TheDriverDomainfunctionalityallowsfor20|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCEimplementationofguestvirtualmachinestouseODA_BASEasthedriverdomainforaccesstovirtualdisks.
OneofthelimitationsofthecurrentimplementationofDriverDomainsonODAislackofsupportforbootpartitioninadiskimageaccessedviadriverdomain.
TheE-BusinessSuiteVMsprovideasinglediskimagewhichincludesbothbootpartitionandtheapplicationpartitiononthesamedisk.
Thisconfigurationeffectivelyexcludesout-of-the-boximplementationDriverDomainsfortheE-BusinessSuiteapplicationsVMs.
DriverDomainscanbeimplementedonE-BusinessSuiteapplicationVMsbymovingoutthebootpartitiontoadifferentdiskimage.
Thefollowingstepsdetailtheprocess:1.
Createanewbootdiskofsize500Minthesamelocationastheoriginal.
ddif=/dev/zeroof=/u01/app/sharedrepo/ebsrepo/.
ACFS/snaps/appvm/VirtualMachines/appvm/boot.
imgoflag=directcount=0bs=1Mseek=500M2.
Attachbootdisktothevmbyupdatingdiskparameterinthevm.
cfgdisk=[u'file:/u01/app/sharedrepo/ebsrepo/.
ACFS/snaps/ebsapp/VirtualMachines/appvm/99a674656e49406382ac741f07a7a582.
img,xvda,w',u'file:/OVS/Repositories/ebsrepo/.
ACFS/snaps/appvm/VirtualMachines/appvm/boot.
img,xvdb,w']3.
Restartapplicationvmoakclistopvmappvmoakclistartvmappvm4.
Partitionthenewdiskandmakeitbootableecho-e"n\np\n1\n\n\nt\n83\na\n1\nw"|fdisk/dev/xvdc5.
Formatthenewpartitionmkfs.
ext4/dev/xvdb16.
Mountthenewfilesystemmount/dev/xvdb1/mnt7.
Copyeverytingunder/boottonewfilesystemcp–dpR/boot/*/mnt8.
Mountthenewfilesystemas/bootumount/mntumount/bootmount/dev/xvb1/boot9.
Installgrubinthenewbootdisk$grubgrub>device(hd0)/dev/xvdbgrub>root(hd0,0)grub>setup(hd0)10.
GettheUUIDofthenewbootpartitionblkid/dev/xvdb11.
Updatethe/bootentryin/etc/fstabwithUUIDofthenewpartitionUUID=cc3635e4-4030-4881-91b0-a8279ddb1ef0/bootext4defaults1221|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCE12.
Editthevm.
cfgtoupdatethedriverdomaininformationforeachdisk.
Thebootpartitionshouldrefertothe/OVS/Repositorieslocationandtherootpartitionshouldrefertooda_baselocationasfollows:disk=[u'file:/u01/app/sharedrepo/ebsrepo/.
ACFS/snaps/appvm/VirtualMachines/ebsdb/471489424d0e46a6a3cd6fc53f4370ee.
img,xvdb,w,oakDom1',u'file:/OVS/Repositories/ebsrepo/.
ACFS/snaps/appvm/VirtualMachines/appvm/boot.
img,xvdb,w']13.
RestarttheapplicationVMoakclistopvmappvmoakclistartvmappvmConclusionOracleDatabaseApplianceisidealforsmalltomediumsizedeploymentsofOracleE-BusinessSuitedeployments.
ItenablestheOracleDatabaseandOracleE-BusinessSuiteApplicationstiertobedeployedinasinglecosteffectiveandflexiblesystemthatisengineeredforsimplicityandreliability.
22|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCEAppendixA:References1.
OracleDatabaseAppliancedocumentation:http://docs.
oracle.
com/cd/E22693_01/index.
htm2.
OracleE-BusinessSuitedocumentation:http://docs.
oracle.
com/cd/V39571_01/current/html/docset.
html3.
MOSKnowledgeDocument1330701.
1:OracleE-BusinessSuiteInstallationandUpgradeNotesRelease12(12.
2)forLinuxx86-644.
MOSKnowledgeDocument1349240.
1:DatabasePreparationGuidelinesforanE-BusinessSuiteRelease12.
2Upgrade5.
MOSKnowledgeDocument1453213.
1:UsingOracle11gRelease2RealApplicationClustersandAutomaticstoragemanagementwithOracleE-BusinessSuiteRelease12.
26.
MOSKnowledgeDocument1566935.
1:ImplementingOracleE-BusinessSuite12.
1DatabasesonOracleDatabaseAppliance7.
MOSKnowledgeDocument380489.
1:UsingLoad-BalancerswithOracleE-BusinessSuiteRelease128.
MOSKnowledgeDocument1147107.
1:DatabasePatchSetUpdateOverlayPatchesRequiredforUsewithPSUsandOracleE-BusinessSuite23|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCEAppendixB:ConfiguringVMsafterFirstBootTheVirtualMachinesareconfiguredduringthefirstboot.
TheE-BusinessSuiteVMshavescriptsthatcanbeusedtoreconfiguretheVMsafterthefirstbootorafterafailureofconfigurationduringfirstboot.
Thesescriptsarepresentinthefollowinglocations:OnbothDataabaseandApplicationsVM:SCRIPTSBASE_DIR:/u01/install/scriptsCONFIGUREDHCP:/u01/install/scripts/configdhcp.
shCONFIGURESTATICIP:/u01/install/scripts/configstatic.
shSCRIPTTOCLEANUPVM:/u01/install/scripts/cleanup.
shOnDatabaseVM:SCRIPTSBASE_DIR:/u01/install/PROD/scripts/STARTSCRIPT:/u01/install/PROD/scripts/startproddb.
shSTOPSCRIPT:/u01/install/PROD/scripts/stopproddb.
shDBVMRECONFIGSCRIPT:/u01/install/PROD/scripts/prodbconfig.
shDBVMCLEANUPSCRIPT:/u01/install/PROD/scripts/proddbcleanup.
shOnApplicationsVM:SCRIPTSBASE_DIR:/u01/install/APPS/scripts/STARTSCRIPT:/u01/install/APPS/scripts/startapps.
shSTOPSCRIPT:/u01/install/APPS/scripts/stopapps.
shAPPSVMRECONFIGSCRIPT:/u01/install/APPS/scripts/appsconfig.
shAPPSVMCLEANUPSCRIPT:/u01/install/APPS/scripts/appscleanup.
shCONFIGUREANEWWEBENTRYPOINT:/u01/install/scripts/configwebentry.
sh2|SOLUTION-IN-A-BOX:BESTPRACTICESFORDEPLOYINGORACLEE-BUSINESSSUITERELEASE12.
2.
XONORACLEDATABASEAPPLIANCEOracleCorporation,WorldHeadquarters500OracleParkwayRedwoodShores,CA94065,USAWorldwideInquiriesPhone:+1.
650.
506.
7000Fax:+1.
650.
506.
7200Copyright2015,Oracleand/oritsaffiliates.
Allrightsreserved.
Thisdocumentisprovidedforinformationpurposesonly,andthecontentshereofaresubjecttochangewithoutnotice.
Thisdocumentisnotwarrantedtobeerror-free,norsubjecttoanyotherwarrantiesorconditions,whetherexpressedorallyorimpliedinlaw,includingimpliedwarrantiesandconditionsofmerchantabilityorfitnessforaparticularpurpose.
Wespecificallydisclaimanyliabilitywithrespecttothisdocument,andnocontractualobligationsareformedeitherdirectlyorindirectlybythisdocument.
Thisdocumentmaynotbereproducedortransmittedinanyformorbyanymeans,electronicormechanical,foranypurpose,withoutourpriorwrittenpermission.
OracleandJavaareregisteredtrademarksofOracleand/oritsaffiliates.
Othernamesmaybetrademarksoftheirrespectiveowners.
IntelandIntelXeonaretrademarksorregisteredtrademarksofIntelCorporation.
AllSPARCtrademarksareusedunderlicenseandaretrademarksorregisteredtrademarksofSPARCInternational,Inc.
AMD,Opteron,theAMDlogo,andtheAMDOpteronlogoaretrademarksorregisteredtrademarksofAdvancedMicroDevices.
UNIXisaregisteredtrademarkofTheOpenGroup.
0115Solution-in-a-box:BestpracticesfordeployingOracleE-BusinessSuiteRelease12.
2.
4onOracleDatabaseAppliance12.
1.
2.
7July2016Author:RamasubramanianAthmanathanContributingAuthors:AndyTremayne,RACPack,E-BusinessSuiteProductTeamCONNECTWITHUSblogs.
oracle.
com/oraclefacebook.
com/oracletwitter.
com/oracleoracle.
com
- generateclusterware相关文档
- managemenclusterware
- pruningclusterware
- integratedclusterware
- 4.8clusterware
- 解决方法clusterware
- highclusterware
Pacificrack：新增三款超级秒杀套餐/洛杉矶QN机房/1Gbps月流量1TB/年付仅7美刀
PacificRack最近促销上瘾了，活动频繁，接二连三的追加便宜VPS秒杀，PacificRack在 7月中下旬已经推出了五款秒杀VPS套餐，现在商家又新增了三款更便宜的特价套餐，年付低至7.2美元，这已经是本月第三波促销，带宽都是1Gbps。PacificRack 7月秒杀VPS整个系列都是PR-M，也就是魔方的后台管理。2G内存起步的支持Windows 7、10、Server 2003\20...
racknerd：美国大硬盘服务器（双路e5-2640v2/64g内存/256gSSD+160T SAS）$389/月
racknerd在促销美国洛杉矶multacom数据中心的一款大硬盘服务器，用来做存储、数据备份等是非常划算的，而且线路还是针对亚洲有特别优化处理的。双路e5+64G内存，配一个256G的SSD做系统盘，160T SAS做数据盘，200T流量每个月，1Gbps带宽，5个IPv4，这一切才389美元...洛杉矶大硬盘服务器CPU：2 * e5-2640v2内存：64G（可扩展至128G，+$64）硬...
香港九龙湾（27元） 2核2G 20元 香港沙田
弘速云是创建于2021年的品牌，运营该品牌的公司HOSU LIMITED（中文名称弘速科技有限公司）公司成立于2021年国内公司注册于2019年。HOSU LIMITED主要从事出售香港VPS、美国VPS、香港独立服务器、香港站群服务器等，目前在售VPS线路有CN2+BGP、CN2 GIA，该公司旗下产品均采用KVM虚拟化架构。可联系商家代安装iso系统。国庆活动 优惠码:hosu10-1产品介绍...
clusterware为你推荐
-
淘宝回应崩了为什么淘宝卖家不回复易烊千玺弟弟创魔方世界纪录王俊凯.易烊千玺编舞吉尼斯记录淘宝门户淘宝网怎么样从个人中心进入首页安徽汽车网安徽省各地车牌号简称是按照什么顺序排的地图应用看卫星地图哪个手机软件最好。sss17.comwww.com17com.com是什么啊？haole16.com玛丽外宿中16全集在线观看 玛丽外宿中16qvod快播高清下载m.2828dy.combabady为啥打不开了，大家帮我提供几个看电影的网址bbs2.99nets.com西安论坛、西安茶馆网、西安社区、西安bbs 的网址是多少？baqizi.cc曹操跟甄洛是什么关系