environmentsbbzs

000066WhitePaperRancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamilyAbstractThiswhitepaperprovidesguidanceonthedeploymentofaRancherKubernetesClusterontotheDellEMCVxFlexfamilyandtheintegrationoftheVxFlexCSIdriverforpersistentvolumes,forcustomersrequiringanon-premisescontainerplatformsolution.
May2020Revisions2RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|000066RevisionsDateDescriptionApril2020InitialreleaseMay2020UpdatedSection3:SolutionArchitectureAcknowledgementsThispaperwasproducedbythefollowing:Author:SanjayPuttaswamySupport:DavidJAdamsandShashikiranChidambaraOthers:SridharSubramanian,ShaliniG,KevinJonesandSimonStevensTheinformationinthispublicationisprovided"asis.
"DellInc.
makesnorepresentationsorwarrantiesofanykindwithrespecttotheinformationinthispublication,andspecificallydisclaimsimpliedwarrantiesofmerchantabilityorfitnessforaparticularpurpose.
Use,copying,anddistributionofanysoftwaredescribedinthispublicationrequiresanapplicablesoftwarelicense.
Copyright2020DellInc.
oritssubsidiaries.
AllRightsReserved.
Dell,EMC,DellEMCandothertrademarksaretrademarksofDellInc.
oritssubsidiaries.
Othertrademarksmaybetrademarksoftheirrespectiveowners.
[5/14/2020][WhitePaper][000066]Tableofcontents3RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|000066TableofcontentsRevisions.
2Acknowledgements.
2Tableofcontents3Executivesummary.
41Introduction.
51.
1Objective.
51.
2Audience.
51.
3Terminology.
52Productoverview.
62.
1VxFlexfamily62.
1.
1VxFlexOS62.
2RancherforKubernetes72.
2.
1HowRancherdeliversproduction-gradeKubernetesatscale.
73Solutioninfrastructure93.
1Logicalarchitecture93.
2Two-layernetworktopology.
104InstallationofRancherKubernetescluster124.
1Requirements124.
2InstallKubernetesclusterusingRKE144.
3InstallRancherserver.
165VxFlexOSContainerStorageInterfacedriver.
195.
1Overview.
195.
2Prerequisites.
195.
3InstallCSIdrivesforVxFlexOS206Conclusion.
23AAppendix.
24A.
1Configurationdetails.
24A.
2RancherKubernetesclusterdetails.
25A.
2.
1cluster.
yamlfiledetails25BTechnicalsupportandresources27B.
1Relatedresources.
27Executivesummary4RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|000066ExecutivesummaryContainertechnologyenablesdevelopmentteamstoquicklyprovisionisolatedapplicationswithoutthetraditionalcomplexities.
Formanycompanies,toboostproductivityandtimetovalue,theuseofcontainersstartswiththedepartmentsthatarefocusedonsoftwaredevelopment.
Kubernetesorchestrationprovidescapabilitiessuchasautoscaling,security,andmanagingcontainerizedapplications.
InordertoruncontainerizedapplicationswithinaKubernetescluster,apersistentandstabledatastoreisrequired,onethatcanoutlastthepodoreventhenodeitisrunningon.
RancherisaKubernetesmanagementplatformthatsimplifiesclusterinstallationandoperations,whethertheyareon-premises,inthecloudorattheedge,givingDevOpsteamsthefreedomtobuildandruncontainerizedapplicationsanywhere.
TheVxFlexfamilyofferskeyvaluepropositionsfortraditionalandcloud-nativeproductionworkloads,deploymentflexibility,linearscalability,predictablehighperformance,andenterprise-graderesilience.
ThispaperdiscusseshowtodeployaKubernetesclusterusingRancherKubernetesEngine(RKE)onaVxFlexFamilyclusterandtheuseoftheVxFlexOSCSIdrivertoprovisionpersistentstorageforKubernetesmanagingcontainerizedapplications.
Introduction5RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|0000661Introduction1.
1ObjectiveThegoalofthispaperistoprovideasimpleapproachtocreatingaKubernetesclusterusingRancherKubernetesEngineandmanagedusingRancherasthecontainerorchestrationlayeronDellEMCVxFlexfamilyplatformtomeettheperformance,scalability,resiliency,andavailabilityrequirements.
ThedocumentalsooutlinesinformationaboutleveragingDellEMCVxFlexOSCSIdrivertodynamicallyprovisionpersistentvolumesonaRanchermanagedKubernetescluster.
1.
2AudienceTheaudienceforthispaperincludessalesengineers,fieldconsultants,ITadministrators,customers,andanyoneelseinterestedinconfiguringanddeployingaKubernetesclusterusingRKEandusingtheVxFlexOSCSIdrivertodynamicallyprovisionpersistentvolumesinaRanchermanagedKubernetescluster.
ReadersareexpectedtohaveanunderstandingandworkingknowledgeofContainers,Kubernetes,andVxFlexOS.
1.
3TerminologyThefollowingtabledefinesacronymsandtermsthatareusedthroughoutthisdocument:TermsanddefinitionsTermDefinitionCACertificateAuthorityCSIContainerStorageInterfaceDNSDomainNameSystemMDMMetaDataManagerOSOperatingSystemPVPersistentVolumePVCPersistentVolumeClaimRKERancherKubernetesEngineSDCStorageDataClientforVxFlexSDSStorageDataServerforVxFlexSSDSolid-StateDiskforVxFlexTLSTransportLayerSecurityVLANVirtualLocalAreaNetworkVMVirtualMachineProductoverview6RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|0000662Productoverview2.
1VxFlexfamilyTheVxFlexsoftware-definedstorageplatformisdesignedtodeliverflexibility,elasticityandsimplicitywithpredictableperformanceandresiliencyatscalebycombiningcomputeaswellashigh-performancestorageresourcesinamanagedunifiedfabric.
VxFlexisanidealplatformformodernizinginfrastructuretoimproveITagility.
TheVxFlexFamilycurrentlyincludesVxFlexReadyNodes,VxFlexappliance,andVxFlexintegratedrack.
VxFlexfamily2.
1.
1VxFlexOSVxFlexOSisthesoftwarefoundationofVxFlexsoftware-definedstorage.
Itisascale-outblockstorageservicethatisdesignedtodeliverflexibility,elasticityandsimplicitywithpredictablehighperformanceandresiliencyatscale.
VxFlexOSconsistsofthefollowingcomponents:StorageDataClient(SDC)Providesfront-endvolumeaccesstoapplicationsandfilesystemInstalledonserversconsumingstorageMaintainspeer-to-peerconnectionstoeverySDSmanagingapoolofstorageStorageDataServer(SDS)Abstractslocalstorage,maintainsstoragepools,andpresentsvolumestotheSDCsInstalledonserverscontributinglocalstoragetotheVxFlexOSclusterProductoverview7RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|000066MetaDataManager(MDM)Overseesstorageclusterconfigurations,monitoring,rebalances,andrebuildsHighlyavailable,independentclusterinstalledonthreeorfivedifferentnodesMayresidealongsideSDCsorSDSs,oronseparatenodesSitsoutsidethedatapathGatewayPerformsinstallationandconfigurationchecksActsasanendpointforAPIcallsandpassesthemtoMDM2.
2RancherforKubernetesRancheristheenterprisecomputingplatformtorunKuberneteson-premises,inthecloudandattheedge.
ItaddressestheoperationalandsecuritychallengesofmanagingmultipleKubernetesclusterseverywhere.
RancheralsoprovidesIToperatorsanddevelopmentteamswithintegratedtoolsforbuilding,deploying,andrunningcloud-nativeworkloads.
Ranchernotonlydeploysproduction-gradeKubernetesclustersfromdatacentertocloudtotheedge,italsounitesthemwithcentralizedauthentication,accesscontrolandobservability.
Rancherletsyoustreamlineclusterdeploymentonbaremetal,edgedevices,privateclouds,publicclouds,orvSphereandsecurethemusingglobalsecuritypolicies.
UseHelmorRancherAppCatalogtodeployandmanageapplicationsacrossanyoralltheseenvironments,ensuringmulti-clusterconsistencywithasingledeployment.
2.
2.
1HowRancherdeliversproduction-gradeKubernetesatscaleRancher'srecipeforproductionqualityKubernetesatscaleProductoverview8RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|000066CertifiedKubernetesDistributionsRanchersupportsanycertifiedKubernetesdistribution.
Foron-premisesworkloads,RancherofferstheRKE-aCNCF-certifiedKubernetesdistributionthatrunsentirelywithinDockercontainers.
Itworksonbare-metalandvirtualizedservers.
RKEsolvestheproblemofinstallationcomplexity,acommonissueintheKubernetescommunity.
WithRKE,theinstallationandoperationofKubernetesisbothsimplifiedandeasilyautomated,anditisentirelyindependentoftheoperatingsystem.
Forthepubliccloud,allthemajordistributionsaresupported,includingEKS,AKS,andGKE.
Foredge,branchanddesktopworkloadsRancheroffersK3s,acertifiedlightweightdistributionofKubernetes.
SimplifiedClusterOperationsRancherprovidessimple,consistentclusteroperationsincludingprovisioning,versionmanagement,visibilityanddiagnostics,monitoringandalerting,andcentralizedaudit.
Security,Policy,andUserManagementRancherletsyouautomateprocessesandappliesaconsistentsetofuseraccessandsecuritypoliciesforallyourclusters,nomatterwheretheyarerunning.
SharedToolsandServicesRancherprovidesarichcatalogofservicesforbuilding,deploying,andscalingcontainerizedapplications,includingapppackaging,CI/CD,logging,monitoring,andservicemesh.
Solutioninfrastructure9RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|0000663SolutioninfrastructureThissolutioncouldbedeployedonanyoftheVxFlexfamily.
Inthissolution,itisdeployedinatwo-layerconfigurationusingcomputeonlynodesdeployedwithVMwareESXihypervisoranddedicatedstorageonlynodestoprovidetherequiredstoragecapacity.
3.
1LogicalarchitectureThediagrambelowshowsthelogicalarchitecturediagramoftheRKEclusterdeployedonatwo-layerVxFlexclustersetupwithfourstorageonlynodesandthreecomputeonlynodes.
LogicalarchitectureEachStorageOnlynodeincludestwoIntelXeonSkylake8-coreprocessors,192GBRAM,andsix3.
84TBSSDs.
FromtheVxFlexOSstandpoint,theRedHatEnterpriseLinux7.
6StorageonlynodesruntheSDScomponentofVxFlexOStoprovidestoragecapacity.
AsingleprotectiondomainiscreatedfromthesefourSDSsystems.
AsinglestoragepooliscreatedwithinthisProtectionDomainfromwhichpersistentstoragevolumescanbeprovisionedforKubernetespods.
ThethreeESXi6.
7ComputeonlynodesruntheSDCcomponentsoftheVxFlexOSinordertoaccessthevolumescreatedinthestoragepool.
TheRKEclusterisdeployedasvirtualmachinesonaVMwarevSphere6.
7cluster.
ForRanchertowork,theminimumhardwarerequirementtorunKubernetesnodecomponentsisoneCPUand1GBofmemory.
ConsideringCPUandMemory,itisrecommendedtohostthedifferentrolesoftheKubernetescluster(etcd,controlplane,andworkers)ondifferentnodessothattheycanscaleindependentlyfromoneanother.
Inthissolution,eachvirtualmachineisconfiguredwith4virtualCPUs,8GBRAM,andthickprovisioneddisksontheVxFlexbackendstorage.
Solutioninfrastructure10RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|000066ThemanagementnodehoststhevCenterappliance,VxFlexOSGateway,DNS,andworkstationforRKE.
Fordetailedconfigurationoftwo-layerVxFlexclusternodes,seeA.
1Configurationdetails.
3.
2Two-layernetworktopologyThefollowingdiagramdepictsthetwo-layernetworkarchitecturethatisbaseduponVxFlexOSbestpractices:NetworkarchitectureSolutioninfrastructure11RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|000066TwoDellS5048FswitchesareconfiguredwithVLTtoprovidefaulttoleranceandenableconnectivitywithotherswitches.
Storagenodes,Computenodes,Management,andotherTeamedNetworksuseLinkAggregationGroups(LAG).
Twodualport25GbMellanoxNICsoneachserverprovide4x25Gbports.
VLAN51and52arededicatedtoSDS-SDScommunication.
BothVLANsareisolatedatswitchleveltoprovidehighavailabilityandavoidasinglepointoffailure.
VLAN53and54arededicatedtoSDS-SDCcommunication.
BothVLANsareisolatedatswitchleveltoprovidehighavailabilityandavoidasinglepointoffailure.
Onstoragenodes,2x10GbportsareNICteamedonVLAN55isdedicatedtoVxFlexOSmanagement.
Oncomputenodes,2x25GbportsareNICteamedtoprovidehighavailability.
ANICteamingnetworkisusedtocreateVirtualNetworksinvSphere.
VLAN56isconfiguredtoprovideconnectivitywiththecustomernetwork,VLAN57isdedicatedvMotion,andVLAN50isdedicatedtoHypervisormanagement.
InstallationofRancherKubernetescluster12RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|0000664InstallationofRancherKubernetesclusterThissectionincludestheKubernetesinstallationusingRKEandtheconfigurationofVxFlexOSCSIdriveronatwo-layerVxFlexcluster.
RanchersupportsKubernetesclustersrunningUbuntu,CentOS,OracleLinux,RancherOS,andRedHatEnterpriseLinux.
ForinformationaboutthesupportedOperatingSystem,Docker,andRancherversions,seeRancher-AllSupportedVersions.
4.
1RequirementsTherequirementsforbuildingRancherKubernetesclusterusingRKEareasfollows:RequirementforRKEclusteronVxFlexFamilyNameVersionDescriptionReferenceRancherserver2.
3.
3RancherserverwillbeusedfromtheWorkstationVM.
https://releases.
rancher.
com/server-charts/latestRancherKubernetesEngine(RKE)1.
0.
0RKEisusedfromWorkstationVM.
https://github.
com/rancher/rke/releases/latestKubectl1.
13.
12KubectltointeractwithKubernetescluster.
https://kubernetes.
io/docs/tasks/tools/install-kubectl/Docker18.
06.
3DockerisinstalledoneachRHELnode.
https://download.
docker.
com/linux/centos/docker-ce.
repo3RHEL7.
6nodeswithSELinuxandswapdisabled7.
6EnsurethenodesmustbeaccessusingSSHandtherequiredportsmustbeopenedbeforeclusterinstallation.
https://rancher.
com/docs/rke/latest/en/os/#portsVxFlexOSCSI1.
1.
3VxFlexOSCSIisusedfromtheWorkstationVM.
https://github.
com/dell/csi-vxflexosTheRKEbinaryisavailableforWindowsandLinuxoperatingsystem.
Inthissolution,RKEisrunfromaLinuxworkstationvirtualmachine.
RKEconnectstothenodesusingSSHkeypairs.
InstallationofRancherKubernetescluster13RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|000066NodesconnectusingSSHkeyTheSSHloginusedfornodeaccessmusthaveaccesstotheDockersocket/var/run/docker.
sockonthemachine,inordertoprovidethisaccess,theaccountmustbeaddedasamemberofthedockergrouponthenodeNote:Therootaccountisnotdisabledbydefault.
ItisrecommendednottouserootastheSSHuser,refertoBugzilla1527565fordetails.
Procedure:1.
CreateLinuxuseraccountoneverynode.
#useradd-m-Gdockerrke#su-rke#mkdir$HOME/.
ssh#chmod700$HOME/.
ssh#touch$HOME/.
ssh/authorized_keys2.
TestDockersocketaccess.
$dockerversionInstallationofRancherKubernetescluster14RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|0000663.
CreateanSSHkeypairusingthefollowingcommandfromtheWorkstationVMwhereRKEbinaryexists:$ssh-keygenThefollowingfilesarecreated:$HOME/.
ssh/id_rsa(SSHprivatekey,keepthissecure)$HOME/.
ssh/id_rsa.
pub(SSHpublickey)4.
CopytheSSHpublickeytotheKubernetesnodestoprovideaccesstothenodes,usingthecreatedSSHkeypair:$cat.
ssh/id_rsa.
pub|sshnode1"cat>>.
ssh/authorized_keys"5.
TestSSHconnectivitybyrunningthefollowingcommandfromtheWorkstationVM,replacing"hostname"witheachoftheKubernetesnodesIPorhostname.
$ssh-i$HOME/.
ssh/id_rsarke@hostnamedockerversion4.
2InstallKubernetesclusterusingRKERKEisafast,simpleKubernetesinstallerusedtoinstallKubernetesonLinuxnodes.
1.
DownloadthelatestRKEinstallerapplicabletotheoperatingsystembeingusedandkeeptheRKEbinaryinarkeuserhomedirectory(/home/rke).
2.
Moverke_linux-amd64binaryasrkeandmaketheRKEbinaryexecutableandconfirm.
$mvrke_linux-amd64rke$chmod+xrke$rke--version3.
Createthecluster.
ymlfilebyrunning".
/rkeconfig"andansweringthequestions.
,thisfilecontainsallinformationrequiredtobuildtheKubernetescluster,suchasnodeconnectioninformationandroles(controlplane,etcd,worker)toapplytoeachnode.
[rke@rancher~]$.
/rkeconfig[+]ClusterLevelPrivateKeyPath[~/.
ssh/id_rsa]:[+]NumberofHosts[1]:3[+]SSHAddressofhost(1)[none]:rk8s-w1.
.
[+]NetworkPluginType(flannel,calico,weave,canal)[canal]:[+]AuthenticationStrategy[x509]:[+]AuthorizationMode(rbac,none)[rbac]:InstallationofRancherKubernetescluster15RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|000066[+]KubernetesDockerimage[rancher/hyperkube:v1.
16.
3-rancher1]:rancher/hyperkube:v1.
13.
12-rancher1[+]Clusterdomain[cluster.
local]:[+]ServiceClusterIPRange[10.
43.
0.
0/16]:[+]ClusterNetworkCIDR[10.
42.
0.
0/16]:[+]ClusterDNSServiceIP[10.
43.
0.
10]:[+]AddaddonmanifestURLsorYAMLfiles[no]:no[rke@rancher~]$4.
Checkthatthecluster.
ymlfileiscreatedinthesamedirectoryasRKEwasrun.
[rke@rancher~]$ls-ltr-rwxrwxr-x1rkerke46135151Dec2504:42rke-rw-r-----1rkerke4717Dec2605:21cluster.
yml[rke@rancher~]$5.
Editthecluster.
ymlfiletoenabletheadditionalfeaturegatesbeforeKubernetesclustercreation.
Thedetailsoftheadditionalfeaturegatesthatareupdatedincluster.
ymlasextra_argsareshowninA.
2.
1cluster.
yamlfiledetails.
6.
BuildtheRKEclusterusingthefollowingcommand:[rke@rancher~]$.
/rkeupINFO[0000]RunningRKEversion:v1.
0.
0INFO[0000]InitiatingKubernetesclusterINFO[0000][certificates]GeneratingadmincertificatesandkubeconfigINFO[0000]SuccessfullyDeployedstatefileat[.
/cluster.
rkestate]INFO[0000]BuildingKubernetesclusterINFO[0000][dialer]Setuptunnelforhost[rk8s-w3]INFO[0000][dialer]Setuptunnelforhost[rk8s-w2]INFO[0000][dialer]Setuptunnelforhost[rk8s-w1].
.
.
.
INFO[0048][addons]SuccessfullysavedConfigMapforaddonrke-ingress-controllertoKubernetesINFO[0048][addons]Executingdeployjobrke-ingress-controllerINFO[0053][ingress]ingresscontrollernginxdeployedsuccessfullyINFO[0053][addons]SettingupuseraddonsINFO[0053][addons]nouseraddonsdefinedINFO[0053]FinishedbuildingKubernetesclustersuccessfully[rke@rancher~]$Note:ItisveryimportanttoaddKubernetesadd-onslikefeature-gates,servicearguments,volumebinds,andenvironmentvariablesbeforebuildingRancherKubernetesclusterusingrkeupcommand.
TheKubernetesfeature-gatesmustbeenabledbeforeinstallingtheCSIDriverforVxFlexOS.
InstallationofRancherKubernetescluster16RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|0000667.
CheckthatthenodesareinareadystateafterasuccessfulKubernetesclustercreation.
Theoutputbelowshowsthatallthenodeshavebothmasterandworkerrolesinstalled.
[rke@rancher~]$kubectlgetnodesNAMESTATUSROLESAGEVERSIONrk8s-w1Readycontrolplane,etcd,worker9hv1.
13.
12rk8s-w2Readycontrolplane,etcd,worker9hv1.
13.
12rk8s-w3Readycontrolplane,etcd,worker9hv1.
13.
12[rke@rancher~]$4.
3InstallRancherserverRancherserverisinstalledusingtheHelmpackagemanagerforKubernetes.
1.
InstallHelmandTillerwithaserviceaccount.
Fordetailedinstruction,seetheHelmQuickstartGuide.
2.
AddtheHelmchartrepositorythatcontainschartstoinstallRancher.
helmrepoaddrancher-latesthttps://releases.
rancher.
com/server-charts/latest3.
CreateanamespaceforRancherascattle-system.
kubectlcreatenamespacecattle-systemThecert-managerisaKubernetesadd-ontoautomatethemanagementandissuanceofTLScertificatesfromvariousissuingsources.
Rancherreliesoncert-managertoissuecertificatesfromRancher'sowngeneratedCAortorequestLetusEncryptcertificates.
4.
Installcert-managerbyfollowingthesesteps:$kubectlapply--validate=false-fhttps://raw.
githubusercontent.
com/jetstack/cert-manager/release-0.
12/deploy/manifests/00-crds.
yaml$kubectlcreatenamespacecert-manager$kubectllabelnamespacecert-managercertmanager.
k8s.
io/disable-validation=true$helmrepoaddjetstackhttps://charts.
jetstack.
io$helmrepoupdate$helminstall\>--namecert-manager\>--namespacecert-manager\>--versionv0.
12.
0\>jetstack/cert-managerInstallationofRancherKubernetescluster17RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|0000665.
Checkthecert-managernamespaceforrunningpodstoverifyitisdeployedcorrectly.
$kubectlgetpods-ncert-managerNAMEREADYSTATUSRESTARTSAGEcert-manager-75cf57777c-ztw9f1/1Running02m2scert-manager-cainjector-f54c57bf8-wkc2z1/1Running02m2scert-manager-webhook-76794c6967-84gb61/1Running02m2s6.
InstallRancherwithHelm.
[root@rancher~]#helminstallrancher-latest/rancher\>--namerancher\>--namespacecattle-system\>--sethostname=rancher.
rack9lab.
comNAME:rancherLASTDEPLOYED:TueJan2103:21:582020NAMESPACE:cattle-systemSTATUS:DEPLOYED.
.
.
.
NOTES:RancherServerhasbeeninstalled.
NOTE:Ranchermaytakeseveralminutestofullyinitialize.
PleasestandbywhileCertificatesarebeingissuedandIngresscomesup.
CheckoutRancherdocsathttps://rancher.
com/docs/rancher/v2.
x/en/Browsetohttps://rancher.
rack9lab.
comHappyContainering!
[root@rancher~]#7.
Checkthecattle-systemnamespaceforrunningpodstoverifyifthenamespaceisdeployedcorrectly.
[root@rancherhelm]#kubectlgetpods–cattle-systemNAMESPACENAMEREADYSTATUSRESTARTSAGEcattle-systemcattle-cluster-agent-85db6685cd-vvft91/1Running07h19mcattle-systemcattle-node-agent-6c8nw1/1Running07h19mcattle-systemcattle-node-agent-jhw2n1/1Running07h19mcattle-systemcattle-node-agent-wh78g1/1Running07h19mcattle-systemrancher-5d549cb9c8-2xmj51/1Running07h23mcattle-systemrancher-5d549cb9c8-8bbzs1/1Running07h23mcattle-systemrancher-5d549cb9c8-bh22p1/1Running07h23mResult:Rancherisupandrunning.
InstallationofRancherKubernetescluster18RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|0000668.
Browsetohttps://rancher.
rack9lab.
comtoaccessthefunctionalRancherserver.
Result:ThethreenodesarelistedinthelocalclusterthatrepresenttheRancherManagementCluster.
VxFlexOSContainerStorageInterfacedriver19RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|0000665VxFlexOSContainerStorageInterfacedriver5.
1OverviewTheCSIdriverforVxFlexOSisaplug-inthatisinstalledintoKubernetestoprovidepersistentstorageusingVxFlexOSstoragesystem.
TheCSIdriverforVxFlexOSandKubernetescommunicateusingtheContainerStorageInterface(CSI)protocol.
TheCSIdriverforVxFlexOSsupportsPersistentvolume(PV)capabilities,DynamicandStaticPVprovisioning,andSnapshotcapabilities.
TheHelmchartinstallstheCSIdriverforDellEMCVxFlexOSusingashellscript.
ThisscriptinstallstheCSIdrivercontainerimagealongwiththerequiredKubernetessidecarcontainers.
ThecontrollersectionoftheHelmchartinstallsthefollowingcomponentsinaStatefulSetinthenamespacevxflexos:CSIdriverforDellEMCVxFlexOSKubernetesProvisioner,whichprovisionsthevolumesKubernetesAttacher,whichattachesthevolumestothecontainersKubernetesSnapshotter,whichprovidessnapshotsupportThenodesectionoftheHelmchartinstallsthefollowingcomponentinaDaemonSetinthenamespacevxflexos:CSIdriverforDellEMCVxFlexOSKubernetesRegistrar,whichhandlesthedriverregistration5.
2PrerequisitesBeforeyouinstalltheCSIDriverforVxFlexOS,verifythatthefollowingrequirementsareinstalledandconfigured:Kubernetesv1.
13.
xDockerconfiguredforbi-directionalmount(MountFlags=shared)KubernetesFeatureGatesenabledforSnapshotsandCSIsupportHelmandTillerthatisinstalledonaMasternodeVxFlexOSSDCinstalledonallKubernetesnodesKubernetessecretcreatedforVxFlexOScredentialsHelmvalues(yamltemplate)forthingslikeVxFlexOSsystemnameorID,defaultgateway,andMDMIPaddresses,defaultstoragepool.
Fordetailedinformation,seeGitHub.
VxFlexOSContainerStorageInterfacedriver20RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|0000665.
3InstallCSIdrivesforVxFlexOS1.
DownloadtheinstallationsourcefilesfromGitHub.
#gitclonehttps://github.
com/dell/csi-vxflexos2.
Createthenamespacecalledvxflexos.
#kubectlcreatenamespacevxflexos3.
CollectinformationfromtheVxFlexOSSDC(StorageDataClient)byrunningtheget_vxflexos_info.
shscript.
4.
Copythecsi-vxflexos/values.
yamlintoafileinthesamedirectoryastheinstall.
vxflexosnamedmyvalues.
yaml,tocustomizesettingsforinstallation.
5.
Editmyvalues.
yamltosettheparameterslikeVxFlexOSsystemnameorID,defaultgatewayandMDMIPaddresses,defaultstoragepool.
6.
Runtheshinstall.
vxflexoscommandtoproceedwiththeinstallation.
[root@rancherhelm]#shinstall.
vxflexosKubernetesversionv1.
13.
12Kubernetesmasternodes:Kubernetesminionnodes:10.
10.
56.
8110.
10.
56.
8210.
10.
56.
83VerifyingtheSDCinstallation.
.
.
.
.
NAME:vxflexosLASTDEPLOYED:TueJan2110:42:492020NAMESPACE:vxflexosSTATUS:DEPLOYED.
.
.
.
running4/4NAMEREADYSTATUSRESTARTSAGEvxflexos-controller-04/4Running031svxflexos-node-k8s5t2/2Running031svxflexos-node-ks5jd2/2Running031svxflexos-node-xvbb52/2Running031sCSIDrivers:NAMEAGEvxflexos41sCSINodeInfos:NAMEAGErk8s-w132srk8s-w213srk8s-w317sStorageClasses:NAMEPROVISIONERAGEvxflexos(default)csi-vxflexos41svxflexos-xfscsi-vxflexos41sVxFlexOSContainerStorageInterfacedriver21RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|000066Noresourcesfound.
installingvolumesnapshotclassvolumesnapshotclass.
snapshot.
storage.
k8s.
io/vxflexos-snapclasscreatedVolumeSnapshotClasses:NAMEAGEvxflexos-snapclass0s[root@rancherhelm]#7.
Checkthevxflexosnamespaceforrunningpodstoverifyitisdeployedcorrectly.
[root@rancher~]#kubectlgetpods-nvxflexosNAMEREADYSTATUSRESTARTSAGEvxflexos-controller-04/4Running031dvxflexos-node-k8s5t2/2Running031dvxflexos-node-ks5jd2/2Running031dvxflexos-node-xvbb52/2Running031d[root@rancher~]#FordetailsinformationaboutCSIdriverinstallation,seeGitHub.
8.
CheckthattheVxFlexOSCSIdriverisrunningontheRancherKubernetesclusterandtheVxFlexOSStorageclassesarelistedintheRanchermenutobeusedbyanyapplicationlaunchedintheRancherKubernetescluster.
8.
TestthedeploymentworkflowofasimplepodonVxFlexOSstorage.
ThehelmchartcontainstwoPersistentVolumeClaimdefinitions.
TheStorageClassvxflexosyieldsamountedext4filesystemandanotherStorageClassvxflex-xfsgivesaxfsfilesystem.
ForthecompletereferencetotestCSIdrivesonVxFlexOS,seeGitHub.
root@rancher~]#kubectlgetpods-ntest-owideNAMEREADYSTATUSRESTARTSAGEIPNODENOMINATEDNODEREADINESSGATESvxflextest-01/1Running66131d10.
10.
56.
82rk8s-w2[root@rancher~]#VxFlexOSContainerStorageInterfacedriver22RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|000066[root@rancherhelm]#kubectlgetpvc-ntestNAMESTATUSVOLUMECAPACITYACCESSMODESSTORAGECLASSAGEpvol0Boundk8s-59c253473c8GiRWOvxflexos71spvol1Boundk8s-59c264333c16GiRWOvxflexos-xfs71s[root@rancherhelm]#9.
VerifythatthepersistentvolumesarecreatedinVxFlexclusterusingtheVxFlexOSGUI.
10.
VerifythefollowingoutputfromRancherUIandconfirmthattheVxFlexOSPersistentVolumesarebeingusedbyKubernetespodsrunningundertestnamespace.
Conclusion23RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|0000666ConclusionTheKubernetesecospherecontinuestogrowinhugestrides,providingmorestability,security,andautomaticservicediscovery.
Streamliningsomeofthebasicoperations,suchastheKubernetesclustersetupandoutlinealongwithdynamicallyprovisioningpersistentstorageusingRancherandtheVxFlexOSCSIonVxFlexfamilyempowersadminstodeploytheirKubernetesenvironmentquicklyfordevelopersandend-users,enablinguninterruptedutilizationofinfrastructure.
Appendix24RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|000066AAppendixA.
1ConfigurationdetailsNote:DellEMCPowerEdgeR740xdserver.
Availablestorageonstoragenodesistheonlydifferencebetweenstorageandcomputenodes.
SSDsonstoragenodesarebeenusedtoformastoragepoolandvolumesthatareusedbycomputenodes.
StorageonlynodesHardwareConfigurationCPUCores2x8IntelXeonGold6134CPU@3.
20GHzMemory12x16GBDDR4NIC2xMLNX25GbE2PConnectX4LXAdpt1xIntel2PX710/2PI350rNDCStorageBOSSS1Controller2x120GBSATASSDDellHBA330controller6x3.
84TBSASSSDoperatingsystemRHEL7.
6FirmwareVersion3.
21.
26.
22VxFlexOS3.
0.
1ComputeonlynodesHardwareConfigurationCPUCores2x8IntelXeonGold6134CPU@3.
20GHzMemory12x16GBDDR4NIC2xMLNX25GbE2PConnectX4LXAdpt1xIntel2PX710/2PI350rNDCStorageBOSSS1Controller2x120GBSATASSDDellHBA330controller2x1.
92TBSASSSDHypervisorESXi6.
7FirmwareVersion3.
21.
26.
22VxFlexOS3.
0.
1Appendix25RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|000066A.
2RancherKubernetesclusterdetailsRancherManagementclusterdetailsComponentsItemsDetailsHardwareVirtualizedHardwarevSphere6.
7CPU4vCPURAM8GBHardDisk40GBNICVMwareVirtualNICSoftwareOperatingSystemRHEL7.
6ContainerRuntimeDocker-18.
06.
3-ceRancherv2.
3.
3RancherKubernetesv1.
13.
12RKEv1.
0.
0VxFlexOSCSIdriverv1.
1.
3A.
2.
1cluster.
yamlfiledetails[rke@rancher~]$catcluster.
yml.
.
.
.
kube-api:image:""extra_args:anonymous-auth:"false"#enable-admission-plugins:"ServiceAccount,NamespaceLifecycle,LimitRanger,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota,DefaultTolerationSeconds,AlwaysPullImages,DenyEscalatingExec,NodeRestriction,EventRateLimit,PodSecurityPolicy"profiling:"false"service-account-lookup:"true"tls-cipher-suites:"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256"feature-gates:"VolumeSnapshotDataSource=true,KubeletPluginsWatcher=true,CSINodeInfo=true,CSIDriverRegistry=true,BlockVolume=true,CSIBlockVolume=true"extra_binds:[]extra_env:[]service_cluster_ip_range:10.
43.
0.
0/16service_node_port_range:""pod_security_policy:falsealways_pull_images:falsesecrets_encryption_config:nullAppendix26RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|000066audit_log:nulladmission_configuration:nullevent_rate_limit:nullkube-controller:image:""extra_args:profiling:"false"address:"127.
0.
0.
1"terminated-pod-gc-threshold:"1000"feature-gates:"VolumeSnapshotDataSource=true,KubeletPluginsWatcher=true,CSINodeInfo=true,CSIDriverRegistry=true,BlockVolume=true,CSIBlockVolume=true"extra_binds:[]extra_env:[]cluster_cidr:10.
42.
0.
0/16service_cluster_ip_range:10.
43.
0.
0/16scheduler:image:""extra_args:profiling:"false"address:"127.
0.
0.
1"feature-gates:"VolumeSnapshotDataSource=true,KubeletPluginsWatcher=true,CSINodeInfo=true,CSIDriverRegistry=true,BlockVolume=true,CSIBlockVolume=true"extra_binds:[]extra_env:[]kubelet:image:""extra_args:streaming-connection-idle-timeout:"1800s"authorization-mode:"Webhook"protect-kernel-defaults:"true"make-iptables-util-chains:"true"event-qps:"0"anonymous-auth:"false"feature-gates:"VolumeSnapshotDataSource=true,KubeletPluginsWatcher=true,CSINodeInfo=true,CSIDriverRegistry=true,BlockVolume=true,CSIBlockVolume=true"tls-cipher-suites:"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256"extra_binds:[]extra_env:[]cluster_domain:cluster.
localinfra_container_image:""cluster_dns_server:10.
43.
0.
10fail_swap_on:falsegenerate_serving_certificate:falsekubeproxy:image:""extra_args:{}extra_binds:[]extra_env:[].
.
[rke@rancher~]$Technicalsupportandresources27RancherKubernetesClusterusingCSIDriverforDellEMCVxFlexfamily|000066BTechnicalsupportandresourcesDell.
com/supportisfocusedonmeetingcustomerneedswithprovenservicesandsupport.
DellEMCVxFlexproductoverviewandWhitepapersonDellEMC.
comprovidesexpertisethathelpstoensurecustomersuccessonDellEMCVxFlexfamily.
StoragetechnicaldocumentsandvideosprovideexpertisethathelpstoensurecustomersuccessonDellEMCstorageplatforms.
B.
1RelatedresourcesNote:Fewlinksmightrequireregistrationtoaccess.
DellEMCVxFlexProductOverviewVxFlexwithKubernetesCSIDriverforVxFlexOSProductGuideRancherProductOverviewRancherInstallationRancherSupportMatrix