中国amazonec2

AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营在AWS上构建无服务器管理的容器应用实践张浙AmazonWebServicesAWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营容器应用在AWS平台上的部署演进无服务管理的容器平台Fargate技术实践Fargate最新技术发布AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营容器应用在AWS平台上的部署演进AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营最开始阶段AmazonEC2AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营然后是Docker开发人员开始在EC2实例上实现应用容器化EC2InstanceContainersAWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营容器技术使得构建和发布云原生应用服务变得更加简单AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营开发人员需要一种更简单的方式来管理大规模的容器集群实例,监控容器的运行及容器化后的业务服务的状态AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营调度和排程集群管理器置放引擎AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营ECSAgentDockerAgentOSEC2Instance开发人员仍然需要关注容器化业务以外的管理工作AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营操作系统管理:软件补丁、系统升级、管理Agent软件集群管理:设定EC2实例集群规模、管理集群资源使用率大规模EC2实例管理仍然是一件很繁杂的工作AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营ECSAgentDockerAgentOSEC2InstanceECSAgentDockerAgentOSEC2InstanceECSAgentDockerAgentOSEC2InstanceElasticContainerServiceAWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营YourContainerizedApplications完全AWS管理无需配置、运行、管理EC2实例弹性集群无感知扩容收缩,只需为您实际消耗的资源付费整合AWS生态和AWS服务生态完全整合:VPCNetworking,ElasticLoadBalancing,IAM权限管理,Cloudwatch以及其他.
AWSFARGATEAWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营AWSContainer服务全景管理服务部署,调度,Scaling&弹性扩展容器化的业务托管服务容器化业务运行平台AmazonElasticContainerServiceAmazonElasticContainerServiceforKubernetesAmazonEC2AWSFargate镜像仓库容器镜像仓库AmazonElasticContainerRegistryAWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营Fargate技术实践AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营示例演示三连线游戏Scorekeep,使用容器部署在Fargate平台FrontendServerContainerAngular+NginxAPIServerContainerJavaInternetPort8080Port5000LoadBalancerDynamoDBSNS一步一步演示配置:计算,网络,存储,权限,日志管理然后实际运行这个小程序!
AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营在Fargate上构造应用AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营定义应用容器属性:容器镜像URL,CPU&内存需求,存储网络设定等等registerTaskDefinitioncreateCluster基础设施隔离边界IAM权限边界runTaskTaskDefination定义任务的一个运行时状态使用Fargate运行模式createServiceElasticLoadBalancer维护多个在线应用副本和弹性负载均衡器整合自动替换不健康的task在Fargate上构造应用AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营TASKDEFINITION{"family":"scorekeep","containerDefinitions":[{"name":"scorekeep-frontend","image":"xxx.
dkr.
ecr.
us-east-1.
amazonaws.
com/fe"},{"name":"scorekeep-api","image":"xxx.
dkr.
ecr.
us-east-1.
amazonaws.
com/api"}]}静态不可变的(Immutable),带有版本控制的文档通过family:version格式识别最多包含10个容器定义所有的容器都会被调度到同一个实例上每个容器定义主要包含:容器名字docker镜像URL(ECR或者公共的镜像仓库)其他的配置…稍后介绍TaskDefinition小片段AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营容器镜像仓库3rdParty私有容器镜像仓库(即将到来!
)公共的容器镜像仓库AmazonElasticContainerRegistry(ECR)AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营计算相关配置AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营CPU&内存设定{"family":"scorekeep","cpu":"1vCpu","memory":"2gb","containerDefinitions":[{"name":"scorekeep-frontend","image":"xxx.
dkr.
ecr.
us-east-1.
amazonaws.
com/fe","cpu":256,"memoryReservation":512},{"name":"scorekeep-api","image":"xxx.
dkr.
ecr.
us-east-1.
amazonaws.
com/api","cpu":768,"memoryReservation":512}]}单位:CPU:cpu-units.
1vCPU=1024cpu-unitsMemory:MBTask级别的资源设定:所有容器可以使用到的CPU和内存的总和必须指定的设定计费基准容器级别的资源设定:定义容器之间的资源可选设定Task级别资源容器级别资源TaskDefinition小片段AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营TASKCPU内存配置组合50种不同的CPU/Memory配置选择可选CPUMemory256(.
25vCPU)512MB,1GB,2GB512(.
5vCPU)1GB,2GB,3GB,4GB1024(1vCPU)2GB,3GB,4GB,5GB,6GB,7GB,8GB2048(2vCPU)Between4GBand16GBin1GBincrements4096(4vCPU)Between8GBand30GBin1GBincrementsAWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营计费方式按秒计费,最小计费时间周期为1分钟只需要为实际使用的计算资源付费基于Task级别的CPU和内存消耗计费AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营网络相关配置AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营FargateTask的awsvpc网络工作模式172.
31.
0.
0/16Subnet172.
31.
1.
0/24InternetVPC中的其他服务EC2LBDBetc.
PrivateIP172.
31.
1.
164FargateTask在subnet中执行在此基础上:创建一个ElasticNetworkInterface(ENI)ENI从当前subnet中获取一个私有的IP地址添加ENI到Task上您的Task可以直接从subnet中获取IP地址了!
您也可以为Task指定公有IP地址通过安全组策略来设定流入和流出流量的安全规则ENIFargateTaskPublic/208.
57.
73.
13/AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营Task网络设定{"family":"scorekeep","cpu":"1vCpu","memory":"2gb","networkMode":"awsvpc","containerDefinitions":[{"name":"scorekeep-frontend","image":"xxx.
dkr.
ecr.
us-east-1.
amazonaws.
com/fe","cpu":256,"memoryReservation":512},{"name":"scorekeep-api","image":"xxx.
dkr.
ecr.
us-east-1.
amazonaws.
com/api","cpu":768,"memoryReservation":512}]}$awsecsrun-task.
.
.
--task-definitionscorekeep:1--network-configuration"awsvpcConfiguration={subnets=[subnet1-id,subnet2-id],securityGroups=[sg-id]}"创建一个ENI并且添加到Task上运行TaskTaskDefinition小片段AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营Task访问InternetTaskENI用来传输所有流入和流出Task的网络流量它也可以被用作:拉取容器镜像(从ECR或者公共的容器镜像仓库)推送日志到CloudwatchTaskENI需要对指定的endpoint可达两种常用的Task访问Internet的设定方式:私有的Task,不允许Internet流量进入,但是允许Task访问Internet公有网络访问的Task,允许Internet流量进入和流出AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营私有TASK的Internet访问设定PublicsubnetPrivatesubnetFargateTaskENIPrivateIP172.
31.
1.
164NATGatewayPublicEIP34.
214.
162.
237InternetGateway172.
31.
0.
0/16172.
31.
2.
0/24172.
31.
1.
0/24DestinationTarget172.
31.
0.
0/16local0.
0.
0.
0/0NATGatewayDestinationTarget172.
31.
0.
0/16local0.
0.
0.
0/0InternetGatewayRouteTablesInternet添加InternetGateway到VPCPublicSubnet配置到InternetGateway的路由规则添加NATGatewayPrivateSubnet配置FargateTask到NATGateway的路由配置安全组允许到internet的流量TypePortDestinationAllTrafficALL0.
0.
0.
0/0OutboundSecurityGroupRulesAWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营OutboundInbound直接Internet访问的TASK设定PublicsubnetFargateTaskPublicIP54.
191.
135.
66InternetGateway172.
31.
0.
0/16172.
31.
2.
0/24DestinationTarget172.
31.
0.
0/16local0.
0.
0.
0/0InternetGatewayRouteTableInternetENI$awsecsrun-task.
.
.
--network-configuration"awsvpcConfiguration={subnets=[public-subnet],securityGroups=[sg-id],}"在Publicsubnet中运行Task为Task分配一个公网IP地址在安全组中指定允许流入的网络流量TypePortSourceHTTP80800.
0.
0.
0/0InboundSecurityGroupRuleTypePortDestinationAllTrafficALL0.
0.
0.
0/0OutboundSecurityGroupRulesassignPublicIp=ENABLEDRunTaskAWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营ELB配置{"family":"scorekeep","cpu":"1vCpu","memory":"2gb","networkMode":"awsvpc","containerDefinitions":[{"name":"scorekeep-frontend","image":"xxx.
dkr.
ecr.
us-east-1.
amazonaws.
com/fe","cpu":256,"memoryReservation":512,"portMappings":[{"containerPort":8080}]},{"name":"scorekeep-api","image":"xxx.
dkr.
ecr.
us-east-1.
amazonaws.
com/api","cpu":768,"memoryReservation":512,"portMappings":[{"containerPort":5000}]}]}$awsecscreate-service.
.
.
--task-definitionscorekeep:1--network-configuration"awsvpcConfiguration={subnets=[subnet-id],securityGroups=[sg-id]}"--load-balancers"[{"targetGroupArn":"","containerName":"scorekeep-frontend","containerPort":8080}]"创建服务TaskDefinitionAWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营面向INTERNET访问的ELB在VPC中的设定PublicsubnetPrivatesubnetFargateTaskENIPrivateIP172.
31.
1.
164:8080ALBPublicIP208.
57.
73.
13:80172.
31.
0.
0/16172.
31.
2.
0/24172.
31.
1.
0/24InternetTask在privatesubnet中分配到了私有的IP地址ALB在publicsubnet中分配到了公网的IP地址确定两个子网的可用区是匹配的设定ALB的securitygroup允许来自互联网的指定流量设定Task的securitygroup允许来自ALB的securitygroup的流量TaskSecurityGroupALBSecurityGroupTypePortSourceHTTP800.
0.
0.
0/0InboundRuleTypePortSourceCustomTCP8080ALBSecurityGroupInboundRuleus-east-1aus-east-1aAWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营存储相关配置AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营磁盘存储基于EBS一次性存储,提供下列存储机制:VolumeStorageWritableLayerStorageAWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营LAYERSTORAGEDocker镜像由多个存储层组成.
最顶层为写入层,用来保存在容器运行产生时的临时文件Fargate允许每个Task使用10GBLayerstorage,包括所有的容器以及镜像使用的存储空间容器内写操作在其他容器中不可见一次性存储(Ephemeral).
Task停止后,容器内数据无法找回ImageLayersWritableLayerImageLayersWritableLayerContainer1Container210GBperTaskAWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营VOLUMESTORAGE跨容器访问的可写存储Fargate为每个Task提供4GBVolumeStorage空间通过TaskDefinition中的volumemount参数进行配置每个容器可以指定不同的containerPaths参数不要指定宿主机的sourcePath参数一次性存储(ephemeral),Task停止后数据无法找回Container1Container24GBVolumeStoragemount/var/container1/data/var/container2/dataAWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营IAM权限相关配置AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营权限层级ClusterPermissionsApplicationPermissionsTaskHousekeepingPermissionsClusterFargateTask集群权限:控制哪些用户可以在指定集群中读取Task状态或运行Task应用权限:允许您的应用容器安全的访问AWS资源应用相关管理(Housekeeping)权限:从ECR拉取镜像向Cloudwatch推送日志创建ENI在ELB上创建/删除targetgroupAWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营集群权限{"Effect":"Allow","Action":["ecs:RunTask"],"Condition":{"ArnEquals":{"ecs:cluster":""}},"Resource":[":*"]}您可以通过定制的IAM规则控制您的集群访问把定制好的权限添加到指定的IAM用户或者角色中演示规则:示例1:允许在指定集群中运行指定的taskdefinition{"Effect":"Allow","Action":["ecs:ListTasks","ecs:DescribeTasks"],"Condition":{"ArnEquals":{"ecs:cluster":""}},"Resource":"*"}示例2:允许在指定集群中获取所有Task的状态信息AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营应用权限您的应用容器需要访问其他的AWS资源,您需要下载AWS认证凭据(AWSACCESSKEY)到Task中的容器里首先,根据您应用的需求,创建一个IAM角色.
在scorekeep示例中,我们的程序需要访问DynamoDB和SNS服务的权限在这个角色中创建和ecs-tasks.
amazonaws.
com的信任关系.
这个设置允许Fargate切换到您定义的角色权限中,并且关联相关认证凭据到您的Task.
在您的taskdefinition中增加TaskRole的ARNFargate会自动为您的Task发布和滚动替换临时凭证在您的应用容器中执行的AWSCLI/SDK调用将会自动获得TaskRole定义权限的认证凭证请使用TaskRole{"family":"scorekeep","cpu":"1vCpu","memory":"2gb","networkMode":"awsvpc","taskRoleArn":"arn:aws.
.
.
role/scorekeepRole","containerDefinitions":[{"name":"scorekeep-frontend","image":"xxx.
dkr.
ecr.
us-east-1.
amazonaws.
com/fe","cpu":256,"memoryReservation":512,"portMappings":[{"containerPort":8080}]},{"name":"scorekeep-api","image":"xxx.
dkr.
ecr.
us-east-1.
amazonaws.
com/api","cpu":768,"memoryReservation":512,"portMappings":[{"containerPort":5000}]}]}TaskDefinitionAWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营应用相关管理(HOUSEKEEPING)权限您需要在您的账号中定义一些特定的权限来运行Fargate引导您的Task并且让他们能够持续运行.
ExecutionRole赋予了下列权限:拉取保存在ECR中的容器镜像向Cloudwatch推送日志ECSServiceLinkedRole赋予了下列权限:ENI管理注册和删除ELBTargetGroupAWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营EXECUTIONROLE需要使用保存在ECR容器镜像或者CloudwatchLogs功能创建一个IAM角色然后增加ECR的只读权限ecr:GetAuthorizationToken&ecr:BatchGetImage或者使用预定义的AmazonEC2ContainerRegistryReadOnly管理规则增加CloudWatchLogs的写权限logs:CreateLogStream&logs:PutLogEvents或者使用预定义的CloudWatchLogsFullAccess管理规则创建和ecs-tasks.
amazonaws.
com.
的信任关系,这个设置的作用是允许Fargate切换到这个角色中把executionrole的arn添加到您的taskdefinition通过ExecutionRole授予权限{"family":"scorekeep","cpu":"1vCpu","memory":"2gb","networkMode":"awsvpc","taskRoleArn":"arn:aws.
.
.
role/scorekeepRole","executionRoleArn":"arn:aws.
.
.
role/scorekeepExecutionRole","containerDefinitions":[{"name":"scorekeep-frontend","image":"xxx.
dkr.
ecr.
us-east-1.
amazonaws.
com/fe","cpu":256,"memoryReservation":512,"portMappings":[{"containerPort":8080}]},{"name":"scorekeep-api","image":"xxx.
dkr.
ecr.
us-east-1.
amazonaws.
com/api","cpu":768,"memoryReservation":512,"portMappings":[{"containerPort":5000}]}]}TaskDefinitionAWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营ECSSERVICELINKEDROLEservice-linkedrole是一种特定的IAM角色类型,可以被直接连接到一种AWS的服务,例如ECS当您创建第一个集群的时候,在您的账号中被自动创建这是一种预定义的规则,并且不可变.
在本场景中,主要涉及到ENI&ELB相关的授权.
您不需要在taskdefinition或者任何API调用的时候显示的指定这个规则.
在这里提及到这个角色类型是为了避免您在IAM界面中看到相关角色类型而产生困惑AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营监控相关配置AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营CLOUDWATCHLOGS配置使用awslogs驱动把您的容器应用中发送到标准输出的信息记录到Cloudwatchlogs中在Cloudwatch中创建一个日志组在taskdefinition中指定LogDriver参数在TaskExecutionRole中添加可写入的权限{"family":"scorekeep",.
.
.
"containerDefinitions":[{"name":"scorekeep-frontend",.
.
.
"logConfiguration":{"logDriver":"awslogs","options":{"awslogs-group":"scorekeep","awslogs-region":"us-east-1","awslogs-stream-prefix":"scorekeep/frontend"}}},{"name":"scorekeep-api",.
.
.
"logConfiguration":{"logDriver":"awslogs","options":{"awslogs-group":"scorekeep","awslogs-region":"us-east-1","awslogs-stream-prefix":"scorekeep/api"}}}]}TaskDefinitionAWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营CLOUDWATCHLOGSLogsTabintheTaskDetailPage在ECS或者Cloudwatch界面中查看应用日志AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营其他监控工具Cloudwatch监控-CPU/Memory使用率指标CloudWatchevent-Task状态发生变化时产生AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营DEMOAWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营{"family":"scorekeep","cpu":"1vCpu","memory":"2gb","networkMode":"awsvpc","taskRoleArn":"arn:aws:…","executionRoleArn":"arn:…","requiresCompatibilities":["FARGATE"],"containerDefinitions":[…]}{"name":"scorekeep-frontend","image":"xxx.
dkr.
ecr…frontend","cpu":256,"memoryReservation":512,"portMappings":[{"containerPort":8080}],"logConfiguration":{"logDriver":"awslogs","options":{"awslogs-group":"scorekeep","awslogs-region":"us-east-1","awslogs-stream-prefix":"scorekeep/frontend"}}}{"name":"scorekeep-api","image":"xxx.
dkr.
ecr…api","cpu":768,"memoryReservation":512,"portMappings":[{"containerPort":5000}],"logConfiguration":{"logDriver":"awslogs","options":{"awslogs-group":"scorekeep","awslogs-region":"us-east-1","awslogs-stream-prefix":"scorekeep/api"}},"environment":[…]#envvar}FINALSCOREKEEPTASKDEFINITIONTaskDefinitionscorekeep-frontendContainerDefinitionscorekeep-apiContainerDefinitionAWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营Fargate最新技术发布AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营TaskMetadata{"Cluster":"default","TaskARN":"arn:aws:ecs:us-west-2:012345678910:task/9781c248-0edd-4cdb-9a93-f63cb662a5d3","Family":"nginx","Revision":"5","DesiredStatus":"RUNNING","KnownStatus":"RUNNING","Containers":[{"DockerId":"731a0d6a3b4210e2448339bc7015aaa","Name":"~internal~ecs~pause","DockerName":"ecs-nginx-5-internalecspause-1234","Image":"amazon/amazon-ecs-pause:0.
1.
0","ImageID":"","Labels":{"com.
amazonaws.
ecs.
cluster":"default","com.
amazonaws.
ecs.
container-name":"~internal~ecs~pause","com.
amazonaws.
ecs.
task-arn":"arn:aws:ecs:us-west-2:012345678910:task/9781c248-0edd-4cdb-9a93-f63cb662a5d3","com.
amazonaws.
ecs.
task-definition-family":"nginx","com.
amazonaws.
ecs.
task-definition-version":"5"},"DesiredStatus":"RESOURCES_PROVISIONED","KnownStatus":"RESOURCES_PROVISIONED","Limits":{"CPU":512,"Memory":512查询Task的状态和容器的运行状态NEW!
{"CreatedAt":"2018-02-01T20:55:08.
366329616Z","StartedAt":"2018-02-01T20:55:09.
058354915Z","Type":"CNI_PAUSE","Networks":[{"NetworkMode":"awsvpc","IPv4Addresses":["10.
0.
2.
106"]}]},"PullStartedAt":"2018-02-01T20:55:09.
372495529Z","PullStoppedAt":"2018-02-01T20:55:10.
552018345Z"}AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营容器健康检查-在TaskDefinition中指定自定义的健康检查命令NEW!
AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营ECS/Fargate托管服务发现NEW!
服务注册:可预置的服务命名自动更新到最近的健康的IP地址和端口号全托管:无需安装部署及维护高可用,高可伸缩性可扩展性:灵活的自动发现边界AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营总结Fargate是ECS服务的一种新的容器运行模式,在这种模式下无需对EC2实例进行管理如果您正在纠结于使用EC2模式还是Fargate模式,可以尝试先从Fargate开始规划您的应用.
这种模式通过把应用从底层虚拟主机完全独立出来的方式,强化了一种比较好的设计实践如果您认为必须要访问和控制底层虚拟主机,那么请您再次考虑一下.
一些特定的原因:特定类型的实例,EC2dedicatedinstances,使用EC2预留实例,GPU,Windows负载告诉我们您的需求,我们想把这些需求加入到Fargate中!
从现在开始使用Fargate!
Fargate完全兼容您现有的容器镜像您只需要在现有的ECStaskdefinitions做细微的修改就可以使用Fargate了.
AWS中国(北京)区域由光环新网运营AWS中国(宁夏)区域由西云数据运营更多信息请参考aws.
amazon.
com/fargate感谢您的聆听