企业创新与研发团队赋能
如果无法正常显示,请先停止浏览器的去广告插件。
1. 企业创新与研发团队赋能
—— XXX DevOps Meetup
Jason Xue (薛军)
资深解决方案架构师,区域经理
xuejx@amazon.com
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
2. 日程
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
3. SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
4. Connectivity 车联网
Novel on-demand services
Marketplace 应用市场
Vehicle as service hub
Multimodal Mobility 多模式交通
Getting from A to B
Electrification 电气化
Increased BEV penetration
Autonomous Driving 自动驾驶
Towards full autonomy
Subscriptions 订阅
Towards
SUM
M I T pay-as-you-go
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
//
5. 软件重新定义汽车:智能网联汽车(ICV)
汽车行业的竞争规则在不断被改写:汽车持续从一个纯硬件产品向软件驱动的电动智能设备发展
汽车要“+物联网”“+大数据”“+智能制造”
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
6. 软件设计开发复杂度困境
软件技术发展的使命之一就是控制复杂度(Complexity)
软件复杂度 = 难以理解 (难以维护和扩展)
无法预测,难以掌控
技术复杂度
业务复杂度
代码的含义模糊 互相依赖
需求规模
软件规模
安全、高性能、高并发、高可用性等需求
项目管理复杂度
团队 文化 进度 工具集
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
7. 如何解决软件研发复杂度?
知识和⼯具
“分⽽治之”
分离职责(Seperation of
Concerns, 参考单一职责原则)
关注接口(定义交互)
隔离业务和技术复杂性
“抽象”
“改善依赖”
设计模式
(GangOfFour)1994
敏捷软件开发(原则模式与实践)
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
埃里克・埃文斯 (Eric Evans)
8. 代码行数急剧膨胀 vs 交付迭代周期
Obidos
胖的快跑不动了!!!
精简二进制代码总大小 1GB+
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
9. 最终交付的是客户想要的吗?
“大多数公司编写这个软件,他们把
这个软件全部运行起来,然后把它们
扔到市场部门,说 ‘ 这里是我们建的,
去写新闻稿 ’ ,那个过程实际上应该是
相反的。”
Jeffrey P. Bezos
创始人及首席执行官
Amazon.com, Inc.
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
10. 亚马逊如何保障围绕客户价值交付?
▪ 机制
• 逆向工作法
▪ 架构
• 微服务架构
• 松耦合应用
• 自服务平台 – 没有门卫
▪ 文化
• 人:我们雇佣建造者、创新者、创业者
• 信念:亚马逊领导力准则
▪ 组织
• 试验
• 双披萨团队
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
11. 复杂的企业现状:双模业务现状
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
12. 创新困境:业务创新精力 vs 支持维护投入
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
13. 什么是 DevOps ?
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
14. 亚马逊服务和组织转型 – Amazon DevOps
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
15. 最创新及最客户至尚的企业是怎么转型的?
组织内思维理念协调一致,
工程技术实践赋能,
自服务工具链助力
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
16. SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
17. 康威定律:
设计系统的组织,其产生的设计和架构等价
于组织间的沟通结构
大家对需求的理解是一致的吗?
交付的结果是价值导向的吗?
团队成员之间协作依赖解决了?
业务需求如何拆分子业务,便于多团队齐头并进?
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
18. SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
19. SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
20. 围绕“开发者”
Operations
Design
Applications
Management
CI/CD
Applications
COTS
Dev,Test,
Prod
Etc.
Infrastructure
Code
Support
Code &
Config
Source Code Repository
/ Service Catalog
J
J
Templates
Landscapes
Infrastructure
Semi-Automated
AMI
Cloud COE / Ops
SUMMIT
IaaS
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DevOps
API
21. Users
Amazon
EKS Control
Plane
Amazon EKS Nodes
Telemetry
Connect
DevOps
CI/CD
Ingress
SRE
SRE
Network & Policy ( Calico )
Storage & Policy ( EBS storage classes )
Managed.
Stakeholder
Tools
Compute & Policy ( Auto Scaling)
Apps
Infrastructure as code
Engineering
Routing
EKS Admin
SUMMIT
21
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
22. 不同开发人员管理边界
Platform
Admins Access Point
Cluster 1
Namespace
Group X
Teams Access Point
Applications
PODs
SUMMIT
Cluster 2
Namespace
Group Y
Name
Space A Name
Space B
PODs PODs
Name
Space C
Name
Space D
Cluster 3
Namespace
Group Z
Name
Space C
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Name
Space D
....
Cluster N
Admin
Namespace
Daemon
sets
Tools
23. AWS 开发者工具箱
CI/CD Tools
AWS
CodeStar
AWS
CodeBuild
AWS
CodeCommit
Infrastructure as Code
AWS
CodeDeploy
AWS
CodePipeline
AWS
CloudFormation
IDE
AWS Cloud Dev.
Kit (CDK)
AWS Cloud9
IDE and DevOps Toolkits
Visual Studio
Code
IntelliJ
PyCharm
Monitoring & Tracing
AWS X-Ray
CLI and Scripting Tools
Visual Studio
Eclipse
VSTS
AWS CLI
Tools for
PowerShell
Amazon
CloudWatch
Web Apps
AWS Elastic
Beanstalk
Languages Mobile
Amazon
Corretto AWS Amplify
SDKs
JavaScript
SUMMIT
Python
PHP
.NET
Ruby
Java
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Go
Node.js
C++
24. 在亚马逊开发人员如何工作? - 开发环境
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
25. AWS 开发工具包和更多的 IDE 集成
AWS Toolkit
for PyCharm
Python
AWS Toolkit
for IntelliJ
Java, Python
AWS Toolkit for
Visual Studio Code
.NET, Node
AWS Toolkit for
Visual Studio
.NET
New
AWS Toolkit
for Webstorm
Node.js
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Toolkit
for Rider
.NET
26. 远程云调试(测试版)
NEW
https://www.jetbrains.com/devops/amazon-aws/
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
27. 在亚马逊开发人员如何工作?-代码审核
PRE-PRODUCTION
TESTING
预生产
DEV/TEST
开发测试
PRODUCTION
生产
CODE REVIEW
代码审核
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
28. Gitflow & 代码审核
https://aws.amazon.com/cn/blogs/devops/using-aws-codecommit-pull-
requests-to-request-code-reviews-and-discuss-code/
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
29. AWS CodeCommit – 合并代码强制满足规则
Nov 20, 2019
NEW
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
30. Amazon CodeGuru :机器学习代码审查和性能建议服务
CodeGuru Reviewer
静态代码分析器
开发 + 审核
具有智能建议的内置
代码审查
SUMMIT
NEW
CodeGuru Profile
性能分析
构建 + 测试
部署
检测和优化昂贵的
代码行
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
预览版
性能评估
识别延迟和性能改进
提高
31. Amazon CodeGuru Reviewer 代码审查
•
•
•
•
•
AWS最佳实践:正确的使用 AWS API
并发:并发结构的正确实现
资源泄漏:正确的资源处理方式
敏感信息泄漏:泄漏个人身份信息、账户信息等
通过数据挖掘发现缺陷代码:通常难以发现的缺陷
Merge
Branch
Make
changes
locally
SUMMIT
Pull
Request
Reviewer
Code
Review
Approval
Amazon CodeGuru Reviewer
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
32. Amazon CodeGuru Reviewer 代码审查效果
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
33. 性能分析可视化-火焰图 & 建议
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
34. SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
35. 在亚马逊开发人员如何工作? - 持续交付
DEV/TEST
开发测试
CODE REVIEW
代码审核
PRE-PRODUCTION
TESTING
预生产
SUMMIT
PRODUCTION
生产
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
36. 亚马逊研发团队的持续交付管道
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
37. 解耦:组织和服务架构
(microservices, 2-pizza teams)
亚马逊研发团队
持续交付经验总结
自动化一切
Automate everything
自服务门户-标准化工具
Standardized tools
各种模版和参考
Belts and suspenders
(governance, templates)
基础设施即代码
Infrastructure as code
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
38. 持续交付开发者工具箱:AWS 原生+第三方
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
39. 直接查看单元/集成测试报告 – AWS CodeBuild
NEW
预览版
支持 JUnit XML 或 Cucumber JSON 格式
比如 Surefire JUnit plugin, TestNG, and Cucumber
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
40. ChatOps 新服务 – ChatBot 聊天机器人
NEW
测试版
•
•
•
•
•
•
•
Amazon CloudWatch
AWS Health/Config
AWS Budgets/System Mgr
AWS Security Hub
Amazon GuardDuty
AWS CloudFormation
AWS Code*
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
41. SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
42. 基础设施选型
AWS CloudFormation
AWS Cloud Development Kit
Terraform
EKSCTL
SUMMIT
AMIs / System Manager
Ansible Chef Packer Puppet Salt
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
43. AWS Cloud Development Kit (AWS CDK)
AWS
CloudFormation
Template
CDK Application
Stack(s)
Construct
Amazon
Simple Queue
Service
NEW
AWS
Lambda
Construct
Amazon
S3 bucket
Amazon
DynamoDB
AWS
CloudFormation
Resources
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
44. IaaC:基于 CDK 的 CI/CD
AWS Cloud
Region
Stack
CDK Application
Stack
Stack
Construct
Stack
Region
Region
Stack
Region
Construct
Stack
Stack
Git push
Git users
Amazon Simple
Queue Service
AWS Lambda
AWS Fargate
Amazon
DynamoDB
AWS
CodePipeline
Region
Stack
Assets
Region
Lambda Runtime Code
Stack
Region
Container image
Stack
SUMMIT
Region
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Stack
45. SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
46. ECS CI/CD
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
47. 容器平台 k8s
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
48. 基于 推模式 的 K8S 持续部署流水线
使用“kubectl” 将任何更改推送到 Kubernetes 集群
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
49. Jenkins – CI/CD with Kubernetes
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
50. K8S CD 流程:Code*+Lambda+ECR
1
6
2
Amazon ECR
3
1
Developer
2
AWS CodeCommit
5
AWS Lambda
Pushes the newly built image
tagged with build id to ECR repo
4 Invokes a Lambda function to
trigger application deployment
5 Leverages Kubernetes Python SDK
to update a deployment
6
AWS CodePipeline
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Triggers an execution of the
pipeline when a new version is
found, builds
a new image with build id
3
4
AWS CodeBuild
Developers continuously integrate
changes into a main branch hosted
within a repo
Fetches new container image
and performs a rolling update
of deployment
51. 基于 拉模式 的 K8S 持续部署流水线(GitOps)
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
•版本控制工具
• GitLab
• Bitbucket
•状态比较工具
• Kubediff
•交付流水线
• Jenkins X
• Argo CD
• Weave Flux
• Spinnaker
52. GitOps 利用 Code* + Weave Flux
https://eksworkshop.com/
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
53. GitOps 利用 ECR + Jenkins X
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
54. SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
55. Key Take Away
• 亚马逊的研发团队DevOps 核心在于:解耦(两个披萨团队和微服
务)、自动化一切、自服务平台、基础设施即代码
• 持续交付要从业务出发,打破团队壁垒,构建高效敏捷交付和数据
反馈闭环
• 工程实践、现代应用架构非常重要
• GitOps 是容器平台 CI/CD 的热点方向
• 亚马逊专业服务团队有专门的 DevOps 赋能服务
• AWS 将亚马逊内部的工具演化成开发者云服务
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
56. AWS DevOps 赋能服务项目
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
57. AWS DevOps 现 状 调 研 和 成 熟 度 评 估
*来自国内研发运营一体化标准
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
58. Github: https://github.com/soldierxue/
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
59. Thank you!
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
60. Thank you!
SUMMIT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.