StackSets Simplifying application management in Kubernetes

1. StackSets Simplifying application management in Kubernetes ScaleUp 360° November 28th 2019 MIKKEL LARSEN @mikkeloscar 2019-11-28

2. $ whoami Mikkel Larsen Software Engineer Cloud Infrastructure (Kubernetes/AWS) @ Zalando SE @mikkeloscar 2 @mikkeloscar

3. “EUROPE’S LEADING ONLINE FASHION PLATFORM” 3

4. WE BRING FASHION TO PEOPLE IN 17 COUNTRIES 17 markets 7 fulfillment centers 26 million active customers 5.4 billion € revenue 2018 250 million visits per month 15,000 employees in Europe 4

5. DEPLOYING TO PRODUCTION https://my-application.io My Application v1 5

6. DEPLOYING TO PRODUCTION 6 https://my-application.io https://preview-v2.my-application.io My Application v1 My Application v2

7. DEPLOYING TO PRODUCTION (BLUE/GREEN) 7 https://my-application.io https://preview-v2.my-application.io My Application v1 My Application v2

8. DEPLOYING TO PRODUCTION (TRAFFIC SWITCHING) https://my-application.io https://preview-v2.my-application.io 1% 99% My Application v1 8 My Application v2

9. DEPLOYING TO PRODUCTION (TRAFFIC SWITCHING) https://my-application.io https://preview-v2.my-application.io 10% 90% My Application v1 9 My Application v2

10. DEPLOYING TO PRODUCTION (TRAFFIC SWITCHING) https://my-application.io https://preview-v2.my-application.io 50% 50% My Application v1 10 My Application v2

11. DEPLOYING TO PRODUCTION (TRAFFIC SWITCHING) https://my-application.io https://preview-v2.my-application.io 100% 0% My Application v1 11 My Application v2

12. APPLICATION STACK IN KUBERNETES Ingress resource name: app-1 Service resource Deployment resource labels: application: app-1 12

13. TRAFFIC SWITCHING ROLLING UPDATE DEPLOYMENT 100% Blue: 75% Green: 25% deployment 13

14. TRAFFIC SWITCHING ROLLING UPDATE DEPLOYMENT 100% Blue: 99% Green: 1% deployment 14

15. SKIPPER TRAFFIC SUPPORT apiVersion: extensions/v1beta1 kind: Ingress metadata: name: my-app annotations: zalando.org/backend-weights: | {"my-app-v1": 90, "my-app-v2": 10} spec: rules: - host: my-app.io ... github.com/zalando/skipper 15

16. MANUAL TRAFFIC SWITCHING BLUE/GREEN DEPLOYMENT IngressTemplate 80% service-v1 deployment-v1 16 20% service-v2 deployment-v2

17. MANUAL TRAFFIC SWITCHING BLUE/GREEN DEPLOYMENT IngressTemplate Need to clean up resources manually :’( 0% service-v1 deployment-v1 17 100% service-v2 deployment-v2

18. TRAFFIC SWITCHING BLUE/GREEN DEPLOYMENT (StackSet) 80% stack: v1 18 20% stack: v2

19. STACKSET DEFINITION apiVersion: zalando.org/v1 kind: StackSet metadata: name: my-app spec: ingress: host: [my-application.io] stackTemplate: spec: version: v1 podTemplate: spec: containers: ... github.com/zalando-incubator/stackset-controller 19

20. DEMO 20

21. ADDITIONAL FEATURES 21

22. PRESCALE STACKS 100% HPA: minReplicas: 3 maxReplicas: 30 stack: v2 stack: v1 22 0%

23. PRESCALE STACKS 0% HPA: minReplicas: 3 maxReplicas: 30 stack: v2 stack: v1 23 100%

24. PRESCALE STACKS Desired: 0% Actual: 100% HPA: minReplicas: 3 maxReplicas: 30 stack: v2 stack: v1 24 Desired: 100% Actual: 0%

25. PRESCALE STACKS Desired: 0% Actual: 100% Desired: 100% Actual: 0% HPA: minReplicas: 3 maxReplicas: 30 stack: v1 25 stack: v2

26. PRESCALE STACKS Desired: 0% Actual: 0% stack: v1 26 Desired: 100% Actual: 100% stack: v2

27. GRADUAL DEPLOYMENTS (AUTOMATIC TRAFFIC SWITCHING) 27

28. INTEGRATE WITH APPLICATION METRICS 28

29. GRADUAL DEPLOYMENTS Desired: 100% Actual: 100% stack: v1 29 Desired: 0% Actual: 0% stack: v2

30. GRADUAL DEPLOYMENTS Desired: 0% Actual: 0% Desired: 100% Actual: 100% stack: v1 30 stack: v1’ (baseline stack) stack: v2

31. GRADUAL DEPLOYMENTS Desired: 1% Actual: 1% Desired: 98% Actual: 98% stack: v1 stack: v1’ (baseline stack) stack: v2 Observe metrics for the two stacks 31

32. GRADUAL DEPLOYMENTS Desired: 10% Actual: 10% Desired: 80% Actual: 80% stack: v1 stack: v1’ (baseline stack) stack: v2 Observe metrics for the two stacks 32

33. GRADUAL DEPLOYMENTS Desired: 10% Actual: 10% Desired: 80% Actual: 80% stack: v1 stack: v1’ (baseline stack) stack: v2 Observe metrics for the two stacks 33

34. GRADUAL DEPLOYMENTS Desired: 0% Actual: 0% Desired: 100% Actual: 100% stack: v1 stack: v1’ (baseline stack) stack: v2 !DEPLOYMENT FAILED! 34

35. OPEN SOURCE StackSet Controller github.com/zalando-incubator/stackset-controller AWS ALB Ingress controller github.com/zalando-incubator/kube-ingress-aws-controller Skipper HTTP Router & Ingress controller github.com/zalando/skipper External DNS github.com/kubernetes-incubator/external-dns 35

36. MIKKEL LARSEN SENIOR SOFTWARE ENGINEER PLATFORM INFRASTRUCTURE mikkel.larsen@zalando.de @mikkeloscar Illustrations by @01k, @kcgreenn, @ntakayama 2019-11-28