Developer Experience at Zalando1

1. Developer Experience at Zalando HANDELSBLATT JAHRESTAGUNG 2019-01-23 HENNING JACOBS @try_except_

2. ZALANDO AT A GLANCE ~ 4.5 billion EUR > 200 million revenue 2017 2 > 15.000 > 70% employees in Europe of visits via mobile devices visits per month > 300.000 > 24 product choices million ~ 2.000 17 brands countries active customers

3. > 200 development teams > 1100 developers 3 Platform

4. YOU BUILD IT, YOU RUN IT The traditional model is that you take your software to the wall that separates development and operations, and throw it over and then forget about it. Not at Amazon. You build it, you run it. This brings developers into contact with the day-to-day operation of their software. It also brings them into day-to-day contact with the customer. - A Conversation with Werner Vogels, ACM Queue, 2006 4

5. ON-CALL: YOU OWN IT, YOU RUN IT When things are broken, we want people with the best context trying to fix things. - Blake Scrivener, Netflix SRE Manager 5

6. DEVELOPER JOURNEY Consistent story that models all aspects of SW dev 6

7. Developer Journey 7

8. Correctness Compliance GDPR Security Cost Efficiency 24x7 On Call Governance Resilience Capacity ... 8 Developer Journey

9. DEVELOPER PRODUCTIVITY Setup Code Build Test Deploy Cloud Native Application Runtime 9 Operate

10. CLOUD NATIVE .. uses an open source software stack to deploy applications as microservices, packaging each part into its own container, and dynamically orchestrating those containers to optimize resource utilization. Cloud native technologies enable software developers to build great products faster. - https://www.cncf.io/ 10

11. CONTAINERS END-TO-END Setup Code Build Test Deploy Cloud Native Application Runtime 11 Operate

12. CONTAINERS 12

13. CONTAINERS 13

14.

15. PLAN & SETUP 15

16. Plan Stories Rules of Play Tech Radar 16

17.

18. Setup Application Bootstrapping 18

19.

20.

21. BUILD & TEST 21

22. CONTINUOUS DELIVERY PLATFORM: BUILD push Git code 22 CDP

23.

24. DEPLOY 24

25. Kubernetes Deploy 25

26. DEPLOYMENT CONFIGURATION ├── deploy/apply │ ├── deployment.yaml │ ├── credentials.yaml # Zalando IAM │ ├── ingress.yaml │ └── service.yaml └── delivery.yaml # Zalando CI/CD 26

27. INGRESS.YAML kind: Ingress metadata: name: "..." spec: rules: # DNS name your application should be exposed on - host: "myapp.foo.example.org" http: paths: - backend: serviceName: "myapp" servicePort: 80 27

28. CONTINUOUS DELIVERY PLATFORM 28

29. CDP: DEPLOY 29

30. CDP: OPTIONAL APPROVAL 30

31. INTEGRATIONS 31

32. CLOUD FORMATION VIA CI/CD "Infrastructure as Code" ├── deploy/apply │ ├── deployment.yaml │ ├── cf-iam-role.yaml │ ├── cf-rds.yaml │ ├── kube-ingress.yaml │ ├── kube-secret.yaml │ └── kube-service.yaml └── delivery.yaml 32 # Kubernetes # AWS IAM Role # AWS RDS Database # CI/CD config

33. POSTGRES OPERATOR Application to manage PostgreSQL clusters on Kubernetes >500 clusters running on Kubernetes 33

34. SUMMARY • Application Bootstrapping • Git as source of truth and UI • 4-eyes principle for master/production • Extensible Kubernetes API as primary interface • OAuth/IAM credentials • PostgreSQL • CloudFormation for proprietary AWS services 34

35. DELIVERY PERFORMANCE METRICS • Lead Time • Release Frequency • Time to Restore Service • Change Fail Rate 35

36. CONTAINERS 36 From "Accelerate: The Science of Lean Software and DevOps"

37. DELIVERY PERFORMANCE METRICS 37 • Lead Time ≙ Commit to Prod • Release Frequency ≙ Deploys/week/dev • Time to Restore Service ≙ MTRS from incidents • Change Fail Rate ≙ n/a

38. “.. means establishing empathy with internal consumers (read: developers) and collaborating with them on the design. Platform product managers establish roadmaps and ensure the platform delivers value to the business and enhances the developer experience.” - ThoughtWorks Technology Radar

39.

40. DEVELOPER SATISFACTION 40

41. DOCUMENTATION "Documentation is hard to find" "Documentation is not comprehensive enough" "Remove unnecessary complexity and obstacles." "Get the documentation up to date and prepare use cases" "More and more clear documentation" "More detailed docs, example repos with more complicated deployments." 41

42. DOCUMENTATION • Restructure following https://www.divio.com/en/blog/documentation/ • Concepts • How Tos • Tutorials • Reference • Global Search • Weekly Health Check: Support → Documentation 42

43.

44. NEWSLETTER "You can now.." • You can now benefit from the most recent Kubernetes 1.12 features, e.g. .. • You can now analyse your Kotlin project with SonarQube and upload your Scala code coverage report to SonarQube 44

45. SIGNAL: ISSUE UPVOTES 45

46. TESTIMONIALS “Useful information, good level of details and pleasant to read. It's one of the few newsletters that I took time to read entirely :)” - a reader, July 2018 46

47. TESTIMONIALS “So, thank you, Team Automata, for listening to our community, taking our upvotes in consideration when developing new solutions and building every day 'the first CI that doesn't suck'.” - a user, October 2018 47

48. LEARNINGS FROM THE PAST 48

49. LEARNINGS FROM THE PAST • Platform as a service vs governance • Platform teams think they are "special", but lack basic customer insights • Developers love autonomy, hard to "go back" once granted • Developers are demanding 49

50. NOTE TO SELF • Establish a product mindset • Listen to your customers • Exploit unfair advantages • Invest into docs & support 50

51. QUESTIONS? HENNING JACOBS HEAD OF DEVELOPER PRODUCTIVITY henning@zalando.de @try_except_ Illustrations by @01k