1. AGILE TESTING @ ZALANDO AGILE TESTING 2019 JAN HEGEWALD HEAD OF PRODUCT OFFER PLATFORM 03.07.2019

2. JAN HEGEWALD Software Engineering since 2002 ISTQB Certified Tester 2009 21 Pairs Of Shoes ~ 7 years Of Agile Experience 6 years In E-Commerce Berlin Born and Based 2 Kids

3. TESTING NO LONGER WORKS AS IT DID

4. TESTING CHALLENGES YESTERDAY TODAY

5. EXAMPLE - PRODUCT OFFER PLATFORM @ ZALANDO PRODUCT PRODUCT OFFER PLATFORM PRICE PRODUCT OFFERS STOCK

6. EXAMPLE - PRODUCT OFFER PLATFORM @ ZALANDO PRODUCT PRICE PRODUCT OFFER PLATFORM STOCK

7. EXAMPLE - PRODUCT OFFER PLATFORM @ ZALANDO Update reason: stock Enriches Campaign Information PRODUCT PRICE PRODUCT OFFER PLATFORM STOCK

8. EXAMPLE - PRODUCT OFFER PLATFORM @ ZALANDO Update reason: stock Enriches Campaign Information PRODUCT PRICE Optimizes Update Delivery Only relevant stock changes PRODUCT OFFER PLATFORM STOCK

9. EXAMPLE - PRODUCT OFFER PLATFORM @ ZALANDO Update reason: stock Enriches Campaign Information PRODUCT PRICE Optimizes Update Delivery Only relevant stock changes PRODUCT OFFER PLATFORM STOCK

10. TESTING CHALLENGES YESTERDAY Inte- grated test TODAY Inte- grated test Integration test Integration test Unit test Unit test

11. TESTING CHALLENGES YESTERDAY TODAY Inte- grated test Inte- grated test Integration test Integration test Unit test Unit test

12. LOVE YOUR INTEGRATION TESTS!

13. BUT IT’S HARD!

14. WITH MICROSERVICES OUR STAGING ENVIRONMENTS BECAME USELESS FOR INTEGRATION

15. STAGING ENVIRONMENTS BECAME EXPONENTIALLY EXPENSIVE V 102 V 63 V 175 V 123 V 15

16. STAGING ENVIRONMENTS BECAME EXPONENTIALLY EXPENSIVE V 102 V 63 V 175 V 124 V 15

17. STAGING ENVIRONMENTS BECAME EXPONENTIALLY EXPENSIVE V 102 V 64 CONTINUOUS DELIVERY MAKES THE PROBLEM WORSE V 177 V 124 V 15 WE NEED TO MOCK DEPENDENCIES

18. AND WHAT ABOUT DATA? AND WHAT ABOUT DATA?

19. MICROSERVICES: SHARED NOTHING

20. IT IS TRICKY TO RELY ON OTHERS’ TEST DATA Production Environment This shall be deployed Build Process Staging Environment

21. IT IS TRICKY TO RELY ON OTHERS’ TEST DATA Production Environment 2. Deploys service to production 1. Runs integration tests against staging Staging Environment Build Process

22. GO FOR CONTRACT TESTING

23. CONTRACT TESTING IN A NUTSHELL Consumer Provider Contract is written and run as a consumer test Only the specific interface used is tested Contract The rest can be changed without breaking tests Consumer

24. BUSINESS MONITORING AND ALERTING DETECT INSTEAD OF PREVENT

25. (BUSINESS) MONITORING PRODUCT PRICE PRODUCT OFFER PLATFORM STOCK End-2-End Business Monitoring

26. NEXT STEP: CHAOS ENGINEERING

27. CHAOS ENGINEERING AIMS AT IMPROVING RESILIENCE NEXT STEP: CHAOS ENGINEERING

28. CHAOS ENGINEERING STEADY STATE EXPERIMENTAL STATE Output Output

29. FROM THE TESTING PYRAMID Inte- grated test Integration test Unit test Monitoring & Alerting

30. … TO THE TESTING DIAMOND Inte- grated test Inte- grated test Integration test Integration test Unit test Unit test

31. ...TO THE TESTING HOUSE Inte- grated test Inte- grated test Inte- grated test Integration test Integration test Integration test Unit test Unit test Unit test Monitoring & Alerting Monitoring & Alerting Chaos Engineering Integrated test Integrated test Integration test Integration test Unit test Unit test

32. ...TO THE TESTING HOUSE Inte- grated test Inte- grated test Inte- grated test Integration test Integration test Integration test Unit test Unit test Unit test Monitoring & Alerting Monitoring & Alerting Chaos Engineering Integrated test Integrated test Integration test Integration test Unit test Unit test

33. WHAT ABOUT AGILE?

34. QUALITY CANNOT BE BUILT IN AT THE END WHAT IS “THE END” IN PRODUCT DEVELOPMENT OR CONTINUOUS DEVELOPMENT? NEXT STEP: CHAOS ENGINEERING

35. WHO IS RESPONSIBLE FOR QUALITY THEN? PRODUCT OWNER CROSS-FUNCTIONAL ENGINEERS Product Owners are accountable for product quality - functional and non-functional Engineers are responsible for product quality AGILE COACH / SCRUM MASTER AGILE TEAM Engineers need to educate Product Owners on quality-related requirements, Product Owner prioritises

36. DevSecQualOps? YES!

37. SO WE DON’T NEED TESTERS ANY MORE?

38. TESTERS CAN BRING VALUABLE EXPERT KNOWLEDGE Testers should have engineering skills They bring expert knowledge on automation, operations, integration testing frameworks and approaches Can educate and support engineering teams in a consulting/agency model

39. WHAT ABOUT AGILE AT SCALE?

40. FACTS & FIGURES

41. SOFTWARE PRODUCT DEVELOPMENT AT ZALANDO 5 tech hubs Across Europe > 2,000 > 200 employees In Tech From > Nations 100 Cross-functional, Agile Delivery Teams

42. COLLABORATION MAKES HIGH QUALITY IMPERATIVE Code becomes shared through end-2-end teams or contribution models In such environments you instantly need high quality standards

43. AUTONOMY NEEDS COMPLIANCE Besides Team autonomy Zalando ensures a common level of quality and compliance: • Standardised Docker/Kubernetes images • Clear responsibility tracking • Offering of e.g. SonarQube from standardised deployment infrastructure • Automatic detection of dependencies with known vulnerabilities

44. SERVICE LEVEL OBJECTIVES (SLOs) MAKE OPERATIONAL QUALITY MANAGEABLE PRODUCT PRODUCT OFFER PLATFORM PRICE PRODUCT OFFERS STOCK Throughput/ Update Latency Read Latency

45. TESTING IN AGILE AT SCALE IS COMPLEX

46. JUNE 2018: ZALANDO LAUNCHED NEW MARKETS IRELAND AND CZECH REPUBLIC

47. COMPLEXITY REQUIRES DIFFERENT TESTING APPROACHES ADAPTED content creation APP and web channels > 20 payment methods FULFILLMENT CENTERS and logistics customer care in NEW languages

48. AGILE TESTING @ ZALANDO - AT A GLANCE Microservice Architecture Complexity moves from the services to the integration Focus in Testing changed Monitoring & Alerting Chaos Engineering Testers Detect Bring Engineering Expert Knowledge on Testing Integrated test Integration test Unit test ● Mocked Integration Tests ● Contract Testing Testing integral part of delivery Rather than Prevent ● Business Monitoring ● Chaos Engineering PO Accountable Engineers Responsible For Quality: DevSecQualOps SLOs Define NF Product Qualities jan.hegewald@zalando.de Agile at Scale ● High quality imperative ● Standards required THANK YOU! www.agil-gefuehrt.de @janhegewald

49. JAN HEGEWALD HEAD OF PRODUCT OFFER PLATFORM jan.hegewald@zalando.de +49 (0) 176 12759037 03.07.2019

50. DISCLAIMER This presentation and its contents are strictly confidential. It may not, in whole or in part, be reproduced, redistributed, published or passed on to any other person by the recipient. The information in this presentation has not been independently verified. No representation or warranty, express or implied, is made as to the accuracy or completeness of the presentation and the information contained herein and no reliance should be placed on such information. No responsibility is accepted for any liability for any loss howsoever arising, directly or indirectly, from this presentation or its contents. 50

51. IMAGE SOURCES https://www.pexels.com/photo/black-and-white-blood-pressure-gauge-business-computer-263370/ https://www.pexels.com/photo/stack-white-business-market-41295/ https://www.pexels.com/photo/two-person-shaking-each-others-hands-872957/ https://www.pexels.com/photo/architecture-atmospheric-bay-bay-bridge-260173/ https://www.pexels.com/photo/yellow-and-green-ride-on-toy-near-green-leaf-trees-1411439/ https://www.pexels.com/photo/black-and-white-business-chart-computer-241544/ https://www.pexels.com/photo/abstract-art-blur-bright-373543/ https://www.pexels.com/photo/close-up-of-telephone-booth-257736/ https://www.pexels.com/photo/dead-end-road-sign-1469196/ https://www.pexels.com/photo/person-holding-silver-colored-accessory-close-up-photo-1050321/ https://www.pexels.com/photo/person-using-black-and-silver-laptop-computer-1193324/ https://www.pexels.com/photo/blur-close-up-code-computer-546819/ https://www.pexels.com/photo/asphalt-blur-car-city-290470/ https://www.pexels.com/photo/action-adults-celebration-clouds-433452/ https://www.pexels.com/photo/arms-bonding-closeness-daylight-1645634/ https://www.pexels.com/photo/person-holding-magnifying-glass-712786/ https://www.pexels.com/photo/everything-is-connected-neon-light-signage-1356300/ https://www.pexels.com/photo/broken-heart-love-sad-14303/ https://www.pexels.com/photo/everything-is-connected-neon-light-signage-1356300/ The rest is property of Zalando 51