HOW TO KILL TWO BIRDS WITH ONE STONE- LEARNING TO RANK WITH MULTIPLE OBJECTIVES
如果无法正常显示，请先停止浏览器的去广告插件。
相关话题：
#zalando
1. HOW TO KILL TWO BIRDS
WITH ONE STONE:
LEARNING TO RANK
WITH MULTIPLE
OBJECTIVES
HAYSTACK EU 2019
ALEXEY KURENNOY
28-10-2019
2. 2
3. MULTI-OBJECTIVE OPTIMISATION IN EVERYDAY LIFE
3
● Product quality vs price
● Hotel location vs facilities
● Job satisfaction vs compensation
● ...
4. ZALANDO AT A GLANCE
~ 5.4
billion EUR
revenue 2018
> 15,500
employees in
Europe
4
> 80%
of visits via
mobile devices
> 300
million
> 27
million
active customers
visits
per
month
> 400,000
product choices
~ 2,000
brands
17
countries
5. OUTLINE
5
● Introduction
● Learning to Rank recap
● Multi-objective optimisation
● LambdaMART recap
● Multi-objective Learning to Rank with LambdaMART
● Experiments
● Conclusion and further steps
6. LTR RECAP
MODEL TRAINING
MODEL
OBJECTIVE
FUNCTION
HISTORICAL DATA
6
RANKING
7. EXAMPLE: NDCG OBJECTIVE
Interaction
7
purchase 3
click 1
no interaction 0
8. WHY A SINGLE OBJECTIVE MAY BE NOT ENOUGH?
● Different types of user feedback (implicit vs explicit)
● Different sources of feedback (user vs annotator)
● E-Commerce: engagement vs post-order experience
● E-Commerce: engagement vs fashionability
8
9. MULTI-OBJECTIVE OPTIMISATION
objective
a dominated solution
objective # 1
9
10. MULTI-OBJECTIVE OPTIMISATION
a Pareto-efficient solution
objective
a dominated solution
objective # 1
10
11. MULTI-OBJECTIVE OPTIMISATION
a Pareto-efficient solution
objective
a dominated solution
the Pareto frontier
objective # 1
11
12. HOW TO SEARCH FOR PARETO SOLUTIONS?
12
13. HOW TO SEARCH FOR PARETO SOLUTIONS?
●
13
Post-augmentation of the sorting rule
14. HOW TO SEARCH FOR PARETO SOLUTIONS?
14
● Post-augmentation of the sorting rule
● Redefining the relevance
15. HOW TO SEARCH FOR PARETO SOLUTIONS?
● Post-augmentation of the sorting rule
● Redefining the relevance
Interaction
15
purchase, fashionable 7
purchase, non-fashionable 3
click 1
no interaction 0
16. HOW TO SEARCH FOR PARETO SOLUTIONS?
16
● Post-augmentation of the sorting rule
● Redefining the relevance
● Scalarization
17. SCALARIZATION
combined objective
17
18. SCALARIZATION
18
● Produces only Pareto optimal solutions
● Allows finding any Pareto optimal solution (under certain assumptions)
● Straightforward to apply when there are more than two objectives
19. HOW TO CHOOSE AMONG PARETO SOLUTIONS?
selected solution
objective # 1
19
20. HOW TO SEARCH FOR PARETO SOLUTIONS?
20
● Post-augmentation of the sorting rule
● Redefining the relevance
● Scalarization
● Constrained optimisation
(Momma et. al. Multi-objective Relevance Ranking. SIGIR 2019)
21. LAMBDAMART RECAP
●
●
A boosting algorithm for optimising ranking metrics (such as NDCG)
Iteration:
○ compute the lambda-gradient
○
21
construct a tree that approximates the lambda-gradient and add it to the ensemble
22. LAMBDAMART RECAP
●
●
A boosting algorithm for optimising ranking metrics (such as NDCG)
Iteration:
○ compute the lambda-gradient
○
22
construct a tree that approximates the lambda-gradient and add it to the ensemble
23. MULTI-OBJECTIVE OPTIMISATION WITH LAMBDAMART
23
24. IMPLEMENTATION
24
● Used Cython
● Parallelisation by means of OpenMP
~5x speed-up (Dual-Core Intel Core i5)
● The speed is on par with the original
implementation
25. OFFLINE EXPERIMENTS: SETUP
● Result pages from Zalando catalog (~50K pages)
● Two objectives:
○ Interaction NDCG
○
●
25
Fashionability NDCG (relevance = fashion score)
Goal: up-sort popular but fashionable articles
26. OFFLINE EXPERIMENTATION: RESULTS
26
27. OFFLINE EXPERIMENTATION: RESULTS
27
● Visibility of popular but non-fashionable articles went UP
● Visibility of fashionable but unpopular articles went UP
● Visibility of fairly popular and relatively fashionable articles went DOWN
28. CONCLUSIONS
●
●
28
A Pareto-efficient solution in ranking can lead to a “mixed” effect
Evaluation in the “multi-objective” space may not be reliable
29. FUTURE STEPS
●
●
29
Experimenting with the “combined” relevance
Experimenting with dataset augmentation
30. P.S.
No birds were killed during the preparation of this talk.
30