从知识图谱到事理图谱

1. 从知识图谱到事理图谱 From Knowledge Graph to Event Evolutionary Graph Ting Liu Research Center for Social Computing and Information Retrieval 2017.11.05，China·Beijing

2. Outline • Motivation of Event Evolutionary Graph • Related Work • Our Efforts on Event Evolutionary Graph • Conclusion

3. Motivation of Event Evolutionary Graph • Most existing knowledge bases foucs on “concepts and their relations”, and failed to mine “event evolutionary logics” • Event evolutionary logics (development principles and patterns between events) are valuable commonsense knowledge, mining this kind of knowledge is crucial for understanding human behaviour and social development Knowledge Graph Event Evolutionary Graph Nodes Are Entities Nodes Are Events Google Knowledge Graph

4. What is Event Evolutionary Graph？ • Event Evolutionary Graph (EEG) ：事理图谱 – EEG is a knowledge base of event evolutionary logics, which describes the event evolutionary principles and patterns – 事理图谱是一个事理逻辑知识库，描述事件之间的演化规律和模式 • Structurally: EEG is a Directed Cyclic Graph, whose nodes are events, and edges stand for the sequential and causal relations (顺承和因果) between events. • Essentially: EEG is a knowledge base of event evolutionary logics, which describes the event evolutionary principles and patterns

5. Applications of Event Evolutionary Graph • EEG can be applied to several downstream tasks, including event prediction, commonsense reasoning, consumption intention mining, dialogue generation, question answering, decision Event Prediction Decision Making System EEG making system and so on. • Large-scale EEG can have big application potentials as traditional Knowledge Graph. Commonsense Reasoning Consumption Intention Mining Question Answering Dialogue Generation

6. Differences and Relations between EEG and KG Research Target Organization Form Main Knowledge Form Determinary of Knowledge Event Evolutionary Graph Knowledge Graph Predicate-Events (谓词性事件) and their relations Noun-Entities (名词性实体) and their relations Directed Graph Directed Graph Event evolutionary logics and transition probability Entities’ attributes and their relations Most event evolutionary logics are not deterministic Most relations between entities are deterministic

7. Event Definition in EEG • Events in ACE: An event is a specific occurrence involving participants. An event is something that happens. An event can frequently be described as a change of state – Traditional event extraction and classification tasks, ACE, KBP – Topic detection and tracking • Events in EEG: – Not specific but abstract events – Represent as general, semantic complete predicate-phrases or segments – “have hot pot”, “watch movies”, “go to the airport” are reasonable event representations – “go to somewhere”, “do things”, “eat”are unreasonable or incomplete events representations

8. Sequential Relation between Events • The sequential relation (顺承关系) between two events refers to their partial temporal orderings After having lunch, Tom paid the bill and left the restaurant. 吃过午饭后，汤姆到前台买单，然后离开了了餐馆。 have lunch sequential pay the bill sequential leave the restaurant

9. Causal Relation between Events • Causal relation (因果关系) is the relation between one event (the cause) and a second event (the effect), where the second event happens as a consequence of the first. • Causal relation is a subset of sequential relation – Satisfy the constraint of partial temporal order The nuclear leak in Japan led to serious ocean pollution 日日本核泄漏漏引起了了严重的海海洋污染。 nuclear leak causal ocean pollution

10. Three Topology Structures of EEG Chain structured EEG under the scenario of “watch movies”.

11. Three Topology Structures of EEG Chain structured EEG under the scenario of “watch movies”.

12. Three Topology Structures of EEG Tree structured EEG under the scenario of “plan a wedding”.

13. Three Topology Structures of EEG Tree structured EEG under the scenario of “plan a wedding”.

14. Three Topology Structures of EEG Cyclic structured EEG under the scenario of “fight”.

15. Three Topology Structures of EEG Cyclic structured EEG under the scenario of “fight”.

16. Outline • Motivation of Event Evolutionary Graph • Related Work • Our Efforts on Event Evolutionary Graph • Conclusion

17. Two Relevant Research Fields • Statistical Script Learning • Event Relation Recognition – Temporal relation recognition – Causal relation recognition

18. Statistical Script Learning • A very relevant research field to EEG • Development Stage – In 1975, American researcher Schank proposed the concept of Script – 2003：Japanese researchers proposed automatic acquisition of script knowledge – 2008-2013：Pioneering work – 2014~now：Recovery and Trend of important script learning papers 7 5 4 development stage 2 0 1975 1977 1985 2003 2008 2009 2010 2012 2014 2015 2016 2017

19. Statistical Script Learning (1) • [IJCAI 1975] Scripts, plans, and knowledge. Roger C. Schank, and Robert P. Abelson, Yale University

20. Statistical Script Learning (2) • [ACL 2008] Unsupervised learning of narrative event chains, Chambers, Jurafsky, Stanford University

21. Statistical Script Learning (3) • [AAAI 2014] Learning scripts as Hidden Markov Models, J. Walker Orr et al, Oregon State Univserity

22. Statistical Script Learning (4) • [ACL 2016] Using sentence-level LSTM language models for script inference, Pichotta and Mooney, University of Texas at Austin

23. Statistical Script Learning (5) • Inferring the correct story ending according to story contexts – [NAACL 2016]A corpus and cloze evaluation for deeper understanding of commonsense stories, Mostafazadeh et al. – [EMNLP 2017]Story Comprehension for Predicting What Happens Next, Snigdha Chaturvedi, Haoruo Peng, Dan Roth, UIUC

24. Temporal Relation Classification • [TACL 2014] Dense event ordering with a multi-pass Architecture, Chambers et al, United States Naval Academy

25. Event-Causality-Driven Stock Prediction • [WSDM 2017] Constructing and embedding abstract event causality networks from text snippets, Sendong Zhao et al., HIT-SCIR Experiments are carried out on long-term (one month) stock price movements using 12,482 events during this period. Focus on predicting the increase or decrease of Standard & Poor’s 500 stock (S&P) index.

26. Outline • Motivation of Event Evolutionary Graph • Related Work • Our Efforts on Event Evolutionary Graph • Conclusion

27. HIT-SCIR: Our Exploration on EEG Construction and Application of Travel Domain EEG

28. Travel Domain EEG Construction

29. Travel Domain EEG Construction Remove html tags, special symbols and so on

30. Travel Domain EEG Construction Segmentation, part-of-speech tagging, and dependency parsing

31. Travel Domain EEG Construction Extract verb-object phrases from the dependency-parsed tree, filter the low- frequency phrases by a proper threshold

32. Travel Domain EEG Construction Every two events from two consecutive sentences are considered as an event pair candidate

33. Travel Domain EEG Construction Regard the sequential relation and direction recognition as two separate supervised binary classification tasks

34. Travel Domain EEG Construction Causality is rare in our experiment corpus, so causality recognition is not covered in this paper.

35. Travel Domain EEG Construction

36. Data Sets and Experiments Results • Experiment corpus – 320,702 question-answering pairs crawled from travel topic on“Zhihu” • Suprevised classification of sequential relation and direction F1 value of sequential relation classification：85.7% F1 value of sequential direction classification ：92.9%

37. Case Study Subgraph in our automatically constructed travel domain EEG under the scenario of “buy train tickets”.

38. Case Study Subgraph in our automatically constructed travel domain EEG under the scenario of “buy train tickets”.

39. Demonstration：Travel Domain EEG Scale of travel domain EEG： 29,825 event nodes, 234,547 directed edges.

40. Travel Domain EEG: Potential Applications • Consumption Intention Mining and Recommendation – Most events can join with ‘want to’, ‘plan to’, ‘will’ and become intention events • want to go to Beijing, want to watch movies, plan to clime Mountain Tai, will have hot pot – Certain events have notably consumption intention, and they can lead to following consumption events; it is of great value to find these events • watch movies, go on a tourist

41. Travel Domain EEG: Potential Applications • Dialogue Generation – go to Beijingàbuy tickets • A: I plan to go to Beijing. B: Have you buy the tickets? – go to Taianàclimb Mountain Tai • A: I want to climb Mountain Tai. B: Then you need to go to Taian first. • Question Answering System – Q: Is there any dos and don’ts if I want to climb Mountain Tai ? – A: Remember to rent a coat, take some water and a flashlight.

42. HIT-SCIR: Our Exploration on EEG Construction and Application of Financial Domain EEG

43. Sample Analysis • Financial news contain lots of event-event causal relations – Explicit causality (with connectives)： “Plasticizer incident led to liquor stocks plummeted.” • “塑化剂事件导致白酒股大跌。” – Implicit causality (no connectives) ： “Baidu Q2 earnings report：net profit increased by 82.9%, stock price rose by 7%.“ • “百度Q2财报：净利同比增82.9%，股价盘后上涨7%” • Except for causal relations, it also contains large quantities of sequential relations – “After nearly half a month’ suspension, IFLYTEK’ stock resumed trading limit.” • “停牌了近半个月的科大讯飞（002230.SZ）复牌，股价开盘即涨停。”

44. Target for Financial Domain EEG • Target – Mine economic changes related (especially stock price movements) sequential and causal event relations from financial news articles, to construct the financial domain EEG • Method – Causal and sequential event relation extraction – Transforming ‘cause-effect pairs’ into graph

45. Extraction Methods for Cause-Effect Pairs • Exploit causal triggers to construct templates and match with regular expressions, to obtain the cause and effect mentions – For example：(.+)(result in|lead to|cause)(.+)(drop|rise) • (.+)(导致|引起|造成)(.+)(下跌|上涨) • Carry out segmentation and POS tagging on the cause and effect mentions • Finally, the sequence of verbs, nouns, and adjectives is regared as the cause and effect after filtering with POS tags

46. Representative Extracted Cause-Effect Pairs • Profits decline of ZTE lead to its stock price dropping sharply. – • Astronomical compensation rumors of ZTE resulted in its stock dropping sharply. – • “双汇并购史密斯菲尔德消息使得双汇发展股价大涨” Meat products production and sales decline, and cost raises caused the bussiness profits to drop. – • “中兴通讯天价赔偿传闻导致股票大跌” The news of Shuanghui’ acquisition of Smith Field made Shuanghui development stock price rise. – • “中兴通讯利润下滑引发股价大跌” “肉制品产、销量下降，成本上升造成肉制品业务利润下降” Related competitors’ entering resulted in the market share of gross profit margin declined. – “相关竞争对手进入导致产品毛利率市场份额下降”

47. Financial Domain EEG • Transforming ‘cause-effect event pairs’ into graph by merging certain events. Merge A and D A：Profit dropped B A D：Business profits decline C C：Production and sales decline, and cost raises B：Stock price drops sharply H G D E F

48. Merge Events by Similarity Computation • Event representation – Bags of verbs and nouns – Bags of verbs, nouns and adjectives – Average word embedding of all words – Average word embedding of verbs, nouns and adjectives • Similarity measure – Jaccard similarity – Cosine similarity

49. Case Study

50. Case Study

51. Data Sources and Scale of Financial EEG • Data sourses：1,362,345 financial news articles – 716,391 individual stocks news articles from Tencent and Netease – 246,499 plate news articles from Netease – 399,455 articles filtered from 10 years’ newspaper articles • Scale of financial domain EEG: – 247,926 event nodes – 154,233 cause-effect pairs/directed edges – 3,111,720 similar pairs/undirected edges

52. Outline • Motivation of Event Evolutionary Graph • Related Work • Our Efforts on Event Evolutionary Graph • Conclusion

53. Conclusion • By farming in various domains for a long time, Knowledge Graph gradually shows its great value. – 知识图谱在各个领域精耕细作，逐渐显露价值 • Knowledge representation still needs breakthrough, and its inference ability needs to be improved. – 知识表示形式有待突破，推理能力有待提高 • Statistical script learning and event relation recognition attracted more and more research attentions. – 统计脚本学习和事件关系识别等事理图谱相关研究越来越吸引研究者关注

54. Conclusion • Event evolutionary graph, whose nodes are predicate phrases and edges are event evolutionary logics, is in the ascendant. – 以“谓词性短语”为节点，以事件演化（顺承、因果）为边的事理图谱方兴未艾 • Event evolutionary graph will play an important role in event prediction and dialogue generation research fields, and improve the interpretability of artificial intelligence systems. – 事理图谱必将在预测、对话等领域发挥重要作用，有力地提升人工智能系统的可解释性

55. Collaborators Bing Qin Professor Big Cilin Ming Liu Associate Professor Big Cilin Xiao Ding Assistant Researcher EEG

56. Thanks！