文本摘要[65-74]旨在将文本或文本集合转换为包含关键信息的简短摘要，是缓解文本信息过载的一个重要手段。文本摘要按照输入类型，可分为单文档摘要和多文档摘要。单文档摘要从给定的一个文档中生成摘要，多文档摘要从给定的一组主题相关的文档中生成摘要。按照输出类型可分为抽取式摘要和生成式摘要。抽取式摘要从源文档中抽取关键句和关键词组成摘要，摘要信息全部来源于原文。生成式摘要根据原文，允许生成新的词语、短语来组成摘要。此外，按照有无监督数据，文本摘要可以分为有监督摘要和无监督摘要。根据输入数据领域，文本摘要又可以分为新闻摘要、专利摘要、论文摘要、对话摘要等等。自动文本摘要可以看作是一个信息压缩的过程，我们将输入的一篇或多篇文档自动压缩为一篇简短的摘要，该过程不可避免地存在信息损失，但要求保留尽可能多的重要信息。自动文摘系统通常涉及对输入文档的理解、要点的筛选以及文摘合成这三个主要步骤。其中，文档理解可浅可深，大多数自动文摘系统只需要进行比较浅层的文档理解，例如段落划分、句子切分、词法分析等，也有文摘系统需要依赖句法解析、语义角色标注、指代消解，甚至深层语义分析等技术。对话摘要是文本摘要的一个特例，其核心面向的是对话类数据。对话类数据有着不同的形式，例如：会议、闲聊、邮件、辩论、客服等等。不同形式的对话摘要在自己的特定领域有着不同的应用场景，但是它们的核心与摘要任务的核心是一致的，都是为了捕捉对话中的关键信息，帮助快速理解对话的核心内容。与文本摘要不同的是，对话摘要的关键信息常常散落在不同之处，对话中的说话者、话题不停地转换。此外，当前也缺少对话摘要的数据集，这些都增大了对话摘要的难度[64]。基于实际的场景，本文提出了阅读理解的距离监督Span-Level对话摘要方案《Distant Supervision based Machine Reading Comprehension for Extractive Summarization in Customer Service》（已发表在SIGIR 2021），该方法比强基准方法在ROUGE-L指标和BLEU指标上提升了3%左右。
2. 文本摘要与对话摘要经典模型介绍
文本摘要从生成方式上可分为抽取式摘要和生成式摘要两种模式。抽取式摘要通常使用算法从源文档中提取现成的关键词、句子作为摘要句。在通顺度上，一般优于生成式摘要。但是，抽取式摘要会引入过多的冗余信息，无法体现摘要本身的特点。生成式摘要则是基于NLG（Natural Language Generation）技术，根据源文档内容，由算法模型生成自然语言描述，而非直接提取原文的句子。目前，生成式摘要很多工作都是基于深度学习中的Seq2Seq模型[44]。最近在以BERT[34]为代表的大量预训练模型出世后，也有很多工作集中在如何利用预训练模型来做NLG任务。下面分别介绍上述两种模式下的经典模型。
为了解决现有对话摘要难以得到指定信息要素以及缺少标注数据的问题，我们提出了一个更灵活的、基于远程监督和阅读理解的抽取式摘要模型（Distant Supervision based Machine Reading Comprehension Model for Extractive Summarization），简称为DSMRC-S，总体结构如下图6所示：图6 DSMRC-S模型的总体结构DSMRC-S由一个基于BERT的MRC（Machine Reading Comprehension）模块、远程监督模块和一个基于密度的提取策略组成。在预处理阶段，对话中的Token会被自动标注，模型会被训练去预测对话中每个Token出现在答案中的概率。然后，基于上一步预测的概率，一个基于密度的提取策略会被用来提取最合适的Span作为答案。我们的方法可以主要分成两部分：①将对话摘要任务转换成阅读理解；②无需额外标注的阅读理解方案。
[1] A. M. Rush, S. Chopra, and J. Weston, “A neural attention model for abstractive sentence summarization,” in Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015.[2] A. See, P. J. Liu, and C. D. Manning, “Get to the point: Summarization with pointer-generator networks,” in Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017.[3] S. Gehrmann, Y. Deng, and A. M. Rush, “Bottom-up abstractive summarization,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018.[4] C. Liu, P. Wang, J. Xu, Z. Li, and J. Ye, “Automatic dialogue summary generation for customer service,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019.[5] S. Chopra, M. Auli, and A. M. Rush, “Abstractive sentence summarization with attentive recurrent neural networks,” in NAACL HLT 2016.[6] Y. Miao and P. Blunsom, “Language as a latent variable: Discrete generative models for sentence compression,” in Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016.[7] D. Wang, P. Liu, Y. Zheng, X. Qiu, and X. Huang, “Heterogeneous graph neural networks for extractive document summarization,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020.[8] M. Zhong, D. Wang, P. Liu, X. Qiu, and X. Huang, “A closer look at data bias in neural extractive summarization models.”[9] Q. Zhou, N. Yang, F. Wei, S. Huang, M. Zhou, and T. Zhao, “Neural document summarization by jointly learning to score and select sentences,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018,[10] J. Cheng and M. Lapata, “Neural summarization by extracting sentences and words,” in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016[11] R. Nallapati, F. Zhai, and B. Zhou, “Summarunner: A recurrent neural network based sequence model for extractive summarization of documents,” in Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence,[12] H. Pan, J. Zhou, Z. Zhao, Y. Liu, D. Cai, and M. Yang, “Dial2desc: End-to-end dialogue description generation,” CoRR, vol. abs/1811.00185, 2018.[13] C. Goo and Y. Chen, “Abstractive dialogue summarization with sentence-gated modeling optimized by dialogue acts,” in 2018 IEEE Spoken Language Technology Workshop, SLT 2018[14] J. Gu, T. Li, Q. Liu, Z. Ling, Z. Su, S. Wei, and X. Zhu, “Speaker-aware BERT for multi-turn response selection in retrieval-based chatbots,” in CIKM ’20[15] K. Filippova, E. Alfonseca, C. A. Colmenares, L. Kaiser, and O. Vinyals, “Sentence compression by deletion with lstms,” in Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015.[16] R. Nallapati, B. Zhou, C. N. dos Santos, C ̧. Gu ̈lc ̧ehre, and B. Xiang, “Abstractive text summarization using sequence-to-sequence rnns and beyond,” in Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, CoNLL 2016,[17] A. Celikyilmaz, A. Bosselut, X. He, and Y. Choi, “Deep communicating agents for abstractive summarization,” in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics[18] R. Paulus, C. Xiong, and R. Socher, “A deep reinforced model for abstractive summarization,” in 6th International Conference on Learning Representations, ICLR 2018[19] L. Zhao, W. Xu, and J. Guo, “Improving abstractive dialogue summarization with graph structures and topic words,” in Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020,[20] Y. Zou, L. Zhao, Y. Kang, J. Lin, M. Peng, Z. Jiang, C. Sun, Q. Zhang, X. Huang, and X. Liu, “Topic-oriented spoken dialogue summarization for customer service with saliency-aware topic modeling,” in Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021[21] Q. Zhou, N. Yang, F. Wei, S. Huang, M. Zhou, and T. Zhao, “A joint sentence scoring and selection framework for neural extractive document summarization,” IEEE ACM Trans. Audio Speech Lang. Process., vol. 28, pp. 671–681, 2020.[22] Y. Chen and M. Bansal, “Fast abstractive summarization with reinforce-selected sentence rewriting,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018.[23] A. Jadhav and V. Rajan, “Extractive summarization with SWAP-NET: sentences and words from alternating pointer networks,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018,[24] S. Narayan, S. B. Cohen, and M. Lapata, “Ranking sentences for extractive summarization with reinforcement learning,” in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018,[25] X. Zhang, M. Lapata, F. Wei, and M. Zhou, “Neural latent extractive document summarization,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,[26] Y. Liu, I. Titov, and M. Lapata, “Single document summarization as tree induction,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019,[27] J. Xu, Z. Gan, Y. Cheng, and J. Liu, “Discourse-aware neural extractive text summarization,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020[28] M. Zhong, P. Liu, Y. Chen, D. Wang, X. Qiu, and X. Huang, “Extractive summarization as text matching,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020[29] Y. Wu, W. Wu, C. Xing, ou, and Z. Li, “Sequential matching network: A new architecture for multi-turn response selection in retrieval-based chatbots,” in ACL 2017,[30] Z.Zhang,J.Li,P.Zhu,H.Zhao,andG.Liu,“Modelingmulti-turn conversation with deep utterance aggregation,” in COLING 2018,[31] X. Zhou, L. Li, D. Dong, Y. Liu, Y. Chen, W. X. Zhao, D. Yu, and H. Wu, “Multi-turn response selection for chatbots with deep attention matching network,” in ACL 2018[32] C. Tao, W. Wu, C. Xu, W. Hu, D. Zhao, and R. Yan, “One time of interaction may not be enough: Go deep with an interaction-over-interaction network for response selection in dialogues,” in ACL 2019[33] M. Henderson, I. Vulic, D. Gerz, I. Casanueva, P. Budzianowski, S. Coope, G. Spithourakis, T. Wen, N. Mrksic, and P. Su, “Training neural response selection for task-oriented dialogue systems,” in Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019[34] J. Devlin, M. Chang, K. Lee, and K. Toutanova, “BERT: pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019,[35] J. Dong and J. Huang, “Enhance word representation for out-of-vocabulary on ubuntu dialogue corpus,” CoRR, vol. abs/1802.02614, 2018.[36] C. Goo and Y. Chen, “Abstractive dialogue summarization with sentence-gated modeling optimized by dialogue acts,” in 2018 IEEE Spoken Language Technology Workshop, SLT 2018,[37] Q. Chen, Z. Zhuo, and W. Wang, “BERT for joint intent classification and slot filling,” CoRR, vol. abs/1902.10909, 2019.[38] L. Song, K. Xu, Y. Zhang, J. Chen, and D. Yu, “ZPR2: joint zero pronoun recovery and resolution using multi-task learning and BERT,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020[39] S. Chuang, A. H. Liu, T. Sung, and H. Lee, “Improving automatic speech recognition and speech translation via word embedding prediction,” IEEE ACM Trans. Audio Speech Lang. Process., vol. 29, pp. 93–105, 2021.[40] C.-Y. Lin, “ROUGE: A package for automatic evaluation of summaries,” in Text Summarization Branches Out. Barcelona, Spain: Association for Computational Linguistics, Jul. 2004, pp. 74–81.[41] K. Papineni, S. Roukos, T. Ward, and W. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics,[42] J. Li, M. Galley, C. Brockett, J. Gao, and B. Dolan, “A diversity-promoting objective function for neural conversation models,” in NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics.[43] Y. Liu and M. Lapata, “Text summarization with pretrained encoders,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019,[44] I.Sutskever,O.Vinyals,andQ.V.Le,“Sequence-to-sequence learning with neural networks,” in Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014[45] D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” in 3rd International Conference on Learning Representations, ICLR 2015,[46] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017,[47] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” J. Mach. Learn. Res., vol. 21, pp. 140:1–140:67, 2020.[48] R.Nallapati, F. Zhai, B. Zhou, “SummaRuNNer: A Recurrent Neural Network Based Sequence Model for Extractive Summarization of Documents.” AAAI 2017.[49] Q. Zhou, N. Yang, F. Wei, S. Huang, M. Zhou, T. Zhao, “Nerual Document Summarization by Jointly Learning to Score and Select Sentences,” ACL 2018.[50] Abigail See, Peter J Liu, and Christopher D Manning. Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368, 2017.[51] Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov and Luke Zettlemoyer. “BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension.” ACL (2020).[52] Zhang, Jingqing, Yao Zhao, Mohammad Saleh and Peter J. Liu. “PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization.” ArXiv abs/1912.08777 (2020): n. pag.[53] Yuan, Lin and Zhou Yu. “Abstractive Dialog Summarization with Semantic Scaffolds.” ArXiv abs/1910.00825 (2019): n. pag.[54] Zou, Yicheng, Lujun Zhao, Yangyang Kang, Jun Lin, Minlong Peng, Zhuoren Jiang, Changlong Sun, Qi Zhang, Xuanjing Huang and Xiaozhong Liu. “Topic-Oriented Spoken Dialogue Summarization for Customer Service with Saliency-Aware Topic Modeling.” AAAI (2021).[55] Brown, Tom B. et al. “Language Models are Few-Shot Learners.” ArXiv abs/2005.14165 (2020): n. pag.[56] Radford, Alec, Jeff Wu, Rewon Child, David Luan, Dario Amodei and Ilya Sutskever. “Language Models are Unsupervised Multitask Learners.” (2019).[57] Radford, Alec and Karthik Narasimhan. “Improving Language Understanding by Generative Pre-Training.” (2018).[58] Mihalcea, Rada and Paul Tarau. “TextRank: Bringing Order into Text.” EMNLP (2004).[59] Hartigan, J. A. and M. Anthony. Wong. “A k-means clustering algorithm.” (1979).[60] Comaniciu, Dorin and Peter Meer. “Mean Shift: A Robust Approach Toward Feature Space Analysis.” IEEE Trans. Pattern Anal. Mach. Intell. 24 (2002): 603-619.[61] Lin, Chin-Yew. “ROUGE: A Package for Automatic Evaluation of Summaries.” ACL 2004 (2004).[62] Papineni, Kishore, Salim Roukos, Todd Ward and Wei-Jing Zhu. “Bleu: a Method for Automatic Evaluation of Machine Translation.” ACL (2002).[63] Ishikawa, Kai, Shinichi Ando and Akitoshi Okumura. “Hybrid Text Summarization Method based on the TF Method and the Lead Method.” NTCIR (2001).[64] Feng, Xiachong, Xiaocheng Feng and Bing Qin. “A Survey on Dialogue Summarization: Recent Advances and New Frontiers.” ArXiv abs/2107.03175 (2021): n. pag.[65] El-Kassas, Wafaa S., Cherif R. Salama, Ahmed A. Rafea and Hoda Korashy Mohamed. “Automatic text summarization: A comprehensive survey.” Expert Syst. Appl. 165 (2021): 113679.[66] Nallapati, Ramesh, Bowen Zhou, Cícero Nogueira dos Santos, Çaglar Gülçehre and Bing Xiang. “Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond.” CoNLL (2016).[67] Shi, Tian, Yaser Keneshloo, Naren Ramakrishnan and Chandan K. Reddy. “Neural Abstractive Text Summarization with Sequence-to-Sequence Models.” ACM Transactions on Data Science 2 (2021): 1 - 37.[68] Fabbri, Alexander R., Irene Li, Tianwei She, Suyi Li and Dragomir R. Radev. “Multi-News: A Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model.” ArXiv abs/1906.01749 (2019): n. pag.[69] Li, Wei and Hai Zhuge. “Abstractive Multi-Document Summarization Based on Semantic Link Network.” IEEE Transactions on Knowledge and Data Engineering 33 (2021): 43-54.[70] DeYoung, Jay, Iz Beltagy, Madeleine van Zuylen, Bailey Kuehl and Lucy Lu Wang. “MSˆ2: Multi-Document Summarization of Medical Studies.” EMNLP (2021).[71] Nallapati, Ramesh, Feifei Zhai and Bowen Zhou. “SummaRuNNer: A Recurrent Neural Network Based Sequence Model for Extractive Summarization of Documents.” AAAI (2017).[72] Narayan, Shashi, Shay B. Cohen and Mirella Lapata. “Ranking Sentences for Extractive Summarization with Reinforcement Learning.” NAACL (2018).[73] Zhong, Ming, Pengfei Liu, Yiran Chen, Danqing Wang, Xipeng Qiu and Xuanjing Huang. “Extractive Summarization as Text Matching.” ACL (2020).[74] Zhang, Jingqing, Yao Zhao, Mohammad Saleh and Peter J. Liu. “PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization.” ArXiv abs/1912.08777 (2020): n. pag.
6. 本文作者
马兵、刘操、今雄、书杰、见耸、杨帆、广鲁等，均来自美团平台/语音交互部。---------- END ----------招聘信息语音交互部负责美团语音和智能交互技术及产品研发，面向美团业务和生态伙伴，提供对语音和口语数据的大规模处理及智能响应能力。经过多年研发积累，团队在语音识别、合成、口语理解、智能问答和多轮交互等技术上已建成大规模的技术平台服务，并研发包括外呼机器人、智能客服、语音内容分析等解决方案和产品，在公司丰富的业务场景中广泛落地；同时我们也非常重视与行业的紧密合作，通过美团语音应用平台已与第三方手机语音助手、智能音箱、智能车机等诸多合作伙伴开展对接，将语音生活服务应用提供给更多用户。 语音交互部长期招聘自然语言处理算法工程师、算法专家，感兴趣的同学可以将简历发送至chenjiansong@meituan.com。