TITLE:
A New Method for Calculating Similarity between Sentences and Application on Automatic Abstracting
AUTHORS:
Wenqian JI, Zhoujun LI, Wenhan CHAO, Xiaoming CHEN
KEYWORDS:
sentence similarity, automatic abstracting, lexrank, sentence-weight computing, redundancy resolution
JOURNAL NAME:
Intelligent Information Management,
Vol.1 No.1,
July
31,
2009
ABSTRACT: Sentence similarity computing plays an important role in machine question-answering systems, machine-translation systems, information retrieval and automatic abstracting systems. This article firstly sums up several methods for calculating similarity between sentences, and brings out a new method which takes all factors into consideration including critical words, semantic information, sentential form and sen-tence length. And on this basis, a automatic abstracting system based on LexRank algorithm is implemented. We made several improvements in both sentence weight computing and redundancy resolution. The system described in this article could deal with single or multi-document summarization both in English and Chinese. With evaluations on two corpuses, our system could produce better summaries to a certain degree. We also show that our system is quite insensitive to the noise in the data that may result from an imperfect topical clustering of documents. And in the end, existing problem and the developing trend of automatic summariza-tion technology are discussed.