Geodesy

Benchmark (surveying)

Philosophy

Similarity (geometry)

Semantic similarity

Operating system

Selection (genetic algorithm)

Economics

Computer science

Natural language processing

Ranking (information retrieval)

Pattern recognition (psychology)

Relevance (law)

Geography

Image (mathematics)

Political science

Multi-document summarization

Autoencoder

Operations management

Artificial intelligence

Sentence

Information retrieval

Novelty

Deep learning

Metric (unit)

Theology

Ground truth

Automatic summarization

Encoder

Expert Systems with Applications

In this paper, we propose SummCoder, a novel methodology for generic extractive text summarization of single documents. The approach generates a summary according to three sentence selection metrics formulated by us: sentence content relevance, sentence novelty, and sentence position relevance. The sentence content relevance is measured using a deep auto-encoder network, and the novelty metric is derived by exploiting the similarity among sentences represented as embeddings in a distributed semantic space. The sentence position relevance metric is a hand-designed feature, which assigns more weight to the first few sentences through a dynamic weight calculation function regulated by the document length. Furthermore, a sentence ranking and a selection technique are developed to generate the document summary by ranking the sentences according to the final score obtained through the fusion of the three sentences selection metrics. We also introduce a new summarization benchmark, Tor Illegal Documents Summarization (TIDSumm) dataset, especially to assist Law Enforcement Agencies (LEAs), that contains two sets of ground truth summaries, manually created, for 100 web documents extracted from onion websites in Tor (The Onion Router) network. Empirical results show that, on DUC 2002, on Blog Summarization, and on TIDSumm datasets, our text summarization approach obtains comparable or better performance than the state-of-the-art methods for different ROUGE metrics.

SummCoder: An unsupervised framework for extractive text summarization based on deep auto-encoders