Abstract
Summarization is the notion of abstracting key content from information sources. The task of summarization has become vital in our everyday lives especially nowadays with the exponentially growing amounts of available information that is generated over the internet. In Natural Language processing, automatic text summarization is the problem of creating short, accurate and fluent text pieces from large textual data. In this thesis, I test state-of-the-art Deep Learning models for summarizing single documents and document clusters. Later, I compare the resulting cluster summaries with principal cluster labeling techniques to determine how accurate the Deep Learning models can be in labeling document clusters. For single-document summarization I chose a model consisted of multiple neural layers based on the encoder-decoder architecture with a novel Recurrent neural cell ( Rotational Unit of Memory-RUM ) in its core. I trained and evaluated the model on a new U.S. patent dataset. For document cluster summarization, I tested a framework that combines a fundamental extractive statistical natural language processing technique ( Maximal Marginal Relevance ) with the abstractive encoder-decoder model to summarize the document clusters. Finally, I evaluated the resulting summaries by comparing them with the top words of the clusters based on the Word Frequency and their TF*IDF scores. The RUM-based encoder-decoder model surpassed all the state-of-the-art models that were tested on the selected dataset and the MMR framework successfully summarized document clusters of up to 100 U.S. patent abstracts.
