This A.I.-powered article reader can extract the text contents from web pages, partition the article into sentences, identify named entities in each sentences, and highlight the sentences that convey the core ideas of the article (in yellow).
The A.I. algorithm behind the reader is based on TextRank, which was originally designed to do extractive summarization (that is, extract sentences from the article as the summary). The summarization process unavoidably loses information in the process, so instead of throwing out unimportant sentences, we highlight the important sentences while keeping the rest intact. The readers can quickly browse the article for the key ideas and entities mentioned, and read the paragraphs that interest them for more contexts and details.
The reader interface also provides an option to manually annotates (highlight) the important sentences (in green), which can be used to create datasets for further fine-tuning the highlighting model. The reader provides full support for English and basic support for Japanese and Chinese.
We open-sourced a demo version of the improved TextRank algorithm at github.com/ceshine/textrank_demo (the actual algorithm used in the reader is a bit more sophisticated).
The technology used in this project:
Manual annotation support (highlight in gree)
Basic Chinese support (no named entity recognition)