Title
An efficient minimum vocabulary construction algorithm for language modeling
Abstract
In learning a new word by a dictionary, we first need to know a set of "basic words" which are frequently appeared in word definitions. It often happens that you cannot understand the word you looked up because there are still some words you do not understand in its definitions or explanations provided by the dictionary. You can keep looking up these new words recursively till they all can be well explained by some basic words you already knew. How to automatically find a minimum set of such basic words to define (or recursively define) the entire vocabulary in a given dictionary is what are going to discuss in this paper. We propose an efficient algorithm to construct the Minimum Vocabulary (MV) using the word frequency information. The minimum vocabulary can be used for language modeling and experimental results demonstrate the effectiveness of using the minimum vocabulary as features in text classification.
Year
DOI
Venue
2012
10.1007/978-3-642-31087-4_31
IEA/AIE
Keywords
Field
DocType
basic word,word frequency information,efficient minimum vocabulary construction,new words recursively,new word,language modeling,efficient algorithm,word definition,minimum set,entire vocabulary,minimum vocabulary
Word lists by frequency,Computer science,Algorithm,Natural language processing,Artificial intelligence,Need to know,Vocabulary,Recursion,Language model
Conference
Citations 
PageRank 
References 
0
0.34
9
Authors
4
Name
Order
Citations
PageRank
Sina Lin182.16
Zengchang Qin243945.46
Zehua Huang300.34
Tao Wan418121.18