Name
Abstract | ||
|---|---|---|
Traditional natural language parsers are based on rewrite rule systems developed in an arduous, time-consuming manner by grammarians. A majority of the grammarian''s efforts are devoted to the disambiguation process, first hypothesizing rules which dictate constituent categories and relationships among words in ambiguous sentences, and then seeking exceptions and corrections to these rules. In this work, I propose an automatic method for acquiring a statistical parser from a set of parsed sentences which takes advantage of some initial linguistic input, but avoids the pitfalls of the iterative and seemingly endless grammar development process. Based on distributionally-derived and linguistically-based features of language, this parser acquires a set of statistical decision trees which assign a probability distribution on the space of parse trees given the input sentence. By basing the disambiguation criteria selection on entropy reduction rather than human intuition, this parser development method is able to consider more sentences than a human grammarian can when making individual disambiguation rules. In experiments, the decision tree parser significantly outperforms a grammarian''s rule-based parser, achieving an accuracy rate of 78% compared to the rule-based parser''s 69%. |
Year | Venue | Keywords |
|---|---|---|
1994 | Natural language parsing as statistical pattern recognition | human grammarian,natural language parsing,natural language,statistical pattern recognition,decision tree parser,statistical parser,individual disambiguation rule,automatic method,parser development method,rule-based parser,endless grammar development process,disambiguation criteria selection,disambiguation process,decision tree,probability distribution,rule based,decision trees,development process,statistical model |
Field | DocType | Volume |
LR parser,Computer science,Artificial intelligence,Natural language processing,Recursive descent parser,Top-down parsing,Pattern recognition,Simple LR parser,GLR parser,LL parser,Parsing,Parser combinator,Machine learning | Journal | abs/cmp-lg/9405009 |
Citations | PageRank | References |
121 | 61.50 | 20 |
Authors | ||
1 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| David M. Magerman | 1 | 726 | 512.15 |