atd-parser ¶This program is a probabilistic adaptive top-down parser. It processes a training terminal symbol sequence from left to right multiple times to synthesize a grammar based on a template grammar (see Top-Down Template Grammar). A training terminal symbol sequence usually consists of parse units immediately following one another, where each parse unit corresponds to an expansion of start nonterminal symbol of a top-down template grammar. The synthesis method is iterative determinization of top-down template grammar.
The parser internally generates adaptive assembler routines for nonterminal symbols of a top-down template grammar. The assembler routines parse terminal symbol sequences for the nonterminal symbols and increment frequencies of productions of a PCFG for the top-down template grammar. Changing production frequencies results in changing spur. An assembler routine can call other assembler routines for nested nonterminal symbols. See Assembler Instruction Set for the description of an instruction set used in the assembler routines. See Assembler Program Structure for the description of their building blocks.
See mk-rg-vit.sh for the description of a script that generates a top-down template grammar for dividing a text into words.
The top-down template grammar provides one-level segmenting a terminal symbol sequence for probabilistic parsing.