Overlapping subword patterns provide an alternative to individual characters. This schematic illustrates how pattern embeddings can be combined into a word representation; the example substrings are illustrative rather than extracted from a trained model. Research source
About this paper
Uses statistically informative subword patterns as building blocks for word representations and neural language modeling.
A regularized pattern-based CRF selects subwords that capture character n-gram structure. Language models built from these patterns are compared with character-based approaches, including a simple sum-of-embeddings architecture.
Citation
BibTeX
@inproceedings{takhanov2017patternsversuscharacters,
title = {Patterns versus Characters in Subword-aware Neural Language Modeling},
author = {Takhanov, Rustem and Assylbekov, Zhenisbek},
year = {2017},
booktitle = {ICONIP 2017},
eprint = {1709.00541},
archivePrefix = {arXiv},
url = {https://arxiv.org/pdf/1709.00541}
}