About this paper

Explores weight sharing in subword-aware language models to reduce model size while improving selected architectures.

The study compares character-, syllable-, and morpheme-aware models. It develops a bottom-up rule for tying embedding layers and shows that suitable reuse can improve morpheme-aware models while substantially reducing their parameter counts.