About this paper

Reproduces the Structurally Constrained Recurrent Network and studies regularization for neural language modeling.

The experiments evaluate dropout, variational dropout, and weight tying. With suitable regularization and optimization, SCRN performs comparably to LSTM on English data and better on the tested non-English data, with results depending on the regularization setting.