About this paper

Explains a geometric relationship between word and context embeddings: reflection in approximately half the dimensions.

Starting from probabilistic assumptions about word vectors and text generation, the analysis connects the pointwise mutual information matrix to a random symmetric Gaussian ensemble. It also motivates a principled way to tie weights in the SGNS model.