A two-dimensional example makes the reflection relation visible: one coordinate is preserved and the other changes sign. Under the paper’s assumptions, the analogous transformation reflects approximately half of the embedding dimensions. Research source
About this paper
Explains a geometric relationship between word and context embeddings: reflection in approximately half the dimensions.
Starting from probabilistic assumptions about word vectors and text generation, the analysis connects the pointwise mutual information matrix to a random symmetric Gaussian ensemble. It also motivates a principled way to tie weights in the SGNS model.
Citation
BibTeX
@article{takhanov2019contextandwordvectors,
title = {Context Vectors are Reflections of Word Vectors in Half the Dimensions},
author = {Assylbekov, Zhenisbek and Takhanov, Rustem},
year = {2019},
journal = {Journal of Artificial Intelligence Research},
url = {https://jair.org/index.php/jair/article/view/11368}
}