The diagonal marks matching independent inputs in a uniform four-point example. The paper bounds variation of the learning gradient across target functions using collision entropy and a parameter measuring pairwise dependence. Gradient arrows are schematic, not experimental results. Research source
About this paper
Relates the information in a learning gradient to pairwise independence and the collision entropy of the input distribution.
The resulting variance bound is applied to Learning with Errors mappings and high-frequency functions. Experiments examine the behavior of gradient-based attacks in these settings.
Citation
BibTeX
@article{takhanov2025gradientinformativeness,
title = {The informativeness of the gradient revisited},
author = {Takhanov, Rustem},
year = {2025},
journal = {Neural Networks},
eprint = {2505.22158},
archivePrefix = {arXiv},
url = {https://arxiv.org/pdf/2505.22158}
}