The Local Learning Coefficient: A Singularity-Aware Complexity Measure

Abstract

This paper introduces the Local Learning Coefficient (LLC), a singularity-aware measure of model complexity grounded in singular learning theory. We provide scalable estimators using SGLD and demonstrate how LLC reveals inductive biases in deep learning. arXiv v1 used a different title and author list and placed greater emphasis on mathematical development. arXiv v2 aligns with the AISTATS camera-ready version and integrates material from arXiv:2402.03698.

Publication
The 28th International Conference on Artificial Intelligence and Statistics (AISTATS 2025)