Relationships between Local Intrinsic Dimensionality and Tail Entropy
James Bailey, Michael E. Houle and Xingjun Ma
The local intrinsic dimensionality (LID) model assesses the complexity of data within the vicinity of a query point, through the growth rate of the probability measure within an expanding neighborhood. In this paper, we show how LID is asymptotically related to the entropy of the lower tail of the distribution of distances from the query. We establish tight relationships for cumulative Shannon entropy, entropy power, and their generalized Tsallis entropy variants, all with the potential for serving as the basis for new estimators of LID, or as substitutes for LID-based characterization and feature representations in classification and other learning contexts.
