Ching Lam CHOIBAKE distils self-knowledge of a network, by propagating samples similarities in each batch.
Preprint 2021
Self-distillation with Batch Knowledge Ensembling Improves ImageNet Classification
Related work
NeurIPS 2026Oral
Shaping Useful Noise: Energy Distributions Predict Visual Pretraining Quality
Synthetic data is useful when it occupies the right real-image likelihood regimes; ELD measures energy shape to predict, diagnose, curate visual pretraining data.