WIP
Research
FisherKD-Unified
Fisher Information-guided adaptive knowledge distillation framework implementing 12+ distillation methods
// DESCRIPTION
FisherKD-Unified is a comprehensive research framework for knowledge distillation, using Fisher Information to guide the distillation process.
Research Goal
Develop principled methods for knowledge transfer that consider parameter importance through Fisher Information analysis.
Implemented Methods (12+)
- Logit-based: KD, DKD, DIST, NKD
- Feature-based: FitNets, AT, PKD, VID
- Relation-based: RKD, CRD, ICKD
- Fisher-based: Fisher KD, Adaptive Fisher
Current Focus
- Reasoning-aware knowledge distillation
- Cross-domain validation (NLP to Vision)
- Layer importance analysis
// HIGHLIGHTS
- 12+ KD methods implemented
- IJCAI/ICML 2026 target
- Cross-domain evaluation