Uni-Layer

Unified Layer Contribution Analysis Toolkit

Uni-Layer provides a unified interface for computing and comparing over 30 layer contribution metrics across 7 categories: magnitude-based, gradient-based, activation-based, Fisher information, Taylor expansion, geometric, and representation similarity metrics. This comprehensive toolkit enables researchers and practitioners to systematically evaluate which layers matter most in deep neural networks for pruning, fine-tuning, and architecture analysis.

The toolkit addresses a common pain point: layer importance metrics are scattered across dozens of papers and codebases with incompatible interfaces and evaluation protocols. Uni-Layer normalizes all metrics to a common scale, provides consistent batch processing, and includes statistical comparison tools (rank correlation, agreement analysis) to understand when different metrics agree or diverge.

Two practical bridges connect Uni-Layer to downstream workflows: a Torch-Pruning bridge that converts layer importance scores into pruning decisions compatible with the Torch-Pruning library, and a PEFT LoRA bridge that uses layer importance to automatically select per-layer LoRA ranks, allocating more adapter capacity to important layers. The LoRA bridge achieves comparable fine-tuning quality with 30-40% fewer total adapter parameters.

The implementation is built on PyTorch with scikit-learn for statistical analysis, and supports both vision (ResNet, ViT) and language (BERT, GPT-2, Llama) model families out of the box.

• 30+ layer contribution metrics from 7 categories in a unified API
• Torch-Pruning bridge for automatic pruning decision generation
• PEFT LoRA rank bridge achieving comparable quality with 30-40% fewer adapter parameters
• Supports both vision (ResNet, ViT) and language (BERT, GPT-2, Llama) models
• Statistical comparison tools for metric agreement and divergence analysis