> IJCNN to ECML-PKDD: Evolving a KD Paper Through Reviewer Feedback

Knowledge Distillation 中级 2026-04-01 06:41 2026-04-01
#paper writing #reviewer feedback #IJCNN #ECML-PKDD #research methodology #peer review

How IJCNN reviewer feedback drove fundamental improvements: from fabricated references and unclear notation to three information-geometric theorems with quantitative predictions. A practical guide to addressing reviewer concerns constructively.

// TABLE_OF_CONTENTS

IJCNN to ECML-PKDD: Paper Evolution Through Review

What Reviewers Found Wrong (IJCNN)

Issue Reviewer Severity
Fabricated references R4 Critical
Unclear method description R3 Major
Theory disconnected from experiments R3, R4 Major
Missing formal definitions R3 Major
Overstated scale claims R4 Major
No cross-domain validation R3 Major

What We Fixed for ECML-PKDD

Fully Resolved (6/13 issues)

  1. Citation quality: All references verified. IJCNN version had fabricated references — unacceptable. ECML-PKDD has every reference checked against actual publications.

  2. Writing structure: Added narrative introduction, clearer method section, consistent notation throughout.

  3. Theory-experiment bridge: New Theorem 3 (three-factor decomposition) provides quantitative predictions (<2.5% gap) that are directly testable. Observed gap: <0.01%. Theory now makes falsifiable claims.

  4. Cross-domain validation: Added CIFAR-100 experiments (2 configurations). Results: -0.78% and +0.90%, average +0.06% — consistent with "no significant benefit."

  5. Goldilocks zone: Now properly defined with statistical test (p=0.12).

  6. Missing definitions: Softmax, sigmoid, entropy now formally defined in Eq. 1-3.

Partially Resolved (3/13)

  1. Scale claims: "124M to 1T" in abstract, but actual training experiments only go to 1.5B. Large-scale validation (27B-1T) is inference-only sampling consistency, not full KD training.

  2. Theorem 4 assumptions: Still unverified. Should be downgraded to Proposition until assumptions are validated.

  3. Entropy compression: de Bruijn identity added but deeper analysis needed.

Still Open (4/13)

  1. No high-entropy token ablation (critical for Theorem 2 claims)
  2. No computational overhead analysis
  3. No notation table
  4. CV experiments still shallow (only CIFAR-100, no ImageNet)

Key Insight: Reviewers Were Right

Both IJCNN reviewers who flagged problems were correct in their core critiques: - R4's "fabricated references" finding was 100% accurate - R3's "reweighting, not selection" identified the exact conceptual fault

Taking reviewer skepticism as a hypothesis rather than a critique to rebut would have accelerated the theoretical resolution. The "three-factor identity" (Theorem 3) came directly from trying to understand WHY the gap was so small — exactly what R3 was asking.

Practical Checklist for Paper Revision

  1. Verify every reference exists (use DOI lookup, not just title search)
  2. Define every symbol on first use
  3. Make theory produce testable predictions with specific numbers
  4. Add at least one cross-domain experiment (even if small)
  5. Be honest about scale: "validated at X scale" vs "trained at Y scale" are different claims
  6. Address reviewer concerns directly, not defensively
< BACK_TO_NOTES()