> When Does LLM Reasoning Help Recommendation? The Semantic Richness Index
A cross-domain study: LLMs improve recommendation only when SRI >= 0.45 (semantic-rich content). On MIND news: +102%. On Amazon Electronics: -50%. Decision rule + 3 efficiency mechanisms yield 81.8% cost reduction.
// TABLE_OF_CONTENTS
Semantic Richness Index for LLM Recommendation¶
Cross-Domain Results¶
| Dataset | Items | SRI | LLM Effect |
|---|---|---|---|
| MIND (news) | 101K | 0.56 | +102% |
| MovieLens | 3.9K | 0.41 | -3% |
| Amazon Movies | 748K | 0.19 | -11% |
| Amazon Electronics | 1.6M | 0.38 | -50% |
| Criteo (encrypted) | 5M | 0.26 | -46% |
Decision rule: If SRI < 0.45, skip LLM integration.
SRI Formula¶
\[\text{SRI} = 0.25 \times \frac{\text{title\_len}}{20} + 0.25 \times \text{vocab\_diversity} + 0.25 \times \frac{\text{cat\_entropy}}{5} + 0.25 \times \text{desc\_coverage}\]
Pearson r=0.60 with LLM benefit. Runs in seconds on any dataset, no training needed.
Three Efficiency Mechanisms¶
- Confidence-based early exit: 0.2ms (trie) vs 166ms (LLM call)
- Trie-guided context compression: 47% token reduction with 2.1% quality improvement
- Score-based caching: Cache user scores (not rankings) for re-ranking new items
Combined: 81.8% cost reduction, 53% carbon savings.
Key Negative Result¶
FrugalGPT cascade routing provides zero benefit for recommendation — confidence scoring that works for classification doesn't transfer to recommendation because difficulty is driven by data sparsity, not semantic complexity.