> DTBI: Why Aggregation Matters 14.5x More Than Orchestration in Oracle Systems
When combining unreliable inference oracles, voting vs weighted averaging can matter 14.5x more than orchestration strategy. A rare-class accuracy threshold (alpha > 0.5) predicts which aggregation wins. Adaptive DTBI learns online via Hedge with O(sqrt(T ln M)) regret.
// TABLE_OF_CONTENTS
DTBI and the Aggregation-Reversal Phenomenon¶
The Central Finding¶
| Dataset | Best Aggregation | RWL | Worst | RWL | Ratio |
|---|---|---|---|---|---|
| NSL-KDD | Weighted average | 6.55 | Voting | 100.28 | 14.5x |
| AI4I | Voting | 0.82 | Weighted avg | ~4.7 | 5.7x |
Proposition 1: Weighted averaging dominates voting when rare-class oracle accuracy \(\alpha < 0.5\). When \(\alpha > 0.5\), voting is preferred.
The Voting-vs-Bayesian Paradox¶
Bayesian fusion is more accurate in isolation but worse in the full pipeline:
| Metric | Bayesian | Voting | Delta |
|---|---|---|---|
| RWL | 4.58 | 1.42 | -69% |
| Escalation rate | 100% | 20% | -80% |
Mechanism: Bayesian fusion produces overconfident beliefs → large risk gaps under asymmetric loss → triggers escalation 100% of the time → wastes budget on 81% NORMAL states.
Budget Phase Transition¶
Below budget=10, DTBI equals cheap baselines. Above budget=20, near-optimal performance. This discontinuous jump suggests a minimum viable budget threshold for VOI-based systems.
Drift Sensitivity¶
| Method | Pre-Drift RWL | Post-Drift RWL | Degradation |
|---|---|---|---|
| DTBI | 2.11 | 11.07 | 5.25x |
| HMM-only | 4.92 | 5.02 | 1.02x |
Fundamental tradeoff: the more a system exploits training distribution structure, the more exposed it is to shift.
Design Principles¶
- Always benchmark aggregation end-to-end, not in isolation
- Measure oracle calibration before selecting aggregation
- Correlated oracle queries waste budget (Proposition 3)
- Use Adaptive DTBI (Hedge, \(O(\sqrt{T \ln M})\) regret) when distribution is unknown