Titans — A Nanosecond Fast Path with an LLM Slow Lane
An event-driven C++20 trading engine built around a hard boundary between a nanosecond fast path and an LLM-scale slow lane — and a measurement harness that refuses to report numbers it cannot resolve. Three of the results in its README are refusals.
// DESCRIPTION
Six orders of magnitude separate a local language model (10–500 ms) from this engine's fast path (single-digit nanoseconds per operation). The design consequence is that the model is never in the request path. It publishes advisories — small, immutable, explicitly expiring facts — into a slot the fast lane samples with a bounded, non-blocking read.
The contract is asserted, not hoped for
Nothing the slow lane does may degrade the fast path, and the test suite proves it. With the slow lane sleeping 200 ms per item — the latency of a real model — the fast lane's p99 is unchanged and the bridge sheds 99.9% of observations. Load shedding is the designed behaviour at that edge, so the drop rate is a reported metric rather than a swallowed error.
The advisory slot is a versioned ring rather than a seqlock because the seqlock version tore 414 times in 2.8M reads and the test caught it.
Measured, with the machine state attached
| Operation | Cost | Throughput |
|---|---|---|
| SPSCQueue::try_push | 1.21 ns | 826 M ops/s |
| SPSCQueue::try_pop | 0.78 ns | 1279 M ops/s |
| ObjectPool::allocate | 1.41 ns | 708 M ops/s |
| operator new/delete (baseline) | 11.71 ns | 85 M ops/s |
| L2OrderBook::update_level | 24.03 ns | 42 M ops/s |
| EventBus::publish (pre-stamped) | 38.26 ns | 26 M ops/s |
What makes this unusual
A trading system that publishes latency figures without stating how they were measured is not
making a checkable claim. Here every figure carries the machine state that produced it, and
scripts/reproduce.sh re-derives all of them and exits with the failure count.
More pointedly: a 40-run series showed six of nine metrics are bimodal across
processes — try_push takes either 1.02 ns or 1.48 ns, decided at process
start and stable for that run. Every published row is one draw from such a distribution. The
second significant digit does not reproduce, and the project says so rather than quoting it.
// HIGHLIGHTS
- Hard fast/slow lane boundary: a hung slow lane leaves fast-lane p99 unchanged
- 1.21 ns SPSC push, 24 ns order-book update, measured with machine state attached
- Seqlock advisory slot tore 414× in 2.8M reads — replaced with a versioned ring
- Six of nine benchmark metrics proved bimodal across processes, and reported as such
scripts/reproduce.shre-derives every published number