Everybody Quotes the Speedup. Nobody Shows You the Knob
Speculative decoding sounds like a free lunch, which should always make you check your wallet. A tiny draft model guesses several tokens ahead, the big model checks them all in one pass, and every correct guess is basically free. Papers cite 2x to 3x and you nod along in the meeting, right up until you're the one implementing it and hit the question nobody put on the slide.
How long should the draft go before you're just burning cycles for style points? What acceptance rate do you actually need before this pays for itself? At what point does a longer draft start losing money because the big model keeps rejecting the tail end of it like a bouncer at a club it doesn't recognize you at?
speculabench answers those with no GPU and no weights, so you can find out you were wrong for free instead of on a production invoice. It models the accept and reject dynamics the way the actual papers do (Leviathan et al. and Chen et al., 2023) and reports the one number you care about: real speedup, measured in units of the expensive forward pass you're trying to avoid.
Turn the Knobs, Watch the Number Move
Three inputs, and they are the whole story.
• Agreement: how often the draft model matches the target. This is the single biggest lever on speedup, and it is the one number most teams never actually measure before shipping.
• Draft length: how many tokens to propose per round. Too short wastes the verification pass. Too long, and the tail gets rejected more than it gets accepted.
• Cost ratio: how expensive the draft pass is relative to the target. A slow draft model quietly eats the savings from every token it gets right.
The Sweet Spot Is a Real Curve, Not a Guess
Sweep draft length at a fixed 80% agreement and watch the tradeoff happen in real numbers: speedup climbs as the draft reaches further ahead, peaks, then falls as the rejected tail stops paying for itself. That turnover point is the entire engineering decision most teams make by vibes and a shrug in standup. Here it's a number you can print and staple to the PR.
The Number
At 90% draft agreement with a draft that costs a tenth of the target, speculative decoding hits 2.86x speedup. At 50% agreement, the same setup limps to 1.41x, and the honest lesson is that a bad draft model doesn't get a discount just because it's small. Small and wrong is still wrong.
7 tests pass in CI across Python 3.9, 3.11, and 3.13. The whole simulation is deterministic, so the number you get is the number I got, no cherry-picked seed hiding in a drawer somewhere.