Speculative Decoding: Theory and Implementation

Show notes

Expert guide to speculative decoding for accelerating LLM inference: draft model selection, acceptance rates, rejection sampling, tree-based speculation…