Speculative Decoding: Theory and Implementation
Expert guide to speculative decoding for accelerating LLM inference: draft model selection, acceptance rates, rejection sampling, tree-based speculation…
1 article in this topic
Expert guide to speculative decoding for accelerating LLM inference: draft model selection, acceptance rates, rejection sampling, tree-based speculation…