Draft Model

1 article in this topic

S Synor·May 28

Speculative Decoding: Theory and Implementation

Expert guide to speculative decoding for accelerating LLM inference: draft model selection, acceptance rates, rejection sampling, tree-based speculation…

9 min read#Speculative Decoding #LLM Inference Acceleration

000