What is Speculative Decoding?
Explains speculative decoding, which pairs a small draft model with a large target model to accelerate LLM inference without changing outputs.
Speculative-Decoding
Local + Frontier Model Collaboration Patterns in Open Source Harnesses
As local LLMs improve, harnesses are learning to pair them with frontier models. A look at the four collaboration patterns already shipping in open source.