Top-K in RAG Search
Top-K in RAG Search In Retrieval-Augmented Generation (RAG), top-k is the number of most relevant document chunks the retriever returns from the vector store for a given query. The “k” is literally just a number — top-3, top-5, top-10, etc. How it works Embed the query into a vector Run a similarity search (cosine, dot product, etc.) against indexed chunks Retriever ranks every chunk by similarity score Top-k says “give me the k highest-scoring ones” Those chunks get stuffed into the LLM’s context as grounding material before generation Choosing k — the tradeoff Too low (k=1, 2): ...