RAG Context Budget: Calculate How Many Chunks Fit

Start with the space left after everything else

A RAG context budget is the room available for retrieved passages in a single model request. It is smaller than the advertised context window because instructions, conversation, tools, and the reply need space too. Budget the passages you will send, not your entire document collection.

The calculation below uses a 64,000-token window as an illustrative endpoint limit. Replace it with your provider’s actual limit. Separate input and output caps can constrain a request even when this total fits.

A worked RAG context budget

64,000 − 2,000 − 6,000 − 2,000 − 8,192 − 6,400 = 39,408 tokens

The 10% margin is a planning assumption, not a required setting. It leaves room for uncertain counts or extra material. The 8,192-token output reserve is also an example, not a promise that every endpoint allows that output length.

Open this 64K RAG budget in the calculator and adjust the numbers for your request.

Convert the remaining budget into chunks

Assume each retrieved chunk contains 600 tokens of passage text plus 20 tokens for its title, source, and separators. Each assembled chunk then costs 620 tokens. Round down when dividing the remaining budget: floor(39,408 ÷ 620) = 63 chunks.

The 63-chunk example uses 39,060 retrieval tokens and leaves 348 tokens unallocated, with the output reserve and safety margin already protected. Open the budget with those 63 chunks included.

This is a capacity ceiling, not a recommended retrieval count. More passages can add irrelevant or conflicting material. Choose retrieval depth using answer quality on your own questions; a larger context allowance alone does not establish better results.

Count the assembled request

1. Measure the text actually sent after retrieval, filtering, and formatting. Include citations, document names, wrappers, and repeated instructions.
2. If overlapping text is already part of each chunk’s token count, do not add the overlap a second time. Repeated text that appears in two submitted chunks still occupies space twice.
3. Use your model’s tokenizer or supported token-counting endpoint. Chunk sizes measured in characters are not reliable token counts.
4. For variable-length chunks, add their actual counts instead of multiplying by an average. Check the full request again before sending it.

Anthropic documents how input and output share context for Claude in its context-window guide. Its token-counting API can estimate supported message payloads. Provider rules differ, so check the endpoint you use.

Common RAG budgeting questions