# i fit 400K of context per agent on 80gb of ram, and it still finds a code hidden at 92% of the prompt

two slots of 409,600 tokens fit on 80gb of ram, and a 373,723 token prompt with three codes hidden inside gets 3 right answers out of 3, read in 195 seconds

- date: 2026-10-19
- type: experiment (local ai)
- hardware: 3090 + 3080 Ti, 80gb
- source: measurements made by the author on his own hardware
- html page: https://pkillsleep.dev/en/labo/400k-contexte-80go/

## what i wanted to know

the model i use natively accepts 262K tokens of context, and you can stretch it twice with YaRN. the question was whether two agents could each get 400K of context on a machine with 80gb of ram, and above all whether the model could still find a piece of info buried at the very bottom of a prompt that long.

## the setup

two slots of 409,600 tokens, the K/V cache in int8, the last 32K tokens of each conversation kept in VRAM and everything else in ram. the service runs with a 72gb memory cap, a lesson i had learned [the hard way the day before](/en/labo/512k-machine-figee/).

## the test

i took a 373,723 token prompt and hid three codes in it, at 5%, 50% and 92% depth, then asked the model to give them back. it gave all three codes right, after reading the prompt in 195 seconds (1,914 tok/s), then answered at 69 tok/s.

## what it costs in ram

**K/V cache in ram, per session**

| | GiB |
|---|---:|
| 131K | 1.16 |
| 262K | 2.32 |
| 400K | 3.63 |


| context | K/V per session |
|---|---:|
| 131K | 1.16 GiB |
| 262K | 2.32 GiB |
| 400K | 3.63 GiB |

with two slots there are in practice three sessions in memory, so about 10.6gb of cache at 400K, versus 7gb at 262K. under real load with two agents, the ram peak went up to around 70gb, just under the cap.

[meme: Serious Cat]

## what i keep

400K per agent works on 80gb if you do the math before launching. but i don't use it as proof that the model reasons well over 400K: it proves it can find a piece of info, which is a different story.

## What these numbers don't tell you

- finding a code in a long prompt works; reasoning for a long time over 400K of context, i didn't measure that, and what people in the community report is that reasoning gives out before retrieval does
- the YaRN stretch applies to every request, even short ones: i didn't measure any quality loss, but i didn't measure anything either way