used gpus, local ai, and a fair amount of stuff breaking along the way

i test things at home, i write down the numbers, i tell you what works and what crashes

the gpu boxhardware
gpu3090 + 3080 Ti, second-hand
cpuRyzen 7 5700X
ram80gb DDR4
power358 W under load
current modelai
modelQwen 3.8 Flash Next, 125B MoE
quantIQ3_XXS, on Strata
solo~110 tok/s
two agents~66 tok/s per agent
context400K per agent
the 24/7 serverservices
whatan old mac that never turns off
formy agents, my notes, my automations
ai100% local, nothing leaves the house
  1. Oct 22, 2026guide · local ai

    your first local llm with llama.cpp, on a 12gb card

    compile llama.cpp for your nvidia card and run a recent model with 256K of context on 12gb, while understanding what each option does

  2. Oct 22, 2026guide · local ai

    running glm 5.3 flash (321B) on two consumer cards with maya

    how to install project maya, pick the right version of the model when your cards have 24gb or less, and the two options that got me 22% more speed, with what they cost

  3. Oct 22, 2026guide · local ai

    qwen 3.8 flash next on strata: a 125B at more than 110 tok/s on a 3090 + a 3080 ti

    how i install strata and run qwen 3.8 flash next on two consumer cards at more than 110 tok/s solo, then how i share it between two agents at the same time

  4. Oct 22, 2026guide · hardware & power

    undervolting a 3090 or a 3080 ti on linux, without losing performance

    the step by step i use on my box: lock the clock, shift the voltage/frequency curve with NVML, check under real load, then make it automatic at boot

  5. Oct 20, 2026local ai

    two coding agents on the same box, from 44 to 66 tok/s each in one evening

    43.8 → 66.4 tok/s per agent

    same machine, same model, one night of tuning: when two agents generate at the same time, each one goes from 43.8 to 66.4 tok/s, and four settings did most of the work

  6. Oct 20, 2026local ai

    glm 5.3 flash (321B) on a 3090 + a 3080 ti: from 26 to 31.7 tok/s, and why i'm still sticking with qwen

    26.1 → 31.7 tok/s

    two maya options give 22% more decode to a 321B model that spills way over into ram, but on a real agent task it stays clearly slower than qwen 3.8 flash next on the same box

  7. Oct 20, 2026local ai

    a 4B model holds 256K of context on a single 12gb 3080 ti, and finds 40 codes out of 40

    136 → 35 tok/s at 254K

    spark-x2.6-4b in q8 with llama.cpp: 136 tok/s empty, still 35 tok/s with 254K tokens of context, and a 12gb card that beats the 3090 because i had capped it less

  8. Oct 20, 2026hardware & power

    i undervolted my 3090 and my 3080 ti on linux, -94 W for almost nothing

    452 W → 358 W

    two coding agents working at the same time, 452 W → 358 W across both cards, the fans go from 91% to 48%, and throughput loses 7%

  9. Oct 19, 2026local ai

    i fit 400K of context per agent on 80gb of ram, and it still finds a code hidden at 92% of the prompt

    3/3 codes at 373,723 tokens

    two slots of 409,600 tokens fit on 80gb of ram, and a 373,723 token prompt with three codes hidden inside gets 3 right answers out of 3, read in 195 seconds

  10. Oct 19, 2026crashes

    i asked for 512K of context × 4 slots on 80gb of ram, and the whole machine froze

    ~23gb of cache too many

    at first i blamed a dead wifi dongle, but it was the ram: the session cache at 512K needed ~23gb more on top of a 40gb model that was already taking up the space, and a simple memory cap would have avoided it all

  11. Oct 19, 2026local ai

    strata 0.2.0 on my rtx 30 cards: +12 to +17% decode, but my agents don't finish any faster

    91.7 → 107.6 tok/s

    a simple engine update takes solo decode from 91.7 to 107.6 tok/s on a 3090 + 3080 Ti, and yet the total time of my agents doesn't move, because they spend their life reading prompts