I got interested in running LLM's locally all the way back when he first LLama "leak" happened. Running them on my laptop, then on my gaming desktop (3070), then adding a 3060 12Gb so I could both run image generation and LLM's at the same time. However, as LLM's are getting better, I decided to put a bit more money into building an AI rig.
Why do I want to run LLM's locally? Well, some things are not worth paying API tokens for. A couple months ago I made an insulting mirror - an AI looks at a webcamera stream and insults everything it sees. This involves effectively streaming videos at an LLM. I'd hate to think of the token cost. But with a local Qwen vision model, it was no drama to prototype and run. Similarly, running experimental AI agents, or vibe-coding speculative things. Sometimes those things have zero value, but are fun to play around with. And the mental "these tokens cost money" adds up quick (and with things like claw's and autoresearch, the actual costs can add up quickly too).
And recently I've been having fun training little stereo depth models, so some extra GPU doesn't hurt.
My original plan was to buy a 3090 to replace the 3060 and 3070, so when once ame up on marketplace for $1200, I messaged the guy. Unfortunately a week later it hadn't arrived and the post hadn't even picked it up. So I messaged him and he said he'd put it out on the porch, confirmed that the NZPost website said it hadn't been picked up and offered me my money back - concluding that it had been stolen from his porch. I said we could wait a few more days in case it arrived.
In the meantime, another good deal of $1500 came along on TradeMe, so I hit the Buy Now button, and that very evening picked it up.
The next evening, I got a message from the original seller. His mother(!) had seen the parcel on the porch and taken it inside. So he scheduled postal for it again, and this time it was picked up
So now I had two GPU's, two big chonking hungry GPU's, and while my PSU was 1000W, I didn't have enough cables to connect to all the power pins. Fortunately on Trademe someone was selling a 1600W PSu for $220, so I snagged that.
Uh oh, it doesn't fit in the case I have. Not even close! So I took a trip to the local e-waste store, and for $40 picked up the largest, most obnoxiously huge PC case I have ever seen...... It barely fits under my desk and has a huge 220mm fan on the side. Plenty of space.
And you know what? Github Copilot has halted new signups to it's plan, Anthropic has removed Claude Code from it's $20 plan, and OpenAI is no longer offering their 2x multiplier. Fortunately Qwen3.6-27B has just been released and it benchmarks as good as Opus 4.5 (the first model that was really useful for agentic coding). I was running Qwen3.6-35B-A3B yesterday and it wa pretty good at vibe-coding, so I'm excited to try the 27B.....