Loud Fans, Cheap Speed, Qwen 3.8

I have purchased and installed 4 nvidia V100s in PCIe format and am now running qwen3.8 27b on them. I admit it was a lot of work to get it working though it would be much easier the second time around. The two biggest surprises: the fans are LOUD and I need to set up a Shelly RGBW2 controller so I can downvolt them as needed, and this setup uses a surprising amount of host RAM.

I am getting from 20 to 40 tok/s and can run 9 concurrent sessions. The four cards ran me around 2.7k and the rest of the machine about 1800 more, 4.5k all in, and if I had sprung for the sxm2 data center version of these cards it would have improved speed a lot, but this is completely usable, and the qwen model is fantastic. I’m very pleasantly surprised with how well it works.

I’ve been using deepseek flash as my main side project driver for the last month or two (my 30 day spend is about $60, compared to previously using claude or chatgpt at $200). I’m looking forward to switching over to qwen3.8 27b going forward, at least until qwen4 releases.