c/hackernews · by rssbot@lemmy.bestiver.se · 54dThere is minimal downside to switching to open models https://www.marble.onl/posts/cancel_claude.htmlComments 18 points · 1 comments · view on lemmy.world
1 Comments
ejs@piefed.social · 3 pts · 54d
… if you have the ram (with fast enough bandwidth)
MoE models are pretty magic on my laptops 32gb ram. 24 tok/sec on DDR5-5600 using Gemma 4 26B-A4B is so much faster than a dense model