c/technology · by yogthos@lemmy.ml · 44dBonsai 27B: The First 27B-Class Model to Run on a Phone https://prismml.com/news/bonsai-27b7 points · 1 comments · view on lemmy.world
1 Comments
vhstape@lemmy.ml · 2 pts · 44d
I'm surprised that the model's benchmark performance isn't that much worse than the base given that it's running with 1-bit quantization. Impressive.