Bonsai 27B: The First 27B-Class Model to Run on a Phone

https://prismml.com/news/bonsai-27b

7 points · 1 comments · view on lemmy.world

1 Comments

vhstape@lemmy.ml · 2 pts · 44d

I'm surprised that the model's benchmark performance isn't that much worse than the base given that it's running with 1-bit quantization. Impressive.