cross-posted from: https://programming.dev/post/51407459
Check what can you use and at what rate of token per seconds would it be... It has examples of many models and quantization levels. Huge resource!
cross-posted from: https://programming.dev/post/51407459
Check what can you use and at what rate of token per seconds would it be... It has examples of many models and quantization levels. Huge resource!
1 Comments
green_red_black@slrpnk.net · -2 pts · 64d
Doctorbllk@slrpnk.net · 3 pts · 64d
I know I will invite ire with this, but I think a self hosted model is relatively acceptable. Get rid of the generative art and stick to things like code and evaluation via a model not being sourced by a massive data center (plus the capability to train a model in a way you may find even more acceptable than a default) and most if not all of the questionable aspects of LLMs fade away.