I switched from ollama to llama.cpp and love it. For such an article, it frustrates me that they don't include the common and popular option in the comparison.
Ollama gained traction by being the first easy llama.cpp wrapper, then spent years dodging attribution, misleading users, and pivoting to cloud, all while riding VC money earned on someone else's engine.
He shares that the people behind llama.cpp don't act poorly (at least in those regards). My experience confirms llama.cpp can run just about any gguf while ollama can only run those that have been customized for ollama, and llama.cpp seems faster (no evidence, just anecdotal).
5 Comments
Flatulent69Iguana@lemmy.world · 11 pts · 23d
llama-cpp
mike_wooskey@lemmy.thewooskeys.com · 5 pts · 23d
I switched from ollama to llama.cpp and love it. For such an article, it frustrates me that they don't include the common and popular option in the comparison.
AstroLightz@lemmy.world · 3 pts · 22d
What's the difference between the two?
mike_wooskey@lemmy.thewooskeys.com · 4 pts · 22d
Here the article that spurred my change: Friends don't let friends use Ollama
He shares that the people behind llama.cpp don't act poorly (at least in those regards). My experience confirms llama.cpp can run just about any gguf while ollama can only run those that have been customized for ollama, and llama.cpp seems faster (no evidence, just anecdotal).
fubarx@lemmy.world · 3 pts · 22d
vLLM.
It's slower to start, but once it gets going, pretty solid.