But in all fairness, it's really llama.cpp that supports AMD.
Now looking forward to the Vulkan support!
Ollama now supports AMD graphics cards
https://ollama.com/blog/amd-preview
https://ollama.com/blog/amd-preview
But in all fairness, it's really llama.cpp that supports AMD.
Now looking forward to the Vulkan support!
4 Comments
sardaukar@lemmy.world · 7 pts · 2y
I've been using it with a 6800 for a few months now, all it needs is a few env vars.
stsquad@lemmy.ml · 5 pts · 2y
That's cool. I've just recently gotten hold of an interesting Ampere system and it's got an AMD card in it. I must give it a spin.
stsquad@lemmy.ml · 1 pts · 2y
I was sadly stymied by the fact the rocm driver install is very much x86 only.
turkishdelight@lemmy.ml · 2 pts · 2y
It's improving very fast. Give it a little time.