Seems like a really cool project. Lowering the barrier to entry of locally run models. As llamacpp supports a ton of models, I imagine it be easy to adapt this for other models other than the prebuilt ones.
LLMs made into single-file executables with llamafile
https://hacks.mozilla.org/2023/11/introducing-llamafile/
4 Comments
kakes@sh.itjust.works · 13 pts · 2y
My first thought is that this makes it a lot easier for a game dev to create a game that uses a local LLM. I'll definitely be keeping an eye on this for my own projects.
Edit: Trying it out now, color me impressed! I opened the file, and it worked instantly without a single finick to be found. Super excited for this!
kakes@sh.itjust.works · 1 pts · 2y
When I upload an image, it seems to be getting re-encoded every time I send a message, which takes a while. I've never really messed around with images in Llama - is this a normal/necessary thing?
kurwa@lemmy.world · 5 pts · 2y
That's pretty dope. I've been imagining some programs where having a small enough locally runnable model could be useful.
RandomLegend@lemmy.dbzer0.com · 4 pts · 2y
Works like a charm!