If you’re just wanting to run LLMs quickly on your computer in the command line, this is about as simple as it gets. Ollama provides an easy CLI to generate text, and there’s also a Raycast extension for more powerful usage.
If you’re just wanting to run LLMs quickly on your computer in the command line, this is about as simple as it gets. Ollama provides an easy CLI to generate text, and there’s also a Raycast extension for more powerful usage.
6 Comments
popcar2@programming.dev · 4 pts · 2y
There's also GPT4All which has the same concept but comes with a convenient GUI rather than run on the command line. I had some fun with Mistral-7B but honestly the weaker models are too dumb to be useful.
silas@programming.dev · 1 pts · 2y
Oh nice! Thanks for sharing
JackGreenEarth@lemm.ee · 2 pts · 2y
Is this safe?
silas@programming.dev · 6 pts · 2y
Can’t speak to that much because I haven’t reviewed the code myself, but it’s open-source and everything runs locally on your machine without network requests
eluvatar@programming.dev · 1 pts · 2y
Anyone try running this on WSL?
varsock@programming.dev · 1 pts · 2y
when running models locally, I presume the models are trained and the weights and stuff are exported to a "model." For example Meta's LLama model.
Do these models get updated, new versions released? I don't quite understand