No, a GPU would be ideal but not everyone has one, especially one with enough VRAM. I have an AMD card with 12gb of VRAM and I can run 7B - 13B models but even the 7B models (which seems to be the lowest that is still good) use a little more than 8gb of VRAM and most people probably have an Nvidia card with 8gb or less. 13B models get very close to using the full 12gb.
it’s gonna take experimentation; there’s a list of all models in the app or on the site and maybe a little googling. you can still use openai too. mistral is solid overall though; good for programming
Where did this come from? Did OpenAI finally release the source code? And where does the training data come from? Is the training data public domain/appropriately licensed?
15 Comments
original_reader@lemm.ee · 13 pts · 2y
Here is the link: https://gpt4all.io/
desmosthenes@lemmy.world · 9 pts · 2y
it works on apple arm architecture surprisingly well; near instantly typically
Empricorn@feddit.nl · 7 pts · 2y
Is running using a GPU a bad thing? I'm new to this...
Fisch@lemmy.ml · 13 pts · 2y
No, a GPU would be ideal but not everyone has one, especially one with enough VRAM. I have an AMD card with 12gb of VRAM and I can run 7B - 13B models but even the 7B models (which seems to be the lowest that is still good) use a little more than 8gb of VRAM and most people probably have an Nvidia card with 8gb or less. 13B models get very close to using the full 12gb.
DoYouNot@lemmy.world · 10 pts · 2y
Not everyone has a dedicated GPU, I would guess. GPUs are good at doing tensor calculations, but they're not the only way.
Sabata11792@kbin.social · 2 pts · 2y
Its better if you have a good GPU, but will not run without a card from the last few years. It can run on CPU but it's much slower.
PlutoniumAcid@lemmy.world · 7 pts · 2y
There's a long listing of models to choose from. How to pick one? What are differences, benefits/drawbacks?
desmosthenes@lemmy.world · 1 pts · 2y
it’s gonna take experimentation; there’s a list of all models in the app or on the site and maybe a little googling. you can still use openai too. mistral is solid overall though; good for programming
gandalf_der_12te@feddit.de · 7 pts · 2y
Where did this come from? Did OpenAI finally release the source code? And where does the training data come from? Is the training data public domain/appropriately licensed?
desmosthenes@lemmy.world · 1 pts · 2y
that’s all on the site - openai offers an api for openai model access
DoucheBagMcSwag@lemmy.dbzer0.com · 0 pts · 2y
Uh....how are they going to pay for the server load?
sane@feddit.de · 9 pts · 2y
DoucheBagMcSwag@lemmy.dbzer0.com · 2 pts · 2y
Oh cool!
null@slrpnk.net · 4 pts · 2y
What server load?
hswolf@lemmy.world · 2 pts · 2y
DoucheBagMcSwag@lemmy.dbzer0.com · 1 pts · 2y
I just read its local