Those smaller models are trained on the larger models. They take just as many resources to train and then you add on more training, so they’re literally worse than the big models.
And what if tech oligarchs weren’t evil and used this new technology to make lives better instead of concentrating capital? We don’t have that either so there’s no use speculating.
Snark aside, I think you’re underestimating just how much data it takes to make these models work. There is no such thing as enough “private” data for training.
There is no such thing as enough “private” data for training.
what the fuck are you on about? You can build local datasets and train additional LORAs locally to have more relevant outputs, and it doesn't go back to the creators of the original weights.
We opted for downloading as many books as possible from a number of pirates sites and stashing them on a usb drive away plan instead. Wife and son have been very busy.
Your only access to information will be through AI and what you get to access will be based on how poor you are. We can stop this trend, we can stop this future, you'd all have to actually do something though, is that beyond you? Doing a bit of a thing to save the future?
17 Comments
sundray@lemmus.org · 15 pts · 23d
I blame Egon.
Skullgrid@lemmy.world · 13 pts · 23d
that last one doesn't add up
reddfugee@lemmy.world · 25 pts · 23d
I think it's directed at the "I don't use commercial LLMs, I just run small local models" crowd
Skullgrid@lemmy.world · 7 pts · 23d
yes and what books are they destroying?
tyler@programming.dev · 16 pts · 23d
Those smaller models are trained on the larger models. They take just as many resources to train and then you add on more training, so they’re literally worse than the big models.
Skullgrid@lemmy.world · -2 pts · 23d
What if your power comes from renewables? And if you're training private data?
Voixe2@lemmy.blahaj.zone · 15 pts · 23d
What if a unicorn gave me the first ethical local model, now all my sins are absolved
ayyy@sh.itjust.works · 4 pts · 22d
And what if tech oligarchs weren’t evil and used this new technology to make lives better instead of concentrating capital? We don’t have that either so there’s no use speculating.
Snark aside, I think you’re underestimating just how much data it takes to make these models work. There is no such thing as enough “private” data for training.
Skullgrid@lemmy.world · 3 pts · 22d
what the fuck are you on about? You can build local datasets and train additional LORAs locally to have more relevant outputs, and it doesn't go back to the creators of the original weights.
ayyy@sh.itjust.works · 1 pts · 22d
You are describing fine tuning, no? Which still requires a base model that was trained on a massive amount of other data.
tyler@programming.dev · 3 pts · 22d
Then they still used just as many resources? All these local models are distillations of the big models.
wieson@feddit.org · 1 pts · 23d
The passages you had memorised.
Bebopalouie@lemmy.ca · 9 pts · 23d
We opted for downloading as many books as possible from a number of pirates sites and stashing them on a usb drive away plan instead. Wife and son have been very busy.
ILikeBoobies@lemmy.ca · 2 pts · 22d
https://www.kavitareader.com/
minorkeys@sh.itjust.works · 3 pts · 22d
Your only access to information will be through AI and what you get to access will be based on how poor you are. We can stop this trend, we can stop this future, you'd all have to actually do something though, is that beyond you? Doing a bit of a thing to save the future?
csolisr@hub.azkware.net · 1 pts · 16d
In related news, I genuinely have no idea of why did that account block me before even interacting with it