Ok, well, that sounds dire, but I don't think "they need to die for us to live" was ever said by anyone who cared much about who lives. I'm gonna keep working on my thing, good luck with your crusade or whatever it is.
That's kind of what I'm thinking. As for deposits, I think there would need to be an agreement that part of membership dues goes toward maintenance and schedulable repairs like replacing wear parts, as well as a pool for mitigating catastrophic damage to equipment. I'm not sure how I would determine "fault" for a piece that broke in someone's field, that seems like getting into insurance territory and I really don't want to model a coop off an insurance company. But I'm stumped as to what the alternative is.
The thing about dangerous equipment certification is, and this is something I grit my teeth at, that it really does need to have some kind of large institutional if not government oversight. I take safety... pretty seriously, and I don't trust me to make the call on whether or not someone can safely operate a machine. That's why I don't let other people operate my tractor. I don't really know how that works in larger ag communities, but in my experience it's just farmers letting farmers borrow tools off the books, and any damages are handled case-by-case. I know deaths and injuries happen because of this but I've never dealt with them firsthand, thankfully. I would love to have the means of producing accurate, helpful safety training material. Based on the farmers I've met, we could use it, though I don't know if most of us would. A lot of farmers take pride in their risk-taking, which I do sympathize with, though I think most farmers are pretty bad at some kinds of risk assessment and a lot of the living ones are coasting on luck.
I think we can stop at "Okay but we need to be able to do that". Capitalism, radical change, those are challenges for sure but the alternative is death. I assume, if I ever get anything off the ground, somebody with a megaphone is going to find a way to have a problem with it, either because they have their own sinister goals, or they are genuinely misguided, or perhaps they have a valid point and I really did miss something. In any case, I think the best approach is to assume what is necessary is possible and try to jump the hurdles as we get to them.
I think it would. I would love to have a coop tool library / repair shop / design shop. Something I want to get into is open-source ag hardware, and a co-op shop would be a good place to pilot designs without one individual having to sink hundreds of hours and thousands of dollars into making a piece of equipment that may or may not work. Even without designing new stuff though, a shared equipment library would be nice.
The downside is that sometimes, people borrow equipment without knowing how to use it, and if you use farming equipment without knowing how to use it, you can destroy thousands of dollars worth of gear and/or get people killed. This happens occasionally even with informal loans among acquaintences, I've had my own close shaves. Almost literally, I've had equipment break and send pieces of metal flying right next to my head. I don't know how I would go about sending out that kind of equipment confident that the users were equipped to use it safely.
That's pretty cool. Do you remember the name of the one in TN? That's where I am and I'd love to talk with them. I know a few growers' co-ops and farmers' markets, but nothing like that. Would be cool to get involved.
I agree, I think logistics is something that benefits from scale and it's a small part of what makes megafarms able to outcompete a more diverse field of small producers. And regardless of scale, it's just something that's hard to reason about and a lot of farmers are bad at it. If I can put together a model distribution cooperative that allows farmers to filter in, and up, and out of the industry as they age, and doesn't just entrench a new power structure that's just as bad as the one it meant to replace, I think we as a species will be a lot better able to weather what's coming.
That is a problem. I generally deal with small local buyers, but it's easier when they have relatively stable needs and expect to be able to buy more or less the same product at more or less the same price, year after year. I think that's going to be less viable as the effects of climate change continue to unfold. What worked ten years ago won't work today, and what works today may not work ten years from now.
Something I've been thinking about is how to have an internet farmers' market that doesn't suck. It's a logistical nightmare even before getting to things like food safety and disease tracking. Since cyclospora has been in the news I've been wondering how to have a better food distribution network that large-scale and small-scale buyers can get local produce from, and hopefully, with enough small-ish producers distributed over a wide area, the product offerings will be stable enough to attract enough buyers to cover the overhead of storage and distribution, quality and safety controls, etc.
But I do think there will have to be a cultural shift. There will still be staple crops, but I think which staples grow where is going to change and keep changing, as will which staples are the most economical for people in a given area. I think that's a mindset shift that will be hard to pull off as long as we keep treating this like a problem we can address tomorrow, because acknowledging that I need to change my eating habits is an acknowledgement that it's a big problem and it's here today. It'll happen eventually as staple prices get more and more unmoored from their historical patterns, but the sooner we can build a climate-change-acknowledging food infrastructure system, and get people on board with it, the blunter the impact will be.
This isn't a problem that's going to go away in the next few decades. If we stop belching CO2 into the atmosphere today (which we also need to do) we'll still have novel, highly erratic weather patterns over the next century or so that growers will need to be able to account for.
We need a better way of creating and distributing adaptable farming methods. No more of this taking out a six- or seven-figure loan for a piece of equipment designed to do the same job for the same crop for decades and hoping it eventually breaks even. Farmers need to be able to adapt their crops and tooling much more quickly and efficiently as we figure out how to feed ourselves and each other in climates that previous generations never anticipated. The better we can do that, the fewer people are going to starve.
I don't know, what was stopping us from solving world hunger and housing before LLMs? There's always some fresh new way for assholes to make life difficult for all of us, for the benefit of the wealthy few. I don't really see what LLMs have to do with that, other than being in vogue at the moment among the grifter class. It's like seeing a pyramid scheme and deciding that the solution is to ban pyramids. The focus should be on the actual malicious behavior, not whatever trinket they're using to disguise it nowadays.
Being kinda dumb never stopped any idea from being good. I don't know about this one though.
Combines are called that cause they combine a lot of the steps of harvesting into one machine. They cut the stalk and remove the grain from all the non-grain parts of the plant. That's a lot of moving parts that all require a decent amount of force, that would all have to be driven by the pedals. I don't know a ton about combines having never used one myself, but I think a lot of those processes benefit from being scaled up, that is, it's more energy efficient per plant to build a machine that can process 50 plants at a time than one that can only handle 5, so you wouldn't gain as much scaling down as you might expect. You could maybe build something that works, technically, but I imagine it would be less of a casual hang and more of a continuous, difficult workout for a team of professional cyclists.
If you wanted to chill and have a drink with friends, you could winnow your grain the old-fashioned(ish) way: you just need to dump your grain out in a stiff breeze. If you have a fan and a hopper, you could just sit around and chat while the hopper empties, and add another bin of grain every few minutes.
I think you're missing the trees for the forest. Yes, all those downsides make the current crop of LLM implementations pretty shitty for humanity, on balance, but none of them are intrinsic to LLMs, they are intrinsic to the current business practices that are attempting to profit from LLMs. LLMs are just math. Often computationally expensive math, sure, though they needn't be. But I don't think it helps anyone to say "LLMs are intrinsically bad and no one should use them" any more than it makes sense to say that blockchains are intrinsically bad and no one should use them (a common sentiment about a decade ago). I think a more realistic approach is to find out what real benefit can be had by using LLMs, and see if that benefit can be realized without all the downsides you mentioned.
To take the case at hand, I think open source code review is probably a pretty good candidate domain for exploration, because there's a large corpus of available text that could be obtained ethically, if LLM trainers bothered to put in the work. It seems like Torvalds isn't concerned with that, and it looks like sashiko only supports LLMs that are known to have been trained on stolen data, which is shitty and I think Torvalds and the Linux maintainers should be taken to task for that. I think LLM makers should get permission from project maintainers before training on their codebases, just as natural-language rightsholders should be asked for permission before using their works for training. I think LLMs trained on ethically sourced data should be permissively licensed and, if they collect revenue, should be expected to kick some of that revenue upstream to the projects they benefit from. I also think such projects could be powered ethically: I have some plans for my own small solar powered server that will run batch jobs when production is higher than my storage capacity. Right now I'm mostly targeting Folding at Home, but I might well run my own LLM trainer if I get enough capacity, and I could see a similar distributed processing network for high-cost jobs from trusted open source LLM projects.
I don't think you're wrong about the scope of the downsides, but I also don't think it makes sense to take them all as a singular block and judge all possible LLM tools by all possible downsides. I think they are problems that can be engineered around with technical and social guidelines for use. And I think it will be down to the open source community to set those guidelines, because I don't know of any other group that has the expertise and the motivation to do so.
I have a straw hat shaped kinda like a Stetson that I wear in the field for the same reason, but only there because I feel silly wearing it in public. I look like a cross between a fake cowpoke and Bob Belcher in his garden.
I think asbestos is a good analogy, but I think your claims that the inherent downsides of LLMs outweigh any possible upsides are as-yet unfounded. I also think it's kind of strange that you assume anyone who thinks otherwise is being disingenuous.
Maybe they are too dangerous for broad use, and we need to regulate them like asbestos, or uranium. Maybe they shouldn't be used outside of a laboratory setting or by anyone who doesn't have extensive training with how to interact with them safely. It seems pretty clear that Torvalds has decided they're worth the downside, and while I don't know if that's a good call, I don't think he's operating in bad faith to the detriment of the kernel project. That doesn't sound like something he would do.
I feel confident that I don't have the expertise to say for certain one way or the other, though the experience I do have with software tools makes me think there's probably an application for them where the downsides can be mitigated to the point where they become worthwhile. I think there's probably some single-purpose or tailored application of LLMs related to textual analysis that don't require the theft of the whole internet, and don't require insane amounts of energy to run. I don't think we have discovered them yet (at least I haven't), but saying "this kind of software is only bad and can have no ethical uses, ever" seems premature.
Yeah, you put your finger on maybe the biggest problem that I don't have a good answer to yet. The ability to interact with LLMs in natural language comes from an analysis of huge tracts of contemporary human writing, and there's not an ethical corpus that can compare in scope to the corpus of just snatching the whole internet. Maybe they don't need to be strictly "natural" language interactions? Maybe a sufficient ethical corpus could be compiled and maintained somehow? I don't know, this is kinda where I get out of my depth.
I think if we, as a community, really put our heads together, we could figure out how to define and make useful user-respecting tools that incorporate LLMs. There are a lot of hard problems, like the massive power consumption, and the ethical use of data, that I don't really know how to solve.
I think a start would be to focus on making smaller, lower-complexity models that are built for purpose, rather than trying to make a jillion-parameter jack-of-all-trades trades model. I think it would also make sense to focus initially on areas where there are already large corpuses of freely available text, like all the writings in the public domain. But I don't really have a good idea of what these tools would be used for, exactly, which is where I'm stuck.
He doesn't really specify use cases beyond Sashiko (I believe that's the project they're discussing), which, if I understand correctly, uses an LLM to review kernel patches using a pre-set list of prompts. The output of those prompts presumably gets added to the PR for anyone who cares to read it.
These outputs sometimes have mistakes in them, so the maintainers must read them with a critical eye, but I think Linus is arguing here that it's sufficiently... "good enough" that including them is helpful to the maintainers, that it catches enough real stuff without wasting too much time on fake stuff. There are always tradeoffs when adding a tool to the stack, and sufficiently rare misfires are probably worth the times it catches real problems that humans miss.
The solution is to make sure those LLM tools help maintainers
instead of just causing them pain. There's no question on that side.
We're not forcing anybody to use it, but I will very loudly ignore
people who try to argue against other people from using it.
And no, AI isn't perfect. But Christ, anybody who points to the
problems at AI had better be looking in the mirror and pointing at
themselves at the same time.
Because it's not like natural intelligence is always all that great either.
The kernel project has been and will continue to be about the technology.
Sure, the social angle of working on open source is important and
often a very motivating part of the project, but in the end that's a
side benefit, not the point of the project.
This is NOT some kind of "social warrior" project, never has been,
and never will be.
In the kernel community we do open source because it results in better
technology, not because of religious reasons.
And so we make decisions primarily based on technical merit. Not fear
of new tools.
IMO it's a pretty clear eyed take. LLMs are tools and, like all tools, are more fit for some purposes than others. And, like many tools, they're hard to understand, easy to misuse, and dangerous when they are misused (intentionally or otherwise). And, like any tool, they can be used for evil if the user has a mind to do so. Just because they are currently being used to do great harm doesn't mean they can't also be put to good use.
I think the FOSS community, in particular the Fediverse community, does itself a disservice by refusing to engage with them at all. They aren't evil in and of themselves, they're just new and selfish people will try to find an angle with them, same as it's always been. I think it's up to us to understand them and try to find a way to use them ethically. Show the world that it's not that we invented scary Artificial Intelligence, it's just that we created a cool, if wonky, search interface for text. Wonky in what way? Well, I think it'd be nice if there were robust FOSS tools and documentation I could use and/or contribute to, and learn about them with a community of my peers working for the common good, instead of having to either take the megacorps at their word or just renounce anything "AI branded" altogether, which doesn't feel like a good long term strategy.
I want to be dead clear about this, because it’s the entire reason Gorai exists: this is not a robotics framework that happens to tolerate AI. It’s a robotics framework designed because of AI. The bet is simple — the robots worth building from here on out are going to be driven, at least in part, by agents. So we designed for that on day one, not as a bolt-on you duct-tape onto the side later.
Seems like a shaky bet. I think I'd rather not have "agents" tightly coupled with my hardware controllers.
second thing, this
Safety enforced at the capability node, not trusted to the agent
Misses the point of why thinking about software as agents is dangerous, particularly in the context of robotics. Two or more agents may simultaneously decide to take actions that, on their own, aren't dangerous, but when performed at the same time can produce a dangerous situation that neither "agent" could foresee. the more agents, the more complicated bodies you have to account for in the system, and it quickly becomes unpredictable.
The software dev -> farmer pipeline is real.
Ok, well, that sounds dire, but I don't think "they need to die for us to live" was ever said by anyone who cared much about who lives. I'm gonna keep working on my thing, good luck with your crusade or whatever it is.
That's kind of what I'm thinking. As for deposits, I think there would need to be an agreement that part of membership dues goes toward maintenance and schedulable repairs like replacing wear parts, as well as a pool for mitigating catastrophic damage to equipment. I'm not sure how I would determine "fault" for a piece that broke in someone's field, that seems like getting into insurance territory and I really don't want to model a coop off an insurance company. But I'm stumped as to what the alternative is.
The thing about dangerous equipment certification is, and this is something I grit my teeth at, that it really does need to have some kind of large institutional if not government oversight. I take safety... pretty seriously, and I don't trust me to make the call on whether or not someone can safely operate a machine. That's why I don't let other people operate my tractor. I don't really know how that works in larger ag communities, but in my experience it's just farmers letting farmers borrow tools off the books, and any damages are handled case-by-case. I know deaths and injuries happen because of this but I've never dealt with them firsthand, thankfully. I would love to have the means of producing accurate, helpful safety training material. Based on the farmers I've met, we could use it, though I don't know if most of us would. A lot of farmers take pride in their risk-taking, which I do sympathize with, though I think most farmers are pretty bad at some kinds of risk assessment and a lot of the living ones are coasting on luck.
I think we can stop at "Okay but we need to be able to do that". Capitalism, radical change, those are challenges for sure but the alternative is death. I assume, if I ever get anything off the ground, somebody with a megaphone is going to find a way to have a problem with it, either because they have their own sinister goals, or they are genuinely misguided, or perhaps they have a valid point and I really did miss something. In any case, I think the best approach is to assume what is necessary is possible and try to jump the hurdles as we get to them.
Thanks! Looks like they are still going, I'll check 'em out.
I think it would. I would love to have a coop tool library / repair shop / design shop. Something I want to get into is open-source ag hardware, and a co-op shop would be a good place to pilot designs without one individual having to sink hundreds of hours and thousands of dollars into making a piece of equipment that may or may not work. Even without designing new stuff though, a shared equipment library would be nice.
The downside is that sometimes, people borrow equipment without knowing how to use it, and if you use farming equipment without knowing how to use it, you can destroy thousands of dollars worth of gear and/or get people killed. This happens occasionally even with informal loans among acquaintences, I've had my own close shaves. Almost literally, I've had equipment break and send pieces of metal flying right next to my head. I don't know how I would go about sending out that kind of equipment confident that the users were equipped to use it safely.
That's pretty cool. Do you remember the name of the one in TN? That's where I am and I'd love to talk with them. I know a few growers' co-ops and farmers' markets, but nothing like that. Would be cool to get involved.
I agree, I think logistics is something that benefits from scale and it's a small part of what makes megafarms able to outcompete a more diverse field of small producers. And regardless of scale, it's just something that's hard to reason about and a lot of farmers are bad at it. If I can put together a model distribution cooperative that allows farmers to filter in, and up, and out of the industry as they age, and doesn't just entrench a new power structure that's just as bad as the one it meant to replace, I think we as a species will be a lot better able to weather what's coming.
That is a problem. I generally deal with small local buyers, but it's easier when they have relatively stable needs and expect to be able to buy more or less the same product at more or less the same price, year after year. I think that's going to be less viable as the effects of climate change continue to unfold. What worked ten years ago won't work today, and what works today may not work ten years from now.
Something I've been thinking about is how to have an internet farmers' market that doesn't suck. It's a logistical nightmare even before getting to things like food safety and disease tracking. Since cyclospora has been in the news I've been wondering how to have a better food distribution network that large-scale and small-scale buyers can get local produce from, and hopefully, with enough small-ish producers distributed over a wide area, the product offerings will be stable enough to attract enough buyers to cover the overhead of storage and distribution, quality and safety controls, etc.
But I do think there will have to be a cultural shift. There will still be staple crops, but I think which staples grow where is going to change and keep changing, as will which staples are the most economical for people in a given area. I think that's a mindset shift that will be hard to pull off as long as we keep treating this like a problem we can address tomorrow, because acknowledging that I need to change my eating habits is an acknowledgement that it's a big problem and it's here today. It'll happen eventually as staple prices get more and more unmoored from their historical patterns, but the sooner we can build a climate-change-acknowledging food infrastructure system, and get people on board with it, the blunter the impact will be.
Oh, they're already trying. It's up to us to make sure they fail.
This isn't a problem that's going to go away in the next few decades. If we stop belching CO2 into the atmosphere today (which we also need to do) we'll still have novel, highly erratic weather patterns over the next century or so that growers will need to be able to account for.
We need a better way of creating and distributing adaptable farming methods. No more of this taking out a six- or seven-figure loan for a piece of equipment designed to do the same job for the same crop for decades and hoping it eventually breaks even. Farmers need to be able to adapt their crops and tooling much more quickly and efficiently as we figure out how to feed ourselves and each other in climates that previous generations never anticipated. The better we can do that, the fewer people are going to starve.
I don't know, what was stopping us from solving world hunger and housing before LLMs? There's always some fresh new way for assholes to make life difficult for all of us, for the benefit of the wealthy few. I don't really see what LLMs have to do with that, other than being in vogue at the moment among the grifter class. It's like seeing a pyramid scheme and deciding that the solution is to ban pyramids. The focus should be on the actual malicious behavior, not whatever trinket they're using to disguise it nowadays.
Being kinda dumb never stopped any idea from being good. I don't know about this one though.
Combines are called that cause they combine a lot of the steps of harvesting into one machine. They cut the stalk and remove the grain from all the non-grain parts of the plant. That's a lot of moving parts that all require a decent amount of force, that would all have to be driven by the pedals. I don't know a ton about combines having never used one myself, but I think a lot of those processes benefit from being scaled up, that is, it's more energy efficient per plant to build a machine that can process 50 plants at a time than one that can only handle 5, so you wouldn't gain as much scaling down as you might expect. You could maybe build something that works, technically, but I imagine it would be less of a casual hang and more of a continuous, difficult workout for a team of professional cyclists.
If you wanted to chill and have a drink with friends, you could winnow your grain the old-fashioned(ish) way: you just need to dump your grain out in a stiff breeze. If you have a fan and a hopper, you could just sit around and chat while the hopper empties, and add another bin of grain every few minutes.
I think you're missing the trees for the forest. Yes, all those downsides make the current crop of LLM implementations pretty shitty for humanity, on balance, but none of them are intrinsic to LLMs, they are intrinsic to the current business practices that are attempting to profit from LLMs. LLMs are just math. Often computationally expensive math, sure, though they needn't be. But I don't think it helps anyone to say "LLMs are intrinsically bad and no one should use them" any more than it makes sense to say that blockchains are intrinsically bad and no one should use them (a common sentiment about a decade ago). I think a more realistic approach is to find out what real benefit can be had by using LLMs, and see if that benefit can be realized without all the downsides you mentioned.
To take the case at hand, I think open source code review is probably a pretty good candidate domain for exploration, because there's a large corpus of available text that could be obtained ethically, if LLM trainers bothered to put in the work. It seems like Torvalds isn't concerned with that, and it looks like sashiko only supports LLMs that are known to have been trained on stolen data, which is shitty and I think Torvalds and the Linux maintainers should be taken to task for that. I think LLM makers should get permission from project maintainers before training on their codebases, just as natural-language rightsholders should be asked for permission before using their works for training. I think LLMs trained on ethically sourced data should be permissively licensed and, if they collect revenue, should be expected to kick some of that revenue upstream to the projects they benefit from. I also think such projects could be powered ethically: I have some plans for my own small solar powered server that will run batch jobs when production is higher than my storage capacity. Right now I'm mostly targeting Folding at Home, but I might well run my own LLM trainer if I get enough capacity, and I could see a similar distributed processing network for high-cost jobs from trusted open source LLM projects.
I don't think you're wrong about the scope of the downsides, but I also don't think it makes sense to take them all as a singular block and judge all possible LLM tools by all possible downsides. I think they are problems that can be engineered around with technical and social guidelines for use. And I think it will be down to the open source community to set those guidelines, because I don't know of any other group that has the expertise and the motivation to do so.
I have a straw hat shaped kinda like a Stetson that I wear in the field for the same reason, but only there because I feel silly wearing it in public. I look like a cross between a fake cowpoke and Bob Belcher in his garden.
I think asbestos is a good analogy, but I think your claims that the inherent downsides of LLMs outweigh any possible upsides are as-yet unfounded. I also think it's kind of strange that you assume anyone who thinks otherwise is being disingenuous.
Maybe they are too dangerous for broad use, and we need to regulate them like asbestos, or uranium. Maybe they shouldn't be used outside of a laboratory setting or by anyone who doesn't have extensive training with how to interact with them safely. It seems pretty clear that Torvalds has decided they're worth the downside, and while I don't know if that's a good call, I don't think he's operating in bad faith to the detriment of the kernel project. That doesn't sound like something he would do.
I feel confident that I don't have the expertise to say for certain one way or the other, though the experience I do have with software tools makes me think there's probably an application for them where the downsides can be mitigated to the point where they become worthwhile. I think there's probably some single-purpose or tailored application of LLMs related to textual analysis that don't require the theft of the whole internet, and don't require insane amounts of energy to run. I don't think we have discovered them yet (at least I haven't), but saying "this kind of software is only bad and can have no ethical uses, ever" seems premature.
Yeah, you put your finger on maybe the biggest problem that I don't have a good answer to yet. The ability to interact with LLMs in natural language comes from an analysis of huge tracts of contemporary human writing, and there's not an ethical corpus that can compare in scope to the corpus of just snatching the whole internet. Maybe they don't need to be strictly "natural" language interactions? Maybe a sufficient ethical corpus could be compiled and maintained somehow? I don't know, this is kinda where I get out of my depth.
I think if we, as a community, really put our heads together, we could figure out how to define and make useful user-respecting tools that incorporate LLMs. There are a lot of hard problems, like the massive power consumption, and the ethical use of data, that I don't really know how to solve.
I think a start would be to focus on making smaller, lower-complexity models that are built for purpose, rather than trying to make a jillion-parameter jack-of-all-trades trades model. I think it would also make sense to focus initially on areas where there are already large corpuses of freely available text, like all the writings in the public domain. But I don't really have a good idea of what these tools would be used for, exactly, which is where I'm stuck.
He doesn't really specify use cases beyond Sashiko (I believe that's the project they're discussing), which, if I understand correctly, uses an LLM to review kernel patches using a pre-set list of prompts. The output of those prompts presumably gets added to the PR for anyone who cares to read it.
These outputs sometimes have mistakes in them, so the maintainers must read them with a critical eye, but I think Linus is arguing here that it's sufficiently... "good enough" that including them is helpful to the maintainers, that it catches enough real stuff without wasting too much time on fake stuff. There are always tradeoffs when adding a tool to the stack, and sufficiently rare misfires are probably worth the times it catches real problems that humans miss.
IMO it's a pretty clear eyed take. LLMs are tools and, like all tools, are more fit for some purposes than others. And, like many tools, they're hard to understand, easy to misuse, and dangerous when they are misused (intentionally or otherwise). And, like any tool, they can be used for evil if the user has a mind to do so. Just because they are currently being used to do great harm doesn't mean they can't also be put to good use.
I think the FOSS community, in particular the Fediverse community, does itself a disservice by refusing to engage with them at all. They aren't evil in and of themselves, they're just new and selfish people will try to find an angle with them, same as it's always been. I think it's up to us to understand them and try to find a way to use them ethically. Show the world that it's not that we invented scary Artificial Intelligence, it's just that we created a cool, if wonky, search interface for text. Wonky in what way? Well, I think it'd be nice if there were robust FOSS tools and documentation I could use and/or contribute to, and learn about them with a community of my peers working for the common good, instead of having to either take the megacorps at their word or just renounce anything "AI branded" altogether, which doesn't feel like a good long term strategy.
Seems like a shaky bet. I think I'd rather not have "agents" tightly coupled with my hardware controllers.
second thing, this
Misses the point of why thinking about software as agents is dangerous, particularly in the context of robotics. Two or more agents may simultaneously decide to take actions that, on their own, aren't dangerous, but when performed at the same time can produce a dangerous situation that neither "agent" could foresee. the more agents, the more complicated bodies you have to account for in the system, and it quickly becomes unpredictable.