Note from OP: I posted this because I find a topic of "agent swarms" super interesting as seen as an engineering challenge. It's just a novel kind of problem yet to be solved. And this article describes what worked well and what not, what gains did they get making certain changes.
But being honest I see some gaps in their reasoning. SQLite is an open source project which most probably means its code already is part of a training set of frontier LLMs. This calls into question whether language models actually invented or just recalled. Also, as noted in the article, they instructed models to build something based on the docs. That means that the goal was to recreate, not to invent. One of the hardest things in software engineering is to figure what to build and this agent swarm has already been given that information, what made the job much easier.
If the benchmark is solved, then why official arc prize page on X https://xcancel.com/arcprize does not mention it? And instead they post results of models that are not even close to 100%?
How is China’s Long March-10B landing different from SpaceX’s Falcon 9? The Falcon 9 lands upright on four deployable legs on a pad or droneship. The Long March-10B has no legs; it uses four hooks to catch a net mounted on a ship, then a cable system absorbs the impact.
LLMs are just another tool. Just too much hyped. If someone is saying that LLMs won't be relevant in a few years is an ignorant. App usage statistics on many platforms show clearly that people got used to these tools.
Being used to modern journalism I was not expecting even a link to schematics, but instead I was given list of all required components and even re-upload of a video showing how to assemble it. What a treasure this website is
For now, finding collections is very manual, primarily through word of mouth. In the future, we are planning to introduce some ways to browse and discover popular collections.
ok, so any recommendations of interesting collections?
Why not?
Why Microsoft develops it's .NET ecosystem? Why Google develops Go/Dart? It costs them lots of money and they give it for free.
The answer is: they don't earn money on it directly, but these tools are a way to tie programmers to their cloud services. If you use .NET you'll probably end up on Azure. If Go - probably you'll use GCP.
So I suspect the same will be with LLMs. At some point they will say: "hey, you can use this LLM however you want, but as you are already using it, then you may want to know our platform is optimized for it"
So are we assuming here that LLMs won't become more efficient over time? GPT-3 has been a frontier model just a few years ago and it's performance blew everyone's mind at that time. I can now run equivalent LLM on my personal computer. Why can't we expect that after a few years Claude Sonnet level of capability won't be possible to accomplish locally?
I wonder why exactly do they want to increase AI compute that fast.
I would understand if their current infrastructure weren't able to handle traffic of core services like Facebook. Then it would be justified because it would mean worse user experience and less profits in effect.
But AI? Have I missed something? As far as I know everything Meta have shown so far were just toys. So why are they in hurry?
Defenders finally have a chance to win, decisively
I'm curious how it will turn out to be in a long term. Are we going to have safer software? Because not only defenders will have a powerful tool, but attackers too. But at the same time, number of bugs is finite... Can we in theory one day achieve literally zero bugs in codebase?
your post is not related to the topic of this community
Note from OP: I posted this because I find a topic of "agent swarms" super interesting as seen as an engineering challenge. It's just a novel kind of problem yet to be solved. And this article describes what worked well and what not, what gains did they get making certain changes.
But being honest I see some gaps in their reasoning. SQLite is an open source project which most probably means its code already is part of a training set of frontier LLMs. This calls into question whether language models actually invented or just recalled. Also, as noted in the article, they instructed models to build something based on the docs. That means that the goal was to recreate, not to invent. One of the hardest things in software engineering is to figure what to build and this agent swarm has already been given that information, what made the job much easier.
If the benchmark is solved, then why official arc prize page on X https://xcancel.com/arcprize does not mention it? And instead they post results of models that are not even close to 100%?
Any chance most of this traffic is just bots hosted on Linux?
lol, so it came to the point when developers need to explain what tools are they using 😆 I don't care how it's done as long as it works as intended
https://www.naijatechguide.com/net-vs-legs-how-chinas-long-march-10b-rocket-landing-is-different-from-spacex-falcon-9.html
LLMs are just another tool. Just too much hyped. If someone is saying that LLMs won't be relevant in a few years is an ignorant. App usage statistics on many platforms show clearly that people got used to these tools.
Being used to modern journalism I was not expecting even a link to schematics, but instead I was given list of all required components and even re-upload of a video showing how to assemble it. What a treasure this website is
ok, so any recommendations of interesting collections?
Why not? Why Microsoft develops it's .NET ecosystem? Why Google develops Go/Dart? It costs them lots of money and they give it for free.
The answer is: they don't earn money on it directly, but these tools are a way to tie programmers to their cloud services. If you use .NET you'll probably end up on Azure. If Go - probably you'll use GCP.
So I suspect the same will be with LLMs. At some point they will say: "hey, you can use this LLM however you want, but as you are already using it, then you may want to know our platform is optimized for it"
So are we assuming here that LLMs won't become more efficient over time? GPT-3 has been a frontier model just a few years ago and it's performance blew everyone's mind at that time. I can now run equivalent LLM on my personal computer. Why can't we expect that after a few years Claude Sonnet level of capability won't be possible to accomplish locally?
According to this chart, it might be hard to find any better deal than Minimax M3. It's just absurdly cheap
https://openrouter.ai/rankings?benchmark=agentic#benchmarks
Hello, mod here, discussions are welcome
I wonder why exactly do they want to increase AI compute that fast.
I would understand if their current infrastructure weren't able to handle traffic of core services like Facebook. Then it would be justified because it would mean worse user experience and less profits in effect.
But AI? Have I missed something? As far as I know everything Meta have shown so far were just toys. So why are they in hurry?
Hopium administered
It amuses me that in a community named literally "ChatGPT" people are downvoting post about new GPT version
I'm curious how it will turn out to be in a long term. Are we going to have safer software? Because not only defenders will have a powerful tool, but attackers too. But at the same time, number of bugs is finite... Can we in theory one day achieve literally zero bugs in codebase?