That's a pretty bad advice for a guy that mentioned he wants a "reputable inbox". Don't host your own. You'll be sending emails and wondering why they're not received because your IP was blacklisted / marked as spam.
Your mind is your body + brain. Your behavior and all of your memories are due to how your synapses and intermediary cells were developed. Similarly, it also depends on the body chemistry.
If you were to fully transplant a brain to a new body and correctly link all the nerves, in theory you could have the same brain in a different body. In practice, this can fail miserably due to differences in all of your sensory connections to the brain and hormone levels.
It's been a while since I looked into neuroscience, so there might be some things I am missing. The conclusion is that altering the chemistry around your body could change you as a person, even if your hypothalamus is the same (i.e., same memories).
I was thinking about getting one, but then I read more about it and realised you can't really use it without the Steam client. Kind of put me off since I have a large part of my library on GOG. Maybe this won't be a problem in the future though.
Dubious open-source practices on the part of Ollama devs. Other than that, LM Studio is using the latest stable llama.cpp rather than the one developed by Ollama, which brings significant speed improvements. You also have a better understanding of what model you're deploying by not using Ollama, and instead looking into the HF repository. For example, Ollama states that they're serving DeepSeek-R1, but pulling this one gives you a distilled 8B billion version that is not actually the DeepSeek-R1 (671B parameters) that one would have expected.
I get it that it might make it easier to use, but you will not learn much by using it. Even worse, competition is even better with performance and similar out-of-the-box capabilities.
Don't use Ollama. If you're on Windows, better try LM Studio and look into HuggingFace models rather than relying on the Ollama repository. Much better.
Dogs were not originally seen as pets in these countries, but rather as work animals. If you were to visit rural areas in an Eastern European country 5-10 years ago, you'd find that people that owned a dog had them chained up next to their fences, used them for protecting their livestock, or used them for hunting. If one were to escape, the village would have an extra litter of puppies that would most likely be dumped in the middle of a field or another village.
I'm not arguing for horror stories like puppy mills, which I am completely against and agree with you in those kinds of cases. But countries like Romania, Bulgaria, Greece, etc. are a bit different. Most of these dogs come from rural areas, which managed to live off of garbage until now.
The alternative to the shelter is euthanasia, but I am honestly glad if a responsible person decides to adopt the animal instead. What you should really be against is people paying to adopt a specific breed from a breeder, not those that free up space in actual shelters.
This is not it, chief. Eastern Europe has lots of problems with street animals, and it's not because people get pets. Most of the time, the animals in these shelters are just "wild" street dogs, which end up forming packs and may attack people.
There's campaigns to neuter dogs and cats to limit the population, but you'll find out that, very frequently, funds will be siphoned off to the pockets of local politicians. Also, some people won't neuter their animals because they believe they won't be useful anymore for defending their yards or catch mice.
And what's the outcome? Dogs and cats end up on the streets, reproduce, and end up increasing the population further, thus propagating the problems.
Western European countries don't struggle as much with this, and even import pets from other countries. The conditions are also much better for animals in these countries. The problem in countries like Romania is corruption and a lack of empathy towards animals, not people owning pets.
Get llama.cpp and try Qwen3.6-35B-A3B. Just came out and looks good. You'll have to look into optimal settings, as it's a Mixture of Experts (MoE) model with only 3B parameters active. That means that the rest can stay in RAM for quick inference.
You could also try the dense model (Qwen3.5-27B), but that will be significantly slower. Put these in a coding harness like Oh-My-Pi, OpenCode, etc. and see how it fares for your tasks. Should be ok for small tasks, but don't expect Opus / Sonnet 4.6 quality, more like better than Haiku.
Although I don't have much of a sway in these votes, I'd still say no. If they're the the only ones interacting on that instance, what stops them from making another instance in a month or two? Do we trigger another vote then to block that one as well? It could be a fun activity to have the monthly "Defederate from Grail's instance", but this doesn't really make sense. I vote to leave the clean-up to mods of instances where this person creates problems. If they don't give up, all they need is yet another account/instance to troll communities here.
There must be something that ensures the response is legitimate. Otherwise, if it's client-side and fully offline, I can just spoof the app to return the response "Yes, over 18". If it's not the government doing the verification, it's Google or Apple, which will give them access to all the "adult" websites you visit. Also, another reason for the EU to push for strict device attestation, without any DIY stuff (i.e., no more GrapheneOS, LineageOS, etc).
I couldn't find a desktop app on the EU's GitHub (another red flag, btw, using GitHub for this). All that seems to be available is code for the Android or iOS apps. Could you share it, if you can?
Even with the Zero Knowledge approach, you will still run an app on a phone (what if I don't have one) that will make some call to the government's servers, which will most likely know what website you're trying to access. We're moving the data mining from some third party to the government, which can be wrongly used later if some idiot comes into power. If it's not making a call to a government's servers, I would be surprised, since you could imagine someone just bypassing this to always return "Over 18".
Even funnier (read "sad"), this initiative will probably rely on Google and Apple to keep it robust, and will likely have no availability on rooted phones or non-Google Play Services ones. It's premature at best to deploy this in a meaningfully safe way.
I agree with you. Crowd-sourcing this type of research would be a completely different goal than what the AI Horde was built for, and would probably not be sustainable with part-time / volunteer researchers. Perhaps it's best for us to just wait until others make more substantial progress.
The goal would still have been inference for the Horde, but with sharing of feedback based on the model's outputs, to align it more with the original one. However, after considering this approach more, I am afraid that the maths behind it makes it impossible to "reconstruct" the original model's manifold, or at least capture the same behaviour in all use cases.
I came here to propose this idea because, to the best of my knowledge, this is only LLM community that actually pushes for sharing of resources. However, I have seen a few days ago a post on the LocalLlama community advocating for sharing of OpenCode sessions in order to crowd-source a fine-tuning dataset, so it seems that more people are having the same thoughts! :)
I will keep an eye out on other advancements, and if I actually end up having some time, perhaps I'll return with some contributions. I agree with you that such a project mostly relies on inference, in which case the AI Horde is not the only one that can provide that capability. What we would need is deploying such a model on HuggingFace, and creating an API endpoint for sharing training data for people that are interested in contributing.
Thanks a lot for offering your thoughts, and taking the time to write such lengthy responses to me! I hope you have a nice weekend!
I hope things get better for you! I would have recommended you get out of your country before they cancel your passport or something, but I'm not sure if it gets any better in other places. Even Europe seems to be speed-running fascism, and it's probably a matter of time before we follow in the US' steps...
Stay strong, and if things get really hairy, consider living off-grid. Gather together with other people that go through this, and make an escape plan. I believe it's becoming more and more sustainable nowadays with the advancements of solar panel technology to live outside civilization well.
Indeed, the quantization described in the Microsoft paper (and even in this NanoQuant paper) severely messes up the behaviour of the model. Even in this newer paper, you'd still incur ~2x performance loss (which is better than what was reported by 1.68 bit paper, if true), in terms of perplexity. However, as per the other paper I have added in the edited post, it is possible to further align a quantized model with the original one. In the end, LLMs are just fancy math that seek to maximize human preferences, and most of the bigger models were just better trained at doing that. With this approach, all we would have to do is just further refine the LoRA weights until we can match the behaviour of the unquantized model, which wouldn't be that expensive if all we have to do is fine-tune a few million parameters. It might be that at the beginning we're seeing worse performance compared to a 3B parameter model, but with more refinement we can further unlock some of the original performance.
Regarding the use of the Horde, I believe that behaviour alignment can't be done without actually using it. Just like corpo-AI are giving away their models so that they can further get data, we could have a similar, but much more compute-efficient, community-driven approach. Models by the people, for the people, if you will. Furthermore, as I mentioned, I think this would be the only community that has the compute and desire to push improvements on such an idea long-term, as it isn't profit-driven.
Let's say that this whole experiment starts with an extreme case, the MiniMax M2.5 model, and we abstract away from any architectural fancy stuff. At ~230B parameters, we would have a 1-bit model size of ~28.75 GB, and, as per Table 2 of NanoQuant, ~23 GB if we were to prune 20% of the weights. This would be enough to fully fit it on a 24GB VRAM GPU. Following this, we could get a well-balanced list (i.e., easy, medium, hard) of reasoning tasks, and fine-tune the LoRA layer to match the output. Heck, we could even tailor this to specific tasks, such as role-playing, coding, etc. It will be a long-term experiment where we might serve two answers (depending on Horde availability), one generated by the quantized model + LoRA and another that is regularly deployed. The user could then choose the model they prefer, and use that information later for further training.
This would indeed be quite cumbersome to set up, and could very well be wasted time. Users might even opt out from this because it could take too much time to help. But hey, I still think it would be a cool experiment to see if consumers could actually use these larger models on regular hardware, and get close to the original performance without paying for all the compute that is needed.
The models themselves would indeed be costly to train if you were to go for the regular approach. You would have to "upscale" the weights to be fp32 from binary, which would make the models only trainable on the usual amount of GPUs. That is because the training process relies on back-propagation, which only makes sense if your operations are differentiable. Since addition is not differentiable, your binary weights would only be updated by 0, so no change.
However, LoRA (16-bit) QLoRA (4/8-bit) fine-tuning can be done on a single GPU, assuming you can fit the model on it. Everything is frozen, except for a separate small network, which is updated during training. This can have BF16 or F32 precision, and would be trained as you would a regular network.
What I am suggesting is to actually leverage bigger models that come out, and attempt to compress them using the proposed algorithm (if it actually scales to bigger models). From there, we could employ some tricks to improve performance, think latent reasoning, community-driven RLHF only on the (Q)LoRA layers, etc. With time, we would be able to pool together a dataset and a pipeline that can be applied to any open-weight model that is released.
But it does sound a bit easier than it would be in practice. This heavily relies on re-purposing the Horde to also store training data (with user consent, of course), user scores, and later introduce a training queue.
Some chap invested a lot of time into making the Skyrim experience nicer. I recommend you check out CHIM :)
Quite a lovely project, but you will have to spend some time to set things up. For example, if you have a good GPU available, you can set up TTS for NPCs, STT for yourself, and then a decent LLM to handle the world interactions. The NPCs then can listen to you talk, follow you, do stuff you tell them (like attack someone, or pick something off the floor), etc. It's something quite revolutionary, if you can spend the time to get it to work. If you're looking for some LLM provider on the cheap, nano-gpt has an 8 dollar per month tier that gives you "fair-use unlimited" access to open source models. Worth a shot!
Note: You won't be able to run all the models and the game on the same computer. The CHIM wiki has some suggestions on the amount of compute needed, and alternatives for the services so that you don't have to run everything locally.
All great on paper, but why would EU leaders fight against Trump? They're all fragmented, trying to hold on to their countries' benefit. Given the most recent decision to cripple the 2035 ban on ICE cars due to pressure from Germany and Italy, I really doubt the leaders here are capable of punishing Trump.
Even if von der Leyen pushed for this (which I doubt, see her behaviour in the first USA EU Deal), the EU would have to act unanimously to do it, and we already know that it is not possible. Given recent polls, it might be possible that Hungary will keep Orban as PM, and other countries will soon vote in their own MAGA-like leaders. See Romania, Bulgaria, and quite a few others in Eastern Europe where corruption is rampant. They'd sell their own mother if it got them a second villa.
I wish to be as hopeful as the author, but the EU leaders keep proving the opposite. In order to survive, the EU should have focused on separating from the US during the Bush era, or even as late as Trump's first mandate. As it is, the EU will probably just wait for Trump's term to be over, and then return to business as usual. I am afraid we won't be seeing any big disturbances to the current world order for a while, unless the AI bubble bursts (which will probably be due to other factors, not EU intervention).
I have a friend who set up a Dreame L10s Ultra. I helped them solder the breakout board, and was there when they flashed the new firmware. Relatively straight forward! Just follow the guide on the website and you should be good.
The robot is now accessible only on the local network, and they got it working in Home Assistant. The only feature that is missing now is direct camera view, which the original robot had. Basically, you could get a live feed of the robot's camers at any time. Looked fun, but it was not necessary.
That's a pretty bad advice for a guy that mentioned he wants a "reputable inbox". Don't host your own. You'll be sending emails and wondering why they're not received because your IP was blacklisted / marked as spam.
Your mind is your body + brain. Your behavior and all of your memories are due to how your synapses and intermediary cells were developed. Similarly, it also depends on the body chemistry.
If you were to fully transplant a brain to a new body and correctly link all the nerves, in theory you could have the same brain in a different body. In practice, this can fail miserably due to differences in all of your sensory connections to the brain and hormone levels.
It's been a while since I looked into neuroscience, so there might be some things I am missing. The conclusion is that altering the chemistry around your body could change you as a person, even if your hypothalamus is the same (i.e., same memories).
I was thinking about getting one, but then I read more about it and realised you can't really use it without the Steam client. Kind of put me off since I have a large part of my library on GOG. Maybe this won't be a problem in the future though.
Dubious open-source practices on the part of Ollama devs. Other than that, LM Studio is using the latest stable llama.cpp rather than the one developed by Ollama, which brings significant speed improvements. You also have a better understanding of what model you're deploying by not using Ollama, and instead looking into the HF repository. For example, Ollama states that they're serving DeepSeek-R1, but pulling this one gives you a distilled 8B billion version that is not actually the DeepSeek-R1 (671B parameters) that one would have expected.
I get it that it might make it easier to use, but you will not learn much by using it. Even worse, competition is even better with performance and similar out-of-the-box capabilities.
Don't use Ollama. If you're on Windows, better try LM Studio and look into HuggingFace models rather than relying on the Ollama repository. Much better.
Dogs were not originally seen as pets in these countries, but rather as work animals. If you were to visit rural areas in an Eastern European country 5-10 years ago, you'd find that people that owned a dog had them chained up next to their fences, used them for protecting their livestock, or used them for hunting. If one were to escape, the village would have an extra litter of puppies that would most likely be dumped in the middle of a field or another village.
I'm not arguing for horror stories like puppy mills, which I am completely against and agree with you in those kinds of cases. But countries like Romania, Bulgaria, Greece, etc. are a bit different. Most of these dogs come from rural areas, which managed to live off of garbage until now.
The alternative to the shelter is euthanasia, but I am honestly glad if a responsible person decides to adopt the animal instead. What you should really be against is people paying to adopt a specific breed from a breeder, not those that free up space in actual shelters.
This is not it, chief. Eastern Europe has lots of problems with street animals, and it's not because people get pets. Most of the time, the animals in these shelters are just "wild" street dogs, which end up forming packs and may attack people.
There's campaigns to neuter dogs and cats to limit the population, but you'll find out that, very frequently, funds will be siphoned off to the pockets of local politicians. Also, some people won't neuter their animals because they believe they won't be useful anymore for defending their yards or catch mice.
And what's the outcome? Dogs and cats end up on the streets, reproduce, and end up increasing the population further, thus propagating the problems.
Western European countries don't struggle as much with this, and even import pets from other countries. The conditions are also much better for animals in these countries. The problem in countries like Romania is corruption and a lack of empathy towards animals, not people owning pets.
Get llama.cpp and try Qwen3.6-35B-A3B. Just came out and looks good. You'll have to look into optimal settings, as it's a Mixture of Experts (MoE) model with only 3B parameters active. That means that the rest can stay in RAM for quick inference.
You could also try the dense model (Qwen3.5-27B), but that will be significantly slower. Put these in a coding harness like Oh-My-Pi, OpenCode, etc. and see how it fares for your tasks. Should be ok for small tasks, but don't expect Opus / Sonnet 4.6 quality, more like better than Haiku.
Although I don't have much of a sway in these votes, I'd still say no. If they're the the only ones interacting on that instance, what stops them from making another instance in a month or two? Do we trigger another vote then to block that one as well? It could be a fun activity to have the monthly "Defederate from Grail's instance", but this doesn't really make sense. I vote to leave the clean-up to mods of instances where this person creates problems. If they don't give up, all they need is yet another account/instance to troll communities here.
There must be something that ensures the response is legitimate. Otherwise, if it's client-side and fully offline, I can just spoof the app to return the response "Yes, over 18". If it's not the government doing the verification, it's Google or Apple, which will give them access to all the "adult" websites you visit. Also, another reason for the EU to push for strict device attestation, without any DIY stuff (i.e., no more GrapheneOS, LineageOS, etc).
I couldn't find a desktop app on the EU's GitHub (another red flag, btw, using GitHub for this). All that seems to be available is code for the Android or iOS apps. Could you share it, if you can?
Even with the Zero Knowledge approach, you will still run an app on a phone (what if I don't have one) that will make some call to the government's servers, which will most likely know what website you're trying to access. We're moving the data mining from some third party to the government, which can be wrongly used later if some idiot comes into power. If it's not making a call to a government's servers, I would be surprised, since you could imagine someone just bypassing this to always return "Over 18".
Even funnier (read "sad"), this initiative will probably rely on Google and Apple to keep it robust, and will likely have no availability on rooted phones or non-Google Play Services ones. It's premature at best to deploy this in a meaningfully safe way.
Oh, I remember this! Thanks for sharing, very nice find! Could be a worthwhile approach once we have the data :)
Apologies for the late reply! Busy days :D
I agree with you. Crowd-sourcing this type of research would be a completely different goal than what the AI Horde was built for, and would probably not be sustainable with part-time / volunteer researchers. Perhaps it's best for us to just wait until others make more substantial progress.
The goal would still have been inference for the Horde, but with sharing of feedback based on the model's outputs, to align it more with the original one. However, after considering this approach more, I am afraid that the maths behind it makes it impossible to "reconstruct" the original model's manifold, or at least capture the same behaviour in all use cases.
I came here to propose this idea because, to the best of my knowledge, this is only LLM community that actually pushes for sharing of resources. However, I have seen a few days ago a post on the LocalLlama community advocating for sharing of OpenCode sessions in order to crowd-source a fine-tuning dataset, so it seems that more people are having the same thoughts! :)
I will keep an eye out on other advancements, and if I actually end up having some time, perhaps I'll return with some contributions. I agree with you that such a project mostly relies on inference, in which case the AI Horde is not the only one that can provide that capability. What we would need is deploying such a model on HuggingFace, and creating an API endpoint for sharing training data for people that are interested in contributing.
Thanks a lot for offering your thoughts, and taking the time to write such lengthy responses to me! I hope you have a nice weekend!
I hope things get better for you! I would have recommended you get out of your country before they cancel your passport or something, but I'm not sure if it gets any better in other places. Even Europe seems to be speed-running fascism, and it's probably a matter of time before we follow in the US' steps...
Stay strong, and if things get really hairy, consider living off-grid. Gather together with other people that go through this, and make an escape plan. I believe it's becoming more and more sustainable nowadays with the advancements of solar panel technology to live outside civilization well.
Indeed, the quantization described in the Microsoft paper (and even in this NanoQuant paper) severely messes up the behaviour of the model. Even in this newer paper, you'd still incur ~2x performance loss (which is better than what was reported by 1.68 bit paper, if true), in terms of perplexity. However, as per the other paper I have added in the edited post, it is possible to further align a quantized model with the original one. In the end, LLMs are just fancy math that seek to maximize human preferences, and most of the bigger models were just better trained at doing that. With this approach, all we would have to do is just further refine the LoRA weights until we can match the behaviour of the unquantized model, which wouldn't be that expensive if all we have to do is fine-tune a few million parameters. It might be that at the beginning we're seeing worse performance compared to a 3B parameter model, but with more refinement we can further unlock some of the original performance.
Regarding the use of the Horde, I believe that behaviour alignment can't be done without actually using it. Just like corpo-AI are giving away their models so that they can further get data, we could have a similar, but much more compute-efficient, community-driven approach. Models by the people, for the people, if you will. Furthermore, as I mentioned, I think this would be the only community that has the compute and desire to push improvements on such an idea long-term, as it isn't profit-driven.
Let's say that this whole experiment starts with an extreme case, the MiniMax M2.5 model, and we abstract away from any architectural fancy stuff. At ~230B parameters, we would have a 1-bit model size of ~28.75 GB, and, as per Table 2 of NanoQuant, ~23 GB if we were to prune 20% of the weights. This would be enough to fully fit it on a 24GB VRAM GPU. Following this, we could get a well-balanced list (i.e., easy, medium, hard) of reasoning tasks, and fine-tune the LoRA layer to match the output. Heck, we could even tailor this to specific tasks, such as role-playing, coding, etc. It will be a long-term experiment where we might serve two answers (depending on Horde availability), one generated by the quantized model + LoRA and another that is regularly deployed. The user could then choose the model they prefer, and use that information later for further training.
This would indeed be quite cumbersome to set up, and could very well be wasted time. Users might even opt out from this because it could take too much time to help. But hey, I still think it would be a cool experiment to see if consumers could actually use these larger models on regular hardware, and get close to the original performance without paying for all the compute that is needed.
The models themselves would indeed be costly to train if you were to go for the regular approach. You would have to "upscale" the weights to be fp32 from binary, which would make the models only trainable on the usual amount of GPUs. That is because the training process relies on back-propagation, which only makes sense if your operations are differentiable. Since addition is not differentiable, your binary weights would only be updated by 0, so no change.
However, LoRA (16-bit) QLoRA (4/8-bit) fine-tuning can be done on a single GPU, assuming you can fit the model on it. Everything is frozen, except for a separate small network, which is updated during training. This can have BF16 or F32 precision, and would be trained as you would a regular network.
What I am suggesting is to actually leverage bigger models that come out, and attempt to compress them using the proposed algorithm (if it actually scales to bigger models). From there, we could employ some tricks to improve performance, think latent reasoning, community-driven RLHF only on the (Q)LoRA layers, etc. With time, we would be able to pool together a dataset and a pipeline that can be applied to any open-weight model that is released.
But it does sound a bit easier than it would be in practice. This heavily relies on re-purposing the Horde to also store training data (with user consent, of course), user scores, and later introduce a training queue.
Some chap invested a lot of time into making the Skyrim experience nicer. I recommend you check out CHIM :)
Quite a lovely project, but you will have to spend some time to set things up. For example, if you have a good GPU available, you can set up TTS for NPCs, STT for yourself, and then a decent LLM to handle the world interactions. The NPCs then can listen to you talk, follow you, do stuff you tell them (like attack someone, or pick something off the floor), etc. It's something quite revolutionary, if you can spend the time to get it to work. If you're looking for some LLM provider on the cheap, nano-gpt has an 8 dollar per month tier that gives you "fair-use unlimited" access to open source models. Worth a shot!
Note: You won't be able to run all the models and the game on the same computer. The CHIM wiki has some suggestions on the amount of compute needed, and alternatives for the services so that you don't have to run everything locally.
Wero is being rolled out slowly in Western Europe. I believe it's already a thing in Germany, France, Belgium, and followed soon by the Netherlands.
All great on paper, but why would EU leaders fight against Trump? They're all fragmented, trying to hold on to their countries' benefit. Given the most recent decision to cripple the 2035 ban on ICE cars due to pressure from Germany and Italy, I really doubt the leaders here are capable of punishing Trump.
Even if von der Leyen pushed for this (which I doubt, see her behaviour in the first USA EU Deal), the EU would have to act unanimously to do it, and we already know that it is not possible. Given recent polls, it might be possible that Hungary will keep Orban as PM, and other countries will soon vote in their own MAGA-like leaders. See Romania, Bulgaria, and quite a few others in Eastern Europe where corruption is rampant. They'd sell their own mother if it got them a second villa.
I wish to be as hopeful as the author, but the EU leaders keep proving the opposite. In order to survive, the EU should have focused on separating from the US during the Bush era, or even as late as Trump's first mandate. As it is, the EU will probably just wait for Trump's term to be over, and then return to business as usual. I am afraid we won't be seeing any big disturbances to the current world order for a while, unless the AI bubble bursts (which will probably be due to other factors, not EU intervention).
I have a friend who set up a Dreame L10s Ultra. I helped them solder the breakout board, and was there when they flashed the new firmware. Relatively straight forward! Just follow the guide on the website and you should be good.
The robot is now accessible only on the local network, and they got it working in Home Assistant. The only feature that is missing now is direct camera view, which the original robot had. Basically, you could get a live feed of the robot's camers at any time. Looked fun, but it was not necessary.