I just updated my setup from LMStudio to llama.cpp with the new QWEN 3.6 27B MTP model and I am getting 80-112 tokens/second, 90 average which is just shocking to me. I am on a 4090 with a context
Window of 64k. It hardly use cloud AI anymore as I rarely need more than 64k if I ensure my first prompt is written like a design document. Multiple prompts are not great so I often just figure out where my initial prompt went wrong, adjust and try again in a fresh session. Way faster this way too. It has really worked out well for me as I am getting just as much done locally for free as I was with hundreds of dollar a month on cloud AI. I am still shocked and grateful it flowed this way.
I am using llamma.cpp with QWEN 3.6 27B MTP, with a 64k context window on a 4090 that OpenCode talks to and then it in term talks to the Unity Game engine via MCP. Getting 80/112 tokens/second work 90 average which is shocking to me as it really does feel as fast as cloud AI (well faster for me as I am in Vietnam and round trips to US data centers really adds up in a session). The only really issue is you pretty much have to one shot prompts as follow up prompts will easily go over the context window size. If I cannot one shot prompts them use cloud AI both that is very rare for my use case. Maybe 1 in 50 or so and only when the tasks touches a lot of large scripts and scenes.
Agreed. I am not longer paying token fees as I am running QWEN 3.6 27B MTP on my 4090 GPU and it is as good and as fast as the frontier models for agentic coding.
The corporations certainly think they are setting themselves up for domination, but that is not how it is trending. The opposite is about to happen as all those over the top irrational exuberance driven data center investments are quickly becoming a liability as Open Source has caught up and Ternary 1.58 bit models are coming to make GPU redundant. Hate to be AI CEO or cloud ai investor right now.
I agree and would add for others reason here that the Luddites issue was not the looms they destroyed, but the out of control inequality that the government was not addressing. We need to stop blaming AI as a society for job loss and instead get the governments to help with the transition which so far they have largely been inactive on.
Wow. I had to go verify this and sure enough, I found a video from 7 years ago where Lucas said this and much more including comparing the American Empire to the Galactic Empire. Wow. I missed that reference as a child.
The panels typically offer both red and near infrared light. The biggest benefit for me which was a surprise benefit I had not even intended to address is my asthma. Specially the near infrared on my lungs has allowed me to stop all asthma medication that I took for over 45 years. It is super effective and there are research papers emerging that support.
Pathological Demand Avoidance is such a passive aggressive term. I find most in the spectrum happily follow a leader provided they are logical, compassionate, considerate and looking for win wins. If the leader is an egotistical boss who sees you as less worthy than them and is demanding you do work that makes no sense and is even perhaps harmful or exploitative, than yeah, I am going to avoidant and not compliant. Wish more were PDA as we would not end up with tyrant leaders.
It is not capitalism and this is not me defending it, but rather pointing out that it is how humanity naturally seems to structure itself. The narcissist / psychopaths / sociopaths tend to get into “leadership” positions and the dumb folks idolize and empower them and thus we end up with the smartest people writing a few paragraphs for the manager who is socially savvy but suffers from the Dunning Kruger Effect. This happens in all economic and political systems including Capitalism and Communism as the Sam people tend to get into positions of power.
Agree. The dumb people out of necessity need to be in groups and those groups are lead by narcissists/ psychopaths / sociopaths who then weaponize the groups ignorance to exploit them for their own gain while using shame and worse to keep the group inline. It leads to the world being led by the wrong people and while we have managed fairly well by most metrics, it is an absolute thorn in the side of humanity that carries existential risks. I sure hope we can change things so that the smartest, kindest people lead instead. Maybe as the power of the group is certainly being eroded in part due to AI allowing smart individuals to climb quietly higher into leadership roles that are not forceful but rather truly leading. We will see. I am hopeful change is coming or we end up destroying everything. Seems like it is on a knife’s edge and has been since we invented nukes.
I am a solo developer on some of these platforms and have to do an annual privacy evaluation which I find painful as the company taking up my valuable time to answer all their BS questions, are the ones breaking privacy. Insane. Further, I had to pay $100 annually to the UK privacy commission as it is required for all UK companies which is also deeply ironic as the same government is requiring you to submit ID for some websites which has made the privacy leaks even more damaging.
As a solo developer I am feeling the brunt of the privacy charades while the big companies just continue to violate privacy unchecked. I have zero desire or intentions to break anyone’s privacy an ask for no information from anyone outside of pulling down their profile to sign in as is required to determine app permission. That is it. No phone numbers, no real names, no biometrics, no photos of ID or anything outside of those username on that platform but I am burdened as if I was collecting all of this and as if I was the one breaking privacy. This world is mental.
Looks like Zuck is all in and has no choice but to support evil, because if/when the evil collapses, so does Zuck. Must be stressful for him as history has shown that this behaviour eventually catches up to them. Sadly, some make it too old age and run entire countries before it catches up which causes a lot of unnecessary suffering.
The irrational exuberance is a drug and Jensen is in part one of the drug dealers who caused it.
I just updated my setup from LMStudio to llama.cpp with the new QWEN 3.6 27B MTP model and I am getting 80-112 tokens/second, 90 average which is just shocking to me. I am on a 4090 with a context Window of 64k. It hardly use cloud AI anymore as I rarely need more than 64k if I ensure my first prompt is written like a design document. Multiple prompts are not great so I often just figure out where my initial prompt went wrong, adjust and try again in a fresh session. Way faster this way too. It has really worked out well for me as I am getting just as much done locally for free as I was with hundreds of dollar a month on cloud AI. I am still shocked and grateful it flowed this way.
I am using llamma.cpp with QWEN 3.6 27B MTP, with a 64k context window on a 4090 that OpenCode talks to and then it in term talks to the Unity Game engine via MCP. Getting 80/112 tokens/second work 90 average which is shocking to me as it really does feel as fast as cloud AI (well faster for me as I am in Vietnam and round trips to US data centers really adds up in a session). The only really issue is you pretty much have to one shot prompts as follow up prompts will easily go over the context window size. If I cannot one shot prompts them use cloud AI both that is very rare for my use case. Maybe 1 in 50 or so and only when the tasks touches a lot of large scripts and scenes.
Agreed. I am not longer paying token fees as I am running QWEN 3.6 27B MTP on my 4090 GPU and it is as good and as fast as the frontier models for agentic coding.
The corporations certainly think they are setting themselves up for domination, but that is not how it is trending. The opposite is about to happen as all those over the top irrational exuberance driven data center investments are quickly becoming a liability as Open Source has caught up and Ternary 1.58 bit models are coming to make GPU redundant. Hate to be AI CEO or cloud ai investor right now.
I agree and would add for others reason here that the Luddites issue was not the looms they destroyed, but the out of control inequality that the government was not addressing. We need to stop blaming AI as a society for job loss and instead get the governments to help with the transition which so far they have largely been inactive on.
That was the part that shocked me as it rings even more true at this present moment.
Wow. I had to go verify this and sure enough, I found a video from 7 years ago where Lucas said this and much more including comparing the American Empire to the Galactic Empire. Wow. I missed that reference as a child.
The panels typically offer both red and near infrared light. The biggest benefit for me which was a surprise benefit I had not even intended to address is my asthma. Specially the near infrared on my lungs has allowed me to stop all asthma medication that I took for over 45 years. It is super effective and there are research papers emerging that support.
Pathological Demand Avoidance is such a passive aggressive term. I find most in the spectrum happily follow a leader provided they are logical, compassionate, considerate and looking for win wins. If the leader is an egotistical boss who sees you as less worthy than them and is demanding you do work that makes no sense and is even perhaps harmful or exploitative, than yeah, I am going to avoidant and not compliant. Wish more were PDA as we would not end up with tyrant leaders.
Red light therapy panel.
It is not capitalism and this is not me defending it, but rather pointing out that it is how humanity naturally seems to structure itself. The narcissist / psychopaths / sociopaths tend to get into “leadership” positions and the dumb folks idolize and empower them and thus we end up with the smartest people writing a few paragraphs for the manager who is socially savvy but suffers from the Dunning Kruger Effect. This happens in all economic and political systems including Capitalism and Communism as the Sam people tend to get into positions of power.
Or cannot see it due to the Dunning Kruger Effect. It is very strong in some.
They are in on it or worse also caught up in the Epstein/Trump honey Trump.
Agree. The dumb people out of necessity need to be in groups and those groups are lead by narcissists/ psychopaths / sociopaths who then weaponize the groups ignorance to exploit them for their own gain while using shame and worse to keep the group inline. It leads to the world being led by the wrong people and while we have managed fairly well by most metrics, it is an absolute thorn in the side of humanity that carries existential risks. I sure hope we can change things so that the smartest, kindest people lead instead. Maybe as the power of the group is certainly being eroded in part due to AI allowing smart individuals to climb quietly higher into leadership roles that are not forceful but rather truly leading. We will see. I am hopeful change is coming or we end up destroying everything. Seems like it is on a knife’s edge and has been since we invented nukes.
Dunning Kruger Effect is strong in Zuck.
It is for the Billionaire class but not everyone else. Would you not agree?
I am a solo developer on some of these platforms and have to do an annual privacy evaluation which I find painful as the company taking up my valuable time to answer all their BS questions, are the ones breaking privacy. Insane. Further, I had to pay $100 annually to the UK privacy commission as it is required for all UK companies which is also deeply ironic as the same government is requiring you to submit ID for some websites which has made the privacy leaks even more damaging.
As a solo developer I am feeling the brunt of the privacy charades while the big companies just continue to violate privacy unchecked. I have zero desire or intentions to break anyone’s privacy an ask for no information from anyone outside of pulling down their profile to sign in as is required to determine app permission. That is it. No phone numbers, no real names, no biometrics, no photos of ID or anything outside of those username on that platform but I am burdened as if I was collecting all of this and as if I was the one breaking privacy. This world is mental.
Looks like Zuck is all in and has no choice but to support evil, because if/when the evil collapses, so does Zuck. Must be stressful for him as history has shown that this behaviour eventually catches up to them. Sadly, some make it too old age and run entire countries before it catches up which causes a lot of unnecessary suffering.
Same here. Canadian living in Vietnam and really could never go back to winters.