I've gotten about 12-15 token/sec on a 2070 Super (8GB) with a build called microFlare + llama.cpp, but it has the MTP stripped out. I bet they're using the MTP to double their base performance
This is right in line with my thinking. You don't teach math to a second grader by handing them a graphing calculator. You need to learn to do things manually first before you start letting the machines do it for you so you understand it
I've gotten 35B-A3B running on a 4GB GPU with 16GB of RAM, the 6B actives should easily fit onto an 8GB card, but yeah, you might could get away with just 64GB of memory, maybe even just 32 if quantized small enough
I've been having this same problem for like 3 weeks now!! Are you using Firefox? That's where I experience the issue, if I try in Chromium it works.
I ended up going to their contact sales page, had the captcha fail there too. But this time it offered me a feedback form. It was ridiculous, but I finally reported it.
I also found if it fails and it gives you a troubleshoot link, you can go to that page and once it fails there it gives you the feedback form too
Being used poorly and mostly resulting in slop, or malicious/unethical/deadly uses, and psychosis
Being forced down the public's throat and put into places where it adds no value, job losses, quality drops, plagiarism
Datacenter/environmental concerns, part prices
So switching to local AI fixes problem 3 (mostly). But we also need the hype bubble to pop so number 2 goes away. Unfortunately the first problem will be hardest to fix and right now requires the public backlash as new norms are being formed and bad actors need to be called out.
That said, I think we're about 1 to 3 years away from the bubble popping and everything starting to balance out to where AI is treated more like just a tool rather than a genie in a box. The hype will fade out and one or two of the big players will die, but at this point I don't think generative AI use is going away
This is bullshit. 4000 hours of watch time was already a near impossible goal for new channels, especially after the past year's UI and algorithm changes. Now they want twice that?!?
It's theft. Even if you only have 100 views, you should be eligible for a cut of the profits you generated with your content.
The market is dying for a new competitor to come in and disrupt their monopoly
Because we sat around and thought we were special. We didn't need unions, we had a rare skill. And now all of our labor power has been sucked away because the people that know nothing about software and infrastructure now think they can do our jobs because they saw a cool tech demo
Can the Gemini assistant set reminders yet? Such a basic function of the old Google assistant and yet Gemini has failed every time I tried it in the past
I think local AI is the future. When the bubble pops and the big companies have to start charging people 5-10x the price for tokens (unsubsidized), people will switch to self-hosted options. Some companies are already surpassing their budgets at the current prices.
Not only that, but you get privacy and security you'll never get from a 3rd party SaaS solution. And there are no quotas or limits you have to worry about.
As people wake up from the AI is magic hype and realize it's just another tool that is good at some things and bad at others, we're going to see some big changes.
I've gotten about 12-15 token/sec on a 2070 Super (8GB) with a build called microFlare + llama.cpp, but it has the MTP stripped out. I bet they're using the MTP to double their base performance
Country music. I can't stand it. The twangs make me cringe.
Johnny Cash and a couple Dolly songs are about all I can enjoy
This is right in line with my thinking. You don't teach math to a second grader by handing them a graphing calculator. You need to learn to do things manually first before you start letting the machines do it for you so you understand it
I've gotten 35B-A3B running on a 4GB GPU with 16GB of RAM, the 6B actives should easily fit onto an 8GB card, but yeah, you might could get away with just 64GB of memory, maybe even just 32 if quantized small enough
There are about 7 countries they will be at war with, and they publicly told us all decades ago this was coming and which ones https://en.wikipedia.org/wiki/Greater_Israel
Don't forget their plans to make homelessness illegal, and then once you're in jail you become a slave
Yes, all the loopholes were put there on purpose
It just pops up inline on the page. Grey box with a text link to open the feedback form at the bottom
I've been having this same problem for like 3 weeks now!! Are you using Firefox? That's where I experience the issue, if I try in Chromium it works.
I ended up going to their contact sales page, had the captcha fail there too. But this time it offered me a feedback form. It was ridiculous, but I finally reported it.
I also found if it fails and it gives you a troubleshoot link, you can go to that page and once it fails there it gives you the feedback form too
There are 3 reasons people hate AI right now:
So switching to local AI fixes problem 3 (mostly). But we also need the hype bubble to pop so number 2 goes away. Unfortunately the first problem will be hardest to fix and right now requires the public backlash as new norms are being formed and bad actors need to be called out.
That said, I think we're about 1 to 3 years away from the bubble popping and everything starting to balance out to where AI is treated more like just a tool rather than a genie in a box. The hype will fade out and one or two of the big players will die, but at this point I don't think generative AI use is going away
This is bullshit. 4000 hours of watch time was already a near impossible goal for new channels, especially after the past year's UI and algorithm changes. Now they want twice that?!?
It's theft. Even if you only have 100 views, you should be eligible for a cut of the profits you generated with your content.
The market is dying for a new competitor to come in and disrupt their monopoly
Because we sat around and thought we were special. We didn't need unions, we had a rare skill. And now all of our labor power has been sucked away because the people that know nothing about software and infrastructure now think they can do our jobs because they saw a cool tech demo
It's completely dead to me now
It's gonna be on YouTube 6 hours later, just wait
Can the Gemini assistant set reminders yet? Such a basic function of the old Google assistant and yet Gemini has failed every time I tried it in the past
I think local AI is the future. When the bubble pops and the big companies have to start charging people 5-10x the price for tokens (unsubsidized), people will switch to self-hosted options. Some companies are already surpassing their budgets at the current prices.
Not only that, but you get privacy and security you'll never get from a 3rd party SaaS solution. And there are no quotas or limits you have to worry about.
As people wake up from the AI is magic hype and realize it's just another tool that is good at some things and bad at others, we're going to see some big changes.
It's a shame there's still no 3.8 MoE model that can run on consumer hardware. Still stuck at 3.6 if you need to leverage offloading experts to CPU
And....? There's nothing illegal about wiping your phone if not already under indictment of something
The network is still alive and well