Crumb
c/opensource · by tatoko556@reddthat.com · 231d

Is there any software that can use it that benefits average user or is it just a waste of silicon???

cross-posted from: https://lemmit.online/post/4242386

This is an automated archive made by the Lemmit Bot.

The original was posted on /r/pcmasterrace by /u/trander6face on 2024-10-24 11:11:47+00:00.

107 points · 25 comments · view on lemmy.world

25 Comments

st3ph3n@midwest.social · 50 pts · 231d (2 replies)

I would have preferred if they had used the die space for the GPU, not this bullshit.

Junkers_Klunker@feddit.dk · 22 pts · 231d (1 reply)

But then, how would ai save our future?

med@sh.itjust.works · 1 pts · 94d

By making vGPU available on Windows on AMD :(

alibloke@feddit.uk · 40 pts · 231d (3 replies)

You can use it for blurring your background on your camera

tatoko556@reddthat.com · 12 pts · 231d

ah that's good

snugglesthefalse@sh.itjust.works · 2 pts · 231d (1 reply)

You can already do that with the lens though?

artyom@piefed.social · 8 pts · 231d

If you have a professional DSLR sure. Not a webcam.

Sparrow_1029@programming.dev · 22 pts · 231d (1 reply)

I got an Asus Vivobook with a pretty new AMD chipset that has one... running Fedora, no Idea how to make something use it either :/

Territorial@piefed.ca · 7 pts · 230d

Also on an Asus laptop with Fedora, after following some obscure instructions from amd I managed to get a Python script to confirm the npu exists and is technically functional... But apparently to do anything approaching useful, I need the slightly fancier npu. What a waste.

sga@piefed.social · 20 pts · 231d (5 replies)
[ removed ]
tatoko556@reddthat.com · 5 pts · 231d (3 replies)

I have heard of ollama before, is this the same thing?

dyathinkhesaurus@lemmy.world · 7 pts · 231d (1 reply)

llama.cpp is not the same thing as ollama. It does a similar thing, but better imo. can multiplex/multithread sessions/conversations with llms (where ollama and lmstudio etc. have to queue them up), stuff like that.

sga@piefed.social · 4 pts · 231d
[ removed ]
tatoko556@reddthat.com · 4 pts · 231d

I checked, ollama does not use npu

Swedneck@discuss.tchncs.de · 2 pts · 230d

So, what? Is that only trained on ethical data?
Because i bloody doubt it

JoMiran@lemmy.ml · 11 pts · 231d (5 replies)

I wonder if they have implemented NPU support into Folding@Home yet.

JRaccoon@discuss.tchncs.de · 5 pts · 231d (4 replies)

There are no plans for it, at least not for a while. Apparently F@H workloads are not suitable for the current NPUs

SapphironZA@sh.itjust.works · 8 pts · 231d (3 replies)

They need hardware that does actual useful work, not hardware accelerated autocorrect.

knightly@pawb.social · 4 pts · 230d (2 replies)

You can do lots of useful work with processors designed for large matrix calculus, but not a lot of folks want to run physics sims for some reason.

ICastFist@programming.dev · 2 pts · 230d (1 reply)

Sounds like it should do well with some advanced 3D stuff? Or wonky physics games like Goat Simulator or Amazing Frog

knightly@pawb.social · 3 pts · 230d

If there are game devs that want to try making a game out of a fluid sim, I'd be all for it. Realistically, most devs will have to wait for better middleware that adds NPU matrix acceleration to existing game engines.

j4k3@piefed.world · 10 pts · 231d

You need a model compiled for the architecture. I saw some for the RK35xx devices when shopping for hardware. I do not think there is software made to split up or run models in general on a NPU. The models must be configured for the physical hardware topology. The stuff that runs on most devices is very small, and these either need a ton of custom fine tuning or they are barely capable of simple tasks.

On the other hand, segmentation models are small, and that makes layers, object identification, and background removal stuff work. Looking at your CPU speed, and available memory, it is unlikely to make much difference. You are also memory constrained for running models, though you could use deepspeed to load from a disk drive too.

hperrin@lemmy.ca · 8 pts · 231d (3 replies)

The only thing I can think of is AI video game upscalers. Other than that, yeah, it’s a waste of silicon.

tatoko556@reddthat.com · 3 pts · 231d (2 replies)

which software exactly (that uses npu instead of gpu)?

hperrin@lemmy.ca · 2 pts · 231d

I don’t think there is one currently, but that’s a potential use.

murvel@feddit.nu · 1 pts · 231d

Co-pilot for example and other LLM types of neural network processing not tied to Nvidia.

calamityjanitor@lemmy.world · 5 pts · 231d

https://github.com/FastFlowLM/FastFlowLM

That RTX 4050 is probably even faster though.