Crumb
c/localllama · by leanleft@lemmy.ml · 15d

model : Ling-3.0-tiny (8b)

https://huggingface.co/inclusionAI/Ling-3.0-tiny

a lightweight hybrid reasoning MoE model with 7.9B total parameters and only 1.3B activated parameters per token. It is designed to deliver strong reasoning and agentic capabilities under a small inference compute footprint, making advanced model capabilities more accessible for local and resource-constrained deployment.

18 points · 8 comments · view on lemmy.world

8 Comments

BeefAndPoultry@lemmus.org · 5 pts · 15d (5 replies)

anyone try this? this might be good for my crappy laptop lol

is it good enough to use with Zoo Code? is it better than Qwen 3.5 4b?

EDIT: woa

https://artificialanalysis.ai/models/ling-3-0-tiny

But not yet supported in llama.cpp https://github.com/ggml-org/llama.cpp/pull/26608

Valmond@lemmy.dbzer0.com · 2 pts · 15d (4 replies)

Would this run on a cpu only pc? Would it be crazy slow if so?

Looking for some small model to cut my teeth on and I only have my laptop at the moment.

BeefAndPoultry@lemmus.org · 3 pts · 15d (3 replies)

I've run Qwen 3.5 4b and Gemma 4 e2b on CPU only, this should be faster than those I think (fewer active parameters). If you have AVX512 or AVX10 then it should help a bit. Still slow compared to a GPU lol.

Valmond@lemmy.dbzer0.com · 1 pts · 14d (2 replies)

How slow 😁😅?

BeefAndPoultry@lemmus.org · 2 pts · 14d (1 reply)

my laptop is crappy, so like 5 tokens per second lol, prompt processing of like 20 tokens per second

I think a decent laptop nowadays, even running CPU only, could probably do like 5x faster

Valmond@lemmy.dbzer0.com · 2 pts · 13d

Thanks for the info!

fozid@lem.radiantfig.fyi · 2 pts · 14d

I haven't tried it yet, but I tested one of their previous ones with CPU interference on an Intel n97, and it was one of the best in terms of t/s performance and also prompt response quality on the metrics I tested against. When this run on llama.cpp I will try and give it a test.

leanleft@lemmy.ml · 1 pts · 15d

personally,
i did one test of ling3-flash and gemma-4-31B side-by-side .
ling3 understood me and had a fantastic answer. gemma must have misunderstood what i was saying.. it wrote a long story-like paragraph that didnt answer my question. but it kinda had 1 bit of insight.
so: (maybe..) don't sleep on ling models !