Domi

u/domi@lemmy.secnd.me
6 posts · 1.3k comments

Recent posts

Recent comments

I have a Strix Halo as well and I have it configured to dynamically allocate everything, so I have run some 100GB+ models as well. The OS itself needs way under a GB without UI so you can get pretty close to the 128GB.

Currently on Fedora, running llama-swap to start the llama.cpp toolboxes by kyuz0, audio.cpp and ComfyUI.

Qwen3.8-27b is my favorite model right now for most things, still playing around with Qwen3.8-Flash-Next but not quite there yet.

Gemma4-31b is also really good with languages and natural writing but I prefer Qwen3.8 for anything programming or logical.

I can only go by what my government tells me but I don't think that's true:

Der Europäische Notruf ist überall in der EU gebührenfrei verfügbar. [...] Die Nummer kann im Notfall selbst bei gesperrtem Mobiltelefon – ohne Netzvertrag, SIM-Card, PIN oder Guthaben – kostenfrei gewählt werden.

Translation:

The European emergency number is available free of charge everywhere in the EU. [...] The number can be dialed free of charge on a locked phone - without contract, SIM, PIN or credit.

Not that I regularily call them but my phone still shows "Emergency calls only" when no SIM is inserted.

Source: https://www.bundeskanzleramt.gv.at/themen/europa-aktuell/2026/02/europaeischer-notruf-112.html

You need to use MTP (Multi token prediction) to get acceptable speeds on a Strix Halo. On llama.cpp you just need to add --spec-type draft-mtp --spec-draft-n-max 3 -np 1 to your arguments.

You should see around 15-20 t/s in coding tasks at Q8, which will still be much slower than Qwen3-Coder-Next-UD-Q8 since it's 27b active parameters instead of 3b.

As for Q8 vs BF16, it usually makes barely to no difference in quality. Speed is higher on Q8 though. Still testing with 3.8 since it just released.

27b fits easily into 128 unified RAM, even unquantized. So something was definitely wrong.

Running 3.8-27b on my Strix Halo box right now.