The other practical side of this is enforceability.

Lets say Linux banned all use of VSCode in contributions, this second. And any Microsoft tools. No exceptions. And it makes sense, because there is an evil blatantly anti-linux entity behind it… What are people gonna do?

Keep using it anyway, and just not disclose it.

This is why linux doesn’t tell people what tools they can and can’t use. Practically, they can’t police what tools people use, just what they contribute. Just like (to a reasonable degree) they don’t try to police people who contributes to the kernel, but instead judge them by their contributions. Other contributors don’t have to like those tools or contributors, but that is how a project like this must operate.

+1

Honestly this should ship with many browsers, be a default for search engines and such.

Yeah, hence the slew of forks.

Not that I blame llama.cpp. They have a huge job for so little dev manpower, and it’s trying to be a jack of all trades rather than something specialized like other frameworks.

Twitter such a black hole, though. Even via lurking with xcancel or similar, it’s just not worth it to me.

Low end hardware is a wide range. The best model for that is going to vary wildly by the specifics of it, like your exact GPU, CPU, CPU RAM size and speed, operating system, and so on.

Reddit’s localllama has deteriorated beyond saving IMO. It’s like all the intelligence/experimenting has drained from there; no one posts any useful data, the same simple help questions get asked to no helpful responses. Even commenters notice it’s full of bots.

I follow some interesting Huggingface users, and lurk in some fragmented LLM Discords, but other than that, I honestly have no idea where to go. I feel kinda cut off from the community.

Actually…

And I really don’t mean to be pedantic. But Unsloth has enshittified some.

• Some of their GGUFs target their own vibecoded PRs first, not necessarily mainline llama.cpp.

• And, more frequently as of recent times, they’re bugged just to get them out the door first, and get the hype.

• The quantization loss is higher than some other quanters now, again, due to the rush.

• There are also some much stronger quantization schemes outside mainline GGUFs now, like ik llama.cpp or exl3.

Unsloth isn’t bad, I’m just saying be careful using them as the defacto quantization source.

I used to love them. But I haven’t downloaded an Unsloth model in some time, and TBH I am not really inclined to…

I would be careful with any “neutering” tools then, depending on your work, because some can mess with Teams, CoPilot, or whatever else you need. Check the options before you run the tools.

Windows Store apps like Teams work fine for me, though, as do IDEs.

Honestly I don’t know which projects are up to date, so I’m afraid to recommend what I vaguely remember.

Search GitHub/Gitlab for “windows debloat” or similar concepts, and sort the results by date updated (e.g. the most recent commit). This is precisely what I’d do with a fresh Windows install, anyway.

On top of that, I go through running services one by one and disable the ones I don’t need.

I disable Defender active protection (except for the firewall/anti tamper) as well, as it absolutely tanks the performance of many applications. As one example I measured, it multiplies the startup time for modded Rimworld, and dramatically slows game ticks. It seems to slow Cyberpunk 2077 as well, but not as reproducibly.

Most “debloaters” won’t do this, as it’s generally not a good idea. But I treat my Windows partition as disposable: nothing sensitive stored, no sensitive data accessible on drives nor networks, data on a separate partition. If it was infected, I could wipe it and reinstall without a care.

Maybe it’s because I neutered my W11 and only dual boot to launch the occasional game or Windows specific driver, but… I haven’t had significant issues with W11? Not that I can remember over the past two years.

No broken updates, no weird copilot things, no excessive background usage, nothing. The only maintenance I do is manually updating graphics drivers.

It wastes none of my time.

The one issue I had was the LG monitor debacle, that tried to install Mcafee, but that was LG’s fault (and I’m never, ever buying an LG display again, as long as I live. I hope you’re happy, LG marketing).

I find this remarkable, because to me it means, Windows doesn’t have to be junk, yet they deliberately configure it that way, and make it extremely difficult to undo. I used a slew of tools neuter it, but I’d have no idea where to start without them.

And to be clear, I use Linux 90% of the time, so I’m not advocating for Windows or anything.

Admittedly, no, but this is slowly improving. Stepfun published their SFT dataset, smaller labs are publishing their datasets for task specific tunes. I believe there was another Chinese lab that published bulk pretraining data, but I can’t find it in my history at the moment.

And, notably, these comparatively tiny labs generally aren’t scraping the internet so abusively like OpenAI/Meta. They don’t need as much. Going by statements in their papers, they tend to use existing archives of web data, buy commercial data, or (more recently) generate a lot synthetic data.

If you run Nemotron, from open datasets.

See the dataset table here. There are sections for pretraining and for specific tasks/topics, like STEM, code and such: https://huggingface.co/nvidia

Chinese models are starting to open their pretraining datasets too, albeit slowly.

Be careful with beellama, as it’s largely vibecoded from what I understand.

That’s not bad per se, but you have to watch out for implementation quirks (eg a model might not inference quite correctly) or questionable features (like TurboQuant, which is sorta obsolete with newer models, and seems to benchmark questionably on older ones).

There are some interesting AMD forks out there though. I wish I could speak to some, but I haven’t tested any, and it’s hard to keep up.

Just for reference, I’m now getting 20 tokens/sec with Deepseek V4 Flash 0731, on a single 3090 desktop, whereas I was only getting 13 in ik llama.cpp, and less in base llama.cpp.

I have not switched to 4.1 in IK yet. All these releases are happening so fast by brain can’t even keep up…

If you are on Nvidia, have you considered using exllama/TabbyAPI with an MoE model?

It already does something more practical: splitting the MoEs in each layer, profiling them, and keeping the “hot” ones on the GPU. Its quantization is also state of the art, way better than llama.cpp, and it’s had KV hadamard rotation implemented years before the TurboQuant.

That’s actually a great idea.

It’ll need a big batch size to hide the latency though (so this only worth it past 8K context, or so?) and then it needs to reload the quantized weights onto the GPU.

It does suck that all paths to better inference are so bad. Bigger GPUs are crazy expensive. Good IGPs are rare. CPU RAM is cost prohibitive. Even NVMe drives are crazy expensive now. Hence there’s only so much that can be done if users have absolutely no options to buy.

this new model, couldn’t universities rebuild it with different contexts and study it in ways you can’t reproduce in other models? like, is k2 horizons a good scientific foundation on which to study machine learning?

Not on a university budget, I don’t think.

But there is all sorts of interesting stuff that can be done with a truly open model other than replicating it.

On top of what others said, Nvidia Nemotron is truly open, with an open dataset.

There have been others (like Olmo), but Nemotron was unique in that it was reasonably useful/cutting edge, while other open models before were basically research artifacts.

Yeah. Kinda feels like OpenClaw and a few ‘trends’ before that.

…The heck happened?

I’ve been under a rock for a bit.

Explain to me: why is there so much hype (and controversy) around some Hyperland Arch Linux config, within like a month? Is this some astroturfing thing?