Thank you for all your insight!! This is really helpful

I need to mess with tabbyapi. Doesn’t help that there’s like 2 tabbys, one is tabbyapi and the other is tabbyml. I am guessing tool support is at its infancy stage.

Damn! Thank you so much. This is very helpful and a great starting point for me to mess about to make the most of my LLM setup. Appreciate it!!

Absolutely disgusting(ly cute)

Dude! That’s so dope. I would really like your insights in how you tuned MoE. That would be a game changer as you can swap out unnecessary layers from the GPU and still get the benefit of using a bigger model and stuff.

Yeah it’s a little hard to do inference with these limited VRAM situations and larger contexts. That’s a massive pain

You’re good. I’m trying to get larger context windows on my models so trying to figure that out and balance token throughput. I do appreciate your insights into the different use cases.

Have you tried larger 70b models? Or compared against larger MoE models?

Not to be that guy (he says as he becomes that guy) but the GPL is not a permissive license, BSD and MIT are. Tho imo GPL is the better and probably best license.

Also what models and use cases did you run it for? And what was your context window?

Lolol yes, it’s a weird straight circle indeed.

However applying those fixes and then learning to fix it is a great way to learn how to troubleshoot and unb0rk your system.

Treat the response like you would of an LLM, it needs to make sense to you, you need to make sure they aren’t messing with you or have given you a fix that only works in their case. Usually the best fixes are the simple ones. And it seems like even with the longer ones you’re able to figure out your simple fixes which is awesome!

Depends on the level of block lists you add. The defaults are pretty sane and it doesn’t need any configuration, you configure your router to use it

Pihole, homeassistant, a music server using moodeaudio

There’s also Zed. And helix.

Amongst all the other great alts here

Also neovim is really dope! Just have to throw that there

And enjoyer

When you really wanna pursue slack

Slackware!

And I use Slackware fwiw

While I agree with your point I don’t think a lot of people will be open to that cause they may not be aware of base distros and all that and in the end it would seem like you’re not listening to them.

OP literally said they want to avoid mint due to bad experiences.

Suggestions?

Oh yes! I mean just empty out the British Museum