Thank you for that explanation. My regex impaired ass thought he wanted to hurt generation[x|y|z].

I’m like “what’d we ever do to you?”

… it should be possible to continue supporting those older cards by setting the compile time information to match your gpus.

I’ll admit that’s probably over my head, but still good advice. The 1080ti works fine for now, but whenever it gets dropped from support, I have a good home for it.

I have a Win7 machine so I can run my older games that don’t play well in linux. Nvidia Rocket Sled Demo on a 1080ti screams.

Probably better to ask on !localllama@sh.itjust.works. Ollama should be able to give you a decent LLM, and RAG (Retrieval Augmented Generation) will let it reference your dataset.

The only issue is that you asked for a smart model, which usually means a larger one, plus the RAG portion consumes even more memory, which may be more than a typical laptop can handle. Smaller models have a higher tendency to hallucinate - produce incorrect answers.

Short answer - yes, you can do it. It’s just a matter of how much RAM you have available and how long you’re willing to wait for an answer.

Yep, I had been hoping for the same thing.

Also, to @projectmoon@lemm.ee, you might want to wait and see what gets announced at Computex next month. Hopefully they announce some new stuff and the current gen prices drop.

Ollama doesn’t currently support mixing CUDA & ROCm. https://github.com/ollama/ollama/issues/3723#issuecomment-2071134571

One thing to keep in mind about adding RAM your speed could drop depending on how many slots you populate. For me, I have a 5700G and with 2x16Gb, it runs at 3200Mhz, but with 4x16Gb(same exact product), it only runs at 1800Mhz. In my case, RAM speed has a huge effect on tokens/sec, if I have a model that has to use some RAM.

You can check AMD’s spec page for your processor, but they don’t really document a lot of this stuff.

So it seems obvious that the US dept of Homeland Security would include a Meta rep on their AI Safety and Security Board. But they didn’t.

Don’t get me wrong, I hate Facebook/Instagram/Meta, but I appreciate their contribution to AI.

It’s out on Ollama already. I’m downloading it now. Can’t wait for the uncensored versions.

Have you tried the guide on AMD’s site? It looks like it’s for Windows, and I don’t know what you’re running. Plus, I use Ollama, so I probably can’t be of much help.

For programing, my favorite is Dolphin-Mixtral, but I’ve had good results with Dolphin-Mistral and Llama2.

Thanks for the info. I’m not on Windows, so I can’t try it. Doesn’t work offline? That’s what I was afraid of.

It’s a great proof of concept. Hopefully we’ll see more text-gen-webui extensions soon.

Thanks for posting. I’m still new to this and had no idea what settings I should be using.

It’s probably a pain to set up in Windows. In Linux, it just works, there’s nothing to set up. I’m using it right now.

OP really should have mentioned their OS.

Edit: Actually, nevermind both my posts. I know DRI_PRIME works by using my APU for regular desktop activity, and routing discrete GPU output in whenever a game is being played. But I don’t know if it’s possible to make it use the dGPU all the time.

Even if it did, it would only work inside the OS, so if you had to boot into the BIOS for anything, you wouldn’t have a display. So for all intents and purposes, it wouldn’t really work.

I just did a quick bing chat search (“does DRI_PRIME work on systems without a cpu with integrated graphics?”) and it says it will work. I can’t check for you because my CPUs all have graphics.

I CAN tell you that some motherboards will support it (my ASUS does) and some don’t (my MSI).

BTW, I’m talking about Linux. If you’re using Windows, there’s a whole series of hoops you have to jump through. LTT did a video a while back.

I just looked at my Steam stats today and 52% is RT3. There may be plenty of games that surpass it in visuals, but game mechanics are a different story, and without a stock market I don’t see much of a point.

I played Tropico 3-5, but gave up after they refused to give us highways and overpasses.

AMD APUs have Video Coding Engine / Unified Video Decoder, while Intel CPUs have QuickSync. FFMPEG’s hardware page says that AMD support is incomplete.

You may want to ask over in !datahoarder@lemmy.ml . This topic often came up back on Reddit, and the general vibe I got was that most people prefer QuickSync. Intel may not be great in a lot of areas, but they are a beast in video encoding/decoding. That being said, I use a Ryzen APU and it’s perfectly fine. There are way more important things to look at when choosing a CPU.

If your performance is slow, I would check your CPU is listed on the chart I linked above. Not all CPUs support all codecs.

Edit: If your CPU doesn’t support the codec, it will still work, it just won’t be accelerated.

You could try Dolphin. It’s the default file manager in KDE, but I think you can install it by itself.

It should be available as an apt package in Ubuntu.

OMG. I originally got the impression (from somewhere) that you couldn’t pass arguments to an alias, so I googled and found that weird function nonsense. Oh well, live and learn. Thanks.

alias hgrep='function _f(){ history | grep $1; };_f'

Because I’m to lazy to type

history | grep whatever_I'm_looking_for

Thanks for posting this. I couldn’t figure out why Steam was broken on my laptop for the last 3 weeks or so. Installing the .deb from the Steam website fixed it. I’m starting to get fed up with Canonical.

You haven’t heard about wearable servers? C’mon get with the times! I’m headed out for a jog now, gotta keep my hashrate up!