From what I’ve seen, it’s definitely worth quantizing. I’ve used llama 3 8B (fp16) and llama 3 70B (q2_XS). The 70B version was way better, even with this quantization and it fits perfectly in 24 GB of VRAM. There’s also this comparison showing the quantization option and their benchmark scores:

Source

To run this particular model though, you would need about 45GB of RAM just for the q2_K quant according to Ollama. I think I could run this with my GPU and offload the rest of the layers to the CPU, but the performance wouldn’t be that great(e.g. less than 1 t/s).

Yes, you can find it here.

Are you using mistral 7B?

I also really like that model and their fine-tunes. If licensing is a concern, it’s definitely a great choice.

Mistral also has a new model, Mistral Nemo. I haven’t tried it myself, but I heard it’s quite good. It’s also licensed under Apache 2.0 as far as I know.

I haven’t tested it extensively, but open webui also has RAG functionality (chat with documents).

The UI it self is also kinda cool and it has other useful features like commands (for common prompts) and searching for stuff online (e.g. with searx). It works quite well with Ollama.

Currently, I only have a free account there. I tried Hydroxide first, and I had no problem logging in. I was also able to fetch some emails. I will try hydroxide-push as well later.

I haven’t heard of Hydroxide before; thank you for highlighting it! Just one question: Does it also require a premium account like the official bridge, or is it also available for free accounts?

It’s great to see such cool FOSS options. Thanks for sharing this – super helpful!

The latest version of Eternity is compatible with Lemmy 0.19. You might need to log out and log back in, though.

However, the new functions aren’t implemented yet (e.g. the new sorting methods)

I think the IzzyOnDroid repo is updated once a day at 6 PM UTC.

You are right. I think it’s time to release it on Google play. I also want to focus on getting the app into the main F-Droid repo as well.

Nice. Thank you

Hi!

Dev here. Would you mind filing a report on this issue via Codeberg? Additional information would be greatly beneficial for me to identify and resolve the problem. If possible, could you also provide me with the relevant logs? This sounds like a serious issue I would like to resolve this ASAP.

I’m so happy to see Infinity for Lemmy in a meme. This made my day. Thank you.

We’ve just lunched !infinityforlemmy@lemdro.id, the official community for the project!

There was an issue with loading subscriptions on the main activity, which I’ve fixed in this commit today (it’s not in the release yet). If you click on the subscriptions menu option once, it will load them and store them in the database, so you will see them after that. Maybe this was the same issue?

Thank you for the fast response!

1. The saved post functionality is not implemented yet. I want to add this functionality in the following days.
2. The “karma” score should be removed, I agree
3. The subscription part is interesting. Is this on the main page or the subscriptions page?
4. I’ve also experienced slow loading times, mostly on the main page. I believe this part of the code causes this issue
5. Sadly, the “cannot fetch user info” error is shown right now when It fails to get the access token (for any reason). If you can provide me with some logs or open an issue about it, I can take a look at it.

Thanks again for your quick review. If you spot any major issues with this build please open an issue on codeberg.

I’ve just published the first alpha release!

UPDATE: I have just released the first alpha version of this project. You can find it here.

You can also use the project link from the post to grab the apk with Obtainium as well.

I am planning on releasing an alpha version in a couple of days. It won’t be anything fancy, just some basic functionality.

I got access to Codeberg’s CI yesterday, so I will set up a pipeline for this project, and I think you will be able to download it with Obtainium this way. I also want to push for an F-Droid or IzzyOnDroid release as well.

I have opened an issue for it!