Gentoo LLMs

[Vieri]

Hi,

I'm starting to look at the oh so many open-source software projects to run LLM models out there, and it's quite overwhelming.
There are some attempts of creating Gentoo ebuilds for some of them (eg. ollama in guru).
I'm particularly interested in the projects that allow to run models from a custom program or via REST APIs.
Not really looking for GUI or webUI.

Any suggestions here in the forum for worthy/updated software that installs from source (no binary packages) and can easily be set up with a custom ebuild?

Regards

[Navar]

https://github.com/rasbt/LLMs-from-scratch , a real Gentoo way.

For creating your own ebuild(s), https://wiki.gentoo.org/wiki/Basic_guide_to_write_Gentoo_Ebuilds

https://ollama.com/download , more "easy" mode. "install" and run local as your limited user (would suggest checking the script instead of blindly running and/or manual install instructions). I doubt Gentoo devs are large enough currently in resources and number, versus all other normal expected packages that may or may not need maintainers, testers, help, etc. to be chasing supporting any specific LLM being available out of the billion or so currently in existance due to crazed fad/demand, e.g., https://huggingface.co/models
_________________
(I don't speak for/represent my current employer)