| View previous topic :: View next topic |
| Author |
Message |
juanatix
n00b
Joined: 13 Jun 2024
Posts: 13
Location: USA
|
 Posted: Fri Apr 11, 2025 5:11 pm Post subject: [SOLVED]Failed to emerge sys-libs/libnvidia-container-1.17.2 |
 |
|
Trying to install app-containers/nvidia-container-toolkit, get the following error. AI told me I needed to patch the source code myself, which didn't sound right, thought I'd ask here.
| Code: |
# emerge -av app-containers/nvidia-container-toolkit
These are the packages that would be merged, in order:
Calculating dependencies... done!
Dependency resolution took 5.63 s (backtrack: 2/20).
[ebuild N ] sys-libs/libnvidia-container-1.17.2:0/1.17.2::gentoo USE="seccomp -static-libs" 0 KiB
[ebuild N ] app-containers/nvidia-container-toolkit-1.17.3-r1:0/1.17.3::gentoo 0 KiB
Total: 2 packages (2 new), Size of downloads: 0 KiB
Would you like to merge these packages? [Yes/No] Yes
>>> Verifying ebuild manifests
>>> Emerging (1 of 2) sys-libs/libnvidia-container-1.17.2::gentoo
>>> Failed to emerge sys-libs/libnvidia-container-1.17.2, Log file:
>>> '/var/tmp/portage/sys-libs/libnvidia-container-1.17.2/temp/build.log'
>>> Jobs: 0 of 2 complete, 1 failed Load avg: 0.55, 0.79, 1.03
*** Resuming merge...
These are the packages that would be merged, in order:
Calculating dependencies... done!
Dependency resolution took 1.01 s.
Total: 0 packages, Size of downloads: 0 KiB
* emerge --keep-going: app-containers/nvidia-container-toolkit-1.17.3-r1
* dropped because it requires <=sys-libs/libnvidia-container-1.17.3, >=sys-
* libs/libnvidia-container-1.17
* Messages for package sys-libs/libnvidia-container-1.17.2:
* ERROR: sys-libs/libnvidia-container-1.17.2::gentoo failed (compile phase):
* emake failed
*
* If you need support, post the output of `emerge --info '=sys-libs/libnvidia-container-1.17.2::gentoo'`,
* the complete build log and the output of `emerge -pqv '=sys-libs/libnvidia-container-1.17.2::gentoo'`.
* The complete build log is located at '/var/tmp/portage/sys-libs/libnvidia-container-1.17.2/temp/build.log'.
* The ebuild environment file is located at '/var/tmp/portage/sys-libs/libnvidia-container-1.17.2/temp/environment'.
* Working directory: '/var/tmp/portage/sys-libs/libnvidia-container-1.17.2/work/libnvidia-container-1.17.2'
* S: '/var/tmp/portage/sys-libs/libnvidia-container-1.17.2/work/libnvidia-container-1.17.2'
* Messages for package app-containers/nvidia-container-toolkit-1.17.3-r1:
* emerge --keep-going: app-containers/nvidia-container-toolkit-1.17.3-r1
* dropped because it requires <=sys-libs/libnvidia-container-1.17.3, >=sys-
* libs/libnvidia-container-1.17
*
* The following 2 packages have failed to build, install, or execute
* postinst:
*
* (sys-libs/libnvidia-container-1.17.2:0/1.17.2::gentoo, ebuild scheduled for merge), Log file:
* '/var/tmp/portage/sys-libs/libnvidia-container-1.17.2/temp/build.log'
* (app-containers/nvidia-container-toolkit-1.17.3-r1:0/1.17.3::gentoo, ebuild scheduled for merge)
*
# |
build.log:
| Code: |
# tail -n 20 /var/tmp/portage/sys-libs/libnvidia-container-1.17.2/temp/build.log
make[2]: Entering directory '/var/tmp/portage/sys-libs/libnvidia-container-1.17.2/work/libnvidia-container-1.17.2/src/nvcgo'
export CGO_CFLAGS="-std=gnu11 -march=native -O2 -pipe"; \
export CGO_LDFLAGS="-Wl,--gc-sections -Wl,-soname,libnvidia-container-go.so.1 -Wl,--gc-sections -Wl,-soname,libnvidia-container-go.so.1 -Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs"; \
go build -o libnvidia-container-go.so -ldflags "-compressdwarf=false -linkmode=external" -buildmode=c-shared .
# nvcgo
./main.go:35:10: cannot define new methods on non-local type CDeviceRule
make[2]: *** [Makefile:40: libnvidia-container-go.so] Error 1
make[2]: Leaving directory '/var/tmp/portage/sys-libs/libnvidia-container-1.17.2/work/libnvidia-container-1.17.2/src/nvcgo'
make[1]: *** [/var/tmp/portage/sys-libs/libnvidia-container-1.17.2/work/libnvidia-container-1.17.2/mk/nvcgo.mk:28: build] Error 2
make[1]: Leaving directory '/var/tmp/portage/sys-libs/libnvidia-container-1.17.2/work/libnvidia-container-1.17.2'
make: *** [Makefile:246: deps] Error 2
* ERROR: sys-libs/libnvidia-container-1.17.2::gentoo failed (compile phase):
* emake failed
*
* If you need support, post the output of `emerge --info '=sys-libs/libnvidia-container-1.17.2::gentoo'`,
* the complete build log and the output of `emerge -pqv '=sys-libs/libnvidia-container-1.17.2::gentoo'`.
* The complete build log is located at '/var/tmp/portage/sys-libs/libnvidia-container-1.17.2/temp/build.log'.
* The ebuild environment file is located at '/var/tmp/portage/sys-libs/libnvidia-container-1.17.2/temp/environment'.
* Working directory: '/var/tmp/portage/sys-libs/libnvidia-container-1.17.2/work/libnvidia-container-1.17.2'
* S: '/var/tmp/portage/sys-libs/libnvidia-container-1.17.2/work/libnvidia-container-1.17.2'
#
|
go version:
| Code: |
> go version
go version go1.24.2 linux/amd64
>
|
|
|
| Back to top |
|
 |
juanatix
n00b
Joined: 13 Jun 2024
Posts: 13
Location: USA
|
| Posted: Fri Apr 11, 2025 7:07 pm Post subject: How I got it to install... |
|
|
I don't know if this was the right way but I managed to get it installed...
1. Create a local overlay if you haven’t already:
| Code: |
mkdir -p /usr/local/portage/sys-libs/libnvidia-container
echo 'PORTDIR_OVERLAY="/usr/local/portage ${PORTDIR_OVERLAY}"' >> /etc/portage/make.conf
|
2. Copy and enter the ebuild directory:
| Code: |
cp -r /var/db/repos/gentoo/sys-libs/libnvidia-container /usr/local/portage/sys-libs/
cd /usr/local/portage/sys-libs/libnvidia-container
|
3. Fix the masters issue
Create (or edit) the overlay metadata file:
| Code: |
mkdir -p /usr/local/portage/metadata
nano /usr/local/portage/metadata/layout.conf
|
Add this line:
This tells Portage your overlay inherits from the main Gentoo tree. Save and exit.
4. Fix missing digest (we'll do this again)
| Code: |
cd /usr/local/portage/sys-libs/libnvidia-container
ebuild libnvidia-container-1.17.3.ebuild manifest
|
5. unpack the source:
| Code: |
ebuild libnvidia-container-1.17.3.ebuild unpack
cd /var/tmp/portage/sys-libs/libnvidia-container-1.17.3/work/libnvidia-container-1.17.3/src/nvcgo
|
6. Patch main.go
| Code: |
cp main.go /tmp/main.go.fixed
vim /tmp/main.go.fixed
|
Look around line 38 I think it was and replace the broken code with this:
| Code: |
// Empty main for c-shared library.
func main() {}
// Local Go wrapper for the C struct
type CDeviceRuleWrapper struct {
ptr *C.struct_device_rule
}
// Convert a C-based DeviceRule to a Go-based cgroup.DeviceRule
func (w *CDeviceRuleWrapper) convert() cgroup.DeviceRule {
r := w.ptr
return cgroup.DeviceRule{
Allow: bool(r.allow),
Type: C.GoString(r._type),
Access: C.GoString(r.access),
Major: func() *int64 { m := int64(r.major); return &m }(),
Minor: func() *int64 { m := int64(r.minor); return &m }(),
}
}
|
Then, toward the bottom around line 109 or so ...
| Code: |
Change this:
for i, cr := range crules {
rules[i] = cr.convert()
}
To this:
for i, cr := range crules {
wrapped := CDeviceRuleWrapper{ptr: &cr}
rules[i] = wrapped.convert()
}
|
7. Edit the ebuild
| Code: |
vim /usr/local/portage/sys-libs/libnvidia-container/libnvidia-container-1.17.3.ebuild
In the PATCHES section, add a new line:
"${FILESDIR}/fix-cgo-wrapper.patch"
Should look like this:
PATCHES=(
"${FILESDIR}/${PN}-1.17.0-fix-makefile-r1.patch"
"${FILESDIR}/fix-cgo-wrapper.patch"
)
|
8. create the patch
| Code: |
diff -u /var/tmp/portage/sys-libs/libnvidia-container-1.17.3/work/libnvidia-container-1.17.3/src/nvcgo/main.go /tmp/main.go.fixed > /usr/local/portage/sys-libs/libnvidia-container/files/fix-cgo-wrapper.patch
|
9. fix the top of fix-cgo-wrapper.patch
| Code: |
vim /usr/local/portage/sys-libs/libnvidia-container/files/fix-cgo-wrapper.patch
Should look like this:
--- a/src/nvcgo/main.go 2024-12-04 04:38:14.000000000 -0500
+++ b/src/nvcgo/main.go 2025-04-11 14:32:51.240950682 -0400
@@ -31,15 +31,21 @@
// Alias for a C-based device_rule struct
type CDeviceRule = C.struct_device_rule
|
The important parts (what tripped me up originally) are the 1st 2 lines. a/src/nvcgo/main.go and b/src/nvcgo/main.go.
10. Finally, regenerate the manifest and install the damn package
| Code: |
cd /usr/local/portage/sys-libs/libnvidia-container
ebuild libnvidia-container-1.17.3.ebuild manifest
emerge -1av =sys-libs/libnvidia-container-1.17.3
|
11. Here's the full patch for reference:
| Code: |
$ cat /usr/local/portage/sys-libs/libnvidia-container/files/fix-cgo-wrapper.patch
--- a/src/nvcgo/main.go 2024-12-04 04:38:14.000000000 -0500
+++ b/src/nvcgo/main.go 2025-04-11 14:32:51.240950682 -0400
@@ -31,15 +31,21 @@
// Alias for a C-based device_rule struct
type CDeviceRule = C.struct_device_rule
+// Go wrapper struct to add methods safely
+type CDeviceRuleWrapper struct {
+ ptr *CDeviceRule
+}
+
// Convert a C-based DeviceRule to a Go-based cgroup.DeviceRule
-func (r *CDeviceRule) convert() cgroup.DeviceRule {
- return cgroup.DeviceRule{
- Allow: bool(r.allow),
- Type: C.GoString(r._type),
- Access: C.GoString(r.access),
- Major: func() *int64 { m := int64(r.major); return &m }(),
- Minor: func() *int64 { m := int64(r.minor); return &m }(),
- }
+func (w *CDeviceRuleWrapper) convert() cgroup.DeviceRule {
+ r := w.ptr
+ return cgroup.DeviceRule{
+ Allow: bool(r.allow),
+ Type: C.GoString(r._type),
+ Access: C.GoString(r.access),
+ Major: func() *int64 { m := int64(r.major); return &m }(),
+ Minor: func() *int64 { m := int64(r.minor); return &m }(),
+ }
}
//export GetDeviceCGroupVersion
@@ -100,8 +106,9 @@
rules := make([]cgroup.DeviceRule, len(crules))
for i, cr := range crules {
- rules[i] = cr.convert()
- }
+ wrapped := CDeviceRuleWrapper{ptr: &cr}
+ rules[i] = wrapped.convert()
+ }
err = api.AddDeviceRules(C.GoString(cgroupPath), rules)
if err != nil {
|
Hope this helps someone. I copied all this after the fact based on my bash history so if something doesn't work reply to the thread and I'll do my best to help. |
|
| Back to top |
|
|
grknight
Retired Dev
Joined: 20 Feb 2015
Posts: 2106
|
| Posted: Fri Apr 11, 2025 8:02 pm Post subject: |
|
|
| juanatix wrote: |
11. Here's the full patch for reference:
| Code: |
$ cat /usr/local/portage/sys-libs/libnvidia-container/files/fix-cgo-wrapper.patch
|
|
If it is just a patch, most of those extra steps could have been avoided if you just created it at /etc/portage/patches/sys-libs/libnvidia-container/fix-cgo-wrapper.patch
The creation of the patch can use the existing, failed ebuild as well. (Since the work is saved in /var/tmp/portage) |
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2935
|
| Posted: Sat Apr 12, 2025 4:05 pm Post subject: |
|
|
I the other thread I told you it compiled fine for me. It turned out I had added it to package.accpet_keywords and it was v1.17.5. I saw your post you deleted in the other thread earlier. I'm sorry didn't have the time to look at it.
Best Regards,
Georgi |
|
| Back to top |
|
|
juanatix
n00b
Joined: 13 Jun 2024
Posts: 13
Location: USA
|
| Posted: Sat Apr 12, 2025 4:15 pm Post subject: |
|
|
If you're talking to me I wasn't part of the other thread.
The issue I'm having now is ollama doesn't use my 1080ti, it's offloading everything to CPU and I don't understand why.
| Code: |
Sat Apr 12 12:08:53 2025
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.230.02 Driver Version: 535.230.02 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce GTX 1080 Ti Off | 00000000:81:00.0 On | N/A |
| 16% 56C P0 62W / 250W | 503MiB / 11264MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 N/A N/A 7179 G /usr/bin/X 253MiB |
| 0 N/A N/A 7234 G /usr/lib64/librewolf/librewolf 133MiB |
| 0 N/A N/A 7411 G ...72,262144 --variations-seed-version 91MiB |
| 0 N/A N/A 7692 G ...erProcess --variations-seed-version 18MiB |
+---------------------------------------------------------------------------------------+
|
| Code: |
> ldd /usr/bin/ollama | grep cuda
libcudart.so.11.0 => /opt/cuda/lib64/libcudart.so.11.0 (0x00007f24f9e00000)
> ldd /usr/lib64/ollama/cuda_v11/libggml-cuda.so | grep cudart
libcudart.so.11.0 => /opt/cuda/lib64/libcudart.so.11.0 (0x00007f4d73c00000)
|
| Code: |
> grep -i "cuda\|error\|llm" ollama_debug.log
2025/04/12 11:53:22 routes.go:1231: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:2048 OLLAMA_DEBUG:true OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/home/sysop/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
time=2025-04-12T11:53:22.439-04:00 level=DEBUG source=sched.go:107 msg="starting llm scheduler"
time=2025-04-12T11:53:22.443-04:00 level=DEBUG source=gpu.go:501 msg="Searching for GPU library" name=libcuda.so*
time=2025-04-12T11:53:22.443-04:00 level=DEBUG source=gpu.go:525 msg="gpu library search" globs="[/usr/bin/libcuda.so* /opt/cuda/lib64/libcuda.so* /usr/lib64/libcuda.so* /opt/cuda/lib64/libcuda.so* /usr/lib64/libcuda.so* /home/sysop/libcuda.so* /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cud /lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
time=2025-04-12T11:53:22.449-04:00 level=DEBUG source=gpu.go:558 msg="discovered GPU libraries" paths=[/usr/lib64/libcuda.so.535.230.02]
initializing /usr/lib64/libcuda.so.535.230.02
CUDA driver version: 12.2
time=2025-04-12T11:53:22.456-04:00 level=DEBUG source=gpu.go:125 msg="detected GPUs" count=1 library=/usr/lib64/libcuda.so.535.230.02
[GPU-41aae151-e334-6117-350f-1ab006f81f09] CUDA totalMem 11169 mb
[GPU-41aae151-e334-6117-350f-1ab006f81f09] CUDA freeMem 10684 mb
releasing cuda driver library
time=2025-04-12T11:53:22.521-04:00 level=INFO source=types.go:130 msg="inference compute" id=GPU-41aae151-e334-6117-350f-1ab006f81f09 library=cuda variant=v12 compute=6.1 driver=12.2 name="NVIDIA GeForce GTX 1080 Ti" total="10.9 GiB" available="10.4 GiB"
initializing /usr/lib64/libcuda.so.535.230.02
CUDA driver version: 12.2
time=2025-04-12T11:53:29.790-04:00 level=DEBUG source=gpu.go:441 msg="updating cuda memory data" gpu=GPU-41aae151-e334-6117-350f-1ab006f81f09 name="NVIDIA GeForce GTX 1080 Ti" overhead="0 B" before.total="10.9 GiB" before.free="10.4 GiB" now.total="10.9 GiB" now.free="10.4 GiB" now.used="480.1 MiB"
releasing cuda driver library
time=2025-04-12T11:53:29.813-04:00 level=DEBUG source=memory.go:108 msg=evaluating library=cud gpu_count=1 available="[10.4 GiB]"
initializing /usr/lib64/libcuda.so.535.230.02
CUDA driver version: 12.2
time=2025-04-12T11:53:29.877-04:00 level=DEBUG source=gpu.go:441 msg="updating cuda memory data" gpu=GPU-41aae151-e334-6117-350f-1ab006f81f09 name="NVIDIA GeForce GTX 1080 Ti" overhead="0 B" before.total="10.9 GiB" before.free="10.4 GiB" now.total="10.9 GiB" now.free="10.4 GiB" now.used="480.1 MiB"
releasing cuda driver library
time=2025-04-12T11:53:29.877-04:00 level=DEBUG source=memory.go:108 msg=evaluating library=cud gpu_count=1 available="[10.4 GiB]"
time=2025-04-12T11:53:29.877-04:00 level=INFO source=server.go:138 msg=offload library=cuda layers.requested=-1 layers.model=33 layers.offload=33 layers.split="" memory.available="[10.4 GiB]" memory.gpu_overhead="0 B" memory.required.full="6.2 GiB" memory.required.partial="6.2 GiB" memory.required.kv="1.0 GiB" memory.required.allocations="[6.2 GiB]" memory.weights.total="4.1 GiB" memory.weights.repeating="3.7 GiB" memory.weights.nonrepeating="411.0 MiB" memory.graph.full="560.0 MiB" memory.graph.partial="677.5 MiB"
time=2025-04-12T11:53:30.026-04:00 level=DEBUG source=server.go:423 msg=subprocess environment="[CUDA_LAUNCH_BLOCKING=1 CUDA_CACHE_PATH=/home/sysop/.cache/nv LD_LIBRARY_PATH=/opt/cuda/lib64:/usr/lib64:/opt/cuda/lib64:/usr/lib64::/usr/bin PATH=/usr/games:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/sysop/bin:/home/sysop/.local/bin CUDA_VISIBLE_DEVICES=GPU-41aae151-e334-6117-350f-1ab006f81f09]"
time=2025-04-12T11:53:30.027-04:00 level=INFO source=server.go:614 msg="waiting for server to become available" status="llm server error"
time=2025-04-12T11:53:30.037-04:00 level=DEBUG source=ggml.go:93 msg="skipping path which is not part of ollama" path=/opt/cuda/lib64
time=2025-04-12T11:53:30.037-04:00 level=DEBUG source=ggml.go:93 msg="skipping path which is not part of ollama" path=/opt/cuda/lib64
time=2025-04-12T11:53:30.278-04:00 level=INFO source=server.go:614 msg="waiting for server to become available" status="llm server loading model"
|
| Code: |
# nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0
|
| Code: |
> groups ollama
video render ollama
|
| Code: |
> ./test_cuda
Found 1 CUDA devices
Device 0: NVIDIA GeForce GTX 1080 Ti, Compute Capability: 6.1
|
| Code: |
> lsmod | grep nvid
nvidia_uvm 1548288 2
nvidia_drm 81920 6
nvidia_modeset 1503232 9 nvidia_drm
nvidia 62107648 552 nvidia_uvm,nvidia_modeset
drm_kms_helper 266240 1 nvidia_drm
drm 786432 10 drm_kms_helper,nvidia,nvidia_drm
video 77824 1 nvidia_modeset
backlight 24576 3 video,drm,nvidia_modeset
i2c_core 135168 6 i2c_designware_platform,i2c_designware_core,drm_kms_helper,nvi ia,i2c_piix4,drm
|
|
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2935
|
| Posted: Sat Apr 12, 2025 4:26 pm Post subject: |
|
|
| juanatix wrote: | | If you're talking to me I wasn't part of the other thread. |
Ah, well, the poster deleted their post which mentioned another thread so I decided it was this one. Maybe it was, but it wasn't theirs.
| juanatix wrote: | The issue I'm having now is ollama doesn't use my 1080ti, it's offloading everything to CPU and I don't understand why.
|
In the other thread I mentioned I couldn't make ollama built locally to use the dGPU, neither Gentoo's ebuild, nor local user build.
One thing you should check is if ollama user is in video group. Another thing - the size of the model. It might not fit in the GPU's RAM.
And if it doesn't work with smaller model, then maybe you too cna't make it use the dGPU. The way I made it to use it was using their docker image and a local instance of open-webui from my home directory. There was a combined image which didn't pick the GPU, there were separate images I couldn't make talking to each other. Most probably due to lack of docker knowledge.
OP in the other thread complained about not being able to install nvidia-container-toolkit because of the same package failing.
Best Regards,
Georgi |
|
| Back to top |
|
|
juanatix
n00b
Joined: 13 Jun 2024
Posts: 13
Location: USA
|
| Posted: Sat Apr 12, 2025 4:34 pm Post subject: |
|
|
| logrusx wrote: | | I the other thread I mentioned I couldn't make ollama built locally to use the dGPU, neither Gentoo's ebuild, nor local build. |
Yea, this has been super frustrating...
| logrusx wrote: | | One thing you should check is if ollama user is in video group. |
It is, I mentioned that in my post: https://forums.gentoo.org/viewtopic-p-8859256.html#8859256
| Code: | > groups ollama
video render ollama
|
| logrusx wrote: | | Another thing - the size of the model. It might not fit in the GPU's RAM. |
Also mentioned in my post. It does fit. GPU has 10.9 GiB free, model required 6.2 GiB.
| Code: | time=2025-04-12T11:53:29.877-04:00 level=DEBUG source=gpu.go:441 msg="updating cuda memory data" gpu=GPU-41aae151-e334-6117-350f-1ab006f81f09 name="NVIDIA GeForce GTX 1080 Ti" overhead="0 B" before.total="10.9 GiB" before.free="10.4 GiB" now.total="10.9 GiB" now.free="10.4 GiB" now.used="480.1 MiB"
releasing cuda driver library
time=2025-04-12T11:53:29.877-04:00 level=DEBUG source=memory.go:108 msg=evaluating library=cud gpu_count=1 available="[10.4 GiB]"
time=2025-04-12T11:53:29.877-04:00 level=INFO source=server.go:138 msg=offload library=cuda layers.requested=-1 layers.model=33 layers.offload=33 layers.split="" memory.available="[10.4 GiB]" memory.gpu_overhead="0 B" memory.required.full="6.2 GiB" memory.required.partial="6.2 GiB" memory.required.kv="1.0 GiB" memory.required.allocations="[6.2 GiB]" memory.weights.total="4.1 GiB" memory.weights.repeating="3.7 GiB" memory.weights.nonrepeating="411.0 MiB" memory.graph.full="560.0 MiB" memory.graph.partial="677.5 MiB"
|
| logrusx wrote: | | The way I made it to use it was using their docker image and a local instance of open-webui from my home directory. |
Docker doesn't recognize or use my GPU, either. Docker is a bloated mess.
| logrusx wrote: | | OP in the other thread complained about not being able to install nvidia-container-toolkit because of the same package failing. |
Yea, this whole exercise has me wondering how much time I'd of saved just running devuan instead. I need to get stuff down and tinkering with all this has wasted a lot of time. |
|
| Back to top |
|
|
sam_
Developer
Joined: 14 Aug 2020
Posts: 2271
|
| Posted: Sat Apr 12, 2025 4:57 pm Post subject: |
|
|
You're free to use whatever you want, but I wouldn't assume that all of these tools are packaged in other distros or are new enough anyway.
FWIW, I didn't see any bug reports for libnvidia-container, but I did just merge a fix for it from its maintainer. |
|
| Back to top |
|
|
juanatix
n00b
Joined: 13 Jun 2024
Posts: 13
Location: USA
|
| Posted: Sat Apr 12, 2025 5:03 pm Post subject: |
|
|
| sam_ wrote: | You're free to use whatever you want, but I wouldn't assume that all of these tools are packaged in other distros or are new enough anyway.
FWIW, I didn't see any bug reports for libnvidia-container, but I did just merge a fix for it from its maintainer. |
You're right, there's no guarantee it would work in devuan, and I shouldn't have said that, I'm just frustrated. I think the 1080ti is 4 generations behind at this point so it's not exactly cutting edge. I've opened up a bug report with Ollama - hopefully they say something other than "did you try Docker?".
Thanks for taking the time to chime in. |
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2935
|
| Posted: Sat Apr 12, 2025 5:07 pm Post subject: |
|
|
Excuse me if I didn't want to go through all that output.
| juanatix wrote: | | Docker doesn't recognize or use my GPU, either. Docker is a bloated mess. |
I can't comment on if it's a bloated mess. there's also podman. Did you run:
And then restart docker? That was all I needed to do on my side.
Best Regards,
Georgi |
|
| Back to top |
|
|
juanatix
n00b
Joined: 13 Jun 2024
Posts: 13
Location: USA
|
| Posted: Sat Apr 12, 2025 5:38 pm Post subject: |
|
|
| juanatix wrote: | there's also podman. Did you run:
| Code: | | nvidia-ctk runtime configure --runtime=docker |
And then restart docker? That was all I needed to do on my side. |
Yep, been down that rabbit hole, too. Doesn't work for podman unfortunately. |
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2935
|
| Posted: Sat Apr 12, 2025 6:29 pm Post subject: |
|
|
I'll try to run it again tomorrow.
Best Regards,
Georgi |
|
| Back to top |
|
|
juanatix
n00b
Joined: 13 Jun 2024
Posts: 13
Location: USA
|
| Posted: Sat Apr 12, 2025 7:33 pm Post subject: |
|
|
Update from the bug report on the Ollama github should anyone else come across this issue:
| Quote: | | ollama finds backends relative to where the ollama binary is installed, by moving up one directory level and appending lib/ollama. Since the executable is /usr/bin/ollama, it expects to find the backends in /usr/lib/ollama. Since the backends are actually in /usr/lib64/ollama, ollama doesn't find them. The quickest workaround would be sudo ln -s /usr/lib64/ollama /usr/lib. |
Fixed it - Ollama now uses my 1080ti. |
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2935
|
| Posted: Sun Apr 13, 2025 3:24 pm Post subject: |
|
|
| juanatix wrote: | Update from the bug report on the Ollama github should anyone else come across this issue:
| Quote: | | ollama finds backends relative to where the ollama binary is installed, by moving up one directory level and appending lib/ollama. Since the executable is /usr/bin/ollama, it expects to find the backends in /usr/lib/ollama. Since the backends are actually in /usr/lib64/ollama, ollama doesn't find them. The quickest workaround would be sudo ln -s /usr/lib64/ollama /usr/lib. |
Fixed it - Ollama now uses my 1080ti. |
That's worth a bug upstream. If I used it, I would file it.
Best Regards,
Georgi |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
