| View previous topic :: View next topic |
| Author |
Message |
Wadewilson
n00b
Joined: 09 Nov 2022
Posts: 56
|
 Posted: Sat Feb 15, 2025 5:43 am Post subject: Tests failing when compiling with portage (ebuild) |
 |
|
Following is the ebuild file for llama.cpp repo that I would like to install to my system:
| Code: | # Copyright 1999-2024 wadewilson619
# Distributed under the terms of the GNU General Public License v2
EAPI=8
inherit cmake-multilib cuda git-r3
EGIT_REPO_URI="https://github.com/ggml-org/llama.cpp.git"
DESCRIPTION="Inference of Meta's LLaMA model (and others) in pure C/C++"
HOMEPAGE="https://github.com/ggerganov/llama.cpp"
LICENSE="MIT"
SLOT="0"
KEYWORDS="~amd64"
IUSE="
address-sanitize
blas
ccache
cuda
curl
hostonly
lto
server
tests
thread-sanitize
undefined-sanitize
unified-memory
utils
"
CPU_FLAGS_X86=( aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sse sse2 sse3 sse4_1 sse4_2 ssse3 )
RESTRICT="userpriv"
DEPEND="blas? ( sci-libs/openblas:= )
cuda? ( dev-util/nvidia-cuda-toolkit )
curl? ( net-misc/curl )
ccache? ( dev-util/ccache )"
RDEPEND="${DEPEND}"
BDEPEND="${DEPEND}"
S="${WORKDIR}/${P}"
src_unpack() {
if [[ ${PV} == 9999 ]]; then
git-r3_src_unpack
git-r3_fetch ${EGIT_REPO_URI} "refs/tags/$(curl -s "https://api.github.com/repos/ggml-org/llama.cpp/releases/latest" | jq -r '.tag_name')"
fi
default
}
src_prepare() {
if use cuda; then
cuda_src_prepare
fi
cmake_src_prepare
}
src_configure() {
if use cuda ; then
export NVCC_CCBIN="$(cuda_gccdir)"
export GGML_CUDA_ENABLE_UNIFIED_MEMORY="$(usex unified-memory)"
fi
local mycmakeargs=(
-DGGML_BLAS="$(usex blas)"
-DGGML_CUDA="$(usex cuda)"
-DGGML_LTO="$(usex lto)"
-DGGML_CCACHE="$(usex ccache)"
-DLLAMA_BUILD_TESTS="$(usex tests)"
-DLLAMA_SANITIZE_THREAD="$(usex thread-sanitize)"
-DLLAMA_SANITIZE_ADDRESS="$(usex address-sanitize)"
-DLLAMA_SANITIZE_UNDEFINED="$(usex undefined-sanitize)"
-DLLAMA_CURL="$(usex curl)"
-DLLAMA_BUILD_COMMON="$(usex utils)"
-DLLAMA_BUILD_SERVER="$(usex server)"
-DCMAKE_INSTALL_PREFIX=${EPREFIX}/opt/${PN}
-DCMAKE_CUDA_ARCHITECTURES="75"
)
cuda_add_sandbox
cmake-multilib_src_configure
}
src_test() {
cuda_add_sandbox
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:"${BUILD_DIR}/bin"
export LD_PRELOAD=/usr/lib/gcc/x86_64-pc-linux-gnu/14/libasan.so.8.0.0:$LD_PRELOAD
cmake-multilib_src_test
}
|
When running | Quote: | | ebuild ./llama-cpp-9999.ebuild manifest clean unpack test install |
tests are failing because of errors raised by Address sanitizer (ASAN) but when I manually compile the repo with address sanitize enabled and run the tests I don't see errors. Why does this happen in portage build system sandbox?
Error thrown by portage build system:
| Code: | =================================================================
==399==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 1 byte(s) in 1 object(s) allocated from:
#0 0x7fbe8a2f8c17 in malloc (/usr/lib/gcc/x86_64-pc-linux-gnu/14/libasan.so.8.0.0+0xf8c17)
#1 0x5602cb50edf8 (/bin/gsed+0x10df8)
#2 0x5602cb50fecc (/bin/gsed+0x11ecc)
SUMMARY: AddressSanitizer: 1 byte(s) leaked in 1 allocation(s).
*
* If you need support, post the output of `emerge --info '=dev-cpp/llama-cpp-9999::greninja'`,
* the complete build log and the output of `emerge -pqv '=dev-cpp/llama-cpp-9999::greninja'`.
* The complete build log is located at '/var/tmp/portage/dev-cpp/llama-cpp-9999/temp/build.log'.
* The ebuild environment file is located at '/var/tmp/portage/dev-cpp/llama-cpp-9999/temp/environment'.
* Working directory: '/var/tmp/portage/dev-cpp/llama-cpp-9999/work/llama-cpp-9999_build-abi_x86_64.amd64'
* S: '/var/tmp/portage/dev-cpp/llama-cpp-9999/work/llama-cpp-9999'
=================================================================
==437==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 43914 byte(s) in 26 object(s) allocated from:
#0 0x7fc720cf8c17 in malloc (/usr/lib/gcc/x86_64-pc-linux-gnu/14/libasan.so.8.0.0+0xf8c17)
#1 0x7fc720736daa (/usr/lib64/libpython3.12.so.1.0+0x136daa)
Indirect leak of 88953 byte(s) in 100 object(s) allocated from:
#0 0x7fc720cf8c17 in malloc (/usr/lib/gcc/x86_64-pc-linux-gnu/14/libasan.so.8.0.0+0xf8c17)
#1 0x7fc720736daa (/usr/lib64/libpython3.12.so.1.0+0x136daa)
SUMMARY: AddressSanitizer: 132867 byte(s) leaked in 126 allocation(s).
|
The specific issue that I am seeing in the logs is "odr-violation" raised by ASAN. Now, I want to understand why the same issue or error is not seen when I compile the repo manually with ASAN enabled.
Also, if I disable address sanitize, all tests are running fine in portage build system sandbox. Before I merge the package I want to be sure if the build is stable, hence I am running the tests.
Log file for failing test case because of ASAN :- https://0x0.st/8ahI.log
Emerge Info :- https://0x0.st/8BoU.txt
Ebuild Package Build log : https://0x0.st/8Bo5.log
Last edited by Wadewilson on Sun Mar 02, 2025 10:19 am; edited 1 time in total |
|
| Back to top |
|
 |
Arniiiii
n00b
Joined: 14 Jan 2025
Posts: 5
Location: Ukraine
|
| Posted: Sun Mar 02, 2025 9:51 am Post subject: Re: Tests failing when compiling with portage (ebuild) |
|
|
I wrote llama.cpp ebuild and I can reproduce asan error without setting the asan USE flag related CMake options, I mean just by adding `-fsanitize=address,leak,undefined` to CFLAGS, CXXFLAGS and LDFLAGS.
I have global `-ggdb3` , so here's test logs: https://paste.gentoo.zip/iH6xTFYC |
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2778
|
| Posted: Sun Mar 02, 2025 11:25 am Post subject: |
|
|
Portage straps the binaries from debug information by default. Also test phase is by default not executed. Otherwise many packages would fail.
I believe debug information is necessary for asan and that's why it's failing. You need to compile the package with several features enabled which I currently don't remember, but one of them I think is nostrip.
Do you really need to test it or you just think it would be a good idea? It's not.
Best Regards,
Georgi |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
