Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[crossdev]Migration/maintnance of an existing Gentoo install
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2278
Location: $TERM

PostPosted: Wed Sep 04, 2024 10:59 am    Post subject: [crossdev]Migration/maintnance of an existing Gentoo install Reply with quote

Hello everyone!

I'm planning to copy up a gentoo install from an old but more powerful machine to a machine with a newer architecture but it's less powerful (typical e.g. of an old server vs a lower end but newer architecture laptop). Both machines are x86_64, however if I chroot into the laptop's gentoo install, it results in segfaults here and there because of incompatible instruction (and -march=native which I intend to keep).

Code:
[17534.512965] python[2202365]: segfault at 10 ip 00007ff4529fcae2 sp 00007ffe29b51ee0 error 4 in libpython3.12.so.1.0[7ff452889000+4be000] likely on CPU 6 (core 3, socket 0)


I've migrated my Gentoo installs several times before, but this time I intend to MAINTAIN it on the server and do no compiling work on the laptop. The answer is crossdev.

I plan to copy up an existing gentoo install into /usr/x86_64-laptop-linux-gnu/, however crossdev installs packages which has /usr/x86_64-laptop-linux-gnu/bin,lib, and the existing gentoo install also has these directories which results in conflicts. Can someone please suggest what to do? Will it be right to overwrite symlinks /lib and /bin to that of crossdev's directories? No packages install any files in /bin/ but I found files in /lib which belongs to sys-apps/baselayout, sys-apps/hwdata, sys-apps/systemd etc...

ROOT= emerge...
Code:
ROOT=/mnt/laptop emerge....


Will not work because the build process tends to copy parts of the static libraries form the host machine which which is -march=native therefore older architecture. There are other problems too because (like emerge: there are no ebuilds built with USE flags to satisfy ">=x11-libs/libdrm-2.4.119[video_cards_radeon]".) because of which I'm forced to think about crossroot.
Setting SYSROOT= will not work since it needs more environment variables to work.
_________________
My blog
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2278
Location: $TERM

PostPosted: Mon Sep 09, 2024 5:27 pm    Post subject: Reply with quote

Question is: if I copy an existing Gentoo installation to /usr/x86_64-X-linux-gnu, do I really need to install the crossdev toolchain (like x86_64-X-linux-gnu/gcc, x86_64-X-linux-gnu/libc etc...)? Or will it work with just crossdev --init-target (because it seems to be working)?
_________________
My blog
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2278
Location: $TERM

PostPosted: Wed Sep 11, 2024 6:00 am    Post subject: Reply with quote

I've realized that the TARGET's glibc conflicts with cross-glibc. I don't know how things are supposed to work with that.
_________________
My blog
Back to top
View user's profile Send private message
pingtoo
Veteran
Veteran


Joined: 10 Sep 2021
Posts: 1307
Location: Richmond Hill, Canada

PostPosted: Sat Nov 23, 2024 4:34 pm    Post subject: Reply with quote

dE_logics wrote:
ROOT= emerge...
Code:
ROOT=/mnt/laptop emerge....


Will not work because the build process tends to copy parts of the static libraries form the host machine which which is -march=native therefore older architecture. There are other problems too because (like emerge: there are no ebuilds built with USE flags to satisfy ">=x11-libs/libdrm-2.4.119[video_cards_radeon]".) because of which I'm forced to think about crossroot.
Setting SYSROOT= will not work since it needs more environment variables to work.
Do you know that highlighted statement in quote as fact or just speculating? Because I have not seen this behaviour before.

In your first post you said
Quote:
I plan to copy up an existing gentoo install into /usr/x86_64-laptop-linux-gnu/, however crossdev installs packages which has /usr/x86_64-laptop-linux-gnu/bin,lib, and the existing gentoo install also has these directories which results in conflicts
I am bit of confuse here because usually stage3 tarball does not create a "/usr/x86_64-laptop-linux-gnu/" so how does this directory come about?

Finally for your objective, I would suggest you use container (docker for example) on the powerful server for building, This will reduce the complexity of setting up cross compiler tool chain.
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2278
Location: $TERM

PostPosted: Sat Nov 23, 2024 5:21 pm    Post subject: Reply with quote

pingtoo wrote:
Do you know that highlighted statement in quote as fact or just speculating? Because I have not seen this behaviour before.


Yes, I've seen it in 2018. And even developers warned me recently! See https://bugs.gentoo.org/939450#c7

pingtoo wrote:
In your first post you said
Quote:
I plan to copy up an existing gentoo install into /usr/x86_64-laptop-linux-gnu/, however crossdev installs packages which has /usr/x86_64-laptop-linux-gnu/bin,lib, and the existing gentoo install also has these directories which results in conflicts
I am bit of confuse here because usually stage3 tarball does not create a "/usr/x86_64-laptop-linux-gnu/" so how does this directory come about?

Finally for your objective, I would suggest you use container (docker for example) on the powerful server for building, This will reduce the complexity of setting up cross compiler tool chain.


Container is like a chroot (but namespaced). So if I use -march=<some other machine>, it will fail in many cases because it does not translate instructions.
_________________
My blog
Back to top
View user's profile Send private message
pingtoo
Veteran
Veteran


Joined: 10 Sep 2021
Posts: 1307
Location: Richmond Hill, Canada

PostPosted: Sat Nov 23, 2024 5:33 pm    Post subject: Reply with quote

dE_logics wrote:
pingtoo wrote:
Do you know that highlighted statement in quote as fact or just speculating? Because I have not seen this behaviour before.


Yes, I've seen it in 2018. And even developers warned me recently! See https://bugs.gentoo.org/939450#c7
I see. So it is mostly relate to build time dependency

Quote:
pingtoo wrote:
In your first post you said
Quote:
I plan to copy up an existing gentoo install into /usr/x86_64-laptop-linux-gnu/, however crossdev installs packages which has /usr/x86_64-laptop-linux-gnu/bin,lib, and the existing gentoo install also has these directories which results in conflicts
I am bit of confuse here because usually stage3 tarball does not create a "/usr/x86_64-laptop-linux-gnu/" so how does this directory come about?

Finally for your objective, I would suggest you use container (docker for example) on the powerful server for building, This will reduce the complexity of setting up cross compiler tool chain.


Container is like a chroot (but namespaced). So if I use -march=<some other machine>, it will fail in many cases because it does not translate instructions.
Yes I also call container as glorified chroot.

If you do the build inside chroot(or container) you should not need cross compile and still using same CPU power as host. So why cross compile still need? is it because you prefer not doing the chroot setup? (as in mounting those pseudo file systems and unmount afterword)
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2278
Location: $TERM

PostPosted: Sat Nov 23, 2024 5:53 pm    Post subject: Reply with quote

Doing a chroot for another machine X on the host Y (with a different CPU) will result in machine X's binaries (which is built using -march=native on machine X) run on Y. Unfortunately the binaries of X and Y are not completely compatible -- binaries of X contain instructions which Y does not understand.

As a result --
Code:
traps: TaskCon~ller #5[195099] trap invalid opcode ip:7f6128f38686 sp:7f611a6c27d0 error:0 in libxul.so[3737686,7f6127ff6000+8723000]
python[2202365]: segfault at 10 ip 00007ff4529fcae2 sp 00007ffe29b51ee0 error 4 in libpython3.12.so.1.0[7ff452889000+4be000] likely on CPU 6 (core 3, socket 0)
traps: protoc[2762543] trap invalid opcode ip:564ee6216980 sp:7ffc2a8311b0 error:0 in protoc


Yup. I got too many of these because of chrooting.
_________________
My blog
Back to top
View user's profile Send private message
pingtoo
Veteran
Veteran


Joined: 10 Sep 2021
Posts: 1307
Location: Richmond Hill, Canada

PostPosted: Sat Nov 23, 2024 6:20 pm    Post subject: Reply with quote

dE_logics wrote:
Doing a chroot for another machine X on the host Y (with a different CPU) will result in machine X's binaries (which is built using -march=native on machine X) run on Y. Unfortunately the binaries of X and Y are not completely compatible -- binaries of X contain instructions which Y does not understand.

As a result --
Code:
traps: TaskCon~ller #5[195099] trap invalid opcode ip:7f6128f38686 sp:7f611a6c27d0 error:0 in libxul.so[3737686,7f6127ff6000+8723000]
python[2202365]: segfault at 10 ip 00007ff4529fcae2 sp 00007ffe29b51ee0 error 4 in libpython3.12.so.1.0[7ff452889000+4be000] likely on CPU 6 (core 3, socket 0)
traps: protoc[2762543] trap invalid opcode ip:564ee6216980 sp:7ffc2a8311b0 error:0 in protoc


Yup. I got too many of these because of chrooting.
But why are you doing -march=native in chroot? that would defeat the idea. You should set your CFLAGS for your -march=<target CPU>

For crossdev setup if you still using -march=native in the /usr/<target>/etc/portage/make.conf you will get same result (none functional binary). GCC can be instructed create any kind of binary as long as it is given right flags. so having crossdev in your idea may or may not do what you want.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54605
Location: 56N 3W

PostPosted: Sat Nov 23, 2024 6:55 pm    Post subject: Reply with quote

segfaults are not incompatible instruction sets
They mean that something tried to access memory it does not own.

Incompabible instruction sets give Illegal Instruction exceptions.

Crossdev is not useful as both systems are amd64.
It should not be possible to install an amd64 target cross toolchain on an amd64 host. That would be a crossdev bug.

A chroot/namespace is the way to go.
You need to discover what -march=native means on the target then use that explicity on the build box.
There is a wiki page about building in a chroot
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
logrusx
Advocate
Advocate


Joined: 22 Feb 2018
Posts: 2475

PostPosted: Sat Nov 23, 2024 7:14 pm    Post subject: Reply with quote

You can safely use the binaries built for the older CPU on the newer. If they are both X86_64 you won't notice a difference.
If they are both X86_64-v2 compatible, the different would be even more marginal.

Best Regards,
Georgi
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2278
Location: $TERM

PostPosted: Sun Nov 24, 2024 4:58 am    Post subject: Reply with quote

By -march=native I imply -march=native on a machine FROM which the ROOT is copied over. In reality I use -march=<machine type>.

I'm not getting segfaults, I'm getting 'invalid opcode' even during compiling. For e.g. chrome, firefox will always fail with -march=<machine type>. QT will never works it results in error 'incompatible processor' and lists instructions which are not available on the build host.

Also the build host is an older system and is an AMD while the target is an intel and newer gen.
_________________
My blog


Last edited by dE_logics on Sun Nov 24, 2024 5:15 am; edited 1 time in total
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2278
Location: $TERM

PostPosted: Sun Nov 24, 2024 5:13 am    Post subject: Reply with quote

Speaking of incompatible instruction I though x86 allowed optional extension instructions -- if on an older processor it executes an older instructions while on a newer one it executes the newer instructions.

But it turned out to be a gimmick.
gcc man page wrote:
-march=cpu-type allows GCC to generate code that may not run at all on processors other than the one indicated.

_________________
My blog
Back to top
View user's profile Send private message
pingtoo
Veteran
Veteran


Joined: 10 Sep 2021
Posts: 1307
Location: Richmond Hill, Canada

PostPosted: Sun Nov 24, 2024 2:28 pm    Post subject: Reply with quote

dE_logics wrote:
By -march=native I imply -march=native on a machine FROM which the ROOT is copied over. In reality I use -march=<machine type>.

I'm not getting segfaults, I'm getting 'invalid opcode' even during compiling. For e.g. chrome, firefox will always fail with -march=<machine type>. QT will never works it results in error 'incompatible processor' and lists instructions which are not available on the build host.

Also the build host is an older system and is an AMD while the target is an intel and newer gen.

X86_64 stuff currently is out of my league, I am using ARM64 primary. So I can only give general advise. The Intel/AMD need someone with more expertise than I am.

However I suggest you provide more information. so we can provide closer to target advise. For example
  • <chroot dir>/etc/portage/make.conf
  • in chroot env, execute emerge --info
  • which package(s) build with error. And the error message (build.log)
  • How the chroot setup. If you have a script share the script, or share generate step by step.
I think with better understand the condition we can provide more concrete suggestions.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54605
Location: 56N 3W

PostPosted: Sun Nov 24, 2024 2:52 pm    Post subject: Reply with quote

AMD and Intel have always had incompatiple instruction set extensions.
Intel started with MMX, AMD added 3dnow!

It's been downhill from there.

See the building in a chroot wiki page for how to handle it.
I would post a link but I'm on my phone.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2278
Location: $TERM

PostPosted: Sun Nov 24, 2024 5:38 pm    Post subject: Reply with quote

NeddySeagoon wrote:
AMD and Intel have always had incompatiple instruction set extensions.
Intel started with MMX, AMD added 3dnow!

It's been downhill from there.

See the building in a chroot wiki page for how to handle it.
I would post a link but I'm on my phone.


Yes, that's what I've been doing with firefox and chromium. I use -mtune instead. A QT build like this is impossible. Cross compile is the only way...
_________________
My blog
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2278
Location: $TERM

PostPosted: Sun Nov 24, 2024 5:43 pm    Post subject: Reply with quote

pingtoo wrote:
dE_logics wrote:
By -march=native I imply -march=native on a machine FROM which the ROOT is copied over. In reality I use -march=<machine type>.

I'm not getting segfaults, I'm getting 'invalid opcode' even during compiling. For e.g. chrome, firefox will always fail with -march=<machine type>. QT will never works it results in error 'incompatible processor' and lists instructions which are not available on the build host.

Also the build host is an older system and is an AMD while the target is an intel and newer gen.

X86_64 stuff currently is out of my league, I am using ARM64 primary. So I can only give general advise. The Intel/AMD need someone with more expertise than I am.

However I suggest you provide more information. so we can provide closer to target advise. For example
  • <chroot dir>/etc/portage/make.conf
  • in chroot env, execute emerge --info
  • which package(s) build with error. And the error message (build.log)
  • How the chroot setup. If you have a script share the script, or share generate step by step.
I think with better understand the condition we can provide more concrete suggestions.


Code:
Portage 3.0.66.1 (python 3.12.7-final-0, !../custom-profiles-gentoo/profiles/mypc/, gcc-14, glibc-2.40-r5, 6.10.7-gentoo-gentoo-dist x86_64)
=================================================================
System uname: Linux-6.10.7-gentoo-gentoo-dist-x86_64-12th_Gen_Intel-R-_Core-TM-_i3-12100-with-glibc2.40
KiB Mem:    32615132 total,    520644 free
KiB Swap:    5242876 total,   5242876 free
Head commit of repository oubliette: dd3a60bb2ba6afa8de4b2ceb2ddcdf2425270e35

Timestamp of repository gentoo: Wed, 20 Nov 2024 00:45:00 +0000
sh bash 5.2_p37
ld GNU ld (Gentoo 2.42 p6) 2.42.0
ccache version 4.10.2 [enabled]
app-misc/pax-utils:        1.3.7::gentoo
app-shells/bash:           5.2_p37::gentoo
dev-build/make:            4.4.1-r100::gentoo
dev-java/java-config:      2.3.4::gentoo
dev-lang/perl:             5.40.0::gentoo
dev-lang/python:           3.11.10_p1::gentoo, 3.12.7_p1::gentoo
dev-lang/rust-bin:         1.81.0-r100::gentoo
dev-util/ccache:           4.10.2-r1::gentoo
sys-apps/baselayout:       2.17::gentoo
sys-apps/sandbox:          2.39::gentoo
sys-apps/systemd:          256.7::gentoo
sys-devel/binutils:        2.42-r2::gentoo
sys-devel/binutils-config: 5.5.2::gentoo
sys-devel/clang:           15.0.7-r3::gentoo
sys-devel/gcc:             14.2.1_p20241116::gentoo
sys-devel/gcc-config:      2.11::gentoo
sys-devel/lld:             15.0.7::gentoo
sys-devel/llvm:            15.0.7-r3::gentoo
sys-kernel/linux-headers:  6.6-r1::gentoo (virtual/os-headers)
sys-libs/glibc:            2.40-r5::gentoo
Repositories:

custom-profiles
    location: /etc/custom-profiles-gentoo
    masters: gentoo
    volatile: True

oubliette
    location: /var/db/repos/oubliette
    sync-type: git
    sync-uri: https://github.com/nabbi/oubliette-overlay
    masters: gentoo
    priority: 50
    volatile: False

gentoo
    location: /usr/portage
    sync-type: git
    sync-uri: https://github.com/gentoo-mirror/gentoo
    priority: 100
    volatile: True

ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=alderlake -mabm -mno-cldemote -mno-kl -mno-pconfig -mno-sgx -mno-widekl -mshstk --param=l1-cache-line-size=64 --param=l1-cache-size=48 --param=l2-cache-size=12288 -fomit-frame-pointer -floop-interchange -floop-strip-mine -floop-block -fgraphite-identity -ftree-loop-distribution -O3 -pipe -flto=1 -fuse-linker-plugin -ffat-lto-objects"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php8.2/ext-active/ /etc/php/cgi-php8.2/ext-active/ /etc/php/cli-php8.2/ext-active/ /etc/php/fpm-php8.2/ext-active/ /etc/php/phpdbg-php8.2/ext-active/ /etc/revdep-rebuild /etc/sandbox.d"
CXXFLAGS="-march=alderlake -mabm -mno-cldemote -mno-kl -mno-pconfig -mno-sgx -mno-widekl -mshstk --param=l1-cache-line-size=64 --param=l1-cache-size=48 --param=l2-cache-size=12288 -fomit-frame-pointer -floop-interchange -floop-strip-mine -floop-block -fgraphite-identity -ftree-loop-distribution -O3 -pipe -flto=1 -fuse-linker-plugin -ffat-lto-objects"
DISTDIR="/var/cache/distfiles/"
EMERGE_DEFAULT_OPTS="--with-bdeps=n --complete-graph --binpkg-respect-use=y --binpkg-changed-deps=n --changed-deps=n --rebuild-if-new-slot=y --load-average 8"
ENV_UNSET="ABI_X86 ACCEPT_LICENSE APACHE2_MODULES APACHE2_MPMS CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH L10N NGINX_MODULES_HTTP PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT SANE_BACKENDS VIDEO_CARDS XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live candy ccache config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="https://mirror.leaseweb.com/gentoo/ http://mirror.csclub.uwaterloo.ca/gentoo-distfiles/"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs"
LEX="flex"
MAKEOPTS="-j12"
PKGDIR="/mnt/systmp/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/tmp"
SHELL="/bin/bash"
USE="a52 aac acpi adns alsa alt-svc amd64 amrenc asm asyncns atm avif bluetooth branding bzip2 cairo cet crypt custom-cflags dav1d dbus declarative dri dts dv dvbpsi editorconfig eglfs enca encode exif faad fdk ffmpeg flac fontconfig gif gles2 gles2-only glslang gmp gstreamer gui heif http2 iconv idn ipv6 jit jpeg jpeg2k jpegxl kde kf6compat lame libdrm libinput libnotify libsamplerate libtirpc lto mad man matroska mikmod minimal mmap modplug mp3 mp4 mpeg mprisi multilib natspec ncurses networkmanager offensive ogg opencl opengl openmp opus orc pam pcre pdf pgo plasma png policykit ppds pulseaudio qml qt6 quicktime raw readline seccomp smp sound spell ssl startup-notification strip svg svt-av1 symlink system-png systemd taglib tcmalloc test-rust theora threads tiff tls-compression truetype twolame uchardet udev udisks unicode upower usb v4l vaapi verify-sig vorbis vpx vulkan wallpaper-metadata wayland webp widgets wxwidgets x264 x265 xft xvid zlib" ABI_X86="64" ADA_TARGET="gcc_12" APACHE2_MODULES="unixd authn_core authz_core authz_host access_compat alias auth_basic authn_file log_config mime log_config proxy proxy_http rewrite proxy_balancer lbmethod_bybusyness lbmethod_byrequests lbmethod_bytraffic lbmethod_heartbeat slotmem_shm env cache dir cgid headers proxy_connect http2 allowmethods auth_digest actions" APACHE2_MPMS="event" CALLIGRA_FEATURES="karbon sheets words" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" GRUB_PLATFORMS="pc efi-64" GUILE_TARGETS="3-0" INPUT_DEVICES="libinput" KERNEL="linux" L10N="en-US en" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LIBREOFFICE_EXTENSIONS="nlpsolver pdfimport" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" NGINX_MODULES_HTTP="access fastcgi gzip gzip_static limit_req rewrite upstream_check proxy" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-2" POSTGRES_TARGETS="postgres16" PYTHON_SINGLE_TARGET="python3_12" PYTHON_TARGETS="python3_11 python3_12" QEMU_SOFTMMU_TARGETS="x86_64" RUBY_TARGETS="ruby32" SANE_BACKENDS="hp" VIDEO_CARDS="intel" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account"
Unset:  ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS


make.conf is empty. I'm using custom profiles instead.

See https://bugs.gentoo.org/944725 for the failed build. Also happens with chromium. The error message and build log is same in chroot.
_________________
My blog
Back to top
View user's profile Send private message
pingtoo
Veteran
Veteran


Joined: 10 Sep 2021
Posts: 1307
Location: Richmond Hill, Canada

PostPosted: Sun Nov 24, 2024 7:23 pm    Post subject: Reply with quote

dE_logics,

I wish I can be more of help. but I can't because I don't know how to parse your current post and the bug report 944725.

And the reason is in chroot there is usually no need to have ROOT=<something> whereas in that bug report you state "ROOT=" I don't know that mean ROOT have a value of something you just want to hide or ROOT have value of blank or no environment variable ROOT at all.

I assume the emerge --info output include in your last post is executed inside chroot environment, It looks fine to me. One suggestion is you might want to enable build binary in EMERGE_DEFAULT_OPTS and possible use --with-bdeps=y instread =n because you want this to be your build environment.

your USE flag have "pgo", this will NOT work in this different CPU setup (Intel vs AMD). Because you target build is Intel machine but executed on AMD, the pgo will run binary for Intel which most likely will fail.

your bug report 955725 is failed for the above "pgo" reason.

For me, in order to understand and make suggestion, I usually need to be sure the environment that got error condition need very obviously (as in verbose), I usually need exactly the command that cause the error and some indication the environment condition. In this case you post the output as I asked, however I don't know if this is inside chroot or not or this is crossdev-emerge output . If you had include the host emerge --info in the same post then I will be able to know for sure. Please note I am not blaming. I just try to explain my way of debugging and a example how I would know for sure. the other possible would be you post include the command prompt which have indicator it is in chroot.
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2278
Location: $TERM

PostPosted: Mon Nov 25, 2024 4:40 am    Post subject: Reply with quote

The bug report is for a cross compile (ROOT=). I did not report a bug with chroot because it would've been invalid since -march=<cpu> was for a different host and the toolchain was not cross compiling. Regardless, it results the same error in chroot.

ROOT point to the target's root directory -- it varies.

Your PGO point might be right -- but that's exactly the reason why I'm cross compiling instead of chrooting. If not firefox, then chromium, if not chromium then QT and KDE always fail.

The emerge --info was in chroot.

Overall, even the devs agree that in this situation (AMD/intel, building on older machines for newer ones etc...) you need to cross compile -- it is the only proper solution. Unfortunately it seems crossdev is unmaintained, the new devs don't have much idea about it and they seem to be unsure on what should be done in these situations.
_________________
My blog
Back to top
View user's profile Send private message
pingtoo
Veteran
Veteran


Joined: 10 Sep 2021
Posts: 1307
Location: Richmond Hill, Canada

PostPosted: Mon Nov 25, 2024 2:26 pm    Post subject: Reply with quote

dE_logics wrote:
Your PGO point might be right -- but that's exactly the reason why I'm cross compiling instead of chrooting. If not firefox, then chromium, if not chromium then QT and KDE always fail.


It is not might be right. It is right in the case of using "PGO" (Profile-Guided Optimization) it mean a phase during build process that will execute binary built by previous phase. In your case it will always result "invalid opcode" no matter using crossdev or not.

The bug report 939450 is refer using "crossdev" usage. therefor the Gentoo dev is based on having crossdev what action require do. They have never said using crossdev to do this what you desire.

In you current desire (migrate binary from AMD to Intel and have PGO enable, the only solution is finding a common ground between AMD and Intel so the binary can be executed on both CPU architectural.

The "common ground" above refer to CFLAGS set to something (which I have no expertise of) so generate opcode can be use on both CPU architectural. [/bug]
Back to top
View user's profile Send private message
logrusx
Advocate
Advocate


Joined: 22 Feb 2018
Posts: 2475

PostPosted: Mon Nov 25, 2024 4:40 pm    Post subject: Reply with quote

AFAIK PGO is hardware dependent and it doesn't make much sense to do it on another computer. Also, as pingtoo says, part of the optimization is to gather profile data by running it. You can't run the newer binary on the older processor. There's another way of doing it, building a special version of the software you want to optimize, let it run for a couple of weeks and it'll gather the data, then supply that data to emerge which'll then produce an optimized binary. However I haven't done it and I can't give an advice or even point you to a relevant resource on the topic.

I'll say it again, build for a generic x86_64 or find the common denominator for both computers.

Best Regards,
Georgi
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2278
Location: $TERM

PostPosted: Tue Nov 26, 2024 8:30 am    Post subject: Reply with quote

Ok. Thanks for clarifying that. I will rebuild without PGO and see after I try changing my CHOST (since it's invalid configuration otherwise).
_________________
My blog
Back to top
View user's profile Send private message
dE_logics
Advocate
Advocate


Joined: 02 Jan 2009
Posts: 2278
Location: $TERM

PostPosted: Tue Nov 26, 2024 9:00 am    Post subject: Reply with quote

Even bash was failing because of that. Thanks for the pointer.
_________________
My blog
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum