| View previous topic :: View next topic |
| Author |
Message |
Siarhei
n00b
Joined: 10 Jan 2016
Posts: 73
Location: Minsk
|
 Posted: Sat Jun 29, 2024 6:43 pm Post subject: [SOLVED] My system is destroyed. /bin/sh: undefinedsymbol... |
 |
|
Hello! After today's system update I can't boot into it. Complete system failure. Got a message while loading the kernel: "/bin/sh: sysmbol lookup error: /bin/sh: undefined symbol: rl_trim_arg_fromkeyse
INIT: Entering runlevel:3". Keyboard and mouse not working. I tried to do as in the article "Fix my Gentoo". Nothing succeeded". On Reddit I found the post about bash and readline. Due to a "readline" build error, the system broke down. Help me please. 
Last edited by Siarhei on Wed Jul 03, 2024 10:34 am; edited 2 times in total |
|
| Back to top |
|
 |
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22617
|
| Posted: Sat Jun 29, 2024 6:56 pm Post subject: |
|
|
As a guess:- Your readline is installed in /usr/lib
- Your /usr is not part of /, and is not mounted by an initramfs, but instead by /etc/fstab.
Is that correct? If so, then the fix is to mount /usr before you run /sbin/init. As an ugly workaround, you could copy the required readline libraries into /lib (assuming you aren't also on a merged-usr system!) long enough to effect deeper repairs. |
|
| Back to top |
|
|
Siarhei
n00b
Joined: 10 Jan 2016
Posts: 73
Location: Minsk
|
| Posted: Sat Jun 29, 2024 7:15 pm Post subject: |
|
|
| Hu wrote: | As a guess:- Your readline is installed in /usr/lib
- Your /usr is not part of /, and is not mounted by an initramfs, but instead by /etc/fstab.
Is that correct? If so, then the fix is to mount /usr before you run /sbin/init. As an ugly workaround, you could copy the required readline libraries into /lib (assuming you aren't also on a merged-usr system!) long enough to effect deeper repairs. |
/usr is part of /. I'll try to describe in more detail what I did. After update system I did "emerge -av --depclean" as ususal. Portage suggested running the command. After rebuilding 8 packages, the system began to generate errors. When I entered the command I received the output "/bin/sh: undefined symbol: rl_trim_arg_fromkeyse". I shouldn't have restarted my computer. I have a fresh system image Calculate Linux Desktop. Maybe I can use it to restore the system? I don't want to reinstall the system, 15 years with it |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22617
|
| Posted: Sat Jun 29, 2024 8:00 pm Post subject: |
|
|
Repair is almost certainly possible, and with 15 years of history at stake, repair seems desirable.
Using a rescue image, please provide the following:- Last 100 lines of /var/log/emerge.log, so we can try to understand what you removed and what you built.
- ls -l / /usr /etc/portage/make.profile /lib*/lib*readline* /usr/lib*/lib*readline*
You may need to run those from outside the chroot, if the programs in Gentoo are badly broken. If so, adjust the paths accordingly. |
|
| Back to top |
|
|
Siarhei
n00b
Joined: 10 Jan 2016
Posts: 73
Location: Minsk
|
| Posted: Sun Jun 30, 2024 5:54 am Post subject: |
|
|
| Hu wrote: | Repair is almost certainly possible, and with 15 years of history at stake, repair seems desirable.
Using a rescue image, please provide the following:- Last 100 lines of /var/log/emerge.log, so we can try to understand what you removed and what you built.
- ls -l / /usr /etc/portage/make.profile /lib*/lib*readline* /usr/lib*/lib*readline*
You may need to run those from outside the chroot, if the programs in Gentoo are badly broken. If so, adjust the paths accordingly. |
emerge.log
https://pastebin.mozilla.org/Gmmn58td
ls -l /mnt/gentoo/
| Code: | drwxr-xr-x 2 root root 4096 июн 29 08:23 bin
drwxr-xr-x 2 root root 4096 ноя 27 2016 boot
drwxr-xr-x 2 root root 4096 окт 10 2009 cache
drwxr-xr-x 2 root root 4096 ноя 27 2016 dev
drwxr-xr-x 120 root root 12288 июн 29 09:01 etc
drwxrwxrwx 3 root root 4096 июн 30 08:05 home
srw------- 1 root root 0 янв 19 2018 kdeinit5__0
srwxr-xr-x 1 root root 0 янв 16 2017 klauncherJ17875.1.slave-socket
drwxr-xr-x 14 root root 4096 мая 18 12:26 lib
lrwxrwxrwx 1 root root 3 июн 8 2019 lib32 -> lib
drwxr-xr-x 7 root root 12288 июн 29 08:23 lib64
drwx------ 2 root root 16384 ноя 27 2016 lost+found
drwxr-xr-x 14 root root 4096 янв 29 2020 media
drwxr-xr-x 3 root root 4096 ноя 27 2022 mnt
drwxr-xr-x 3 root root 4096 сен 25 2009 net
drwxr-xr-x 13 root root 4096 мая 8 18:20 opt
dr-xr-xr-x 2 root root 4096 ноя 27 2016 proc
drwx------ 50 root root 4096 июн 29 09:12 root
drwxr-xr-x 2 root root 4096 дек 5 2019 run
drwxr-xr-x 2 root root 12288 июн 29 08:23 sbin
dr-xr-xr-x 2 root root 4096 ноя 27 2016 sys
drwxr-xr-x 5 root root 4096 дек 5 2019 tmp
drwxrwxr-x 15 portage portage 4096 июн 30 2019 usr
drwxr-xr-x 13 root root 4096 мая 8 17:40 var |
ls -l /mnt/gentoo/usr
| Code: | drwxr-xr-x 3 root root 81920 июн 29 09:01 bin
drwxr-xr-x 421 root root 53248 июн 29 08:37 include
drwxr-xr-x 3 root root 4096 апр 5 2009 kde
drwxr-xr-x 105 root root 98304 июн 29 09:01 lib
drwxr-xr-x 115 root root 200704 июн 29 09:01 lib64
drwxr-xr-x 24 root root 4096 июн 29 08:21 libexec
drwxr-xr-x 10 root root 4096 июн 8 2019 local
-rw-r--r-- 1 root root 943 дек 4 2016 my.cnf
drwxrwxr-x 175 portage portage 4096 мая 3 2020 portage
drwxr-xr-x 3 root root 4096 мар 25 2009 qt
drwxr-xr-x 2 root root 12288 июн 29 08:22 sbin
drwxr-xr-x 303 root root 12288 июн 29 08:35 share
drwxr-xr-x 7 root root 4096 июн 29 08:39 src
lrwxrwxrwx 1 root root 8 мар 22 2009 tmp -> /var/tmp
drwxr-xr-x 6 root root 4096 июн 17 2008 x86_64-pc-linux-gnu |
ls -l /mnt/gentoo/etc/portage/make.profile
| Code: | | /mnt/gentoo/etc/portage/make.profile -> ../../usr/portage/profiles/default/linux/amd64/23.0/split-usr/desktop/plasma |
ls -l /mnt/gentoo/lib*/lib*readline*
| Code: | | No such file or directory |
ls -l /mnt/gentoo/usr/lib*/lib*readline*
| Code: | lrwxrwxrwx 1 root root 16 июн 29 09:01 /mnt/gentoo/usr/lib64/libreadline.so -> libreadline.so.8
lrwxrwxrwx 1 root root 18 июн 29 09:01 /mnt/gentoo/usr/lib64/libreadline.so.8 -> libreadline.so.8.1
-rwxr-xr-x 1 root root 347120 июн 29 09:01 /mnt/gentoo/usr/lib64/libreadline.so.8.1
lrwxrwxrwx 1 root root 16 июн 29 09:01 /mnt/gentoo/usr/lib/libreadline.so -> libreadline.so.8
lrwxrwxrwx 1 root root 18 июн 29 09:01 /mnt/gentoo/usr/lib/libreadline.so.8 -> libreadline.so.8.1
-rwxr-xr-x 1 root root 327604 июн 29 09:01 /mnt/gentoo/usr/lib/libreadline.so.8.1
|
P.s. But I didn't delete anything. "depclean" gave an error and asked to run the command[/code]
| Code: | | emerge --newuse --update --autounmask=n --deep --keep-going --with-bdeps=y --quiet-build=n --regex-search-auto=y @world |
|
|
| Back to top |
|
|
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 9607
Location: beyond the rim
|
| Posted: Mon Jul 01, 2024 12:49 pm Post subject: |
|
|
Something very wrong going on there. According to emerge.log you were downgrading a ton of packages, including readline. This could very well be the reason for your problems if the mentioned symbol was introduced by that newer readline version you had installed previously. Don't know if you maybe got an outdated mirror , accidentally used an old snapshot or maybe your whole /usr tree got messed up somehow. Maybe check emerge --info for the repository timestamp. Also /mnt/gentoo/usr should not be owned by portage:portage, but that is probably just cosmetic.
In general, never ever run a large scale update without using --pretend first. If you'd see a ton of downgrades that should trigger an alarm to not proceed.
As a solution, probably best to obtain some binaries of the previously installed versions of all those downgraded packages first to fix any symbol errors, then remerge those packages one by one and making sure the correct version gets installed this time. |
|
| Back to top |
|
|
Siarhei
n00b
Joined: 10 Jan 2016
Posts: 73
Location: Minsk
|
| Posted: Mon Jul 01, 2024 2:54 pm Post subject: |
|
|
| Genone wrote: | Something very wrong going on there. According to emerge.log you were downgrading a ton of packages, including readline. This could very well be the reason for your problems if the mentioned symbol was introduced by that newer readline version you had installed previously. Don't know if you maybe got an outdated mirror , accidentally used an old snapshot or maybe your whole /usr tree got messed up somehow. Maybe check emerge --info for the repository timestamp. Also /mnt/gentoo/usr should not be owned by portage:portage, but that is probably just cosmetic.
In general, never ever run a large scale update without using --pretend first. If you'd see a ton of downgrades that should trigger an alarm to not proceed.
As a solution, probably best to obtain some binaries of the previously installed versions of all those downgraded packages first to fix any symbol errors, then remerge those packages one by one and making sure the correct version gets installed this time. |
Thank you for your advice. But I do not know how to restore the system without a working chroot. I cannot boot into the system |
|
| Back to top |
|
|
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 9607
Location: beyond the rim
|
| Posted: Mon Jul 01, 2024 3:40 pm Post subject: |
|
|
| Siarhei wrote: | | Thank you for your advice. But I do not know how to restore the system without a working chroot. I cannot boot into the system |
You don't need a working chroot for the first step. Ideally all you need is a copy of the correct libreadline.so.8.2 binary that you can put into /mnt/gentoo/usr/lib/ and change the symlink libreadline.so.8 to point to libreadline.so.8.2 instead of the 8.1 version. That should fix the "undefined symbol" problem and allow you to at least get into the chroot (unless there are more problems we can't see yet).
Obtaining said copy may or may not be tricky though if you didn't have FEATURES=buildpkg (or buildsyspkg or unmerge-backup) enabled already when that version was installed, don't know which version is available on the public Gentoo binhost.
There are other options to solve this, but this would be the least invasive. |
|
| Back to top |
|
|
Siarhei
n00b
Joined: 10 Jan 2016
Posts: 73
Location: Minsk
|
| Posted: Mon Jul 01, 2024 5:27 pm Post subject: |
|
|
| Genone wrote: | | Siarhei wrote: | | Thank you for your advice. But I do not know how to restore the system without a working chroot. I cannot boot into the system |
You don't need a working chroot for the first step. Ideally all you need is a copy of the correct libreadline.so.8.2 binary that you can put into /mnt/gentoo/usr/lib/ and change the symlink libreadline.so.8 to point to libreadline.so.8.2 instead of the 8.1 version. That should fix the "undefined symbol" problem and allow you to at least get into the chroot (unless there are more problems we can't see yet).
Obtaining said copy may or may not be tricky though if you didn't have FEATURES=buildpkg (or buildsyspkg or unmerge-backup) enabled already when that version was installed, don't know which version is available on the public Gentoo binhost.
There are other options to solve this, but this would be the least invasive. |
OK, I'll do that. I hope it helps. Thank you for your help. |
|
| Back to top |
|
|
Siarhei
n00b
Joined: 10 Jan 2016
Posts: 73
Location: Minsk
|
| Posted: Mon Jul 01, 2024 7:27 pm Post subject: |
|
|
| It didn't work out. Most likely, the ibreadline.so.8.2 is not suitable. The system won't boot anyway |
|
| Back to top |
|
|
sam_
Developer
Joined: 14 Aug 2020
Posts: 1948
|
| Posted: Mon Jul 01, 2024 10:47 pm Post subject: |
|
|
| What happened? |
|
| Back to top |
|
|
Siarhei
n00b
Joined: 10 Jan 2016
Posts: 73
Location: Minsk
|
|
| Back to top |
|
|
kimchi_sg
Advocate
Joined: 26 Nov 2004
Posts: 3038
|
|
| Back to top |
|
|
sam_
Developer
Joined: 14 Aug 2020
Posts: 1948
|
| Posted: Tue Jul 02, 2024 6:26 pm Post subject: |
|
|
| Try doing bash too. |
|
| Back to top |
|
|
Siarhei
n00b
Joined: 10 Jan 2016
Posts: 73
Location: Minsk
|
| Posted: Wed Jul 03, 2024 10:33 am Post subject: |
|
|
And I saved the system! At least I've loaded into it. I realized that I love gentoo very much.
Genone, your advice saved my nerves.
Solution: "copy of the correct libreadline.so.8.2 binary that you can put into /mnt/gentoo/usr/lib/ and change the symlink libreadline.so.8 to point to libreadline.so.8.2 instead of the 8.1 version"
It remains to bring the updates back to normal
Thank you all for your help |
|
| Back to top |
|
|
|
Portage & Programming
