| View previous topic :: View next topic |
| Author |
Message |
Saundersx
Apprentice
Joined: 11 Apr 2005
Posts: 294
|
 Posted: Tue Jan 23, 2018 9:34 pm Post subject: objtool problems when compiling newest kernel |
 |
|
I compile my own kernel (as opposed to genkernel) and these started popping up recently (4.14.14-15)
| Code: | CC arch/x86/kernel/signal.o
arch/x86/kernel/signal.o: warning: objtool: do_signal()+0x78: return with modified stack frame
CC arch/x86/kernel/signal_compat.o
arch/x86/kernel/signal_compat.o: warning: objtool: copy_siginfo_to_user32()+0x21: sibling call from callable instruction with modified stack frame
CC arch/x86/kernel/traps.o
CC arch/x86/kernel/idt.o
arch/x86/kernel/idt.o: warning: objtool: idt_setup_from_table()+0xb3: stack state mismatch: reg1[3]=-1+0 reg2[3]=6-16
CC arch/x86/kernel/irq.o
CC arch/x86/kernel/irq_64.o
CC arch/x86/kernel/dumpstack_64.o
CC arch/x86/kernel/time.o
CC arch/x86/kernel/ioport.o
CC arch/x86/kernel/dumpstack.o
arch/x86/kernel/dumpstack.o: warning: objtool: __die()+0xa9: return with modified stack frame
CC arch/x86/kernel/nmi.o
CC arch/x86/kernel/ldt.o
/bin/sh: line 1: 21423 Segmentation fault ./tools/objtool/objtool orc generate --no-fp "arch/x86/kernel/ldt.o"
make[2]: *** [scripts/Makefile.build:321: arch/x86/kernel/ldt.o] Error 139
make[1]: *** [scripts/Makefile.build:579: arch/x86/kernel] Error 2
make: *** [Makefile:1028: arch/x86] Error 2
|
I have an Intel laptop and it happily compiles away without even so much as a warning. Reading into it I found this https://bugs.gentoo.org/642924 but after some testing it appears that march=native is not sneaking through. However comment #15 https://bugs.gentoo.org/642924#c15 implies this is an AMD issue. After further digging I found there are more objtool patches in the pipeline that haven't made their way into the kernel yet https://lkml.org/lkml/2018/1/23/450
However this install is certainly showing it's age, I use distcc and previously ccache/cross-compiling in the past so there may be some orphaned files floating around causing problems.
Simple question before I end up spending hours doing an unnecessary deep dive into my setup: Does anyone else with AMD (particularly piledriver) have these issues? |
|
| Back to top |
|
 |
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Tue Jan 23, 2018 9:45 pm Post subject: |
|
|
| Code: | | binutils-config --linker ld.bfd |
better? |
|
| Back to top |
|
|
Saundersx
Apprentice
Joined: 11 Apr 2005
Posts: 294
|
| Posted: Tue Jan 23, 2018 9:58 pm Post subject: |
|
|
| Nope, same warnings/error. |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Tue Jan 23, 2018 10:29 pm Post subject: |
|
|
| then disable CONFIG_STACK_VALIDATION while waiting for the patch. |
|
| Back to top |
|
|
Saundersx
Apprentice
Joined: 11 Apr 2005
Posts: 294
|
| Posted: Tue Jan 23, 2018 10:33 pm Post subject: |
|
|
objtool crashing probably isn't an issue (I hope) so I just kept running make -j1 until it finished. One more thing failed
| Code: | | /bin/sh: line 1: 23672 Segmentation fault ./tools/objtool/objtool orc generate --no-fp "mm/memory.o" |
Just going to run this kernel as is. |
|
| Back to top |
|
|
Saundersx
Apprentice
Joined: 11 Apr 2005
Posts: 294
|
| Posted: Tue Jan 23, 2018 10:34 pm Post subject: |
|
|
| Just saw your post after mine, I'll try that next time if .16 comes out first. |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Tue Jan 23, 2018 10:38 pm Post subject: |
|
|
the real answer now 
| Quote: | commit f41b2d7ee791e845147a3e3cafb25493dca6870a
Author: Josh Poimboeuf <jpoimboe@redhat.com>
Date: Mon Jan 15 08:17:07 2018 -0600
objtool: Fix seg fault with gold linker
commit 2a0098d70640dda192a79966c14d449e7a34d675 upstream.
Objtool segfaults when the gold linker is used with
CONFIG_MODVERSIONS=y and CONFIG_UNWINDER_ORC=y.
With CONFIG_MODVERSIONS=y, the .o file gets passed to the linker before
being passed to objtool. The gold linker seems to strip unused ELF
symbols by default, which confuses objtool and causes the seg fault when
it's trying to generate ORC metadata.
Objtool should really be running immediately after GCC anyway, without a
linker call in between. Change the makefile ordering so that objtool is
called before the linker.
|
Wait for it, in case you want it already: https://cdn.kernel.org/pub/linux/kernel/v4.x/ChangeLog-4.14.15 |
|
| Back to top |
|
|
Adarion
Tux's lil' helper
Joined: 22 Aug 2005
Posts: 78
|
| Posted: Fri Feb 02, 2018 1:23 pm Post subject: |
|
|
Strange. I just pulled in 4.14.16 and objtool still messes up totally.
stack state mismatch
can't decode instruction
return with modified stack frame
can't find jump dest instruction at
sibling call from callable instruction with modified stack frame
Error messages all over it.
In the end it segfaults. 
objtool belongs to the kernel sources. Did the kernel devs not notice anything (in the linked changelog some fix was announced for 4.14.15 - but I'm with 4.14.16 here)? Can't be just my machine, right?
All regardless of gcc version (6.4.0-r1 or 7.3.0 (should have retpoline support)). So since 4.14.13 I was unable to compile ANY new kernel, which really sucks. I don't need KPTI (all AMD and/or ancient machines that aren't affected) but retpolines and stuff would be nice for a start. I do NOT want 4.4.xxx or 4.9.xx since those usually don't have recent amdgpu/radeon stack and the likes.
Is there anything as of early February that a Gentoo user can do about it?
_________________
stop tcpa, swpatents, corrupt politicians and other scary stuff |
|
| Back to top |
|
|
Tyler_Durden
Apprentice
Joined: 27 Jul 2004
Posts: 189
Location: Germany
|
| Posted: Fri Feb 02, 2018 3:45 pm Post subject: |
|
|
| Adarion wrote: | Strange...
Is there anything as of early February that a Gentoo user can do about it? |
4.15 is not an option? switch to the bfd linker does not help?
_________________
Gentoo 17.1 x86_64
Intel Core i9-9900K
Asus MAXIMUS XI HERO
AMD Radeon 6800XT
64GB DDR4
Samsung SSD 970 EVO Plus 1TB
8x Seagate Archive (SATA-RAID 64TB)
Digital Devices Cine S2 V6.5 DVB Adapter |
|
| Back to top |
|
|
Tony0945
Watchman
Joined: 25 Jul 2006
Posts: 5127
Location: Illinois, USA
|
|
| Back to top |
|
|
assemblerhead
n00b
Joined: 23 Jan 2016
Posts: 36
Location: TX,US
|
| Posted: Sat Feb 03, 2018 1:44 am Post subject: Thanks, a workaround that works. :D |
|
|
Bug in Question :
https://bugs.gentoo.org/645774
Switching the linker to ld.bfd and disabling "[ ] Compile-time stack metadata validation" got rid of the errors.
Also disabled :
[ ] Remove the kernel mapping in user mode
[ ] Avoid speculative indirect branches in kernel
The kernel compiled with AMD Piledriver set as CPU.
The 'Spectre / Meltdown' cure is a mess of conflicting patches it seems.
Thanks to the person who posted a link in/at the bug report. |
|
| Back to top |
|
|
Tony0945
Watchman
Joined: 25 Jul 2006
Posts: 5127
Location: Illinois, USA
|
| Posted: Sat Feb 03, 2018 11:01 am Post subject: Re: Thanks, a workaround that works. :D |
|
|
| assemblerhead wrote: | disabling "[ ] Compile-time stack metadata validation" got rid of the errors.
Also disabled :
[ ] Remove the kernel mapping in user mode
[ ] Avoid speculative indirect branches in kernel
|
These are respectively:
CONFIG_STACK_VALIDATION
CONFIG_PAGE_TABLE_ISOLATION
CONFIG_RETPOLINE
STACK_VALIDATION can't be directly altered by make menuconfig.
| Code: | Symbol: STACK_VALIDATION [=y] │
│ Type : boolean │
│ Prompt: Compile-time stack metadata validation │
│ Location: │
│ -> Kernel hacking │
│ -> Compile-time checks and compiler options │
│ Defined at lib/Kconfig.debug:369 │
│ Depends on: HAVE_STACK_VALIDATION [=y] │
│ Selected by: UNWINDER_ORC [=y] && <choice> && X86_64 [=y] |
Selecting any of the three unwinders doesn't change STACK_VALIDATION which is enabled by HAVE_STACK_VALIDATION
| Code: | Symbol: HAVE_STACK_VALIDATION [=y] │
│ Type : boolean │
│ Defined at arch/Kconfig:798 │
│ Selected by: X86 [=y] && X86_64 [=y] |
How did you shut that off? Direct editing of .config? Or is CONFIG_X86 turned off in non-multilib profiles?
[Moderator edit: changed [quote] tags to [code] tags to preserve output layout. -Hu] |
|
| Back to top |
|
|
philmiv
n00b
Joined: 18 Jan 2011
Posts: 8
|
| Posted: Sat Feb 03, 2018 3:07 pm Post subject: |
|
|
I just managed to compile the 4.15 kernel without objtool segfaults / warnings by setting only:
Processor type and features -> Processor family -> Generic_x86_64 (previously it was AMD_Excavator)
I did that since I want to try the ORC unwinder, which depends on stack validation. |
|
| Back to top |
|
|
assemblerhead
n00b
Joined: 23 Jan 2016
Posts: 36
Location: TX,US
|
| Posted: Sat Feb 03, 2018 4:09 pm Post subject: The requested info ... |
|
|
@ Tony0945
Verified they are Disabled :
[ ] Compile-time stack metadata validation
[ ] Remove the kernel mapping in user mode
[ ] Avoid speculative indirect branches in kernel
Copied this direct from the .config :
| Code: |
# Kernel Performance Events And Counters
CONFIG_HAVE_STACK_VALIDATION=y
|
| Code: |
# Compile-time checks and compiler options
# CONFIG_STACK_VALIDATION is not set
|
No direct editing of the .config file was done.
The kernel compiled ... I am using it, no problems so far.
Also ...
I have sys-libs/binutils-libs USE="64-bit-bfd multitarget" installed as well.
Don't know if that makes a difference. |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 23082
|
| Posted: Sat Feb 03, 2018 5:41 pm Post subject: |
|
|
| Tony0945 wrote: | | STACK_VALIDATION can't be directly altered by make menuconfig. |
That depends on other configuration options. With the right combination of other menuconfig choices, STACK_VALIDATION can be changed in menuconfig. | Tony0945 wrote: | | Code: |
│ Selected by: UNWINDER_ORC [=y] && <choice> && X86_64 [=y] |
Selecting any of the three unwinders doesn't change STACK_VALIDATION which is enabled by HAVE_STACK_VALIDATION
|
I get the same output, and that && <choice> && piece is confusing. However, you have the dependency relations a bit confused. You stated if HAVE_STACK_VALIDATION, then force STACK_VALIDATION. As I understand it, the relation is actually, if !HAVE_STACK_VALIDATION, then force !STACK_VALIDATION; if HAVE_STACK_VALIDATION, then permit (but do not force) STACK_VALIDATION.
If UNWINDER_ORC is enabled (and your output shows it is enabled for you), then that forces STACK_VALIDATION on. If you choose a different unwinder, you will be permitted to disable STACK_VALIDATION. I confirm that it does not automatically disable STACK_VALIDATION when a different unwinder is selected.
| Tony0945 wrote: | | How did you shut that off? Direct editing of .config? Or is CONFIG_X86 turned off in non-multilib profiles? |
Select an unwinder other than UNWINDER_ORC. Go to STACK_VALIDATION. Choose N.
CONFIG_X86 should be enabled for both x86 and amd64 profiles, both multilib and no-multilib (but at least on vanilla-sources, the kernel is wholly unaware of your profile; I think that applies to gentoo-sources also). The no-multilib profiles can be used without CONFIG_IA32_EMULATION, but CONFIG_X86 is a general flag indicating you are on an Intel/AMD/etc. family, rather than ARM, SPARC, etc. |
|
| Back to top |
|
|
assemblerhead
n00b
Joined: 23 Jan 2016
Posts: 36
Location: TX,US
|
| Posted: Sat Feb 03, 2018 9:49 pm Post subject: |
|
|
Just thought of this ... should have explained this in an earlier posting.
I modify the vendor support for CPUs in 'menuconfig'
A good number of the config choices change when you enable expert config in 'make menuconfig'
Enable expert config in 'make menuconfig'
| Code: |
[*] Configure standard kernel features (expert users) --->
|
Then modify this :
| Code: |
[*] Supported processor vendors --->
--- Supported processor vendors
[ ] Support Intel processors
[*] Support AMD processors
[ ] Support Centaur processors
|
I was surprised at the difference that made. 'dmesg' output looks a lot different.
A lot easier to understand where the problems are. |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Sat Feb 03, 2018 11:19 pm Post subject: Re: Thanks, a workaround that works. :D |
|
|
| assemblerhead wrote: | | Thanks to the person who posted a link in/at the bug report. |
You're welcome  |
|
| Back to top |
|
|
Tony0945
Watchman
Joined: 25 Jul 2006
Posts: 5127
Location: Illinois, USA
|
| Posted: Sat Feb 03, 2018 11:49 pm Post subject: |
|
|
| philmiv wrote: | Processor type and features -> Processor family -> Generic_x86_64 (previously it was AMD_Excavator)
I did that since I want to try the ORC unwinder, which depends on stack validation. |
My Kaveri should have built with excavtor also but Bulldozer (bdver1) was as high as it would go.
I went the route you did, starting with generic and ending at bulldozer. Same with Bristol Ridge.
Should beat generic. |
|
| Back to top |
|
|
Tony0945
Watchman
Joined: 25 Jul 2006
Posts: 5127
Location: Illinois, USA
|
| Posted: Sun Feb 04, 2018 12:53 am Post subject: |
|
|
| Hu wrote: | If UNWINDER_ORC is enabled (and your output shows it is enabled for you), then that forces STACK_VALIDATION on. If you choose a different unwinder, you will be permitted to disable STACK_VALIDATION. I confirm that it does not automatically disable STACK_VALIDATION when a different unwinder is selected.
| Tony0945 wrote: | | How did you shut that off? Direct editing of .config? Or is CONFIG_X86 turned off in non-multilib profiles? |
Select an unwinder other than UNWINDER_ORC. Go to STACK_VALIDATION. |
That works. Thanks, again, Hu. Once I did that, CONFIG_MNATIVE worked fine.
It is a bit of a hack because the stack validation is a good thing. Detailed explanation is found in /usr/src/linux/tools/objtool/Documentation/stack-validation.txt
I see what the objtool talk is all about and it seems likely that recent patches have neglected the macro in places that bdver1 and below don't reach.
EDIT: Surprisingly, the altered kernel seems to boot noticeably slower. That is, OpenRC is slower. The messages used to race across the screen too fast to read on the Bristol Ridge.
Maybe gcc-6.4.0 's Bristol Ridge optimizations aren't as good as the BDVER1 optimizations. |
|
| Back to top |
|
|
Adarion
Tux's lil' helper
Joined: 22 Aug 2005
Posts: 78
|
| Posted: Thu Feb 22, 2018 4:42 pm Post subject: |
|
|
| Tyler_Durden wrote: | | Adarion wrote: | Strange...
Is there anything as of early February that a Gentoo user can do about it? |
4.15 is not an option? switch to the bfd linker does not help? |
Yes, but usually I wait for a few point releases until the worst fallout of a new major kernel release has settled.
Selecting anything else than ORC unwinder did not help (frame pointer unwinder). By the way, this is also located in kernel hacking and ORC seems to be the (recommended) default.
Also in terms of any further GCC settings, linker settings and so on, I usually stay with the defaults (unless I think I know what I am doing, and I do not have that feeling here ).
Today I compiled successfully on my Kabini system - what actually seemed to help (gcc 6.4.0-r1) was to switch CPU type in the Kernel config to amd64 generic (x86_64) - no specific CPU type. Voila! It compiled, I booted it and I had all the fancy new stuff in. Still only moderate Spectre countermeasures since I wasn't using gcc 7.3.x for that, but okay.
At the very moment my AMD FX-6300 is also compiling gentoo-sources:4.15.4 with the same generic x86_64 setting and NOT complaining (and seems regardless of other settings so far)! (Please stay so!)
Moments ago I tried with "gcc autodetect native CPU" setting in CPU type and it failed again with countless error messages littered along the compilation process.
So for me the kernel config
Processor type and features -> Processor family -> Generic-x86-64
aka CONFIG_GENERIC_CPU [=y]
"solved" the issue. Of course I am not entirely happy to run something that might be using no optimizations, but well, it's a start.
I'll give it a try with gcc 7.3.x later.
_________________
stop tcpa, swpatents, corrupt politicians and other scary stuff |
|
| Back to top |
|
|
Tony0945
Watchman
Joined: 25 Jul 2006
Posts: 5127
Location: Illinois, USA
|
| Posted: Thu Feb 22, 2018 7:18 pm Post subject: |
|
|
| Adarion wrote: | | Selecting anything else than ORC unwinder did not help (frame pointer unwinder). By the way, this is also located in kernel hacking and ORC seems to be the (recommended) default. . |
What's needed is to remove unwinding entirely. Which is a little higher in the menu.
https://cateee.net/lkddb/web-lkddb/STACK_VALIDATION.html |
|
| Back to top |
|
|
BGazotti
n00b
Joined: 10 Dec 2021
Posts: 1
Location: Brazil
|
| Posted: Fri Dec 10, 2021 1:37 pm Post subject: |
|
|
| I'm somewhat sorry for reviving this thread, but just wanted to let y'all know that this issue is back in 5.15. Using -march=native or -march=bdver2 on my FX-6300 leads to tons of objtool warnings and an unbootable kernel. I'm compiling it again with -march=bdver1 and so far there's no warnings being produced, so I'm optimistic about that. Curiously, 5.14 didn't have that problem. I'll also go to kernel.org and report a bug there. |
|
| Back to top |
|
|
sam_
Developer
Joined: 14 Aug 2020
Posts: 2120
|
| Posted: Fri Dec 10, 2021 5:11 pm Post subject: |
|
|
| BGazotti wrote: | | I'm somewhat sorry for reviving this thread, but just wanted to let y'all know that this issue is back in 5.15. Using -march=native or -march=bdver2 on my FX-6300 leads to tons of objtool warnings and an unbootable kernel. I'm compiling it again with -march=bdver1 and so far there's no warnings being produced, so I'm optimistic about that. Curiously, 5.14 didn't have that problem. I'll also go to kernel.org and report a bug there. |
Could you update this post with a link to the bug when you report it?
Thanks! |
|
| Back to top |
|
|
Goverp
Advocate
Joined: 07 Mar 2007
Posts: 2202
|
| Posted: Fri Dec 10, 2021 6:46 pm Post subject: |
|
|
Interesting. I had the problem on my bdver4 laptop some time back, and cured it with "mcpu=bdver2 mtune=bdver4"! Works fine on kernel 5.15.7 ...
_________________
Greybeard |
|
| Back to top |
|
|
Goverp
Advocate
Joined: 07 Mar 2007
Posts: 2202
|
| Posted: Fri Oct 06, 2023 1:25 pm Post subject: |
|
|
It's back again!
I built kernel 6.5.5 with -march=bdver4, and couldn't boot the result. I tried again and only just noticed an objtool message whizzing past. Retry with
| Code: | make clean
make > make.log |
made it much easier to see a screenful of stderr messages.
Following BGazotti's post, I've just compiled 6.5.6 (which showed the same problem) with
| Code: | | -march=bdver1 -mtune=bdver4 |
and that cures the messages. Haven't tried booting with it yet.
Note I can't say if the problem was or wasn't there earlier; due to finger trouble I'd been mistakenly using "-mtune=bdver4" instead of "-march=bdver4" for some time before 6.5.5.
<edit>Yup, that boots OK.
_________________
Greybeard |
|
| Back to top |
|
|
|
Kernel & Hardware
