| View previous topic :: View next topic |
| Author |
Message |
christoph_peter_s
Tux's lil' helper
Joined: 30 Nov 2015
Posts: 108
|
 Posted: Fri Jun 14, 2019 10:25 am Post subject: [solved] Howto run a 32bit daemon on AMD64? |
 |
|
Hi guys,
I want to use my file server as Flexlm license server. The daemons are kind of ancient...
| Code: | catilina /usr/local/flexlm/bin # file lmgrd
lmgrd: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-lsb.so.3, for GNU/Linux 2.6.9, stripped |
I have read the "AMD64/32-bit Chroot Guide" - but this leaves quite a few open questions... It states: "For the most common uses you do not need do build a 32-bit chroot system" and as an alternative: "you need to use emulation libraries to make them (32 bit applications) working"
So I got the impression, that I first should try to use the mentioned emulation libraries. But I could not find any pointer on how to do so. The latest relevant forum entry dates back to 2007 and points to a package, which seems to have vanished from the portage tree a long time ago...
I anybody out there, how can explain the steps, which should be done to make these 32 programs working.
A big Thank You & Best regards
Peter
PS: I use a plain multilib profile (default/linux/amd64/17.0 (stable)) - and IA32 emulation is enabled in the kernel...
PS2: I have tried | Code: | | ln -s ld-linux.so.2 ld-lsb.so.3 |
in lib32. But that was not sufficient. It still complained: | Code: | catilina /lib32 # /usr/local/flexlm/bin/lmgrd -h
-bash: /usr/local/flexlm/bin/lmgrd: No such file or directory |
Last edited by christoph_peter_s on Sat Jun 15, 2019 12:07 am; edited 1 time in total |
|
| Back to top |
|
 |
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54578
Location: 56N 3W
|
| Posted: Fri Jun 14, 2019 5:09 pm Post subject: |
|
|
christoph_peter_s,
The emulation libraries were the old way of doing multilib on a multilib install.
They have been gone a long time.
What does | Code: | | lddtree /usr/local/flexlm/bin/lmgrd |
tell?
Your error message means that something that /usr/local/flexlm/bin/lmgrd needs is not found.
Check that you have all the things that lddtree lists.
Its possible that you need to turn on the 32 bit versions of dependencies which you have installed as 64 bit only just now.
Thats what the ABI_X86= option does. Don not set that in make.conf.
If its missing 32 bit support on some packages that is your problem, set ABI_X86= on a per package basis.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
christoph_peter_s
Tux's lil' helper
Joined: 30 Nov 2015
Posts: 108
|
| Posted: Fri Jun 14, 2019 5:13 pm Post subject: |
|
|
First the output of lddtree:
| Code: | catilina /lib32 # lddtree /usr/local/flexlm/bin/lmgrd
/usr/local/flexlm/bin/lmgrd (interpreter => /lib/ld-lsb.so.3)
libpthread.so.0 => /lib32/libpthread.so.0
ld-linux.so.2 => /lib64/ld-linux.so.2
libm.so.6 => /lib32/libm.so.6
libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/32/libgcc_s.so.1
libc.so.6 => /lib32/libc.so.6
libdl.so.2 => /lib32/libdl.so.2 |
I am a bit clueless, on what that does imply. But there are two things, that seem curious: first the interpreter points to /lib/ld-lsb.so.3, which should be 64 bit. And so does ld_linux.so.6, which points to /lib64...
To the best of my knowledge, I have never messed around with ABI_X86
| Code: | catilina /etc/portage # cat make.conf | grep ABI
#ABI_X86="64" |
I still think, some background explanations might be helpful not only to me. Maybe there is some others out there, how want to run some ugly stuff, which is beyound Your control...
edit: I searched for the ldd linked libs...
| Code: | catilina /etc/portage # ls -l /lib64 | grep ld-lsb
catilina /etc/portage # ls -l /lib32 | grep libpthread
-rwxr-xr-x 1 root root 153236 May 22 12:36 libpthread-2.29.so
lrwxrwxrwx 1 root root 18 May 22 12:36 libpthread.so.0 -> libpthread-2.29.so
catilina /etc/portage # ls -l /lib32 | grep libm
-rwxr-xr-x 1 root root 824656 May 22 12:36 libm-2.29.so
lrwxrwxrwx 1 root root 12 May 22 12:36 libm.so.6 -> libm-2.29.so
-rwxr-xr-x 1 root root 17744 May 22 12:36 libmemusage.so
catilina /etc/portage # ls -l /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/32 | grep libgcc_s
-rw-r--r-- 1 root root 132 Feb 26 19:33 libgcc_s.so
-rw-r--r-- 1 root root 798352 Feb 26 19:33 libgcc_s.so.1
catilina /etc/portage # ls -l /lib32 | grep lic.so
catilina /etc/portage # ls -l /lib32 | grep libc.so
lrwxrwxrwx 1 root root 12 May 22 12:36 libc.so.6 -> libc-2.29.so
catilina /etc/portage # ls -l /lib32 | grep libdl.so
lrwxrwxrwx 1 root root 13 May 22 12:36 libdl.so.2 -> libdl-2.29.so |
So ld-lsb.so.3 seems missing ...
edit 2: more experimentation:
| Code: | catilina /etc/portage # ln -s /lib64/ld-linux.so.2 /lib64/ld-lsb.so.3
catilina /etc/portage # /usr/local/flexlm/bin/lmgrd -h
Segmentation fault
catilina /etc/portage # ln -s /lib32/ld-linux.so.2 /lib32/ld-lsb.so.3
catilina /etc/portage # /usr/local/flexlm/bin/lmgrd -h
Segmentation fault |
So it now seems to find everything - but maybe not the correct stuff... |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54578
Location: 56N 3W
|
| Posted: Fri Jun 14, 2019 5:59 pm Post subject: |
|
|
christoph_peter_s,
/ | Code: | lib32 # lddtree /usr/local/flexlm/bin/lmgrd
/usr/local/flexlm/bin/lmgrd (interpreter => /lib/ld-lsb.so.3)
libpthread.so.0 => /lib32/libpthread.so.0
ld-linux.so.2 => /lib64/ld-linux.so.2
libm.so.6 => /lib32/libm.so.6
libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.0/32/libgcc_s.so.1
libc.so.6 => /lib32/libc.so.6
libdl.so.2 => /lib32/libdl.so.2 |
lists the files that lmgrd depends on and in turn, the things that they depend on too.
e.g. libpthread.so.0 => /lib32/libpthread.so.0 means that libpthread.so.0 will be looked for as | Code: | | /lib32/libpthread.so.0 |
If you only have the 64 bit version it will be missing.
| Code: | | ls /lib32/libpthread.so.0 |
will tell if you have it or not.
To continue the example | Code: | | equery b libpthread.so |
will tell the package that provides the file, if its installed. In this case its sys-libs/glibc.
Next | Code: | | emerge sys-libs/glibc -pv |
will tell if you have both 64 bit and 32 bit versions. sys-libs/glibc is a bad example, On a multilib install you get both, its not optional.
Heh. Those dependencies are all provided by toolchain packages, so you will have them at one version or another.
If you have differett versions, that will be a problem.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
christoph_peter_s
Tux's lil' helper
Joined: 30 Nov 2015
Posts: 108
|
| Posted: Fri Jun 14, 2019 6:32 pm Post subject: |
|
|
Dear Neddy,
Thank You very much. I think I understand the basic procedure. And indeed as the last error message has changed from "No such file or directory" to "Segmentation fault" it looks like the necessary libraries are there. Not necessarily the correct ones, but nothing is completely missing...
| Code: | catilina /etc/portage # equery b libpthread.so
* Searching for libpthread.so ...
cross-armv6j-unknown-linux-gnueabihf/glibc-2.28-r6 (/usr/armv6j-unknown-linux-gnueabihf/usr/lib/libpthread.so -> ../../lib/libpthread.so.0)
cross-armv7a-unknown-linux-gnueabihf/glibc-2.28-r6 (/usr/armv7a-unknown-linux-gnueabihf/usr/lib/libpthread.so -> ../../lib/libpthread.so.0)
sys-libs/glibc-2.29-r2 (/usr/lib32/libpthread.so -> ../../lib32/libpthread.so.0)
sys-libs/glibc-2.29-r2 (/usr/lib64/libpthread.so -> ../../lib64/libpthread.so.0) |
Even the ones for the Raspi cross chain are there...
Glibc has the flags multiarch and multilib set... also no surprise.
One other question: these 32 bit daemons seem to be compiled with an stone old gcc 4.9.x - could the ABI changes, that we had when upgrading to gcc 5/6 be relevant?
Best regards
Peter |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54578
Location: 56N 3W
|
| Posted: Fri Jun 14, 2019 7:21 pm Post subject: |
|
|
christoph_peter_s,
Yes. The ABI changes can break things, especially the USE=pie default change.
You can't normally mix pie and non pie code.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
christoph_peter_s
Tux's lil' helper
Joined: 30 Nov 2015
Posts: 108
|
| Posted: Fri Jun 14, 2019 7:57 pm Post subject: |
|
|
Hi Neddy,
OK, so the first and slim approach seems to be dead.
Do You think, it is possible, to build a chroot using an old non-PIE profile and pre gcc-5? I don't have too many security concerns, as only that license server would run under these potentially risky conditions.
Best regards
Peter |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54578
Location: 56N 3W
|
| Posted: Fri Jun 14, 2019 8:26 pm Post subject: |
|
|
christoph_peter_s,
How much effort do you want to put into this?
I would start with an up to date chroot and switch off the default USE=pie. Then follow the news item for the pie change to turn it off again everywhere it matters.
Maybe thats enough?
There is the mole traditional approach to the chroot. You put everything in the chroot you application needs to run, then | Code: | | chroot /new/root <program> |
You don't have to build your own binaries. I think an old multilib stage3 will do. As you want a 32 bit chroot. An old 32 bit stage3 might be better.
That changes the chroot command. You need | Code: | | linux32 chroot /new/root <program>[ |
so you get a faked 32 bit kernel inside the chroot.
I have a collection of Old Gentoo stuff. It was donated by another forum contributor. I just host i
Oh, a 32 bit chroot won't have a lib32/ , its all 32 bit.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
christoph_peter_s
Tux's lil' helper
Joined: 30 Nov 2015
Posts: 108
|
| Posted: Fri Jun 14, 2019 9:55 pm Post subject: |
|
|
Some progress... I downloaded the latest i686 stage3 tarball and just followed https://wiki.gentoo.org/wiki/Project:AMD64/32-bit_Chroot_Guide, but did not update so far. Furthermore I copied that flexlm stuff into the chroot directory. Then I entered and checked:
| Code: | ulysses /usr/local/flexlm/bin # ldd lmgrd
linux-gate.so.1 (0xf7f91000)
libpthread.so.0 => /lib/libpthread.so.0 (0xf7f66000)
libm.so.6 => /lib/libm.so.6 (0xf7e6a000)
libgcc_s.so.1 => /usr/lib/gcc/i686-pc-linux-gnu/8.3.0/libgcc_s.so.1 (0xf7e4b000)
libc.so.6 => /lib/libc.so.6 (0xf7c67000)
libdl.so.2 => /lib/libdl.so.2 (0xf7c61000)
/lib/ld-lsb.so.3 => /lib/ld-linux.so.2 (0xf7f92000) |
Everything was there execpt for /lib/ld-lsb.so.3. So I had to
| Code: | | ln -s /lib/ld-linux.so.2 /lib/ld-lsb.so.3 |
And a first big success: lmgrd is starting up without a segmentation fault.
According to Neddy's advice I will now disable PIE and then do a world upgrade (or something like that). I got a pretty decent i7-2600 Sandy Bridge in my file server (and lots of RAM), so even a full world re-emerge could give me any headache. I just have to have a plan for a systematic approach. I guess, first I save a copy to a safe location, so that I cannot ruin, what I have now.
But definitely it is a good success. Even though quite a bit of research will be necessary until the license daemon is up and running as an autostarted service in that chroot...
Best regards and Many Thanks to Neddy
Peter |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54578
Location: 56N 3W
|
| Posted: Fri Jun 14, 2019 10:27 pm Post subject: |
|
|
christoph_peter_s,
Make a new chroot. Copy only the thinys you need out of the old chroot into the new one.
If it all works, job done.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
christoph_peter_s
Tux's lil' helper
Joined: 30 Nov 2015
Posts: 108
|
| Posted: Sat Jun 15, 2019 12:12 am Post subject: |
|
|
(a lazy) success!
Didn't change anything in the chroot. No update, nothing.
I just fixed my start-and-stop script for the license daemon - and it just works.
I will wait until I have a backup (runs only once a day), and then I might try the update of the chroot.
I am _very_ happy that I got my license server up again...
PS: this is what I love Gentoo for: I can make everything working. it might take a while, it might be tough to learn what's necessary - but in the end there seems to be a solution for every problem...
Last edited by christoph_peter_s on Sat Jun 15, 2019 12:48 am; edited 1 time in total |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22696
|
| Posted: Sat Jun 15, 2019 12:46 am Post subject: |
|
|
| Normally, I discourage people from freezing a system at a point in time. However, in this case, freezing the chroot may be a better option. Presumably, you are restricting access to the license manager in a way that prevents attempts to exploit it, so updates to bring in hardening changes are unnecessary. You aren't offering any services other than the license manager, and since you cannot rebuild it to adapt to any changes brought in through updates, there is limited value in updating supporting libraries. |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54578
Location: 56N 3W
|
| Posted: Sat Jun 15, 2019 11:37 am Post subject: |
|
|
christoph_peter_s,
What Hu said.
My suggestion to create a new stripped down chroot will minimise the the attack surface presented by the old software as you won't have out of date junk lying around.
However, I'm more paranoid than some.
If everything is locked down, there is no external access, that may be good enough for you.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
christoph_peter_s
Tux's lil' helper
Joined: 30 Nov 2015
Posts: 108
|
| Posted: Sat Jun 15, 2019 11:49 am Post subject: |
|
|
Hi Neddy,
I followed most of Your advice (mostly on Arm, e.g. using distcc and the package.env mechanism). So I am pretty inclined to follow You here to. But I need some pointer on how to start. If I understand You right, then You propose to uninstall a lot of the stuff, that came with the stage3 tarball. Is there any list, on what should be there. Or even better: what should be ripped off first.
But the next thing I do, is writing a small tut on how to tame an Asus P8Z68 mobo, which refused to switch off - until I fixed the DSDT...
Best regards
Peter |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54578
Location: 56N 3W
|
| Posted: Sat Jun 15, 2019 12:12 pm Post subject: |
|
|
christoph_peter_s,
You need your application and everything listed in the output of lddtree. That should be about all.
You may need /proc, /dev and friends.
That's why I suggested you create a new chroot and copy things over. Its not much and if I'm wrong and you do need other things, you still have them to hand.
Its not a case of just copying the files. You need to reproduce the directory structure inside the chroot too.
e.g. | Code: | | mkdir /usr/local/flexlm2 |
will be your new chroot.
You need | Code: | | libpthread.so.0 => /lib/libpthread.so.0 (0xf7f66000) |
So that goes into | Code: | | /usr/local/flexlm2/lib/libpthread.so.0 |
and so on.
If there are symbolic links pointing to /lib/libpthread.so.0, you may need them, you may not.
If lmgrd has any config files in /etc ... or anywhere else, they will be handy.
The idea in to create a chroot that only contains the needed files for the licence server and nothing else.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
christoph_peter_s
Tux's lil' helper
Joined: 30 Nov 2015
Posts: 108
|
| Posted: Sat Jun 15, 2019 12:46 pm Post subject: |
|
|
Hi Neddy,
ah, now I understand, what You were trying to propose!
And You say, that these files laying around in the chroot could possibly be helpful to someone, who exploits a vulnerability in that crappy flexlm daemon?! Well from my own experience I know, that with early versions of Flexlm (v6.x or lower) it took less then ten minutes to derive a keygen from the daemon. Well it took a bit of training to be that fast, but is was so incredibly easy. So basically I think that all the flexlm stuff is really crappy. And I really doubt, that anybody at Globetrotter (and what came after it) really cared on buffer overflows or other security related aspects...
So I have no doubts, that if some talented cracker starts attacking a server through the flexlm service, then he will succeed. If however somebody is so deep within my network, then most likely all is lost already. Well, I try to be a too easy target anyway. The most valuable box is the backup server. Maybe I have to work a bit on the security of that box (not that it should be too lousy, but one can always do better). In a real worst case scenario the backup server should stay clean if the other boxes were compromised.
But I disgress...
Best regards
Peter |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54578
Location: 56N 3W
|
| Posted: Sat Jun 15, 2019 1:54 pm Post subject: |
|
|
christoph_peter_s,
If someone breaks into a standard stage3 chroot via flexlm, they will find a complete working toolchain to help them break out of the chroot.
If they are stuck in the chroot with very little, it gets harder.
It all depends on your level of paranoia and your entire security setup. Its your call if its worth the effort in the context of your environment.
Security is not to keep the resourceful, determined well funded targeted attacker out. They will use other ways anyway.
Its the layers of an onion to deter script kiddies and opportunistic intruders. The idea is to make them give up and find an easier target.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
|
Gentoo on AMD64
