QEMU/KVM questions

costel78 · Guru Joined: 20 Apr 2007 Posts: 407

I decided to buy a Raspberry Pi 4B and I want to run Gentoo on it and hope to be able to port some fpc/lazarus app on it.
Confusion come with all qemu/kvm/crossdev stuff, as I hope to compile almost everything on my desktop and use getbinpkg on Raspberry itself.
The plan is to have a qemu virtual machine using chroot and, also, use distcc to cross-compile using crossdev on the same machine (amd64, Intel 6700K CPU, 32GB RAM).

The issue is that qemu is about 8-10 time slower than normal CPU performance.
My question is: Are there *ANY* options to improve performance ? Neddy Seagoon said on this forum that QEMU/KVM performance is almost the performance of the real machine, but I have no ideea how to archive that.

Current setup:
1. app-emulation/qemu compiled with static-user useflag
2. qemu installed on chroot: ROOT=/mnt/linux/arm64 emerge --usepkgonly --oneshot --nodeps qemu
3. distcc setup both on server (my desktop) and client (qemu chroot)
4. crossdev is setup:

mike155 · Posted: Tue Aug 06, 2019 6:48 pm Post subject:

I'm afraid you can't use KVM if you want to emulate an Raspberry Pi on your x86 or amd64 desktop.

See: https://unix.stackexchange.com/questions/340912/qemu-with-kvm-with-differing-host-guess-architectures

NeddySeagoon · Posted: Tue Aug 06, 2019 7:18 pm Post subject:

costel78,

QEMU on a AMD Phenom(tm) II X6 1090T is about the same speed as a Pi3.

Once you move away from native compiling, success varies.
Pure cross compiling is fastest but not all build system support it. A few are even cross compile hostile. They attempt to execute build products during the course of the build.

Another option is cross distcc. You build on the Pi with the help of distcc. This is not 100%. A few packages build but produce broken code.
A few (like gcc) cannot use distcc.

It doesn't matter if you use QEMU in a chroot or as the basis of a KVM, the arm64 CPU is still emulated in software and its is horribly slow.

I suspect the KVM route is slowest as you will have an arm64 kernel too. In the chroot, the hosts native kernel and services are used.

Feel free to use my BINHOST to get a leg up.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

costel78 · Guru Joined: 20 Apr 2007 Posts: 407

Thank you for yours support!

@NeddySeagoon: unfortunately there are quite a few use-flags differences between your BINHOST and my setup, especially python-3.7 is required.
Can you, please, provide a list of packages which fail at runtime if they are compiled with distcc/crossdev ?
I manage to compile gcc on qemu but with --disable-bootstrap configure option and, at least until now, avoided successfully llvm, clang and rust.

Given proper cooling, how realistic is to compile entire gentoo on Raspberry Pi 4, with swap and PORTAGE_TMPDIR on external ssd, connected via USB ?
_________________
Sorry for my English. I'm still learning this language.

NeddySeagoon · Posted: Wed Aug 07, 2019 6:53 pm Post subject:

costel78,

Until the kernel is fixed the Pi4 in 64 bit mode can only use the bottom 1G RAM.
That rules out some of the bigger packages but it seems you don't want them anyway.

My Pi4 has the Power over Ethernet HAT, which incorporate a fan. There is no heatsink on the CPU.
It uses the 'On Demand' CPU governor and always runs at 1.5GHz.
The same effect should be achievable with a good passive heatsink or even on the Pi4
That's a cut down Socket5 heatsink (Pentium on AMD k6 series)

I built all of Gentoo except Rust and FIrefox on a Pi3, so it will be a bit faster on a Pi4.
I used cross distcc a lot too.

The list of things that don't work with distcc/crossdev is not fixed. You need to try it.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

costel78 · Guru Joined: 20 Apr 2007 Posts: 407

NeddySeagoon,

The PoE HAT available in my area use all GPIO pins so I will use a chipset heatsink from 486 era, about 15mm height and a noctua 40mm 5V fan. The radiator will also cover USB3 chip as I read it also get very warm.
I will try to match use-flags to be able to use your chromium and libreoffice binary packages.

Thank you for your support!
_________________
Sorry for my English. I'm still learning this language.

NeddySeagoon · Posted: Thu Aug 08, 2019 7:21 pm Post subject:

costel78,

The PoE hat covers the GPIO pins but only uses a few.
I have fitted extenders so I can still use the serial port. All the pins poke through that HAT.

If you don't have a PoE network switch, the PoE HAT is a very expensive fan.

I'm open to suggestions for USE flags. That does not mean I will follow suggestions but its always harmless to ask.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

costel78 · Guru Joined: 20 Apr 2007 Posts: 407

Good news. After I disabled all Intel's kernel mitigation I have:

NeddySeagoon · Posted: Fri Aug 09, 2019 4:16 pm Post subject:

costel78,

I try to add the ~arm64 keyword to things. That means my USE flags are a bit of a mess, so I don't mind suggestions than would make things more user friendly.

I have USE=docs set globally, which is a bid odd but I need to build test documentation
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.