| View previous topic :: View next topic |
| Author |
Message |
LinuxEnjoyer
n00b
Joined: 15 Mar 2023
Posts: 5
|
 Posted: Wed Mar 15, 2023 4:30 pm Post subject: [SOLVED] Podman can't run foreign architecture images |
 |
|
Hello, I hope I put this in the right place. x
I can't run foreign architecture images with podman (test command from arch docs here https://wiki.archlinux.org/title/Podman#Foreign_architectures):
| Code: |
[user@machine /home/user]# podman run --arch aarch64 'docker.io/alpine:latest' arch
{"msg":"exec container process (missing dynamic library?) `/bin/arch`: No such file or directory","level":"error","time":"2023-03-15T16:20:51.000472949Z"}
|
That's using crun, using runc gives the same error, but without the (missing dynamic library?) extra information.
Seems like a classic case of not-really-statically-linked-although-it-should-be, but I can't figure out why it happens.
The same thing happens when I try to build an image for a foreign architecture, arm64 specifically (host is amd64, yes podman and docker both accept arm64 and aarch64 interchangeably).
| Code: |
[user@machine /home/user]# emerge -pv qemu
These are the packages that would be merged, in order:
Calculating dependencies... done!
Dependency resolution took 1.13 s.
[ebuild R ] app-emulation/qemu-7.2.0::gentoo USE="aio alsa bzip2 curl fdt filecaps gnutls gtk jpeg ncurses nls opengl oss pam pin-upstream-blobs png pulseaudio sdl slirp smartcard static-user udev usb vhost-net vnc xattr -accessibility -bpf (-capstone) -debug -doc -fuse -glusterfs -infiniband -io-uring -iscsi -jack -jemalloc -lzo -multipath -nfs -numa -plugins -python -rbd -sasl -sdl-image (-selinux) -snappy -spice -ssh -static -systemtap -test -usbredir -vde -virgl -virtfs -vte -xen -zstd" PYTHON_TARGETS="python3_10 -python3_9 -python3_11" QEMU_SOFTMMU_TARGETS="aarch64 arm x86_64 -alpha -avr -cris -hppa -i386 -loongarch64 -m68k -microblaze -microblazeel -mips -mips64 -mips64el -mipsel -nios2 -or1k -ppc -ppc64 -riscv32 -riscv64 -rx -s390x -sh4 -sh4eb -sparc -sparc64 -tricore -xtensa -xtensaeb" QEMU_USER_TARGETS="aarch64 aarch64_be arm x86_64 -alpha -armeb -cris -hexagon -hppa -i386 -loongarch64 -m68k -microblaze -microblazeel -mips -mips64 -mips64el -mipsel -mipsn32 -mipsn32el -nios2 -or1k -ppc -ppc64 -ppc64le -riscv32 -riscv64 -s390x -sh4 -sh4eb -sparc -sparc32plus -sparc64 -xtensa -xtensaeb" 0 KiB
Total: 1 package (1 reinstall), Size of downloads: 0 KiB
|
Qemu is built with static user.
| Code: |
[user@machine /home/user]# ls -lah /proc/sys/fs/binfmt_misc
total 0
drwxr-xr-x 2 root root 0 Mar 15 15:04 .
dr-xr-xr-x 1 root root 0 Mar 15 15:04 ..
-rw-r--r-- 1 root root 0 Mar 15 15:04 qemu-aarch64
-rw-r--r-- 1 root root 0 Mar 15 15:04 qemu-aarch64_be
-rw-r--r-- 1 root root 0 Mar 15 15:04 qemu-arm
--w------- 1 root root 0 Mar 15 15:04 register
-rw-r--r-- 1 root root 0 Mar 15 15:04 status
|
binfmt_misc has the appropriate entries, kernel can run misc binaries, podman can run amd64 images but not aarch64 images.
Does anyone know I've forgotten that makes me unable to run foreign architecture images through podman?
Last edited by LinuxEnjoyer on Wed Apr 05, 2023 11:09 am; edited 1 time in total |
|
| Back to top |
|
 |
pingtoo
l33t
Joined: 10 Sep 2021
Posts: 932
Location: Richmond Hill, Canada
|
| Posted: Wed Mar 15, 2023 6:25 pm Post subject: |
|
|
LinuxEnjoyer,
I think the article you pointed are missing leading. It missing a step of | Code: | | podman cp /usr/bin/qemu-aarch64 image:/usr/bin/qemu-aarch64. |
According to pod man page | Code: | --arch=ARCH
Override the architecture, defaults to hosts, of the image to be pulled. For
example, arm. Unless overridden, subsequent lookups of the same image in
the local storage will match this architecture, regardless of the host. |
is for pulling architecture specificed image. It is not used running the image. |
|
| Back to top |
|
|
LinuxEnjoyer
n00b
Joined: 15 Mar 2023
Posts: 5
|
| Posted: Wed Apr 05, 2023 11:09 am Post subject: |
|
|
I managed to solve this, the problem came from strange default binfmt flags as well as a seemingly broken qemu-aarch64[ static binary as well as default config that's not friendly to containers.
First off, the default `OC`-flags causes issues when running in containers, when those were changed to `FP` I instead got segfaults, which was an improvement, the binary was at least available to the container.
Editing the docker image shouldn't be necessary with a properly configured binfmt, and indeed no distro suggests that. It seems that the ebuild for the binary is busted or at least broken-by-default for purposes of using it for running in containers.
So the solution is to get a working version of qemu-<arch>-static, can easily be sourced from the Arch repos for example, move that into /usr/bin/qemu-aarch64. Edit /etc/init.d/qemu-binfmt and change flags from OC to FP, then restart the service. |
|
| Back to top |
|
|
pingtoo
l33t
Joined: 10 Sep 2021
Posts: 932
Location: Richmond Hill, Canada
|
| Posted: Wed Apr 05, 2023 3:21 pm Post subject: |
|
|
LinuxEnjoyer,
I am interesting to learn more about the binfmt, can you share some pointers about the 'FP' flags? where can I find the reference about the "flags" field in binfmt and why 'FP' will work.
Thanks. |
|
| Back to top |
|
|
LinuxEnjoyer
n00b
Joined: 15 Mar 2023
Posts: 5
|
| Posted: Thu Apr 06, 2023 3:56 pm Post subject: |
|
|
I think the best source is the kernel docs here https://docs.kernel.org/admin-guide/binfmt-misc.html.
I tried a few different versions, but as makes sense when checking out those docs, having the `F` flag is necessary for this to work. Exactly why is beyond me, I'm guessing how docker/podman sets up their virtualization, I'm guessing that they're doing some mounts/chroots that makes the binary unavailable under some circumstances.
I'm using `OCF` as flags now and that works fine! |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Other Things Gentoo
