Use crossdev to directly update other system's packages

[M95D]

Hi.

I have a very low power devboard (RPi3 clone) and a normal desktop. I run Gentoo on the desktop and I also tried to run it on the devboard, but emerging packages is simply too slow and there's not enough RAM.
Gentoo on the devboard is installed on a micro-SD card.
Can I mount the SD card in some directory on the desktop (like /usr/<arch>) and use crossdev to update packages directly on the SD card without any binpkg or distcc?

Thanks.

[pingtoo]

M95D,

[M95D]

No network mount. Turn off the devboard, take the card out, put it in the desktop's card reader, mount the card into some directory and run crossdev/emerge/something to directly update packages that are installed on the card and install new ones.

Chroot can't be used because the desktop machine is x86 and the devboard is arm. The desktop won't be able to run gcc or any other native binaries from the card without emulation. Qemu and emulations are slow. I would prefer to use crossdev or some other cross-toolchain to build packages directly from the host Gentoo system.

https://wiki.gentoo.org/wiki/Cross_build_environment describes how to build a base system. It doesn't say if the same commands (especially <arch>-emerge) can be used to update a system that was previously used (no longer "clean" - user added files, various packages installed, config files edited, defaults changed, etc.).

I can live with the fact that some packages can't be built on a x86 host. I want to be able to update at least the very large packages like llvm, gcc, libre/openoffice and firefox.

[pingtoo]

M95D,

Thanks for the clarification.

Just like /proc, /sys and /dev, chrossdev install everything in /usr/<arch> it can be bind-mounted into <chroot>/usr/<arch>.

Have a look of this gentoo wiki about cross container It is very out dated. but the concept is correct you can follow it in principle. You don't have to use container. Although personally I prefer container over chroot but this is matter of taste.

just a idea, may be you can just dump the SD as image (dd if=/dev/<sdcard> of=/path/to/sd-image bs=1M) mount the image and work on it. Once you are done, you re-image it back to SD.

[NeddySeagoon]

M95D,

Crossdev won't build everything.

Cross compiling requires a few changed lines in make.conf from native but otherwise, your idea works.
Crossdev will expect to build into /usr/<tuple>

Chroot works too ... sort of.
You need QEMU built with USE=static-user and host binfmt support.
Install the QEMU binary into the target root. It will emulate the target CPU to run arm code on the host.
As its register level hardware emulation, its slow, but you can use all of the hosts resources.

See Further Reading for your options and more details.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[M95D]

[NeddySeagoon]

M95D,

/usr/<Target_CHOST>/ is your RPi3 clone host target root filesystem.
Copy its etc/portage make.conf somewhere safe because you need to read it later.

Mount you SD card filesystem tree rooted at /usr/<Target_CHOST>/

Compare its make.conf with the one you saved and make the adjustments required for cross compiling.

I use two make.conf files one called make.conf_native and the other, make.conf_cross, with a symbolic link called make.conf to choose between them.

When you call

[M95D]

I tried. It worked, but for very few packages.
Worked: gcc, llvm
Didn't work: rust, all packages dependent on rust (such as librsvg), perl, nodejs (really dissapointed by this!).

Because of librsvg, I couldn't even get as far as having a working X.

I put the card back into the devboard and it will probably take a week to build nodejs.

[pingtoo]

I wonder what failed.

I can think of why for perl, it is because the perl build system require targeted mini-perl (or something alike) to complete. But not sure if other is similar.

Did you build QEMU with static-user? and setup binfmt? if you do then may be we can help to find out what else make the build fail.

[NeddySeagoon]

M95D,

Share a build log and your target system specs.

gcc knows to not bootstrap when cross compiling.
Perl is cross compiling hostile.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[ddavidson72]

Thank you people for this work. Based on what I am reading, I think am trying to accomplish the exact same objectives. I had the same problems, and I learned how to solve most of them from your posts here. Thanks again.

I am also following the instructions in the crossdev wiki. There seems to be the superceding article for the aforementioned crossdev build article, but I have also reviewed this one extensively. The crossdev wiki article I am reading is here:
https://wiki.gentoo.org/wiki/Crossdev

I think I almost have something working. The thing I am still confused about is how to manage the different and differing portage repositories and profiles among the 2 platforms. For example, it feels like I am juggling 3 repositories:

1. /var/db/repos/gentoo on the x86_64 workstation. I am using the workstation to cross compile arm64 packages and kernels, trying to leverage the workstation's multiprocessing power to save time.

2. /var/db/repos/gentoo repository on the microSD card. The microSD card is where I want to save the outputs of the cross compiled kernels and packages. I would be fine using this one exclusively for the microSD card, ensuring it's segmented from the workstation's repositories.

3. The recommended /var/db/repos/crossdev separate overlay repository, also on the x86_64 workstation, which is recommended from the crossdev article I mentioned earlier from the gentoo wiki. The wiki recommended using the "master = gentoo"

To save time, I wanted to try to start out with the arm64 stage3 archive already extracted on my microSD card's root file system. The stage3 archive for the arm64 platform has /etc/portage/make.profile, which is a symbolic link to:
../../var/db/repos/gentoo/profiles/default/linux/arm64/23.0/desktop

...which doesn't exist. This link points to nothing, because the stage3 archive doesn't contain the portage tree. So like I have done for many of my gentoo installations, I downloaded the latest portage tree, placing it into the microSD card at this path: /var/db/repos/gentoo (I renamed the directory "portage" to "gentoo").

That seems okay so far - when I get on the workstation this resolves to the path I would expect. If I mounted the microSD card root file system on /usr/aarch64-unknown-linux-gnu, the make.profile symbolic link ends up resolving to:

[pingtoo]

ddavidson72,

I think you are almost there but may be miss a little bit when you execute emerge command.

Please have a look at the script I posted It seems to be same as your steps but may be my script step order can give you some insight.

As the post stated, I did not complete test it. so let me know if you run into problem. we can try to work out the detail.

[ddavidson72]

[pingtoo]

ddavidson72,

Thank you very much for your detail information. Very much appreciated.

I think we may not sharing same concept (or at least some detail not same)

I think you use "repository" for two different entities. One is Portage's repo structure for crossdev, the other is Gentoo's repo (a.k.a portage snapshot) which is Gentoo developer sharing ebuild for what is know as Gentoo system.

And then there is PORTAGE_CONFIGROOT which from your post seems to link (associate) it with portage snapshot in some way I am not sure.

So I will clarify what is in my posted script and hopefully this will help our communication. I will have some very obviously Gentoo concept in my description so please bear with me, it is intent to make sure I share my point of view so it will not be assumed.

In my script, there are three entities, The host (X86 Gentoo), the crossdev construct and CHROOT construct (where target platform binary reside)

Each for of three entities have its own Gentoo Portage environment (which have its own etc/portage/*)

The idea in my script is using the CHROOT as location to store target platform environment it is not meant to be use for building unless there is some need to execute target native binary during package build process.

Primary we want to use crossdev toolchain to build, because it is running Host native binary so it will be much faster compare to the QEMU emulated environment.

So the script try to make crossdev environment as close as CHROOT environment, however let the crossdev-emerge put the build result into CHROOT instead the default /usr/<crossdev-taget/

Why crossdev environment need to close to CHROOT because crosdev-emerge use information in /usr/<crossdev-target/etc/portage/* for build options but use CHROOT/var/lib/portage and CHROOT/var/db/pkg for install related decision

I am not sure I understand your concern of "corruption". The Portage tree used during emerge is read-only there will not be anything introduce into Portage tree no matter which of the those three environment.

You may have notice I use "snapshot" refer to Portage tree. This is because you don't need to make three environments share same "version". As you know Gentoo is a rolling system, every day Gentoo change a little bit. So the idea is take a snapshot of the rolling changes so it will be fixated of that moment and you build process now can cross several days without worry about chasing Gentoo's changes.

my script bind mount var/db/repos/gentoo and var/cache/distfiles is because my practice. I don't update my gentoo system. I only rebuild entire system when I upgrade (the upgrade usually involve hardware changes). So my portage tree always fixed on the initial build process day. That way my system is always consist. if later I have miner requrement changes and simple targeted emerge run into problem then it usually mean that portage snapshot have a bug. I will either patch it myself or find a near future snapshot that resolve my issue and stay there.

As using symbolic link, I also try this trick in early day of learning, but later I found it is too big of effort try to keep it working as thing change, so I abandoned the idea.

A little explain for my early paragraph about "repository" and "construct". A Portage "construct" consist few parts,
- etc/portage/*
- var/db/pkg/*
- var/db/repos/*
- var/lib/portage/*
- var/cache/{binpkgs,distfiles}
I have left out some other part for portage because they are not related in this conversation.

The crossdev only have etc/portage/* because by design it is not expecting something install in to crossdev environment.

The CHROOT usually don't need var/db/repos/* because we don't expect to build anything in there. but in when package build process require target native execution you going to need to build that package in the CHROOT therefor just setup ahead in case this need happen.

The Host (Gentoo) have set of Portage "construct" for its own maintenance, then there is another set for crossdev. because it is the Host that maintain crossdev integrity so it need to have a var/db/repos/crossdev to isolate another set of compiler tool chain. So to not pollute Host compiler tool chain.

I am not sure I cover everything because I kind lost my thoughts, English is not my first language, it take a toll on me when I try to put together ideas in words and at same time I need to find the right "word" and right sentences. So let me know if I created something confusing and I will try to explain where I can.

[ddavidson72]

Hello again Pingtoo,

Thank you also for sharing more information and more context.

Yes, I think we're still on the same sheet of music and trying to accomplish very similar objectives. I think we're going about it almost the same way, and I also noticed that your script isn't really using the actual binary qemu chroot, but you're right, just setting up the construct. I noticed that qemu-aarch64 was being copied, but I don't think it's being executed as part of your script, is it? And then I can see the binfmt service is being used, and I was needing to start that in order to chroot using qemu binaries. But for this, I don't think we're doing that yet, and I think I'm trying to do something very similar like you are trying. Like everybody has mentioned, the qemu chroot is very slow, and I think we're trying to find something a little more optimal and scalable. Maybe the minor difference here is that you like to use your process to do just fresh, up-to-date releases/builds, where my goals are a little different and I want to do some incremental updates. It feels like it's the same thing though, especially starting off with a fresh build- I have the same 3 things I am juggling, like your setup:
1. I have the x86_64 that I am running and then installing the crossdev package on that to cross-compile.
2. I have the crossdev construct. I am using it to emerge packages and cross-compile kernel images for the SoC.
3. I also have the SoC construct. Like it was mentioned here, I was going to keep this on the microSD card and mount it on /usr/aarch64-unknown-linux-gnu/, but I was also just working with it in a directory, like you were also doing with your /choot directory, but I wasn't mounting the other mount points /proc, /sys, /dev, etc. That seems to work okay too- I can compile packages.

I think there still is "building" going on because the "emerge-aarch64-unknown-linux-gnu" command is still going to "build" binaries for the other arch, and thus "building" (compiling) is still taking place in both our environments. That's also the Gentoo way, right?!

Yes, you're right, I was being very ambiguous about referring the the portage snapshot, and using this synonymous with the portage tree, the repository, etc. I was thinking about this later, and remembered that the make.profile really doesn't point at the portage ebuilds, but to the profile for that arch. And I think you're right- it should be read only. But profiles are nested in the tree, and the symbolic link for the make.profile usually resolves to something in the tree.

So I think we're totally on the same sheet of music.

For the corruption concerns, I was a little concerned about the corruption because of what I read on the gentoo wiki page for crossdev-

from: https://wiki.gentoo.org/wiki/Crossdev (at the time of this post) - there is this warning under the "Crossdev Overlay" heading:

[pingtoo]

ddavidson72,

Thank you very much!

As long as you are comfortable with that script and you understand what it is doing you are good as gold.

The "script" is really not mine per se. it is target for the topic poster. it use qemu and crossdev because the thread OP want to do it that way. So first part of script was trying to setup a CHROOT, second part is to configure crossdev to make sure it match target setup. Having CHROOT environment ready is for case where build want to use target native binary as part of build process then the package will have to utilize qemu for a slow build, but the script make it ready, so it is easiler to switch between crossdev or chroot build.

I come up that script is because I am in middle of design my catalyst stage4 build process, so I was study catalyst code and found the code logic match the topic poster to large degree. so I manage to write down each step. it more of extract of execution logs.

Good luck with you journey with RPI .