| View previous topic :: View next topic |
| Author |
Message |
lars_the_bear
Tux's lil' helper
Joined: 05 Jun 2024
Posts: 126
|
 Posted: Sun Jun 16, 2024 7:14 pm Post subject: |
 |
|
| szatox wrote: |
Faster installation is not a backup.
|
Well, perhaps 'backup' was the wrong term. But 'Keeping copies of installations artefacts that have to be built from source so I don't have to built them from source if I have to reinstall' seemed like a bit of a mouthful.
BR, Lars. |
|
| Back to top |
|
 |
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3216
|
| Posted: Sun Jun 16, 2024 9:06 pm Post subject: |
|
|
I'm lost now. Do you _want_ to reinstall your system multiple times in a short time?
Anyway, emerge has FEATURES="buildpkg" which builds packages automagically every time you install or upgrade something, and stores them in its cache for later reuse. Good to have when testing upgrades, since it allows for a fast rollback without losing progress on everything else your system was doing.
It's also a great timesaver when running multiple machines with very similar systems, since you can build once and then install everywhere.
| Quote: | | Keeping copies of installations artefacts that have to be built from source so I don't have to built them from source if I have to reinstall |
Restoring from actual backup will likely be faster than reinstalling, it also protects your customized settings and users' data, and you should already have some backup solution configured and scheduled, so no extra work required.
_________________
Make Computing Fun Again |
|
| Back to top |
|
|
lars_the_bear
Tux's lil' helper
Joined: 05 Jun 2024
Posts: 126
|
| Posted: Mon Jun 17, 2024 7:23 am Post subject: |
|
|
| szatox wrote: | | Restoring from actual backup will likely be faster than reinstalling, it also protects your customized settings and users' data, and you should already have some backup solution configured and scheduled, so no extra work required. |
Fair enough; but I've never even considered backing up system stuff before. I have robust backup schemes for my business data, but it's just data. I've never felt the need to back up binaries or configuration in the past.
When I screw up a Fedora installation -- which I do all the time -- I've generally found it quicker just to reinstall from scratch, than to find out what I broke. It never takes more than an hour. I suspect that Gentoo needs a different approach, because reinstallation from scratch takes a comparatively long time. Hence my question.
BR, Lars. |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54483
Location: 56N 3W
|
| Posted: Mon Jun 17, 2024 8:52 am Post subject: |
|
|
lars_the_bear,
There are several backup strategies for Gentoo.
1. Just /home - you can work the lest out if you need to.
2. home/ /etc/portage/ and /var/lib/portage/world - That's enough to rebuild your install
3. Add in binary packages to 2. To speed up picking up the pieces.
4. A stage 4 which is fastest.
A disk image is a bad idea in these days of solid state storage. It all depends how long your system can be down
See also my signature :)
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
lars_the_bear
Tux's lil' helper
Joined: 05 Jun 2024
Posts: 126
|
| Posted: Mon Jun 17, 2024 10:26 am Post subject: |
|
|
Hi
Do 'binhosts' in general respect CPU types and flags?
What I mean is: I have three computers all set up with the same profile and USE flags, but they have slightly different CPUs. Could I use a single binary host for all three?
I appreciate that if I had one x64, one ARM, and one Sparc I probably couldn't share binaries. But all my computers have Intel I7 CPUs -- of different vintage. The output of 'cpuid2cpuflags' is very slightly different between the machines. For example, one shows 'avx2', but the others don't.
I'm wondering how Portage/emerge copes with this kind of situation. I have no clue at all whether a binary built with 'avx2' CPU flag would work on one without that support. My guess is that most things would work, but a few wouldn't. Would Portage recognize when things had to be rebuilt in cases of subtle distinction like this?
BR, Lars. |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 21970
|
| Posted: Mon Jun 17, 2024 12:25 pm Post subject: |
|
|
That depends on the choices of the binhost administrator. If the binhost creates binaries that use only features available on all the CPUs in question, then yes, you can share the binaries. If the binhost is allowed to use all the features of its host CPU, and that CPU supports features the client's don't, then the binhost may (or may not) produce binaries that fail on the clients.
Find the greatest common subset of features across all the participating hosts. Configure the binhost to use only those features.
Portage only recognizes what it can see in configuration flags. If an upstream package silently enables avx2 support despite your setting not to use avx2, Portage will not know about it, and so cannot help you. Such packages deserve a bug report, because they should enable only what they are told they can use. |
|
| Back to top |
|
|
lars_the_bear
Tux's lil' helper
Joined: 05 Jun 2024
Posts: 126
|
| Posted: Tue Jun 18, 2024 8:56 am Post subject: |
|
|
| Hu wrote: |
Find the greatest common subset of features across all the participating hosts. Configure the binhost to use only those features.
|
I kind-of wish I'd known about this, before I started five days of compiling 
My first installation took five days. I used the binary packages the build generated to populate a binhost. When I came to install on a new machine, using the same binhost, it was much quicker -- maybe five hours, rather than five days. So, that's good.
But...
In my first installation, I did what it said in the Gentoo Handbook -- I used 'cpuid2cpuflags' to configure compiler settings. Since my first build machine had avx2 support in the CPU, I guess I got binaries that needed avx2 support. I was quite surprised by the number of packages that had to be built from source, even though I had already provided binaries, because the machine I was installing did not have avx2 support. At least these packages depend on it: libnss, libaom, ffmpeg, flac.
It's too late to do it all again, I guess, but...
What should I have done, to get binaries that worked on all CPUs? Or, at least, on all Intel i7 CPUs, which is all I have? Should I simply have missed out the 'cpuid2cpuflags' step? I guess I probably should have used '--march=native' in my CFLAGS setting, although that seems not to have been a problem.
BR, Lars. |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54483
Location: 56N 3W
|
| Posted: Tue Jun 18, 2024 9:15 am Post subject: |
|
|
lars_the_bear,
Welcome to the school of hard knocks ... or experience is what you get just after you needed it. :)
You way find some ideas here as well as some of the problems.
You should not use -march=native at all, except as a test on each system to see what it means.
Likewise with cpuid2cpuflags.
CFLAGS tells gcc what instructions it can choose from. It may choose not to use them all.
CPUFLAGS tells the build system that is can use optimised code that uses these instruction sets.
You need settings that are the lowest common denominator to generate code that will run everywhere.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
lars_the_bear
Tux's lil' helper
Joined: 05 Jun 2024
Posts: 126
|
| Posted: Tue Jun 18, 2024 10:21 am Post subject: |
|
|
| NeddySeagoon wrote: |
You should not use -march=native at all, except as a test on each system to see what it means.
Likewise with cpuid2cpuflags.
|
I guess I've been doing this stuff long enough that I should have thought through the implications of the use of these flags. I always assumed I would need to install more than once, and on multiple machines.
I suppose what I didn't appreciate -- and should have -- is how darned long it takes to compile most of a complete Linux installation from scratch, on a 2012 laptop.
The other thing I did not appreciate -- and probably couldn't have done at the time -- is how few of the binaries in the stock Gentoo distribution would be suitable for my set-up. All the behemoth packages -- firefox, libreoffice, vlc, Gimp -- had to be built from source.
What I should have done, knowing what I know now, is to run the complete installation, with lowest-common-denominator settings, in a VM on one of my business machines. Then I could have populated a binhost with the binaries, and used those for the older machines. Even with the additional set-up, I think this would have been quicker by days.
Oh, well. As you say, experience is what you gain by not having any experience.
BR, Lars. |
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3216
|
| Posted: Tue Jun 18, 2024 10:26 am Post subject: |
|
|
| Quote: | | Since my first build machine had avx2 support in the CPU, I guess I got binaries that needed avx2 support. I was quite surprised by the number of packages that had to be built from source, even though I had already provided binaries, because the machine I was installing did not have avx2 support. At least these packages depend on it: libnss, libaom, ffmpeg, flac. |
You can expect multimedia and cryptography applications to use extensions for better performance, though still not as good as running on GPU.
The other stuff should be fine. Allowing the code to be built for a cpu with extensions doesn't do much if there is no code which would actually take advantage of those extensions.
_________________
Make Computing Fun Again |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Other Things Gentoo
