View previous topic :: View next topic |
Author |
Message |
BlackEye l33t
Joined: 04 Dec 2002 Posts: 756 Location: Germany
|
Posted: Thu Nov 20, 2008 11:26 am Post subject: [Administrative] Howto updating multiple servers? |
|
|
Hello,
this is an administrative question.
I have a xen-server with 3 domUs on it (beside some other gentoo systems). All these systems are installed with gentoo. What I'm searching for is a guide how can I best updating the whole set of these servers in a simple way.
I can execute "emerge -avuDN @world" on all systems parallel. But this means to compile all and everything several times which don't have to be.
Is there a way to share the installed packaged so that I can reuse the compiled packaged on other systems too? Whats about USE-Flag differences then? Is there a guide which describes the steps I need to do to have a better life with multiple gentoo systems?
Regards,
Martin |
|
Back to top |
|
|
frostschutz Advocate
Joined: 22 Feb 2005 Posts: 2977 Location: Germany
|
Posted: Thu Nov 20, 2008 11:51 am Post subject: |
|
|
If the configuration (that is to say, architecture, cflags, useflags, list of installed software) is identical, you can build binary packages on one system and emerge the ready to use binaries on the other systems. If the configuration is different the best you could do is use a shared local ccache / distcc or something, as every machine will get its own custom build. That's the downside of Gentoo customization, naturally if you distribute binaries they can no longer be customized. |
|
Back to top |
|
|
BlackEye l33t
Joined: 04 Dec 2002 Posts: 756 Location: Germany
|
Posted: Thu Nov 20, 2008 12:13 pm Post subject: |
|
|
do portage automaticly notice about the compatibility of an prebuild package?
I use almost the same configuration on my systems. However - little differences are normal. One server uses Apache and Mysql and thus have +apache +mysql in use-flags of several packages. However - these should not affect the basic packages like glibc, gcc and all the large thingys...
If I could try to share there binary packages - do portage know of the compatibility of such a package? |
|
Back to top |
|
|
djinnZ Advocate
Joined: 02 Nov 2006 Posts: 4831 Location: somewhere in L.O.S.
|
Posted: Thu Nov 20, 2008 1:20 pm Post subject: |
|
|
yes with -k option the packages compiled with different use flags will be rebuilted, with -K are rejected and with -K --nodeps will be installed without check. Note than the packages built for a different arch will be rejected but the cflags and ldflags used are ignored.
the two cents of mine:
I have one server, one laptop and two identical desktops, all with amd cpus so I use the x86 arch and the same cflags for all.
To be sure and to minimize the update time my solution is to use a dedicated disk on the server.
On this disk I have 5 partitions, 3 with the image to build (mounted as /mnt/gentoo/server, desktop and laptop), one for shared tmp and ccache (I use no more ccache but I think can be easy shared on the same machine and I have used so, but with a very old version of the gcc) and one for the shared portage tree, sources and binaryes.
I start to compile from the server image because it have the smallest use flag set later i compile on the desktop and endly for the laptop. For the server i start by example (I am not able to report the exact commands but is only to explain) a chroot /mnt/gentoo/server: | layman -S && emerge --sync && emerge -aDNuv world && emerge --depclean && revdep-rebuild && | later a chroot /mnt/gentoo/server: | mount server:/mnt/gentoo/server/usr/portage/packages /usr/portage/packages && layman -S && emerge --sync && emerge -aDNukv world && emerge --depclean && revdep-rebuild && umount /usr/portage/packages && eix -I --only-names | xargs quickpkg | and endly with the laptop.
Later on every client i mount the packages dir via nfs (because is possible a library break on wget at update time) and update via a sequence of emerge -NDuK worlds command (in fact I use a script to read direcly in the /var/db of the source what must be updated and use the -K and the --nodeps options passed to single emerge comkmands).
The trick is to use the -k option (and using the binaries builted for the minimal use and later build the binaries with the new assets, not directly with the feature buildpkg or the -B/-b option) so all the packages different in the use set will be recompiled and -K to install.
With this approach i am able to update in a max time of one half hour all the computers without any risk to have a longer stop time due to rebuilds requred to fix library. By using a shared ccache you will minimize more the compile time.
The other benefit is than with this approach I have ever a full backup (and more usable tha a "stage 4" tar many GB big) of the packages in case of failure. (and this is the reason to make the binaries support a priority in the portage developement IMHO by adding filtering on the -march cflag by example, but this is another question)
Verify the commands I have reported I am not in my office and my memory fails often. _________________ scita et risus abundant in ore stultorum sed etiam semper severi insani sunt
mala tempora currunt...mater stultorum semper pregna est
Murpy'sLaw:If anything can go wrong, it will - O'Toole's Corollary:Murphy was an optimist |
|
Back to top |
|
|
BlackEye l33t
Joined: 04 Dec 2002 Posts: 756 Location: Germany
|
Posted: Thu Nov 20, 2008 1:58 pm Post subject: |
|
|
thanks for your suggestions. This looks very nice..
I was now able to copy the binary of gcc and glibc to my other server and minimize my compile-time to a few minutes. Thats quite nice.
Not I only need to find a nice solution hot I can provide these binary packaged to all my servers in my xen domain. Maybe I will use one donU with nfs-server as the source since the dom0 should not be used for more than it have to be.
Thanks for the suggestions
Regads,
Martin |
|
Back to top |
|
|
djinnZ Advocate
Joined: 02 Nov 2006 Posts: 4831 Location: somewhere in L.O.S.
|
Posted: Thu Nov 20, 2008 2:48 pm Post subject: |
|
|
The best thing of the update via chroot and binaries is than the compile time is not so important (in fact I sync the portage daily and compile at low priority and not cure if the rebuilt will take an entire day) and in the case our "beloved" devels will do something wrong (as in the case of the recent com_err ss mismatch) you have all the time to blaspheme and correct or wait for the next update to fix without concerns because the computers are ever fully operational.
The strange thing is than so the update is more secure (in the requesto of the stop time and for the risk of library mismatch) than with a binary distribution.
I have never used xen but you can think on a local remount also or to mount direcly from the base chroot the target root and using the evironment variables ROOT and PORTAGE_CONFIG_ROOT to install (or recover), the only reason to not use the traditional way is because a mount will not be affected by library conflict and mismatch but the call to a single program to download the packages (as wget) can ever fail in the update, is rare and can be prevented with the new version 2.2 of portage, not sure if really work with binaries so is better prevent mistakes.
I have also forget than I use and emerge --metadata not emerge --sync for the successive builds because the portage tree is the same and obliviously i have a local portage copy on every running systems, updated only before installing the binaries. I waste some space on the disk with those multiple backups but the unsecurity will cost more than an HD.
Endly note than only share the ccache between different copies of the same compiler will work, so make you sure than the gcc has ever the same use flags and cflags everywere. The risk is to have inconsistant binaries compiled, i have warned you. _________________ scita et risus abundant in ore stultorum sed etiam semper severi insani sunt
mala tempora currunt...mater stultorum semper pregna est
Murpy'sLaw:If anything can go wrong, it will - O'Toole's Corollary:Murphy was an optimist |
|
Back to top |
|
|
|