Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[Administrative] Howto updating multiple servers?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
BlackEye
l33t
l33t


Joined: 04 Dec 2002
Posts: 756
Location: Germany

PostPosted: Thu Nov 20, 2008 11:26 am    Post subject: [Administrative] Howto updating multiple servers? Reply with quote

Hello,

this is an administrative question.
I have a xen-server with 3 domUs on it (beside some other gentoo systems). All these systems are installed with gentoo. What I'm searching for is a guide how can I best updating the whole set of these servers in a simple way.
I can execute "emerge -avuDN @world" on all systems parallel. But this means to compile all and everything several times which don't have to be.
Is there a way to share the installed packaged so that I can reuse the compiled packaged on other systems too? Whats about USE-Flag differences then? Is there a guide which describes the steps I need to do to have a better life with multiple gentoo systems?

Regards,
Martin
Back to top
View user's profile Send private message
frostschutz
Advocate
Advocate


Joined: 22 Feb 2005
Posts: 2977
Location: Germany

PostPosted: Thu Nov 20, 2008 11:51 am    Post subject: Reply with quote

If the configuration (that is to say, architecture, cflags, useflags, list of installed software) is identical, you can build binary packages on one system and emerge the ready to use binaries on the other systems. If the configuration is different the best you could do is use a shared local ccache / distcc or something, as every machine will get its own custom build. That's the downside of Gentoo customization, naturally if you distribute binaries they can no longer be customized.
Back to top
View user's profile Send private message
BlackEye
l33t
l33t


Joined: 04 Dec 2002
Posts: 756
Location: Germany

PostPosted: Thu Nov 20, 2008 12:13 pm    Post subject: Reply with quote

do portage automaticly notice about the compatibility of an prebuild package?
I use almost the same configuration on my systems. However - little differences are normal. One server uses Apache and Mysql and thus have +apache +mysql in use-flags of several packages. However - these should not affect the basic packages like glibc, gcc and all the large thingys...
If I could try to share there binary packages - do portage know of the compatibility of such a package?
Back to top
View user's profile Send private message
djinnZ
Advocate
Advocate


Joined: 02 Nov 2006
Posts: 4831
Location: somewhere in L.O.S.

PostPosted: Thu Nov 20, 2008 1:20 pm    Post subject: Reply with quote

yes with -k option the packages compiled with different use flags will be rebuilted, with -K are rejected and with -K --nodeps will be installed without check. Note than the packages built for a different arch will be rejected but the cflags and ldflags used are ignored.

the two cents of mine:
I have one server, one laptop and two identical desktops, all with amd cpus so I use the x86 arch and the same cflags for all.
To be sure and to minimize the update time my solution is to use a dedicated disk on the server.
On this disk I have 5 partitions, 3 with the image to build (mounted as /mnt/gentoo/server, desktop and laptop), one for shared tmp and ccache (I use no more ccache but I think can be easy shared on the same machine and I have used so, but with a very old version of the gcc) and one for the shared portage tree, sources and binaryes.
I start to compile from the server image because it have the smallest use flag set later i compile on the desktop and endly for the laptop. For the server i start by example (I am not able to report the exact commands but is only to explain) a
chroot /mnt/gentoo/server:
layman -S && emerge --sync && emerge -aDNuv world && emerge --depclean && revdep-rebuild &&
later a
chroot /mnt/gentoo/server:
mount server:/mnt/gentoo/server/usr/portage/packages /usr/portage/packages && layman -S && emerge --sync && emerge -aDNukv world && emerge --depclean && revdep-rebuild && umount /usr/portage/packages && eix -I --only-names | xargs quickpkg
and endly with the laptop.
Later on every client i mount the packages dir via nfs (because is possible a library break on wget at update time) and update via a sequence of emerge -NDuK worlds command (in fact I use a script to read direcly in the /var/db of the source what must be updated and use the -K and the --nodeps options passed to single emerge comkmands).
The trick is to use the -k option (and using the binaries builted for the minimal use and later build the binaries with the new assets, not directly with the feature buildpkg or the -B/-b option) so all the packages different in the use set will be recompiled and -K to install.
With this approach i am able to update in a max time of one half hour all the computers without any risk to have a longer stop time due to rebuilds requred to fix library. By using a shared ccache you will minimize more the compile time.
The other benefit is than with this approach I have ever a full backup (and more usable tha a "stage 4" tar many GB big) of the packages in case of failure. (and this is the reason to make the binaries support a priority in the portage developement IMHO :twisted: by adding filtering on the -march cflag by example, but this is another question)

Verify the commands I have reported I am not in my office and my memory fails often. :wink:
_________________
scita et risus abundant in ore stultorum sed etiam semper severi insani sunt:wink:
mala tempora currunt...mater stultorum semper pregna est :evil:
Murpy'sLaw:If anything can go wrong, it will - O'Toole's Corollary:Murphy was an optimist :wink:
Back to top
View user's profile Send private message
BlackEye
l33t
l33t


Joined: 04 Dec 2002
Posts: 756
Location: Germany

PostPosted: Thu Nov 20, 2008 1:58 pm    Post subject: Reply with quote

thanks for your suggestions. This looks very nice..
I was now able to copy the binary of gcc and glibc to my other server and minimize my compile-time to a few minutes. Thats quite nice.
Not I only need to find a nice solution hot I can provide these binary packaged to all my servers in my xen domain. Maybe I will use one donU with nfs-server as the source since the dom0 should not be used for more than it have to be.

Thanks for the suggestions :)

Regads,
Martin
Back to top
View user's profile Send private message
djinnZ
Advocate
Advocate


Joined: 02 Nov 2006
Posts: 4831
Location: somewhere in L.O.S.

PostPosted: Thu Nov 20, 2008 2:48 pm    Post subject: Reply with quote

The best thing of the update via chroot and binaries is than the compile time is not so important (in fact I sync the portage daily and compile at low priority and not cure if the rebuilt will take an entire day) and in the case our "beloved" devels will do something wrong (as in the case of the recent com_err ss mismatch) you have all the time to blaspheme and correct or wait for the next update to fix without concerns because the computers are ever fully operational. :twisted:
The strange thing is than so the update is more secure (in the requesto of the stop time and for the risk of library mismatch) than with a binary distribution.

I have never used xen but you can think on a local remount also or to mount direcly from the base chroot the target root and using the evironment variables ROOT and PORTAGE_CONFIG_ROOT to install (or recover), the only reason to not use the traditional way is because a mount will not be affected by library conflict and mismatch but the call to a single program to download the packages (as wget) can ever fail in the update, is rare and can be prevented with the new version 2.2 of portage, not sure if really work with binaries so is better prevent mistakes.

I have also forget than I use and emerge --metadata not emerge --sync for the successive builds because the portage tree is the same and obliviously i have a local portage copy on every running systems, updated only before installing the binaries. I waste some space on the disk with those multiple backups but the unsecurity will cost more than an HD.

Endly note than only share the ccache between different copies of the same compiler will work, so make you sure than the gcc has ever the same use flags and cflags everywere. The risk is to have inconsistant binaries compiled, i have warned you.
_________________
scita et risus abundant in ore stultorum sed etiam semper severi insani sunt:wink:
mala tempora currunt...mater stultorum semper pregna est :evil:
Murpy'sLaw:If anything can go wrong, it will - O'Toole's Corollary:Murphy was an optimist :wink:
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum