Gentoo cluster install questions.

pklizest · n00b Joined: 01 Aug 2010 Posts: 1

Hi, I have been wanting to install setup a Gentoo cluster for sometime, but I have a collection of about 30 odd computers with various broken bits that have been donated to me, and this has led me to reinstall Gentoo multiple times on these box. However this is painfully slow due to the Internet connection at my lab being junk. I was hoping there is a relatively easy way to slap a base install on the various computers quickly without being forced to install manually each time. I would then go back in and modify stuff as needed. I have both 32bit and 64bit 86x archs computers, and I also have some old sun servers.

I was going to add the latests portage and stage3 to the minimal Gentoo iso, and put a stripped down version of glis on the cd as well. But before I make a few different isos learn how to bash script and refactor iso properly, I wanted to ask if anyone had a better way to do this.

tl;dr, How do i unattended install gentoo with no internets?

glis=Gentoo Linux install script; http://glis.sourceforge.net/
_________________
Bitches don't know about my houston hackerspace.

pappy_mcfae · Posted: Wed Dec 22, 2010 8:34 am Post subject:

Sure. Set up one machine with a generic Gentoo install.

Once the install is done, do a stage-4 to it, and use the stage-4 to populate the other computers. That way, you only have to do the hard work once. With the stage-4, you copy your finished system to all the machines. Once that's done, just customize the kernels and make.conf, and go from there.

What is a stage-4? It is a form of a backup that uses tar to create a tarball. The tarball created is a working Gentoo installation. A working Gentoo installation is defined as a fully functional base Gentoo operating system one will have when done following the Gentoo Handbook. By creating the backup and distributing it to the other machines, all you have to do is to untar, chroot, setup the boot loader, and you're done.

It saves a lot of time...believe me.

Cheers,
Pappy
_________________
This space left intentionally blank, except for these ASCII symbols.

delenda · n00b Joined: 28 Dec 2010 Posts: 4

A few questions...How is your cluster physically interconnected? By switches, routers (locally but fast) ? Or direct to the internet ?
Perhaps some computers can be booted through LAN? Diskless install using PXE boot

Do you require 64bit software, or could everything be 32bit?

[Edit]
Just found this: http://www.gentoo-wiki.info/HOWTO_Create_A_Build_Host

delenda · n00b Joined: 28 Dec 2010 Posts: 4

[Edit - 2: must be talking to meself]

I'm trying to build my own cluster..I don't have like "about 30 odd computers", more like 03...but generally, it's the same principle.

Now here's an idea, create a general live-cd (or dvd usb-stick/hdd/...) on your main (quickest) computer...
(see links)...Either setup an local DNS server identifying each node (dhcp/MAC adress), or adapt the basic image, e.g. change every host name, et cetera, before imaging it to the cd/dvd/usb-stick....Then boot the node from its personalized LiveCD/LiveUSB/LiveDVD....

Each node would adress the server and log on..The cluster grows....

Some thoughts: if I would were to use my most powerful node only, leaving the remaining nodes unused, I'd spare the electric bill.

One out of my 03 nodes, has now depreciated hardware, IDE but still a fully functional hard drive :cry:

So, if you do have "odd hardware" - hmmm

30odd computers, so probably - , why bother to install it "fully optimized"? I mean CFLAGS like -march=pentium4, would only complicate everthing.

One could alter the livecd to boot and expand into ram, thus utilizing only cpu-ram-eth, omitting any other "odd" hardware. Heck linux doesn't use much ram, as win#!#. A stripped version - bootable toolchain, no X, gnome,gtk,kde ..- uses far less..

Included bonus, a minimal "livecd-boot" would be easier to maintain. A new toolchain could destroy a outdated compilation farm. Updating a few livecd-stages, burning those to discs/sticks would save a lot of time in stead of duplicating this to each nodes hard disc....

If you really want to recuperate some of the 30-odd computer hardware - eg huge disc space - , one could easily focus those boot-discs. (eg disc - nfs server posibilities - or set it up locally...)

Another idea: since most of those nodes would not be used graphically, one could easily tap into it's graphic abilities, I mean ram...I'd use it as tmp or as as a ramdisc containing the toolchain....swapping would probably not be really used.

Links:
http://en.gentoo-wiki.com/wiki/Build_Your_Own_LiveCD_or_LiveDVD
https://forums.gentoo.org/viewtopic-t-296892.html
http://en.gentoo-wiki.com/wiki/Using_Graphics_Card_Memory_as_Swap

PS: I only used a stage2/stage3 installation
PPS: I'm currently unsuccessful building an 1 personalized LiveCD, but still trying...

Raptor85 · Apprentice Joined: 29 Dec 2007 Posts: 208

pappy's way is the way to go, after that you're going to want to set up a binhost on one of your machines so you only need compile once. If they're identical machines you're golden, if not, you'll want to carefully check all drivers needed for each machine, and the "common" architecture (probably 686), you'll lose performance somewhat on machines that support more, but it will save you a lot of time and leave you with only one common image to maintain.

I have distcc set up across all of mine, one of these days i need to test it by re-compiling openoffice across all the nodes. I wonder if there's a record for fastest compile of that yet

pgu · l33t Joined: 30 Jul 2009 Posts: 721 Location: Oslo, Norway

Is there a way to run all the nodes in a cluster diskless and share all the binaries (assuming all x86_64 compatible nodes)?
I used to do this in the good old days using SunOS. /var, /tmp, /etc was unique as /bin/ /lib etc was share among all the nodes in the cluster.

That way adding a node in the cluster would be simply to duplicate the unique part on the server, enable PXE in the new node and boot.

pgu · l33t Joined: 30 Jul 2009 Posts: 721 Location: Oslo, Norway

delenda · n00b Joined: 28 Dec 2010 Posts: 4

Building a stage-4 and extracting it to the local hard disc, or creating a bootable disc/stick is basically the same... :wink:

Only the destination differs.

wim-x · Posted: Sat Jan 08, 2011 12:09 pm Post subject:

pgu · l33t Joined: 30 Jul 2009 Posts: 721 Location: Oslo, Norway

wim-x · Posted: Tue Jan 18, 2011 6:53 pm Post subject:

pgu · l33t Joined: 30 Jul 2009 Posts: 721 Location: Oslo, Norway

I would assume /etc being unique due to /etc/fstab, /etc/hostname etc?

I would like to set up a front-node which serves a cluster of nodes to be used for simulations. As I overclock the nodes they tend to be disposed pretty rapidly, hence I would like to make the installation as easy as possible. Enable PXE or insert an USB stick and boot, then reformat the local hard drives which are used for scratch storage during simulation. Linux and software resides on the front-node.

pgu · l33t Joined: 30 Jul 2009 Posts: 721 Location: Oslo, Norway

At home I have a couple diskless mini-itx machines. They boot from the server, and nfs mount the file system, but nothing is shared among the clients in this setup. Each machine have their own file system tree on the server.

But I would like to share to make it easier to add nodes to the cluster. New software would be available to all nodes.

wim-x · Posted: Thu Jan 20, 2011 5:32 pm Post subject: