Gentoo on WRT

SLBMEH · Posted: Fri Apr 07, 2006 11:58 am Post subject: Gentoo on WRT

Has anyone tried to install Gentoo on a WRT device such as a Linksys WRT54G? That's what I'm beginning to work on at the moment and if anyone has successfully accomplished it any/all advice is greatly appreciated.
_________________
Steve - Semper Fi

guduri · Posted: Fri Apr 07, 2006 12:21 pm Post subject:

Not that I know of. I am assuming you already looked into OpenWrt. It works really well.
_________________
Power is about what you can control. Freedom is about what you can unleash.

Redhatter · Posted: Fri Apr 07, 2006 12:24 pm Post subject:

Ohh god no...

OpenWRT would definately be a better choice. Gentoo will not run on a WRT54G, or any other Broadcom-based wireless router. It's simply not our focus.

These are machines with very little memory (32MB) and no storage options whatsoever -- Gentoo simply is not intended for such crippled hardware.

You're welcome to try port Gentoo to such a device, but you're on your own... none of us on the Gentoo/MIPS team have the time or desire to support them. Gentoo Embedded, might have an interest, but you'd have to ask them.
_________________
Stuart Longland (a.k.a Redhatter, VK4MSL)
I haven't lost my mind - it's backed up on a tape somewhere...

Gentoo/MIPS Cobalt developer, Mozilla herd member.

a7thson · Posted: Mon Apr 10, 2006 5:52 pm Post subject:

A newbie speaks, so please bear with me here...

It's quite possible to do this, but it definitely requires some "out of box" thinking, that box being the little router itself. I picked up one of the nice WRT54GS models (a v2.1) reasonably on Ebay and have been getting a crash course in embedded through this process, with the goal of a workable Gentoo system mounted off a remote filesystem.

guduri · Posted: Mon Apr 10, 2006 6:23 pm Post subject:

a7thson,

What you are saying sound nice as an experimental set. But in practical use, that may be a bit painful. Here are the reasons:

1) There are two points of failure now. The router and the machine exporting the filesystem.
2) A router usually needs to have 24/7 uptime. Now this is also required of the second system.

P.S: Frankly, I would myself try this with the linksys router and the slug (NSLU2 from Linksys. A nifty little device).
_________________
Power is about what you can control. Freedom is about what you can unleash.

a7thson · Posted: Mon Apr 10, 2006 9:25 pm Post subject:

1clue · Advocate Joined: 05 Feb 2006 Posts: 2569

It's not quite so far out there as you might think.

I haven't experimented with this, but we have a few managed switches, firewalls and routers in our office. They all allow for a DHCP boot and configuration.

The thing is here that each component should be redundant in a high availability environment. That means that you would use more than one router, in this example, and would boot from one of at least two DHCP servers which are each appropriately configured to supply the correct boot image to the router's network card.

The idea is called failover. (sorry, not trying to sound snotty if you already know this, just trying to be helpful if you're not aware)

Anyway, the idea is that any component is expendable, and if that component were to vanish in a puff of smoke you would only need to replace it, with no particular hurry since something else has taken over its function.

This happy state of affairs exists in one or two spots on my network, but alas routers are still not in those spots.

a7thson · Posted: Tue Apr 11, 2006 1:23 am Post subject:

a7thson · Posted: Thu Apr 20, 2006 4:29 pm Post subject:

For anyone still following along, I made some small amount of progress on this notion of getting Gentoo onto the WRT54G series router. Although the WRT54G itself *will* run a standard compiler, the conditions are far from ideal for this and it is very time-consuming. Surprisingly however, the box never died and it maintained my two connections, in fact I was even able to connect to and use the web interface of OpenWrt a few times while all this nonsense was going on. It should not compromise your router as everything is being done in an isolated chroot, so the worst you can probably do is to write some nastiness to NVRAM, or run it out of memory. Either power cycle or a hard reset should fix any issue and you shouldn't play with OpenWrt in the first place if either of those steps or the possibility of blowing up the router scare you.

If there is interest in this, I will do an informal HOWTO when I have access to my bash history again. For now some high-level details and suggestions will have to suffice. I quickly discovered after a few broken filesystems, that the stage3 compiled by the gentoo-embedded folks was for me a dead end. Doing a system build from scratch ended up to be much more promising as I wanted a 0.9.28 uClibc.

before going further, these are the packages that I've gotten to compile so far, for my v2.1 WRT54GS:

Some of these are not particularly useful and were produced as part of the experimental attempt to build "emerge -e system". Other parts are far more encouraging. Building this set will yield a usable yet basic Gentoo chroot environment.

You first need to prepare the minimal OpenWrt. Gentoo will be resource-intensive on the small but serviceable router and we want as much memory available as possible especially if you try onboard compilation (it works; I built uclibc-0.28, coreutils-5.2-r1, and a few others natively but I would not expect the router to do much else when running a compile job). I built OpenWrt from buildroot but a precompiled binary should work just as well. You'll need to use ipkg (itsy package management system) for OpenWrt to include some extra kernel modules: support for NFS is required, and support for loopback file system is recommended.

What I've done so far, I would divide into 3 primary stages: bootstrap, minimal, and gentoo-base. I built a very minimal root with uclibc, busybox, a toolchain and some basic networking. Essentially I then went through from "emerge -e system" noting what compiled, what nearly compiled, and what broke.

To follow along, you need to build a cross-compiler for the mipsel-base Broadcom chip in the router. You then need to set up a proper cross-compilation environment after the basic bootstrap. I'll detail the process and give the scripts and settings I used to get this far in the hopes that others interested in this will try to replicate this on their own boxes, but wanted to let someone know that I have not abandoned the idea by any means, and intend to take it further as time permits. Feel free to message me if you want to get involved further, this could actually be an interesting project.
_________________
i7-3610QM | E5-2670 | FX-8300

devilbush · n00b Joined: 15 Mar 2006 Posts: 13

Just FYI, I know of at least one person who is still following along. =)

I run Squawker's gentoo tarball on 2 of my 3 slugs and Open-WRT on my WRT54G, so this thread is tremendously interesting to me!

I wish I had the bandwidth to help, but I'm working two programming jobs and also currently struggling with getting my x800 running under XGL on my ~amd64 build in my spare time (haha spare time) - though I'm about to give up and just buy a NVidia 7900 and be done with it. Maybe after I make a spendy visit to Newegg I'll be able to kick in some time and try with you.

Best of luck! It is very impressive that you have come this far!

a7thson · Posted: Sat Apr 22, 2006 5:33 pm Post subject:

a7thson · Posted: Sun Apr 23, 2006 2:36 pm Post subject:

Gentoo on Linksys WRT54G series HOWTO-in-progress
Part One: Building Blocks

This isn't a full HOWTO, as I'm not yet to the point where that's a useful exercise, but I'll give some instructions as best I recall, for those who might want to follow along and perhaps advance the process. I quickly discovered after a few broken filesystems, that the stage3 compiled by the gentoo-embedded folks was for me a dead end. Doing a system build from scratch ended up to be much more promising.

Disclaimer / Caveat Emptor:
Following along with this tutorial should not be dangerous but this is experimental so if you b0rk the router and it becomes a nice brick, I hope you have a JTAG... I don't . As I said earlier this should not compromise your router as everything we're doing is being done in an isolated chroot, and we're doing this initial work outside the router itself. At worst a misbehaving program might inadvertently write junk to NVRAM, alter your routing table or temporarily lock you out of your router . A power cycle or hard reset should fix all of these issues in short order. If you are already willing to play with OpenWrt, you have already stepped past the bounds of normal use of a stock router. I also warn that the Linksys routers and router modification in general is a BIG topic and there is a lot to learn. This is as safe as I know how to make it, if you have further suggestions to make this foolproof they are most welcome.

Main rationale for the work:
OpenWrt is great, and I've only scratched the surface of what's possible with it. But the limits of a binary-based distribution are already too well-known to need comment; it is probably the reason many came to Gentoo in the first place! What we want to accompilsh is to have a usable experimental testbed allowing us to build newer versions of packages we want to work with. Since we can keep the "life-support" going on the router, and can fix nearly anything we do to the Wrt within the chroot, dangerous things like automating blocking of bad hosts, creating firewall rules on-the-fly or adding dynamically to hosts.deny cannot potentially lock us out of our own router :-)

as easily, if we do someething really dumb, or if a script goes awry.

Moving along then...
For the purposes of this HOWTO, I would divide this installation into 3 primary stages: bootstrap, minimal, and gentoo-base. These are preceeded by a preparatory step, in which we emerge, configure and set up our cross-compilation environment, and install OpenWrt itself on the router. This HOWTO will cover the preparatory step and bootstrap of the cross-compiler, as it is in actual fact the largest part of the whole task, the rest is largely experimentation and getting used to watching emerges fail :-D

This is intended as a work-in-progress. It is my hope that others will be interested in doing this for themselves, will follow, expand on, and correct what I lay out here, and can advance further into making this a serious application. Some of this will be elementary to people experienced with OpenWrt, but quite frankly I don't care, as this is intended to be a tutorial that anyone reasonably experienced in Linux should be able to follow even if OpenWrt is new to them.

Preparation: Installing OpenWrt.

OpenWrt is a fairly straightforward install if you know what you're doing and are comfortable with makefile-based build environments like the uClibc buildroot provides. It's (in my opinion!) not nearly as nice as an ebuild but it does get the job done. I am not going to attempt to teach how to work with OpenWrt and configure it and assume either A) you've done that, been there before or B) you'll learn about OpenWrt later, right now I want to see Gentoo on a router. I'll try to cover the minimal knowledge of OpenWrt which is needed to make it through the HOWTO, and the remaining configuration and customization is left to your discretion.

For OpenWrt, we have two basic options:
a) Install a pre-compiled kernel and core filesystem, and download prebuilt packages from official and/or unofficial OpenWrt sites.
b) Download and install the OpenWrt buildroot, configure and compile the kernel and critical maintenance filesystem ourselves, and selectively compile our own packages, installing them locally to the user-writable file system provided by OpenWrt.

I chose to follow the second route. If you choose the second method as well, go and grab the buildroot from here
http://downloads.openwrt.org/whiterussian/rc4/whiterussian_rc4.tar.bz2 (I used RC4, but now RC5 is available, which is here http://downloads.openwrt.org/whiterussian/rc5/whiterussian_rc5.tar.bz2. You'll need to unpack this somewhere where you have ample space - the build process will consume at least 1-2GB between the files it will download and the compile process, possibly more if you install a large number of packages. I chose to put it on an LVM2 volume and bind this to /opt/cross/openwrt. So extract the tarball:

GNUtoo · Veteran Joined: 05 May 2005 Posts: 1919

a7thson · Posted: Mon Apr 24, 2006 11:53 pm Post subject:

22bsti · Posted: Mon May 01, 2006 3:41 pm Post subject:

i have been wondering about this for some time now, i definately may need to try it with my ever decreasing free time i have gentoo on my nslu2 atm and this looks very intersting

GNUtoo · Veteran Joined: 05 May 2005 Posts: 1919

if my memory is good the wrt54 use the 43xx chip
broadcom 43xx driver are avaliable in portage
They will be avaliable in the 2.6.17 kernel
(there are also the chip of airport extreme)

i think you'll need to work on the memory footprint for gentoo
we need to test the memory footprint and the performance overhead(for example between -O2 and -Os (there isn't that much difference between O2 and O3 and i had Kdevelop break with O3 and not with O2))

you also need to compare the compressions such as:
-jffs2
-squashfs

there was a paper on the net for the squashfs and others standard compression filesystem

I don't know if using unionfs is a good idea because of the problems asociated to this thing

casso · Posted: Mon Jun 04, 2007 5:27 am Post subject: Still Listening

Hi,
Just to let you know that I appreciated you post very much. I have a large interest in Linux Embedded. I have successfully modified two X-BOXes to run Gentoo, and am working on getting a GameCube to run Gentoo as well. Of course, the GameCube is rather difficult since the network adapter causes such a huge bottleneck.

Having said that, I wish to tackle Gentoo on my ASUS WL-500g Premium. I have two USB ports at the ready on this device, although I will probably build over NFS, or even NBD to create my system. I am going to start with OpenWRT buildroot as a stepping stone into an embedded system for the router. Then I plan to follow the information you have given here to get it running some components of Gentoo, while using the least amount of resources. If DD-WRT (which I am currently using) can occupy only 4MB of RAM when being used, then maybe I can get everything I want running under Gentoo inside about 16MB of RAM, with all my packages, and some swap space just in case on a 2GB USB stick.

I appreciate your post so very much. I'm glad to know I'm not alone and that I don't have to start at square one like you did. I would be that keen to start there, so again thanks for saving me.

Michael Cassaniti
_________________
Need assistance with any of the following? Just PM me and I will see what I can do

LDAP, Kerberos , SSH, Samba, PPP, DHCP, NTP, autofs, CUPS

GNUtoo · Veteran Joined: 05 May 2005 Posts: 1919

why not:
-using FreeWRT as base(re-use the configuration tool for busybox,for the cross compilers...)
-using gentoo baselayout(specialy baselayout2) and configuration

freewrt is a modular firmware for routers
you can make your own squashfs or jffs2 image with the busybox option of your choice and the packages of your choice
gentoo desktop isn't enough modular for such embedded thing because busybox comes without all the option you want or don't want.
so if we want a modular distribution we could look at Freewrt and merge both gentoo and Freewrt...

i've a stupid question...is the source code of the packages changed for openwrt or freewrt in order to have a smaller footprint?
in the case we should have a different repository...

i'd love to have gentoo packages on my wrt54gs such as openssh with skeys or use flags for the packages
FreeWRT already has a similar system giving the choice of different packages with support for different database or systems
for example snort can be compiled with:
-debug
-inline
-mysql
-portgres sql

a7thson · Posted: Tue Jun 05, 2007 8:41 pm Post subject:

I agree that FreeWRT or another firmware might be a better solution nowadays... it was over a year ago since I did the original experiment w/OpenWRT, and I used what was available at the time. NFS mount of a stage3 mipsel could easily be performed with any number of base firmwares without a great number of changes, via a similar method - in fact any firmware that has NFS and loopback support to mount a disk image for swap should do the trick. Luckily there are more powerful routers out there now which contain niceties like USB/CF support and the like which may make this whole process unnecessary, ... but it will still work.

My memory may be a bit foggy, and I've been inactive here for several months (looks like I need to update my sig too :-D

), a few inquiries about this project aside, but I'll offer what assistance I can here or via PM and have corresponded with a handful of people since writing the original HOWTO. I did end up with a cross-compiled OpenSSH from Gentoo portage incidentally (via cross-compiler with a ROOT= prefix, onto the NFS mounted root), along with a handful of other packages, but never got to bootstrapping a kernel, also python bootstrap compiles itself with python at one point, and thus would not support cross-compiling (as of 2.4.2 anyway), so stock portage on Wrt54G was a wash -- "emerge system" is impossible without python. Also there were some issues with the (now) old uclibc version I was using, but it's difficult to know how far an active project like that has come since I tried this. Unfortunately an untimely hard drive crash and finishing a masters' degree interfered with taking it much further at the time - a full-time job later, and the rest is a blur :-)

though I hope to return to it again. It's all rather academic until we can actually get to a gentoo base emerge system, then think about a kernel and the necessary jffs2/squashfs utilities to bootstrap ourselves and our fresh new kernel into the flash :-)

and move off the NFS root.
_________________
i7-3610QM | E5-2670 | FX-8300

GNUtoo · Veteran Joined: 05 May 2005 Posts: 1919

abandoning is a bad idea and that wasn't what i meant
but Working in coordination with FreeWrt is a better idea and could be great for both them and us
because FreeWRT is a gentoo-like firmware adding some little things such as:
*more package =>benefit for both
*having a gentoo-ish configuration such as configuration in /etc/conf.d/ and using the gentoo baselayout could be great
*improoving their build system to suport use flags and similar(they have already basic support for that)

at the end we could have a firmware where you could choose between the gentoo way and the FreeWRT way

the thing that we must keep in mind is to stay embedded because there are devices that work without an external filesystem while supporting an external filesystem as an option(could be ram or usb or network filesystems)

we should also offer the option of compiling the kernel ourselves in order to support not common things such as:
->the linksys devices that has a pc-card or pcmcia interface for the nozimi card...we could add any other cards and add their drivers such as usb2.0 cards,Gigabit cards...others wireless cards
->the routers with a mini-pci card...here we could support others cards such as ralink ones with the rt2x00 driver in ap mode(the problem here is that mabe it's broken in the last git relase...i wasn't able to get help on the irc because i don't know if the interface that is ra0 correspond to the old wlan_master0 one)
->add support for a lot of usb devices such as network cards

GNUtoo · Veteran Joined: 05 May 2005 Posts: 1919

abandoning is a bad idea and that wasn't what i meant
but Working in coordination with FreeWrt is a better idea and could be great for both them and us
because FreeWRT is a gentoo-like firmware adding some little things such as:
*more package =>benefit for both
*having a gentoo-ish configuration such as configuration in /etc/conf.d/ and using the gentoo baselayout could be great
*improoving their build system to suport use flags and similar(they have already basic support for that)

at the end we could have a firmware where you could choose between the gentoo way and the FreeWRT way

the thing that we must keep in mind is to stay embedded because there are devices that work without an external filesystem while supporting an external filesystem as an option(could be ram or usb or network filesystems)

we should also offer the option of compiling the kernel ourselves in order to support not common things such as:
->the linksys devices that has a pc-card or pcmcia interface for the nozimi card...we could add any other cards and add their drivers such as usb2.0 cards,Gigabit cards...others wireless cards
->the routers with a mini-pci card...here we could support others cards such as ralink ones with the rt2x00 driver in ap mode(the problem here is that mabe it's broken in the last git relase...i wasn't able to get help on the irc because i don't know if the interface that is ra0 correspond to the old wlan_master0 one)
->add support for a lot of usb devices such as network cards

by the way i realy wait for a firmware that can offer a 2.6 kernel with the broadcom cards with the 43xx driver.,..
do you know if it's already possible to have the master mode with theses cards?
because openwrt has 2.6 branches that have been relased but within theses branches the wireless doen't work

a7thson · Posted: Wed Jun 06, 2007 1:24 am Post subject:

GNUtoo · Veteran Joined: 05 May 2005 Posts: 1919

a7thson · Posted: Wed Jun 06, 2007 3:51 pm Post subject:

No I haven't had a chance to play with FreeWrt yet. Have you successfully compiled python with its cross-compiler? I couldn't do so with my cross-compiler toolchain back when I wrote this howto, so assume it would still have to be done natively. Otherwise no python bootstrap, thus no python, thus no portage. Since python compiles and runs python to build python in its build process :-)

casso · Posted: Fri Jul 06, 2007 12:20 am Post subject: FreeWRT tested

Hi,
Sorry it has been so long since my last reply. I was working on another project that is almost complete. I sucessfully installed FreeWRT to my ASUS router and found it useful enough. Problem with most of these WRT distros is that they all use a 2.4 kernel, and I need (yes need) a 2.6 kernel for some additional things I want my router to do.

Briefly, I want my router to do the following:

To implement the first point, I will use dd-wrt, which is, from my perspective, the Fedora of WRT distros. I will build a 2.6 kernel and kexec to go with dd-wrt, modifying the original image to fit these inside. Lastly, I will use another computer to load my Gentoo system from an external hard drive, and be able to use Gentoo on my router as if it was any other mipsel computer.

Because I want to be able to have the choice of either a basic distro suitable for a network appliance, and a full Gentoo distro, I have chosen to use kexec to switch between. If I wanted to have just the full distro, I could have made a firmware image that would boot my Gentoo system easily. I could have included the kernel and initramfs image within the firmware image (I have space enough), but I opted to keep these outside for a few reasons, one was to allow an easy upgrade of the Gentoo kernel without the need to reflash the firmware.

This thread seems to be looking at running Gentoo on the router as if it was any other mipsel based computer. Here I am looking at running both a network appliance setup (using dd-wrt) and Gentoo on the same hardware, but not simultaneously. I may later look more heavily into GNAP and embedded Gentoo, moving away from dd-wrt and across to a router that runs both systems as Gentoo. I still want to have an easy to configure system for the firmware image, one that uses either jffs or nvram to make changes to its configuration.
_________________
Need assistance with any of the following? Just PM me and I will see what I can do

LDAP, Kerberos , SSH, Samba, PPP, DHCP, NTP, autofs, CUPS