i think i broke /dev

erik258 · Posted: Thu Jan 04, 2007 6:17 pm Post subject: i think i broke /dev

I was moving a diskless client's root directory and seem to have broken things. I tried to reassemble, but couldn't get the computer working entirely correctly (but i'm close!). The problem appears to be in /dev but i'm not sure.

for each terminal started in /etc/inittab, i get (on boot)
Couldn't open //dev/tty1
and so on through "//dev/tty11". consolefont fails as well. no login comes up (of course, agetty can't open the tty devices) and i can't ssh into the box either (also makes sense). Then in a little bit, for each of the consoles...
INIT: Id "c1" respawning too fast: disabled for 5 minutes
and so on for the other consoles, followed by
INIT: no more processes left in this runlevel
Init then continues to kill the cX consoles every five minutes + a little bit.

I'd like a really quick fix for this problem, and was wondering if anyone had an idea. I have tried copying the /dev folder over from a clean install as well as using the same copy command (cp -R -v -p) to copy them to the directory but neither worked, nor did deleting /dev entirely and hoping for udev to create all on boot.

Anybody have suggestions? I'd rather not have to reinstall, being ever so lazy. The box isn't a production end or important computer in any way, just a diskless netboot test box. Therefore i have convenient access to the filesystem ; )
_________________
Configuring a Firewall? Try my iptables configuration
LinuxCommando.com is my blog for linux-related scraps and tidbits. Stop by for a visit!

harrisonmetz · Posted: Thu Jan 04, 2007 6:38 pm Post subject:

What kernel does it boot there, and you have udev, not devfs or any old stuff?

didymos · Posted: Thu Jan 04, 2007 6:48 pm Post subject:

You should only need /dev/console and /dev/null to boot. I don't know why copying over a device tree wouldn't fix that. You do anything unnatural to /etc/inittab lately?
_________________
Thomas S. Howard

erik258 · Posted: Thu Jan 04, 2007 6:53 pm Post subject:

it's all pretty much new, in fact i just got done with a system update. udev 1.03, vanilla 2.6.17.13 kernel.
_________________
Configuring a Firewall? Try my iptables configuration
LinuxCommando.com is my blog for linux-related scraps and tidbits. Stop by for a visit!

erik258 · Posted: Thu Jan 04, 2007 7:06 pm Post subject:

harrisonmetz:
unfortunately, no. here'sthe relevant section of my /etc/inittab (but i did copy all of these directories around a bit, although i think i used cp -R -p on every copy so permissions/special files should be fine i would think).

didymos · Posted: Thu Jan 04, 2007 7:09 pm Post subject:

It's bizarre that copying /dev doesn't work for another reason, because that error message usually means there's some sort of problem creating this or that tty/ttyS, either because of inittab or it not being in /dev.
_________________
Thomas S. Howard

didymos · Posted: Thu Jan 04, 2007 7:18 pm Post subject:

Did I misunderstand, or did you have a working diskless station before this? If it wasn't working before, wouldn't some sort of initrd/initramfs be needed to supply init and getty and whatnot?
_________________
Thomas S. Howard

erik258 · Posted: Thu Jan 04, 2007 7:32 pm Post subject:

crw------- 1 root tty 4, 0 Aug 3 09:23 tty0
crw------- 1 root tty 4, 1 Aug 3 09:23 tty1

those look just as they do on working systems...

and to clarify, yes it was working before, and is using PXElinux and an NFS-mounted root to boot.

Any other parts of the filesystem suspect? How do those permissions look?
_________________
Configuring a Firewall? Try my iptables configuration
LinuxCommando.com is my blog for linux-related scraps and tidbits. Stop by for a visit!

didymos · Posted: Fri Jan 05, 2007 7:45 am Post subject:

Well, maybe it is a permissions thing, as tty0 and tty1 have 660 on my system, whereas yours are only 600.
_________________
Thomas S. Howard

erik258 · Posted: Fri Jan 05, 2007 10:57 pm Post subject:

i'm not sure, it looks like the permissions get changed around a bit when the terminal devices are in use.

i bit the bullet and rebuilt the brand new system, which is now back up and running. Funny, i still can't figure out how i broke it.
_________________
Configuring a Firewall? Try my iptables configuration
LinuxCommando.com is my blog for linux-related scraps and tidbits. Stop by for a visit!

didymos · Posted: Sat Jan 06, 2007 10:50 pm Post subject:

Yeah, I forgot about the whole dynamic permissions stuff. I guess it'll just remain a mystery. Unless, of course, you accidentally do it again. Or anyone else does, for that matter, since we have no idea what might trigger it.
_________________
Thomas S. Howard

boydo · Posted: Sun Feb 04, 2007 5:22 pm Post subject:

hmmmm... Anything further on this? I have the exact same problem after rsyncing from one system to another. I have done this successfully at least 3 or 4 times, but this time I have the same messages on boot as was first posted here by erik258.

for each terminal started in /etc/inittab, i get (on boot)
Couldn't open //dev/tty1
and so on through "//dev/tty11". consolefont fails as well. no login comes up (of course, agetty can't open the tty devices) and i can't ssh into the box either (also makes sense). Then in a little bit, for each of the consoles...
INIT: Id "c1" respawning too fast: disabled for 5 minutes
and so on for the other consoles, followed by
INIT: no more processes left in this runlevel
Init then continues to kill the cX consoles every five minutes + a little bit.

I have checked the permissions of the /dev/tty?, and they are 600 and 660, exactly the same as my laptop. So thinking it's not a permissions issue. The only thing I can think of at this point is that since I did a genkernel (I do this for new hardware and then do a custom kernel later) that maybe its something in the initramfs. So to troubleshoot, I am in the process of building a new kernel now. Just wondering if there are any other suggestions out there????

erik258 · Posted: Sun Feb 04, 2007 7:36 pm Post subject:

I don't know why this happens. I have successfully copied /dev and the rest of the filesystem too for new computers and haven't had this problem again.
_________________
Configuring a Firewall? Try my iptables configuration
LinuxCommando.com is my blog for linux-related scraps and tidbits. Stop by for a visit!

boydo · Posted: Sun Feb 04, 2007 10:01 pm Post subject:

In chrooted environment I re-emerged udev and hotplug. That didn't work. The I re-emerged baselayout. Did the trick! Now I know the how, but not the why...

Thanks,

Boyd