View previous topic :: View next topic |
Author |
Message |
therik n00b
Joined: 14 Jul 2024 Posts: 7
|
Posted: Sat Jul 20, 2024 11:38 pm Post subject: Accidentally deleted /sys /dev and /proc |
|
|
Hello,
I did stupid. I just rm -rfed /usr/bin/initramfs while my /proc, /sys and /dev were still rbinded into it.
The machine is still running fairly OK, I manually recreated /dev/null and can even use firefox.
Is this something I need to be worried about?
One thing that particularly worries me is some of the special files that were mounted:
Code: | # mount |grep 'on /sys'
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
efivarfs on /sys/firmware/efi/efivars type efivarfs (rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime)
tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
tracefs on /sys/kernel/debug/tracing type tracefs (rw,nosuid,nodev,noexec,relatime)
|
Specifically, /sys/firmware/efi/efivars.
Did I just brick my board?
Other stuff that is mounted now is this:
Code: |
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=33,pgrp=1,timeout=0,minproto=5,maxproto=5,direct)
none on /proc type proc (rw,relatime)
|
Code: | devtmpfs on /dev type devtmpfs (rw,nosuid,size=4096k,nr_inodes=8219528,mode=755,inode64)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,inode64)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,nosuid,nodev,relatime,pagesize=2M)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
|
Is it safe to reboot the machine? |
|
Back to top |
|
|
therik n00b
Joined: 14 Jul 2024 Posts: 7
|
Posted: Sun Jul 21, 2024 2:34 pm Post subject: |
|
|
So, to answer my own questions, I did a reboot and it was mostly OK.
I had to redo the boot order and boot entries, because they got wiped from efivars, but turns out that the motherboard is sensible and didn't let the OS erase its own firmware.
I haven't noticed any other damage so far. |
|
Back to top |
|
|
pjp Administrator
Joined: 16 Apr 2002 Posts: 20484
|
Posted: Sun Jul 21, 2024 3:06 pm Post subject: |
|
|
/sys, /dev, and /proc are created dynamically, so most of that should be fine.
In some situations, a couple of entries in /dev may be required before they are created dynamically. For example: Code: | $ tar tvJf stage3-amd64-nomultilib-openrc-20240714T170402Z.tar.xz ./dev/
drwxr-xr-x root/root 0 2024-07-14 15:36 ./dev/
crw------- root/root 5,1 2024-07-14 15:36 ./dev/console
crw-rw-rw- root/root 1,3 2024-07-14 15:36 ./dev/null | However, that you apparently removed something else requiring you to re-create boot entries makes me wonder how you removed these items and whether or not anything else was removed. _________________ Quis separabit? Quo animo? |
|
Back to top |
|
|
Hu Administrator
Joined: 06 Mar 2007 Posts: 22613
|
Posted: Sun Jul 21, 2024 3:10 pm Post subject: |
|
|
As OP showed, there were various magic filesystems mounted under /sys, and the rm -r would have traversed into those. One of those, efivarfs, is commonly associated with firmware boot entry management.
Recursing into pstore might have deleted valuable data. All the others look to me like pseudo-filesystems that would be fully recovered by the reboot. |
|
Back to top |
|
|
pjp Administrator
Joined: 16 Apr 2002 Posts: 20484
|
Posted: Sun Jul 21, 2024 3:21 pm Post subject: |
|
|
Thanks. I didn't realize anything under /sys was ever not dynamic. I thought the rm might have been more broad than the listed directories. _________________ Quis separabit? Quo animo? |
|
Back to top |
|
|
therik n00b
Joined: 14 Jul 2024 Posts: 7
|
Posted: Sun Jul 21, 2024 7:07 pm Post subject: |
|
|
Hu wrote: | Recursing into pstore might have deleted valuable data. |
Is pstore just to hold crash dumps and logs?
If it's just some potential logs from previous kernel crashes then that's not an issue, it's just my home machine. But it's nice to know that this exists, I'll definitely be checking this regularly.
Thanks for your help. So far it looks like a very cheap lesson.
pjp wrote: | However, that you apparently removed something else requiring you to re-create boot entries makes me wonder how you removed these items and whether or not anything else was removed. |
I'm pretty sure it was just /proc /sys and /dev, although it was a very close shave.
I'm not 100% sure what exactly happened, since part of it was from a script but I'm sure I had /proc /dev and /sys mounted into /usr/src/initramfs/{dev,proc,sys}, and I had my main ssd mounted somewhere in there too. I chrooted in, ran something and it hanged, and I suspended the chroot, but didn't bother killing it, since I was about to reboot anyway.
Then I ran a script that would normally build a new kernel and put it in /boot, but part of that script involves deleting the previous /usr/src/initramfs and rebuilding it again. And although the script does check and umount anything mounted in there before deleting it, it didn't check whether the umount actually succeeded and proceeded with deleting the folder.
Obviously, the umount failed cause there was the suspended process still hanging from inside the chroot there.
I'm VERY grateful that the hanged process didn't have anything open from my root ssd, so it got unmounted cleanly by a sheer luck.
Lessons learned. |
|
Back to top |
|
|
Hu Administrator
Joined: 06 Mar 2007 Posts: 22613
|
Posted: Sun Jul 21, 2024 9:32 pm Post subject: |
|
|
I think pstore is just dumps and logs, yes. However, there may be uses of pstore that I do not know about.
I suggest making deeper changes to protect against this in the future. The quick and cheap change would be to make all those mounts read-only, such that any attempt to rm anything in there fails with Read-only filesystem (EROFS). Another cheap change would be to modify your unmounting script so that failure to umount is fatal to the script.
The most invasive change, which would also reduce the write workload, would be to migrate to the kernel's initramfs generation facility, which takes a list file and generates the initramfs from the files in that list - with no need to create a temporary chroot that mirrors what will be in the initramfs. See Documentation/filesystems/ramfs-rootfs-initramfs.rst for how to use this. If you need more help, please ask. |
|
Back to top |
|
|
figueroa Advocate
Joined: 14 Aug 2005 Posts: 3005 Location: Edge of marsh USA
|
Posted: Tue Jul 23, 2024 4:52 am Post subject: |
|
|
Restore from last night's backup!
Truth be told, I don't backup the content of /proc, /dev/ or /sys, just the directories themselves. But the greatest comfort you can have is a nice warm backup. _________________ Andy Figueroa
hp pavilion hpe h8-1260t/2AB5; spinning rust x3
i7-2600 @ 3.40GHz; 16 gb; Radeon HD 7570
amd64/23.0/split-usr/desktop (stable), OpenRC, -systemd -pulseaudio -uefi |
|
Back to top |
|
|
machinewaiCEO n00b
Joined: 24 Jul 2024 Posts: 9 Location: Turkey
|
Posted: Mon Aug 05, 2024 5:48 pm Post subject: |
|
|
This is the most linux thing I have read all day _________________ We will destroy the jedi. |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|