View previous topic :: View next topic |
Author |
Message |
superdeez n00b
Joined: 10 May 2015 Posts: 61
|
Posted: Sat Mar 18, 2023 8:55 am Post subject: Clone an entire filesystem |
|
|
When I built my current system for my root drive, I used an SSD that (while free) was on the small side and finally I ran out of space.
Not sure if dd would work if the two drives are of different sizes. When I've used cp before I've found it doesn't copy over *everything* (though it's been good enough when I've moved my home dir around between different drives). What's the best way to attempt to copy over my entire / and /boot filesystems?
root is ext4, boot is ext2. |
|
Back to top |
|
|
sdauth l33t
Joined: 19 Sep 2018 Posts: 659 Location: Ásgarðr
|
Posted: Sat Mar 18, 2023 10:46 am Post subject: |
|
|
rsync -avHAX /mnt/oldssd_root/ /mnt/newssd_root/
rsync -avHAX /mnt/oldssd_boot/ /mnt/newssd_boot/
should do to the job. I used that a few weeks ago when moving from ext4 to btrfs.
then restore UUID of your old boot & root partition to the new one. (With tune2fs -U *UUID* ..)
Otherwise, since the old one is smaller, you could simply dd the old one to the new one and then expand the root partition. |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54642 Location: 56N 3W
|
Posted: Sat Mar 18, 2023 12:12 pm Post subject: |
|
|
superdeez,
dd copies everything, including the empty space. That's a bad thing for an SSD because even if its erased empty space on the source, it will incur a write on the destination.
rsync is good, it only copies the content of the filesystems.
You must do the copy as root, while the source filesystem is not mounted. If its in use, files open for writing will be corrupt in the copy.
You can update the UUIDs but they are supposed to be Universally Unique ... the kernel will hate you for two identical UUIDs in the same system at the same time, so I advise against it.
Chroot into the copy, once its done and update all the UUID consumers. That's future proof.
BIOS booting users will need to reinstall grub to the MBR as that is outside any filesystems. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
superdeez n00b
Joined: 10 May 2015 Posts: 61
|
Posted: Sat Apr 15, 2023 3:41 pm Post subject: |
|
|
I will come back and update this thread with the (apparently) successful conclusion.
I had never ever thought of using rsync to mirror a filesystem like that. I'd always used some variation of cp -R.
It worked like a charm. I was replacing a 128GiB ssd that was unused and free to me. Since my early Linux days I've used one hdd for root and another, larger (4TiB, in this case) ssd for my home dir. Though I may get flak for this, I used a low hours 500GiB mechanical hdd I had in a laptop that bit the dust last year. Very slightly slower, but I already own it and it only has around 2500 hours on it.
No UUIDs or anything here, my machines all still use lilo as a bootloader.
But rsync -avHAX worked perfectly, then I just chrooted into the new HDD's root, ran lilo and it all works.
I only had to run one srync command as I mounted both /boots before running it and it just copied that all as well.
Thank you to all who replied. |
|
Back to top |
|
|
dmpogo Advocate
Joined: 02 Sep 2004 Posts: 3459 Location: Canada
|
Posted: Sat Aug 24, 2024 5:31 pm Post subject: |
|
|
NeddySeagoon wrote: | superdeez,
dd copies everything, including the empty space. That's a bad thing for an SSD because even if its erased empty space on the source, it will incur a write on the destination.
rsync is good, it only copies the content of the filesystems.
You must do the copy as root, while the source filesystem is not mounted. If its in use, files open for writing will be corrupt in the copy.
You can update the UUIDs but they are supposed to be Universally Unique ... the kernel will hate you for two identical UUIDs in the same system at the same time, so I advise against it.
Chroot into the copy, once its done and update all the UUID consumers. That's future proof.
BIOS booting users will need to reinstall grub to the MBR as that is outside any filesystems. |
Can you clarify what "You must do the copy as root, while the source filesystem is not mounted. If its in use, files open for writing will be corrupt in the copy." means, I.e. how I can use rsync on unmounted source ? |
|
Back to top |
|
|
Hu Administrator
Joined: 06 Mar 2007 Posts: 22870
|
Posted: Sat Aug 24, 2024 5:53 pm Post subject: |
|
|
I think that was a confusing phrasing. If the filesystem is mounted read-write, then its superblock may be inconsistent (bad when using dd; irrelevant when using rsync). Additionally, if it is mounted read-write and any applications have files open for write, then those files may also be inconsistent. To guard against both cases, the source filesystem should be mounted -o ro (read-only), so that the kernel enforces that no application can write to any file in that filesystem. In my experience, though I am not aware of a hard guarantee in this regard, the superblock tends to be consistent for a read-only filesystem, which benefits dd. As Neddy says though, dd is not a good choice if your filesystem can be copied by rsync. You might use dd if the filesystem is damaged and you are trying to create a clone for recovery purposes. |
|
Back to top |
|
|
C5ace Guru
Joined: 23 Dec 2013 Posts: 488 Location: Brisbane, Australia
|
Posted: Sat Aug 24, 2024 10:34 pm Post subject: |
|
|
I use this batch file to backup a running laptop with MBR and grub2 via WIFI to my desktop"
Code: | #!/bin/sh
echo TOSH-Backup start
date | tee /var/log/TOSH-Backup.log
/usr/bin/rsync -crtpogvlHDisuz --rsh=ssh --progress --delete --stats --exclude=/dev/* --exclude=/tmp/* --exclude=/proc/* --exclude=/run/* --exclude=/sys/* --exclude=/home/*/.cache/mozilla/* --exclude=/home/*/.cache/thumbnails/* --exclude=/home/*/.cache/thunderbird/* --exclude=/root/.cache/mozilla/* --exclude=/root/.cache/thumbnails/* --exclude=/root/.cache/thunderbird/* --exclude=/var/cache/distfiles/* --exclude=/var/db/repos/gentoo/* root@tosh.itw.lan:/ /home/backups/TOSH-FULL-Backup-21-08-2024/ | tee -a /var/log/TOSH-Backup.log
date | tee -a /var/log/TOSH-Backup.log
echo TOSH-Backup end |
To restore to the running laptop:
Code: | #!/bin/sh
echo TOSH-Restore start
date | tee /var/log/TOSH-Restore.log
/usr/bin/rsync -crtpogvlHDisuz --rsh=ssh --progress --delete --stats --exclude=/dev/* --exclude=/tmp/* --exclude=/proc/* --exclude=/run/* --exclude=/sys/* --exclude=/home/*/.cache/mozilla/* --exclude=/home/*/.cache/thumbnails/* --exclude=/home/*/.cache/thunderbird/* --exclude=/root/.cache/mozilla/* --exclude=/root/.cache/thumbnails/* --exclude=/root/.cache/thunderbird/* --exclude=/var/cache/distfiles/* --exclude=/var/db/repos/gentoo/* /home/backups/TOSH-FULL-Backup-21-08-2024/ root@tosh.itw.lan:/ | tee -a /var/log/TOSH-Backup.log
date | tee -a /var/log/TOSH-Restore.log
echo TOSH-Restore end |
To restore to a new drive, I boot the laptop using "Minimum Installation CD", create, format and mount the required partitions, then run TOSH-Restore. When completed, I 'chroot' and run 'grub-install /dev/sda' and 'grub-mkconfig -o /boot/grub/grub.cfg'. _________________ Observation after 30 years working with computers:
All software has known and unknown bugs and vulnerabilities. Especially software written in complex, unstable and object oriented languages such as perl, python, C++, C#, Rust and the likes. |
|
Back to top |
|
|
Goverp Advocate
Joined: 07 Mar 2007 Posts: 2187
|
Posted: Sun Aug 25, 2024 10:56 am Post subject: |
|
|
FWIW I take backups and copy disks and so forth in Single User mode:
IIUC that should mean the only source of processes that can update things is the terminal on which I'm running (as root), so if there is background I/O activity, it's my fault.
I presume/hope there's a systemd equivalent. _________________ Greybeard |
|
Back to top |
|
|
dmpogo Advocate
Joined: 02 Sep 2004 Posts: 3459 Location: Canada
|
Posted: Sun Aug 25, 2024 5:42 pm Post subject: |
|
|
Sorry for somewhat hijacking the thread, but I just resized/moved partitions and /home filesystem around to make extra space for root / . (*) Now I am in position to delete and recreate larger root partition and expand the filesystem.
What stops is a question whether I can do it on life partition (should be, second step is definitely doable), but more that I will get a new PARTUUID and that is how root is build in into my EFI stub kernel parameters. So I can rebuild kernel with new parameter, but is there a way to change manually PARTUUID the way filesystem UUID can be changed with tune2fs ? Or are some advance versions of fdisk ( I tend to use this primitive one) allow setting PARTUUID when creating the partition ?
The danger is of course that if anything goes wrong I'll be unbootable
(*) having experienced remarkable hardware problems on the way, but that is another story |
|
Back to top |
|
|
Hu Administrator
Joined: 06 Mar 2007 Posts: 22870
|
Posted: Sun Aug 25, 2024 7:10 pm Post subject: |
|
|
As I read the interactive help of fdisk, you can enter expert mode, then use u to change the partition UUID. |
|
Back to top |
|
|
dmpogo Advocate
Joined: 02 Sep 2004 Posts: 3459 Location: Canada
|
Posted: Sun Aug 25, 2024 9:50 pm Post subject: |
|
|
Hu wrote: | As I read the interactive help of fdisk, you can enter expert mode, then use u to change the partition UUID. |
Right, thanks ! Never new there is something interesting behind 'x' command
Last thing, to convince oneself that I can delete and recreate the increased partition on the mounted drive despite the warnings (yes, with the same starting block, of course) |
|
Back to top |
|
|
szatox Advocate
Joined: 27 Aug 2013 Posts: 3477
|
Posted: Mon Aug 26, 2024 8:31 am Post subject: |
|
|
You can.
Partition table is already loaded to memory, your system wont even see the change until you rescan (though fdisk should trigger it automatically after you write changes; if it fails, partprobe is your friend). _________________ Make Computing Fun Again |
|
Back to top |
|
|
dmpogo Advocate
Joined: 02 Sep 2004 Posts: 3459 Location: Canada
|
Posted: Mon Aug 26, 2024 10:51 pm Post subject: |
|
|
szatox wrote: | You can.
Partition table is already loaded to memory, your system wont even see the change until you rescan (though fdisk should trigger it automatically after you write changes; if it fails, partprobe is your friend). |
Thanks for confirmation, I thought that it should be fine, but ... It worked fine at the end. I even did not mistype PARTUUID I needed to have Did not know about partprobe, useful info |
|
Back to top |
|
|
|