| View previous topic :: View next topic |
| Author |
Message |
eltech
Guru
Joined: 05 Nov 2002
Posts: 582
Location: New York
|
 Posted: Thu Dec 28, 2006 5:37 pm Post subject: backup script to a mounted hard drive. |
 |
|
Hello all ..
I'm trying to create a backup script that will completely back up my whole hard drive that I can then restore from in the event of a hard drive failure.
So I mounted a 2nd drive since I dont have a tape drive and Im using the DUMP command using the following script.
| Code: |
#!/bin/sh
TAPE=/dev/hdb2
FILESYSTEMS="/"
NETWORKFILESYSTEMS=
if [ "$FILESYSTEMS" ]; then
echo Backing up these local filesystems: $FILESYSTEMS
fi
if [ "$NETWORKFILESYSTEMS" ]; then
echo Backing up these network filesystems: $NETWORKFILESYSTEMS
fi
if [ "$FILESYSTEMS" ]; then
for filesystem in $FILESYSTEMS
do
echo "Dumping $filesystem..."
dump 0 -u -B4194304 -f $TAPE $filesystem
done
fi
if [ "$NETWORKFILESYSTEMS" ]; then
for filesystem in $NETWORKFILESYSTEMS
do
echo "Tar'ing $filesystem..."
tar cf $TAPE -C $filesystem $filesystem
done
fi
|
Problem is that once its complete
| Code: |
Backing up these local filesystems: /
Dumping /...
DUMP: Date of this level 0 dump: Thu Dec 28 12:10:47 2006
DUMP: Dumping /dev/hda2 (/) to /dev/hdb2
DUMP: Label: /
DUMP: Writing 10 Kilobyte records
DUMP: mapping (Pass I) [regular files]
DUMP: mapping (Pass II) [directories]
DUMP: estimated 1346256 blocks on 0.32 tape(s).
DUMP: Volume 1 started with block 1 at: Thu Dec 28 12:10:50 2006
DUMP: dumping (Pass III) [directories]
DUMP: dumping (Pass IV) [regular files]
DUMP: 67.29% done at 3019 kB/s, finished in 0:02
DUMP: Closing /dev/hdb2
DUMP: Volume 1 completed at: Thu Dec 28 12:18:37 2006
DUMP: Volume 1 1506040 blocks (1470.74MB)
DUMP: Volume 1 took 0:07:47
DUMP: Volume 1 transfer rate: 3224 kB/s
DUMP: 1506040 blocks (1470.74MB) on 1 volume(s)
DUMP: finished in 467 seconds, throughput 3224 kBytes/sec
DUMP: Date of this level 0 dump: Thu Dec 28 12:10:47 2006
DUMP: Date this dump completed: Thu Dec 28 12:18:37 2006
DUMP: Average transfer rate: 3224 kB/s
DUMP: DUMP IS DONE
|
I mount hdb2 to /media/backups
but when trying to set TAPE to this location I get an error that its a directory and not a device. Wont work.
When i specify /dev/hdb2 when mounted, the dump completes, but i'm not sure its successful because when I do go into the /media/backups folder and do a ls, there isnt anything, But i get this error.
| Code: | | EXT3-fs error (device hdb2): ext3_readdir: bad entry in directory #2: rec_len is smaller than minimal - offset=0, inode=0, rec_len=0, name_len=0 |
Now the drive is hdb2 because when i tried using hdb and hdb1 i got the same error.
I have formatted the partitions as ext3: mke2fs -j so I'm not sure what to do at this point .. |
|
| Back to top |
|
 |
erik258
Advocate
Joined: 12 Apr 2005
Posts: 2650
Location: Twin Cities, Minnesota, USA
|
| Posted: Thu Dec 28, 2006 6:47 pm Post subject: |
|
|
lots of people don't like dump and it may be broken. In fact, Linus himself seems to have something against it.
If i wanted to do this I think I would just dd if=/dev/hd-part-to-backup of=/backupdisk-mountpoint/date.diskimage.image
you can then mount with mount -o loop /backupdisk-mountpoint/date.diskimage.image /mnt/backup and you have a complete fs intact. I did this with windows to migrate unbeknownst to a new drive (it almost worked, but windows caught on when it noticed it was on SATA and not IDE bus ; ) )
The downside is you have an image the size of your disk partition. You might be able to compress it though if you wanted - it should compress fairly well.
_________________
Configuring a Firewall? Try my iptables configuration
LinuxCommando.com is my blog for linux-related scraps and tidbits. Stop by for a visit! |
|
| Back to top |
|
|
eltech
Guru
Joined: 05 Nov 2002
Posts: 582
Location: New York
|
| Posted: Thu Dec 28, 2006 6:51 pm Post subject: |
|
|
| erik258 wrote: | lots of people don't like dump and it may be broken. In fact, Linus himself seems to have something against it.
If i wanted to do this I think I would just dd if=/dev/hd-part-to-backup of=/backupdisk-mountpoint/date.diskimage.image
you can then mount with mount -o loop /backupdisk-mountpoint/date.diskimage.image /mnt/backup and you have a complete fs intact. I did this with windows to migrate unbeknownst to a new drive (it almost worked, but windows caught on when it noticed it was on SATA and not IDE bus ; ) )
The downside is you have an image the size of your disk partition. You might be able to compress it though if you wanted - it should compress fairly well. |
thanks for the response.. Not a problem, but we are talking about a system running and in production, dd cant run at this stage.. correct? the drive cannot be in use which dd is being usd on .. am i correct?
also i would need to be able to restore such a backup to another drive.. how would this be possible? |
|
| Back to top |
|
|
erik258
Advocate
Joined: 12 Apr 2005
Posts: 2650
Location: Twin Cities, Minnesota, USA
|
| Posted: Thu Dec 28, 2006 6:53 pm Post subject: |
|
|
i don't know, but i think so. I am going to find out.
edit: that operation is going right now and although the computer's bogged down a bit (also running BOINC numbercruncher and many firefoxes) the problem could certainly be remedied with a better nice setting for the dd command.
_________________
Configuring a Firewall? Try my iptables configuration
LinuxCommando.com is my blog for linux-related scraps and tidbits. Stop by for a visit! |
|
| Back to top |
|
|
erik258
Advocate
Joined: 12 Apr 2005
Posts: 2650
Location: Twin Cities, Minnesota, USA
|
| Posted: Thu Dec 28, 2006 6:57 pm Post subject: |
|
|
I just did that very thing, a disk-to-disk transfer running dd on a live root fs and dumping it to an image file on another disk. It worked wonderfully.
Remember, dd is running in memory. It just has to read those sectors on the hard drive. It doesn't need to read them all at once or anything so it can play nice with running programs. It could be outdated by the time it's finished though, if the system changes while dd is running.
I was able to mount the image as described and then could navigate around in it, and everything was still there. Try it, it's neat!
_________________
Configuring a Firewall? Try my iptables configuration
LinuxCommando.com is my blog for linux-related scraps and tidbits. Stop by for a visit! |
|
| Back to top |
|
|
erik258
Advocate
Joined: 12 Apr 2005
Posts: 2650
Location: Twin Cities, Minnesota, USA
|
| Posted: Thu Dec 28, 2006 7:09 pm Post subject: |
|
|
you can restore a backup of a partition like this (it's really a partition dump i guess, even moreso than dump would do) by creating a partition of the same size on another disk and then dd if=fs.image of=/dev/newdisk-partition-special-file
That's how i copied ntfs without write support for ntfs ; ) you still have write support for the drive itself.
_________________
Configuring a Firewall? Try my iptables configuration
LinuxCommando.com is my blog for linux-related scraps and tidbits. Stop by for a visit! |
|
| Back to top |
|
|
eltech
Guru
Joined: 05 Nov 2002
Posts: 582
Location: New York
|
| Posted: Thu Dec 28, 2006 7:21 pm Post subject: |
|
|
erik,
sounds like a success can you outline step by step how you got this done?
Also this will be a dd running in the early AM when only logging will really change so it will be a sufficient enough backup ..
I would really appreciate a step by step outline if you could...
Thanks! |
|
| Back to top |
|
|
erik258
Advocate
Joined: 12 Apr 2005
Posts: 2650
Location: Twin Cities, Minnesota, USA
|
| Posted: Thu Dec 28, 2006 8:20 pm Post subject: |
|
|
it's so very easy. but please note, i've never used this to back up anything, and am not actually in charge of any production boxes other than my own. Im telling you that for your own good, so that you don't think I can really assure you this fits your needs.
I)Cloning a partition
1) choose the partition to clone
2) choose a storage space (hmm, better not be on same partition, might get a bad image file .. but that's a pointless backup anyways.)
3) execute the dd.
II)Restoring from Clone
1) having the right partition destination
2) execute the dd.
I)Cloning a partition
1) choose the partition to clone
of course, you already know what partition you want to copy? It can't be the partition on which you intend to store the image of course. What you may not know is the size you'll need to use for the partition to put it back on another hard drive. that's the tricky part, especially if the cylinder size on the new drive doesn't match up with the one on the old drive. Here, look at this website i completed to document my attempts with windows XP. I am still confident it would work if only i had put it on PATA, or if windows was more like linux. BUT I HAVE NEVER RESTORED A LINUX PARTITION LIKE THIS for backup purposes so you'll have to test it. before something fails, preferably.
since it's very important the partition be the same size (perhaps minimum, perhaps exact) and have the same cylinder size I highly recommend you put that info in your created image by capturing the output of fdisk -l somewhere on the partition you intend to back up. just save this output in a file:
| Quote: | pascal dan # fdisk -l
Disk /dev/sda: 160.0 GB, 160041885696 bytes
255 heads, 63 sectors/track, 19457 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 3648 29302528+ 7 HPFS/NTFS
/dev/sda2 3649 3651 24097+ 83 Linux
/dev/sda3 3652 5476 14659312+ 83 Linux
/dev/sda4 5477 19457 112302382+ 5 Extended
/dev/sda5 5477 5599 987966 82 Linux swap / Solaris
|
now you know all you need to know to put this back on a new hard drive.
2) choose a storage space
It better be on a different partition. it could be on the same disk, but not the same partition. You'll need the free space to store the whole partition, not just the used amount.
3) execute the dd.
this part is also documented in the above - linked website. This command would clone a copy of the hard drive partition /dev/sda3, and save it in /mnt/backup/root.image (has to be moutned, on a different partition)
| Code: | | dd if=/dev/sda3 of=/mnt/backup/root.image |
you now have an exact replica of the data on that partition, filesystem metadata and all. Restore's as easy as... well, read on.
II)Restoring from Clone
1) having the right partition destination
look at the website to get exact info on what needs to be the same on the new partition. The relevant lines are also reddened above, in the example given here.
2) execute the dd.
this is equally simple. you can use any unix with dd and acces to your new hard drive partition. in this example, we have set up /dev/sda3 on our new disk with the exact data from fdisk, like i have said many times over. I do believe that's iportant.
| Code: | | dd if=/mnt/backup/root.image of=/dev/sda3 |
Drawbacks
This method of backup will replicate filesystem errors; copying the files from the filesystem might avoid them. The image is as large as the partition and cumbersome. It is painfully slow to bzip and the process takes full cpu usage. If bzipped, you wouldn't be able to read it until uncompressed, and you'd need lots of space to swap the data around in.
_________________
Configuring a Firewall? Try my iptables configuration
LinuxCommando.com is my blog for linux-related scraps and tidbits. Stop by for a visit! |
|
| Back to top |
|
|
eltech
Guru
Joined: 05 Nov 2002
Posts: 582
Location: New York
|
| Posted: Thu Dec 28, 2006 8:29 pm Post subject: |
|
|
very nice I will toy with this now..
can this also serve as a way to back up say the whole hdb or hda disk rather than partitions?
What format should the whole hdb be in (ext3, ext2)?
Thank you again .. your help has been outstanding.. 
eltech
Last edited by eltech on Thu Dec 28, 2006 8:40 pm; edited 1 time in total |
|
| Back to top |
|
|
erik258
Advocate
Joined: 12 Apr 2005
Posts: 2650
Location: Twin Cities, Minnesota, USA
|
| Posted: Thu Dec 28, 2006 8:35 pm Post subject: |
|
|
i believe it can, eletch, i beleve it can. I beleive it could be used successfully on just about any block device, too.
_________________
Configuring a Firewall? Try my iptables configuration
LinuxCommando.com is my blog for linux-related scraps and tidbits. Stop by for a visit! |
|
| Back to top |
|
|
eltech
Guru
Joined: 05 Nov 2002
Posts: 582
Location: New York
|
| Posted: Thu Dec 28, 2006 8:40 pm Post subject: |
|
|
| erik258 wrote: | | i believe it can, eletch, i beleve it can. I beleive it could be used successfully on just about any block device, too. |
erik, outstanding ..
What format should the whole hdb be in (ext3, ext2)? |
|
| Back to top |
|
|
erik258
Advocate
Joined: 12 Apr 2005
Posts: 2650
Location: Twin Cities, Minnesota, USA
|
| Posted: Thu Dec 28, 2006 8:43 pm Post subject: |
|
|
it doesn't matter the filesystem on the hard drive you're backing up. as long as you can write to it, you can put the image file on any partition with any filesystem. The device files for the hard drive partitions work on a lower level than filesystems do.
_________________
Configuring a Firewall? Try my iptables configuration
LinuxCommando.com is my blog for linux-related scraps and tidbits. Stop by for a visit! |
|
| Back to top |
|
|
eltech
Guru
Joined: 05 Nov 2002
Posts: 582
Location: New York
|
| Posted: Thu Dec 28, 2006 8:48 pm Post subject: |
|
|
| erik258 wrote: | | it doesn't matter the filesystem on the hard drive you're backing up. as long as you can write to it, you can put the image file on any partition with any filesystem. The device files for the hard drive partitions work on a lower level than filesystems do. |
understood .. so I will post back results, what i did .. what worked and if it was a 100% success .. I'm sure this will help someone along the way .. |
|
| Back to top |
|
|
erik258
Advocate
Joined: 12 Apr 2005
Posts: 2650
Location: Twin Cities, Minnesota, USA
|
| Posted: Thu Dec 28, 2006 8:52 pm Post subject: |
|
|
i hope so...
_________________
Configuring a Firewall? Try my iptables configuration
LinuxCommando.com is my blog for linux-related scraps and tidbits. Stop by for a visit! |
|
| Back to top |
|
|
erik258
Advocate
Joined: 12 Apr 2005
Posts: 2650
Location: Twin Cities, Minnesota, USA
|
| Posted: Thu Dec 28, 2006 9:18 pm Post subject: |
|
|
my bzipping that image finally got done. It took about an hour, was a 5 gig partition and a fast computer. Shrank down to 2 gigs. Thre was lots of free space.
_________________
Configuring a Firewall? Try my iptables configuration
LinuxCommando.com is my blog for linux-related scraps and tidbits. Stop by for a visit! |
|
| Back to top |
|
|
eltech
Guru
Joined: 05 Nov 2002
Posts: 582
Location: New York
|
| Posted: Fri Dec 29, 2006 1:03 am Post subject: |
|
|
| erik258 wrote: | | my bzipping that image finally got done. It took about an hour, was a 5 gig partition and a fast computer. Shrank down to 2 gigs. Thre was lots of free space. |
well, i did the backup portion to an image (*.image) and it was about 8gb, it took about 1 hour; didnt try bzipping it yet. I will need to bzip it though, to move to a smb share. I will try to put the image onto a hard drive tomorrow and see how this works out ..
for this testing its all being done in a vmware; so lets hope i get real world results ..
I will post back .. |
|
| Back to top |
|
|
erik258
Advocate
Joined: 12 Apr 2005
Posts: 2650
Location: Twin Cities, Minnesota, USA
|
| Posted: Fri Dec 29, 2006 5:53 am Post subject: |
|
|
I'd like to know why this kind of thing isnt more common. It seems like a good backup solution, albeit somewhat unyieldy in file size.
_________________
Configuring a Firewall? Try my iptables configuration
LinuxCommando.com is my blog for linux-related scraps and tidbits. Stop by for a visit! |
|
| Back to top |
|
|
eltech
Guru
Joined: 05 Nov 2002
Posts: 582
Location: New York
|
| Posted: Fri Dec 29, 2006 8:10 pm Post subject: |
|
|
 erik, we did it .. it works... i completely backed up the whole drive to another drive, fdisk the new drive and created only 1 partition .. dd'ed the backup drive to the original (emulate new hard drive) and viola .. success! It booted to grub, then off it went into the environment fully working ..
so dd is a great way to back up a whole drive, but yes size is the issue here; but for me, its not that big an issue during the times of the backup and the move over our network ..
erik, thanks for your help ..! |
|
| Back to top |
|
|
erik258
Advocate
Joined: 12 Apr 2005
Posts: 2650
Location: Twin Cities, Minnesota, USA
|
| Posted: Fri Dec 29, 2006 10:40 pm Post subject: |
|
|
no problem. i am really glad to see one of 'my' designs going into a production box  and i never use smilies.
_________________
Configuring a Firewall? Try my iptables configuration
LinuxCommando.com is my blog for linux-related scraps and tidbits. Stop by for a visit! |
|
| Back to top |
|
|
|
Other Things Gentoo
