Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Using dd to clone a drive [fast & simple]
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
Gentree
Watchman
Watchman


Joined: 01 Jul 2003
Posts: 5350
Location: France, Old Europe

PostPosted: Fri Apr 29, 2005 7:26 pm    Post subject: Using dd to clone a drive [fast & simple] Reply with quote

After buying a new HD I wish to move the whole contents of my current drive to a new drive : mbr, partition table, partitions (mixed ntfs ext3 R4), the lot.

The disks are similar 80G / 120G Seagate IDE.

Is it feasible to just do a byte copy using dd to clone the old disk?
Code:

time dd if=/dev/hdc of=/dev/hda bs=80M count=1024

TIA Gentree. 8)
_________________
Linux, because I'd rather own a free OS than steal one that's not worth paying for.
Gentoo because I'm a masochist
AthlonXP-M on A7N8X. Portage ~x86


Last edited by Gentree on Wed Jun 22, 2005 2:36 pm; edited 2 times in total
Back to top
View user's profile Send private message
nixnut
Bodhisattva
Bodhisattva


Joined: 09 Apr 2004
Posts: 10974
Location: the dutch mountains

PostPosted: Fri Apr 29, 2005 7:36 pm    Post subject: Reply with quote

I don't think so. Since they're different sizes, sectors and cylinders would be different. I'm not sure, but I think you'd end up with a partition table that has nothing to do with the rest of the data on your disk, even if it ended up being a valid partition table, which I doubt.
_________________
Please add [solved] to the initial post's subject line if you feel your problem is resolved. Help answer the unanswered

talk is cheap. supply exceeds demand
Back to top
View user's profile Send private message
Sm1
Apprentice
Apprentice


Joined: 02 Dec 2003
Posts: 251
Location: Ames, IA

PostPosted: Fri Apr 29, 2005 8:06 pm    Post subject: Reply with quote

Look into using partimage. I have used it to make partition images for drives, but i'm not sure if it will work for an entire drive
Back to top
View user's profile Send private message
r3pek
Retired Dev
Retired Dev


Joined: 17 Sep 2003
Posts: 568
Location: Lisbon - Portugal

PostPosted: Fri Apr 29, 2005 9:56 pm    Post subject: Reply with quote

yes i can.... a simple
Code:
dd if=/dev/hda of=/dev/hdb
would do it.... i use this to clone machines over the network :) usign ssh.... just make sure that the destination dist is at least the same size as the origin...
Back to top
View user's profile Send private message
Gentree
Watchman
Watchman


Joined: 01 Jul 2003
Posts: 5350
Location: France, Old Europe

PostPosted: Fri Apr 29, 2005 10:31 pm    Post subject: Reply with quote

Thanks ,

that's even better/simpler than I expected. I just did the filrst 18M (7 min 50 = 72M/s ave r/w nice)

after that I tried parted to see the partitions and it looks good, but I get an error msg from parted.

Code:
(parted) p                                                               
Error: Invalid partition table on /dev/ide/host0/bus0/target0/lun0/disc - wrong
signature 0
Ignore/Cancel? i


I guess that may be because half the partitions listed in the part-tab are missing. Since it's fairly quick I'll do the rest.

Thanks for the replied.
8)
_________________
Linux, because I'd rather own a free OS than steal one that's not worth paying for.
Gentoo because I'm a masochist
AthlonXP-M on A7N8X. Portage ~x86
Back to top
View user's profile Send private message
Gentree
Watchman
Watchman


Joined: 01 Jul 2003
Posts: 5350
Location: France, Old Europe

PostPosted: Sat Apr 30, 2005 8:06 am    Post subject: Reply with quote

Yesss!

perfect clone.

All I needed to do was edit grub.conf and /etc/fstab on the new disk and c/hdc/hda/g

Its quite wierd to be running an identical system knowing its on other drive. Anyway I'm posting from the new system so all looks to be perfect. (Have not tried the win2k boot yet)

One point worth noting: 86 min for whole 80G drive. This is much slower than my first copy of 18G in 7.5 min , so it would seem to be a lot faster to define a large block size (within RAM limits) as I did in my first attempt.

@r3pek: it would be interesting to see if this gives an increase over a network but if you are on std 100Mb ethernet it may not be significant.

Thanks for your help anyway. 8)
_________________
Linux, because I'd rather own a free OS than steal one that's not worth paying for.
Gentoo because I'm a masochist
AthlonXP-M on A7N8X. Portage ~x86
Back to top
View user's profile Send private message
r3pek
Retired Dev
Retired Dev


Joined: 17 Sep 2003
Posts: 568
Location: Lisbon - Portugal

PostPosted: Sat Apr 30, 2005 6:37 pm    Post subject: Reply with quote

It saves me all the trouble of open one server, put a new scsi card on it, add the new disc, make the copy, remove the disc, remove the scsi card, close the server and put the disk on the other server :)
Back to top
View user's profile Send private message
lemino
n00b
n00b


Joined: 16 Mar 2004
Posts: 61
Location: Gothenburg, Sweden

PostPosted: Wed Jun 22, 2005 12:59 pm    Post subject: Problem resizing reiserfs-partition. Reply with quote

Hi. I used the command

dd if=/dev/hda of=/dev/hdb

and it seems as if it has worked out well. A clone is created, and allthough it doesn't behave in the exaktly same manner as the old one, it works. For some reason it hangs on shutdown though... The problem is that I would like to resize the root-partition on my new drive (seagate 120) since it has not only take the content of the old disks (seagate 20) root-partition, but also its shape and size. Thus I'm left with one boot-partition, swap and a small root on my new drive + lots of free space I can't use. I have tried to make the reiserfs-root consume the remaining free space without success. Any tips?
Back to top
View user's profile Send private message
syg00
l33t
l33t


Joined: 23 Aug 2004
Posts: 907
Location: Brisbane, AUS

PostPosted: Wed Jun 22, 2005 1:15 pm    Post subject: Reply with quote

I don't like the concept of a "byte-level" copy - unless you can guarantee similar architecture (and by implication, size).
Makes more sense to concentrate on data replication - stage4 seems to work o.k.
Back to top
View user's profile Send private message
Gentree
Watchman
Watchman


Joined: 01 Jul 2003
Posts: 5350
Location: France, Old Europe

PostPosted: Wed Jun 22, 2005 2:15 pm    Post subject: Reply with quote

Yes , I agree.

I had a seagate 80 to copy to a seagate120 of the same family so it was pretty close.

It obviously left me with an 80G setup on the 120 but I just needed to use parted to increase the extended partition upwards and create new paritions to use the space.

I would not have wanted another 40 SeaGigs on the exisiting partitions anyway.

It is a bit of a hack so it has to be used with care but it is beautifully compared to duping all the partitions I had , formatting them , copying the contents and redoing my grub setup.

That simple command did the whole lot , perfectly. 8)
_________________
Linux, because I'd rather own a free OS than steal one that's not worth paying for.
Gentoo because I'm a masochist
AthlonXP-M on A7N8X. Portage ~x86
Back to top
View user's profile Send private message
lemino
n00b
n00b


Joined: 16 Mar 2004
Posts: 61
Location: Gothenburg, Sweden

PostPosted: Thu Jun 23, 2005 2:46 pm    Post subject: Reply with quote

I get what seems to be a nice cloon but it is a little slow at startup and it hangs when trying to stop fcron during shutdown. Otherwise it seems to be working alright. Ideas?
Back to top
View user's profile Send private message
Gentree
Watchman
Watchman


Joined: 01 Jul 2003
Posts: 5350
Location: France, Old Europe

PostPosted: Thu Jun 23, 2005 3:08 pm    Post subject: Reply with quote

Does the old partition still work as before?

make a new line in grub.conf so you can boot one or the other and compare directly.

You could dig around out of curosity , but if it seems not be the same boot to a live CD and just do

cp -a src_partn dest_partn

no sense in messing around. It was supposed to save time!

8)
_________________
Linux, because I'd rather own a free OS than steal one that's not worth paying for.
Gentoo because I'm a masochist
AthlonXP-M on A7N8X. Portage ~x86
Back to top
View user's profile Send private message
duderonomy
Guru
Guru


Joined: 20 Mar 2004
Posts: 349
Location: SF Bay Area

PostPosted: Thu Jun 23, 2005 3:50 pm    Post subject: Re: Using dd to clone a drive [fast & simple] Reply with quote

Gentree wrote:
Is it feasible to just do a byte copy using dd to clone the old disk?

I am interested in this topic. My experience is that
identical drives will work just fine however there
are definitely some issues with different size drives
when doing a whole disk copy: `dd if=/dev/hda of=/dev/hdb`

Has anyone found useful links that describe the details of
partitions on the x86 architecture? I'd like to learn what is
so special about resizing a partition that requires special
software.
Back to top
View user's profile Send private message
bonbons
Apprentice
Apprentice


Joined: 04 Sep 2004
Posts: 250

PostPosted: Thu Jun 23, 2005 4:10 pm    Post subject: Re: Using dd to clone a drive [fast & simple] Reply with quote

duderonomy wrote:
Gentree wrote:
Is it feasible to just do a byte copy using dd to clone the old disk?

I am interested in this topic. My experience is that
identical drives will work just fine however there
are definitely some issues with different size drives
when doing a whole disk copy: `dd if=/dev/hda of=/dev/hdb`

Has anyone found useful links that describe the details of
partitions on the x86 architecture? I'd like to learn what is
so special about resizing a partition that requires special
software.
As much as I know problems araise when you have partitions or OS that still rely on Cylinder/Sector metrics instead of the linear LBA.

I guess reading about old-time DOS, BIOS Disk size restrictions and similar will provide the information you are seeking for. Check the partition tools and their developper documentation (there information must be available!)
Back to top
View user's profile Send private message
rutski89
Guru
Guru


Joined: 14 Mar 2005
Posts: 468
Location: United States N.Y.

PostPosted: Thu Jun 23, 2005 5:55 pm    Post subject: Reply with quote

I don't like the idea of byte copying for cloning and re-use another a new drive. Its perfectly fine for backing up a drive beforeing trying some seriously dangerous operations, but other than that I wouldn't trust it as more than a hack or quick fix; reinstalling will produce better results.
_________________
<< ^ | ~ >>
Back to top
View user's profile Send private message
duderonomy
Guru
Guru


Joined: 20 Mar 2004
Posts: 349
Location: SF Bay Area

PostPosted: Thu Jun 23, 2005 7:45 pm    Post subject: Reply with quote

rutski89 wrote:
I don't like the idea of byte copying for cloning and re-use another a new drive. Its perfectly fine for backing up a drive... reinstalling will produce better results.
No worries, man. This is not necessarily true, otherwise
using GHOST would not be a "best practice". Block copy is valid.

Block copy is a popular method used by large data installations
to make backups (i.e. to get the job done within prescribed
windows of time, due to ever increasing and very large
amounts of data. Parsing through a file list and making all
those translations to the block locations is more time-consuming
than direct block copy.).

Problems arise when things I do not know about, such as
offsets I heard about in the NTFS file system, do not get
satisfied after a copy. I have more empirical work to do
before I can comment on block copying to different sized
partitions or whole disk block copies to disks with differing
geometries. I want to sort this out, sooner than later.
Back to top
View user's profile Send private message
canal
Apprentice
Apprentice


Joined: 28 Sep 2004
Posts: 203

PostPosted: Thu Jun 23, 2005 9:27 pm    Post subject: Re: Using dd to clone a drive [fast & simple] Reply with quote

bonbons wrote:
As much as I know problems araise when you have partitions or OS that still rely on Cylinder/Sector metrics instead of the linear LBA.

Very true. But... there are exactly zero such OSes today. Why so ? Easy: you can not have HDD bigger then 8GB with different C/S formula. All HDDs >8GB in size must have 255 heads and 63 sectors/track. This is not an option - this is requirement.

bonbons wrote:
I guess reading about old-time DOS, BIOS Disk size restrictions and similar will provide the information you are seeking for. Check the partition tools and their developper documentation (there information must be available!)

It's still available if you'll search long enough. It's not really useful today.

As for changes in partition sizes... This is not problem with partitions! You can change size of partition with fdisk easily. No problem. But.... Filesystem must be changed as well - and this is quite a problem.

rutski89 wrote:
I don't like the idea of byte copying for cloning and re-use another a new drive. Its perfectly fine for backing up a drive beforeing trying some seriously dangerous operations, but other than that I wouldn't trust it as more than a hack or quick fix; reinstalling will produce better results.
Of course - you room will be warmer due to high CPU usage. This is the only difference, of course (but see above about changes in filesystem size!).

Good way to duplicate system is with "cp -aiv" command - we have system transferred 4 (or may be 5? not sure) times this way. RedHat 6.2 with kernel 2.6.10 and stuff. Works just fine still. Anyone who claims you can not just trasfer your system with "dd" or "cp -aiv" to new HDD is just not knowleadgeable enough: Linux is quite cloneable, this is not Windows XP!

duderonomy wrote:
whole disk block copies to disks with differing
geometries. I want to sort this out, sooner than later.

There are two fundamental truths a far as "disks with differing geometries" are concerned.
1. Never ever try to move partitions between two HDDs with different geometries (without special tools) - this is problematic.
2. Forget abut this for all modern systems: all HDDs bigger then 8GiB have the same geometry. This is biggest size accessible without LBA and to turn LBA on geometry must be: 255 heads and 63 sectors/track. After that all HDDs are interchangeable.
Back to top
View user's profile Send private message
rutski89
Guru
Guru


Joined: 14 Mar 2005
Posts: 468
Location: United States N.Y.

PostPosted: Fri Jun 24, 2005 12:00 am    Post subject: Re: Using dd to clone a drive [fast & simple] Reply with quote

canal wrote:
rutski89 wrote:
I don't like the idea of byte copying for cloning and re-use another a new drive. Its perfectly fine for backing up a drive beforeing trying some seriously dangerous operations, but other than that I wouldn't trust it as more than a hack or quick fix; reinstalling will produce better results.
Heh, I'm sure it works just fine. I'd personally rather re-install, but if I had to do it fast then I wouldn't hesitate or worry about doing a block copy.
_________________
<< ^ | ~ >>
Back to top
View user's profile Send private message
Corona688
Veteran
Veteran


Joined: 10 Jan 2004
Posts: 1204

PostPosted: Fri Jun 24, 2005 1:36 am    Post subject: Reply with quote

The way I generally do drive copies is:
Code:
dd if=/dev/hda of=/dev/hdb bs=512 conv=sync

The conv=sync part is very useful for getting data off of drives that are failing -- absolutely necessary, in fact. With this flag, when it fails to read a sector from the input drive, it will write a sector of all zeros to the output drive; without it, it'll just skip over it without writing anything on the output, making your output image a different size from the input!
_________________
Petition for Better 64-bit ATI Drivers - Sign Here
http://www.petitiononline.com/atipet/petition.html
Back to top
View user's profile Send private message
EASYdoor
Tux's lil' helper
Tux's lil' helper


Joined: 21 Aug 2004
Posts: 79
Location: Novo mesto

PostPosted: Sun Aug 21, 2005 1:41 pm    Post subject: hi all Reply with quote

I can confirm that using the dd clone method works ONLY with EXACT geometry from the drive that u are cloning.
I tried to clone IBM 41.1 GB onto MAXTOR 40 GB....well everything went well and the "no space left on device" ....when i've manually mounted the drive every data that i needed was there.../boot /root ....everything but when i've restarted the system fschk found an error and i was foreced to do it manually which later on hang it self....so no good :(

i've figured it out that you can manually change CHS (Cylinder/Head/Sector) using fdisk (advanced options) or cfdisk.
Well if i do that i'll have to do an extra empty partition & mount to use it. I don't want that, so I'm currently backing up files with cp -aiv method. Now the question? Do i have to copy the mbr data or anything else to succsesfully boot my new system??

i did my own partitions, little different than the previous system /boot now 100M & swap 1G & /root everything else (200 GB seagate)
i've mounted /dev/hdh1 /b1 && /dev/hdg1 /b2 + root partition & copied everything with -aiv option. Do I have to do anything else?!??

like chroot & install grub???
_________________
Pejt na pir! Ne, ne,...se boljs,....JST grem na pir ;)


Last edited by EASYdoor on Sun Aug 21, 2005 3:29 pm; edited 1 time in total
Back to top
View user's profile Send private message
Kaapeli
Tux's lil' helper
Tux's lil' helper


Joined: 27 Dec 2004
Posts: 110
Location: Oulu, Finland

PostPosted: Sun Aug 21, 2005 3:29 pm    Post subject: Re: hi all Reply with quote

EASYdoor wrote:
I can confirma that using the dd clone method works ONLY with EXACT geometry from the drive that u are cloning.
I tried to clone IBM 41.1 GB onto MAXTOR 40 GB ....well everything went well and the "no space left on device " ....when i've manually mounted the drive every data that i needed was there.../boo /root ....everything but when i've restarted the system fschk found an error and i was foreced to do it manually which later on hang it self....so no good :(


So, you tried to clone a bigger disk on a smaller disk? This doesn't work, since obiviously you can't fit the whole image on the target disk and the image is not complete. However, if you were copying from smaller disk to bigger one, there wouldn't be any problems.

However, I prefer using cp -av /source /target since that allows you to copy the files onto a different filesystem. And it will also fix any fragmentation problems and therefore increase the disk performance compared to the dd clone. Furthermore, the disks are rarely identical, so you would end up having some empy space on the target disk. And I also don't think it makes any sense to copy the free disk space ;)

And what comes to the MBR data, it's enough to reinstall the bootloader and make sure the proper partition is bootable.
_________________
Temperatures
Back to top
View user's profile Send private message
cybrjackle
Apprentice
Apprentice


Joined: 09 Jan 2003
Posts: 248
Location: USA

PostPosted: Sun Aug 21, 2005 3:45 pm    Post subject: Reply with quote

Code:

time rsync --progress --stats -av /source /target
Back to top
View user's profile Send private message
EASYdoor
Tux's lil' helper
Tux's lil' helper


Joined: 21 Aug 2004
Posts: 79
Location: Novo mesto

PostPosted: Sun Aug 21, 2005 6:11 pm    Post subject: Reply with quote

Quote:
time rsync --progress --stats -av /source /target


why time command if i only want to copy...

hehe, if u manually override the cylinder then in fdisk everything is ok but hetn u make mke2fs -j /dev/hdg3 ....hdg write_intr error1 & DriveREady SeekComplete Error & stuff....
but anyway i'll give it a shoot :)
_________________
Pejt na pir! Ne, ne,...se boljs,....JST grem na pir ;)
Back to top
View user's profile Send private message
cybrjackle
Apprentice
Apprentice


Joined: 09 Jan 2003
Posts: 248
Location: USA

PostPosted: Sun Aug 21, 2005 9:23 pm    Post subject: Reply with quote

I just time everything, its a habbit :)
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum