Convert JBOD to Software RAID-1 maintaining data


Converting to raid1 from a bare partition structure: Overview: Starting system: 1 drive: /dev/sda Materials: 2 spare drives Finished system: 2 drives, all partitions RAID1, original /dev/sda as spare

  1. Insert spare drive as /dev/sdb
  2. Find out what the existing partition structure is:
    root@kepler:~# fdisk -l
    
    Disk /dev/sda: 160.0 GB, 160000000000 bytes
    255 heads, 63 sectors/track, 19452 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x000a29dc
    
       Device Boot      Start         End      Blocks   Id  System
    /dev/sda1   *           1       18660   149879808   83  Linux
    /dev/sda2           18660       19453     6367233    5  Extended
    /dev/sda5           18660       19453     6367232   82  Linux swap / Solaris
    
    Disk /dev/sdb: 160.0 GB, 160041885696 bytes
    255 heads, 63 sectors/track, 19457 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x00000000
    
    Disk /dev/sdb doesn't contain a valid partition table
    root@kepler:~#
    
  3. Now create a compatible (NOT exact) structure on the target. Note that I definitely prefer a /boot partition on it's own, se we're going to fix that too -- use your favorite partitioning tool, and create a result that looks something like this:
    
    root@kepler:~# fdisk -l
    
    Disk /dev/sda: 160.0 GB, 160000000000 bytes       ((**UNMODIFIED**))
    255 heads, 63 sectors/track, 19452 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x000a29dc
    
       Device Boot      Start         End      Blocks   Id  System
    /dev/sda1   *           1       18660   149879808   83  Linux
    /dev/sda2           18660       19453     6367233    5  Extended
    /dev/sda5           18660       19453     6367232   82  Linux swap / Solaris
    
    Disk /dev/sdb: 160.0 GB, 160041885696 bytes       ((**New Partitions**)
    255 heads, 63 sectors/track, 19457 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x00000000
    
       Device Boot      Start         End      Blocks   Id  System
    /dev/sdb1               1          36      289138+  fd  Linux raid autodetect
    /dev/sdb2              37       18659   149589247+  fd  Linux raid autodetect
    /dev/sdb3           18660       19452     6369772+   5  Extended
    /dev/sdb5           18660       19452     6369741   fd  Linux raid autodetect
    root@kepler:~# 
    
  4. Construct degraded raidsets for each of the new partitions
    root@kepler:# mdadm --create /dev/md0 --level=1 --raid-devices=2 missing /dev/sdb1
    mdadm: Note: this array has metadata at the start and
        may not be suitable as a boot device.  If you plan to
        store '/boot' on this device please ensure that
        your boot-loader understands md/v1.x metadata, or use
        --metadata=0.90
    Continue creating array? y
    mdadm: Defaulting to version 1.2 metadata
    mdadm: array /dev/md0 started.
    root@kepler:# mdadm --create /dev/md1 --level=1 --raid-devices=2 missing /dev/sdb2
    mdadm: Note: this array has metadata at the start and
        may not be suitable as a boot device.  If you plan to
        store '/boot' on this device please ensure that
        your boot-loader understands md/v1.x metadata, or use
        --metadata=0.90
    Continue creating array? y
    mdadm: Defaulting to version 1.2 metadata
    mdadm: array /dev/md1 started.
    root@kepler:# mdadm --create /dev/md2 --level=1 --raid-devices=2 missing /dev/sdb5
    mdadm: Note: this array has metadata at the start and
        may not be suitable as a boot device.  If you plan to
        store '/boot' on this device please ensure that
        your boot-loader understands md/v1.x metadata, or use
        --metadata=0.90
    Continue creating array? y
    mdadm: Defaulting to version 1.2 metadata
    mdadm: array /dev/md2 started.
    root@kepler:#
    
  5. Verify correct creation of degraded raidsets:
    root@kepler:~# cat /proc/mdstat
    Personalities : [raid1] 
    md2 : active raid1 sdb5[1]
          6368705 blocks super 1.2 [2/1] [_U]
          
    md1 : active raid1 sdb2[1]
          149588151 blocks super 1.2 [2/1] [_U]
          
    md0 : active raid1 sdb1[1]
          289126 blocks super 1.2 [2/1] [_U]
          
    unused devices: 
    root@kepler:~#
    
  6. Format file systems onto the md devices as appropriate:
    root@kepler:~# mke2fs /dev/md0
    mke2fs 1.41.12 (17-May-2010)
    Filesystem label=
    OS type: Linux
    Block size=1024 (log=0)
    Fragment size=1024 (log=0)
    Stride=0 blocks, Stripe width=0 blocks
    72288 inodes, 289124 blocks
    14456 blocks (5.00%) reserved for the super user
    First data block=1
    Maximum filesystem blocks=67633152
    36 block groups
    8192 blocks per group, 8192 fragments per group
    2008 inodes per group
    Superblock backups stored on blocks: 
            8193, 24577, 40961, 57345, 73729, 204801, 221185
    
    Writing inode tables: done                            
    Writing superblocks and filesystem accounting information: done
    
    This filesystem will be automatically checked every 36 mounts or
    180 days, whichever comes first.  Use tune2fs -c or -i to override.
    root@kepler:~# mke2fs -j /dev/md1
    mke2fs 1.41.12 (17-May-2010)
    Filesystem label=
    OS type: Linux
    Block size=4096 (log=2)
    Fragment size=4096 (log=2)
    Stride=0 blocks, Stripe width=0 blocks
    9355264 inodes, 37397037 blocks
    1869851 blocks (5.00%) reserved for the super user
    First data block=0
    Maximum filesystem blocks=4294967296
    1142 block groups
    32768 blocks per group, 32768 fragments per group
    8192 inodes per group
    Superblock backups stored on blocks: 
            32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
            4096000, 7962624, 11239424, 20480000, 23887872
    
    Writing inode tables: done                            
    Creating journal (32768 blocks): done
    Writing superblocks and filesystem accounting information: done
    
    This filesystem will be automatically checked every 38 mounts or
    180 days, whichever comes first.  Use tune2fs -c or -i to override.
    root@kepler:~# mkswap /dev/md2
    mkswap: /dev/md2: warning: don't erase bootbits sectors
            on whole disk. Use -f to force.
    Setting up swapspace version 1, size = 6368700 KiB
    no label, UUID=eddc5aa2-35ad-45f1-97db-41a9fb76dbab
    root@kepler:~# 
    
  7. Copy filesystem data ... in this case, we are copying a root filesystem, and we do NOT want to copy any of the operating system special directories' contents... so we will use 'cp -a' for each NON-system directory in root... e.g. :
    
    root@kepler:/# ls /
    bin   dev  home        lib    lost+found  mnt        opt   root  selinux  sys  usr  vmlinuz
    boot  etc  initrd.img  lib64  media       oldkepler  proc  sbin  srv      tmp  var
    
    //Note we omit dev lost+found mnt sys proc tmp  //Note: be careful with /media & contents as well
    
    root@kepler:/# cp -a bin home lib opt root selinux usr vmlinuz boot etc initrd.img lib64 media oldkepler sbin srv var /mnt
    
  8. Create the directories we did not copy in step 6 so they are present
    when boot the operating system onto the new drives ... and remember to fix up
    the modes so they're right too
    
    root@kepler:/mnt# mkdir dev mnt sys proc tmp
    root@kepler:/mnt# cd /
    root@kepler:/# ls -ldah dev mnt sys proc tmp
    drwxr-xr-x  17 root root 3.0K Dec 14 08:54 dev
    drwxr-xr-x  13 root root 4.0K Dec 14 09:49 mnt
    dr-xr-xr-x 119 root root    0 Dec 11 14:56 proc
    drwxr-xr-x  13 root root    0 Dec 11 14:56 sys
    drwxrwxrwt   2 root root 4.0K Dec 14 09:50 tmp
    root@kepler:/# cd -
    /mnt
    root@kepler:/mnt# chmod 755 dev mnt sys ; chmod 777 tmp ; chmod +t tmp ; chmod 555 proc
    root@kepler:/mnt# ls -ldah dev mnt sys proc tmp
    drwxr-xr-x 2 root root 4.0K Dec 14 09:48 dev
    drwxr-xr-x 2 root root 4.0K Dec 14 09:48 mnt
    dr-xr-xr-x 2 root root 4.0K Dec 14 09:48 proc
    drwxr-xr-x 2 root root 4.0K Dec 14 09:48 sys
    drwxrwxrwt 2 root root 4.0K Dec 14 09:48 tmp
    root@kepler:/mnt# 
    
  9. Copy/move any other partitions if you are rearranging your partition layout in this process.
    In this example, I am breaking boot out to it's own partition
    
    root@kepler:/mnt# ls -ldah boot
    drwxr-xr-x 3 root root 4.0K Dec 14 08:52 boot
    root@kepler:/mnt# mv boot bootold ; mkdir -m0744 boot ; ls -ldah boot
    drwxr--r-- 2 root root 4.0K Dec 14 10:48 boot
    root@kepler:/mnt# mount /dev/md0 /mnt/boot
    root@kepler:/mnt# mv bootold/* boot/
    root@kepler:/mnt# rmdir bootold
    
  10. Generate the appropriate /etc/mdadm/mdadm.conf file in the new root
    
    root@kepler:/mnt# mdadm --detail --scan >>/mnt/etc/mdadm/mdadm.conf 
    root@kepler:/mnt# cat /mnt/etc/mdadm/mdadm.conf 
    # mdadm.conf
    #
    # Please refer to mdadm.conf(5) for information about this file.
    #
    
    # by default, scan all partitions (/proc/partitions) for MD superblocks.
    # alternatively, specify devices to scan, using wildcards if desired.
    DEVICE partitions
    
    # auto-create devices with Debian standard permissions
    CREATE owner=root group=disk mode=0660 auto=yes
    
    # automatically tag new arrays as belonging to the local system
    HOMEHOST 
    
    # instruct the monitoring daemon where to send mail alerts
    MAILADDR root
    
    # definitions of existing MD arrays
    
    # This file was auto-generated on Fri, 14 Dec 2012 08:52:15 -0600
    # by mkconf 3.1.4-1+8efb9d1+squeeze1
    ARRAY /dev/md0 metadata=1.2 name=kepler:0 UUID=766506ff:e42e6a66:ce138bda:4a066256
    ARRAY /dev/md1 metadata=1.2 name=kepler:1 UUID=688eed4a:2d64e8bc:e5e6cce4:ec26897e
    ARRAY /dev/md2 metadata=1.2 name=kepler:2 UUID=20b9a69a:7ac619bc:e7814e3a:2de54ae0
    root@kepler:/mnt#
    
  11. Fix the /etc/fstab in the target root
    Before:
    root@kepler:/mnt/etc# cat fstab 
    # /etc/fstab: static file system information.
    #
    # Use 'blkid' to print the universally unique identifier for a
    # device; this may be used with UUID= as a more robust way to name devices
    # that works even if disks are added and removed. See fstab(5).
    #
    #                
    proc            /proc           proc    defaults        0       0
    # / was on /dev/sda1 during installation
    UUID=6b9da7ff-3f61-4d6c-b8b2-d010641e918c /               ext3    errors=remount-ro 0       1
    # swap was on /dev/sda5 during installation
    UUID=9a0cfe1c-0801-45b1-83ab-b870185be9d9 none            swap    sw              0       0
    /dev/scd0       /media/cdrom0   udf,iso9660 user,noauto     0       0
    root@kepler:/mnt/etc#
    
    After:
    root@kepler:/mnt/etc# cat fstab 
    # /etc/fstab: static file system information.
    #
    # Use 'blkid' to print the universally unique identifier for a
    # device; this may be used with UUID= as a more robust way to name devices
    # that works even if disks are added and removed. See fstab(5).
    #
    #                
    proc            /proc           proc    defaults        0       0
    # / was on /dev/sda1 during installation
    /dev/md1 /               ext3    errors=remount-ro 0       1
    /dev/md0 /boot           ext2    errors=remount-ro 0       1
    # swap was on /dev/sda5 during installation
    /dev/md2 none            swap    sw              0       0
    /dev/scd0       /media/cdrom0   udf,iso9660 user,noauto     0       0
    root@kepler:/mnt/etc#
    
  12. Edit target /boot/grub/device.map
    Before:
    (hd0)   /dev/disk/by-id/scsi-1ATA_ST3160812AS_5LSEGGJL
    After
    (hd0)   /dev/sda
    (hd1)   /dev/sdb
    ----note: you MAY want to keep the '/dev/disk/by-id/' naming, but you will
        need to do further edits of /boot/grub/device.map as you proceed with the
        steps below
    
  13. Chroot into new target, reinstall grub, update grub.cfg / rebuild initrd
    root@kepler:/mnt/boot/grub# mount --bind /dev /mnt/dev
    root@kepler:/mnt/boot/grub# mount --bind /sys /mnt/sys
    root@kepler:/mnt/boot/grub# chroot /mnt
    root@kepler:/# grub-install /dev/sdb
    root@kepler:/# update-grub
    Generating grub.cfg ...
    Found linux image: /boot/vmlinuz-2.6.32-5-amd64
    Found initrd image: /boot/initrd.img-2.6.32-5-amd64
    grep: /proc/mounts: No such file or directory
    grep: /proc/swaps: No such file or directory
    done
    root@kepler:/# exit
    root@kepler:/mnt/boot/grub# umount /mnt/sys
    root@kepler:/mnt/boot/grub# umount /mnt/dev
    
  14. Shutdown - rearrange drives, remove source drive, and attempt boot to new system (Note that for our example here, this means remove sda, then move sdb to sda, insert spare sdb)
    If all goes well, you're now up & running on all degraded raid1 md sets
    root@kepler:~# df -ah
    Filesystem            Size  Used Avail Use% Mounted on
    /dev/md1              141G   29G  105G  22% /
    tmpfs                 4.0G     0  4.0G   0% /lib/init/rw
    proc                     0     0     0   -  /proc
    sysfs                    0     0     0   -  /sys
    udev                  4.0G  160K  4.0G   1% /dev
    tmpfs                 4.0G     0  4.0G   0% /dev/shm
    devpts                   0     0     0   -  /dev/pts
    /dev/md0              274M   20M  240M   8% /boot
    root@kepler:~# swapon -s
    Filename                                Type            Size    Used    Priority
    /dev/md2                                partition       6368696 0       -1
    root@kepler:~# 
    
  15. Partition the new spare drive that will be our raid partner
    root@kepler:~# sfdisk -d /dev/sda | sfdisk /dev/sdb
    Checking that no-one is using this disk right now ...
    OK
    
    Disk /dev/sdb: 19457 cylinders, 255 heads, 63 sectors/track
    
    sfdisk: ERROR: sector 0 does not have an msdos signature
     /dev/sdb: unrecognized partition table type
    Old situation:
    No partitions found
    New situation:
    Units = sectors of 512 bytes, counting from 0
    
       Device Boot    Start       End   #sectors  Id  System
    /dev/sdb1            63    578339     578277  fd  Linux raid autodetect
    /dev/sdb2        578340 299756834  299178495  fd  Linux raid autodetect
    /dev/sdb3     299756835 312496379   12739545   5  Extended
    /dev/sdb4             0         -          0   0  Empty
    /dev/sdb5     299756898 312496379   12739482  fd  Linux raid autodetect
    Warning: no primary partition is marked bootable (active)
    This does not matter for LILO, but the DOS MBR will not boot this disk.
    Successfully wrote the new partition table
    
    Re-reading the partition table ...
    
    If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
    to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
    (See fdisk(8).)
    root@kepler:~# 
    
  16. Add the target partitions into the degraded raidsets
    root@kepler:~# mdadm --manage /dev/md0 --add /dev/sdb1
    mdadm: added /dev/sdb1
    root@kepler:~# mdadm --manage /dev/md1 --add /dev/sdb2
    mdadm: added /dev/sdb2
    root@kepler:~# mdadm --manage /dev/md2 --add /dev/sdb5
    mdadm: added /dev/sdb5
    root@kepler:~# cat /proc/mdstat
    Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
    md0 : active raid1 sdb1[2] sda1[1]
          289126 blocks super 1.2 [2/2] [UU]
          
    md1 : active raid1 sdb2[2] sda2[1]
          149588151 blocks super 1.2 [2/1] [_U]
          [>....................]  recovery =  0.3% (471552/149588151) finish=52.6min speed=47155K/sec
          
    md2 : active raid1 sdb5[2] sda5[1]
          6368705 blocks super 1.2 [2/1] [_U]
            resync=DELAYED
          
    unused devices: 
    root@kepler:~# 
    
  17. Rewrite grub onto new spare drive
    root@kepler:~# grub-install /dev/sdb
    Installation finished. No error reported.
    
Last update:
2012-12-16 04:39
Author:
dnorth
Revision:
1.0
Average rating:0 (0 Votes)

You cannot comment on this entry

Chuck Norris has counted to infinity. Twice.

Records in this category

Tags