FreeBSD ZFS Vol3 Mirror ZFS

A ZFS mirror consists of two or more disks where all information is written to all members of the vdev. Reads are performed in parallel from all disks, thus increasing read performance. The mirror will have the capacity of the smallest device, and up to N-1 disks can fail without data loss. Additionally, any single-disk vdev can be transformed into a mirror using the zpool attach command. In this article, we will explain how to create a mirror and how to expand a vdev to a mirror.

Before continuing with the article, it is advisable to review this previous one so that all concepts are clear.

We check the available disks in the system:

root@freeBSD:~ # camcontrol devlist

<BHYVE SATA DISK 001>              at scbus0 target 0 lun 0 (ada0,pass0)  
<BHYVE SATA DISK 001>              at scbus1 target 0 lun 0 (ada1,pass1)  
<BHYVE SATA DISK 001>              at scbus2 target 0 lun 0 (ada2,pass2)  
<BHYVE SATA DISK 001>              at scbus3 target 0 lun 0 (ada3,pass3)  
<BHYVE SATA DISK 001>              at scbus4 target 0 lun 0 (ada4,pass4)

We verify that the system disk is ada0:

root@freeBSD:~ # zpool status

  pool: zroot  
 state: ONLINE  
  scan: none requested  
config:  
  
        NAME        STATE     READ WRITE CKSUM  
        zroot       ONLINE       0     0     0  
          ada0p3    ONLINE       0     0     0  
  
errors: No known data errors

We create the partitions that ZFS will use on the next two disks that will form the mirror:

root@freeBSD:~ # gpart create -s GPT ada1

ada1 created

root@freeBSD:~ # gpart add -t freebsd-zfs -a 4k ada1

ada1p1 added, but partition is not aligned on 8192 bytes

root@freeBSD:~ # gpart show ada1

=>     40  2097072  ada1  GPT  (1.0G)  
       40  2097072     1  freebsd-zfs  (1.0G)

root@freeBSD:~ # gpart create -s GPT ada2

ada2 created

root@freeBSD:~ # gpart add -t freebsd-zfs -a 4k ada2

ada2p1 added, but partition is not aligned on 8192 bytes

root@freeBSD:~ # gpart show ada2

=>     40  2097072  ada2  GPT  (1.0G)  
       40  2097072     1  freebsd-zfs  (1.0G)

We create the mirror pool indicating the partitions to be used:

zpool create mypool mirror /dev/ada1p1 /dev/ada2p1

We check its status:

root@freeBSD:~ # zpool status mypool

  pool: mypool  
 state: ONLINE  
  scan: none requested  
config:  
  
        NAME        STATE     READ WRITE CKSUM  
        mypool      ONLINE       0     0     0  
          mirror-0  ONLINE       0     0     0  
            ada1p1  ONLINE       0     0     0  
            ada2p1  ONLINE       0     0     0  
  
errors: No known data errors

We can see that there is 832M since it is two 1G disks in mirror mode.

root@freeBSD:~ # zfs list mypool

NAME     USED  AVAIL  REFER  MOUNTPOINT  
mypool   276K   832M    96K  /mypool

Now we will create a pool with one disk and later expand it to mirror:

root@freeBSD:~ # gpart create -s GPT ada3

ada3 created

root@freeBSD:~ # gpart add -t freebsd-zfs -a 4k ada3

ada3p1 added, but partition is not aligned on 8192 bytes

root@freeBSD:~ # zpool create mypool2 /dev/ada3p1
root@freeBSD:~ # zpool status mypool2

  pool: mypool2  
 state: ONLINE  
  scan: none requested  
config:  
  
        NAME        STATE     READ WRITE CKSUM  
        mypool2     ONLINE       0     0     0  
          ada3p1    ONLINE       0     0     0  
  
errors: No known data errors

We prepare the next disk to be attached to the vdev that forms mypool2:

root@freeBSD:~ # gpart create -s GPT ada4

ada4 created

root@freeBSD:~ # gpart add -t freebsd-zfs -a 4k ada4

ada4p1 added, but partition is not aligned on 8192 bytes

We add the second disk:

root@freeBSD:~ # zpool attach mypool2 ada3p1 ada4p1

And voilà, vdev converted to mirror:

root@freeBSD:~ # zpool status mypool2

  pool: mypool2  
 state: ONLINE  
  scan: resilvered 312K in 0 days 00:00:00 with 0 errors on Thu Nov 12 22:18:11 2020  
config:  
  
        NAME        STATE     READ WRITE CKSUM  
        mypool2     ONLINE       0     0     0  
          mirror-0  ONLINE       0     0     0  
            ada3p1  ONLINE       0     0     0  
            ada4p1  ONLINE       0     0     0  
  
errors: No known data errors

If one of the disks fails, we just need to replace it:

root@freeBSD:~ # zpool status mypool

  pool: mypool  
 state: DEGRADED  
status: One or more devices could not be used because the label is missing or  
        invalid.  Sufficient replicas exist for the pool to continue  
        functioning in a degraded state.  
action: Replace the device using 'zpool replace'.  
   see: http://illumos.org/msg/ZFS-8000-4J  
  scan: none requested  
config:  
  
        NAME                      STATE     READ WRITE CKSUM  
        mypool                    DEGRADED     0     0     0  
          mirror-0                DEGRADED     0     0     0  
            10132353784689973799  FAULTED      0     0     0  was /dev/ada1p1  
            ada1p1                ONLINE       0     0     0  
  
errors: No known data errors

We prepare the new disk:

root@freeBSD:~ # gpart create -s GPT ada2

ada2 created

root@freeBSD:~ # gpart add -t freebsd-zfs -a 4k ada2

ada2p1 added, but partition is not aligned on 8192 bytes

We replace the damaged one:

root@freeBSD:~ # zpool replace mypool 10132353784689973799 ada2p1
root@freeBSD:~ # zpool status mypool

  pool: mypool  
 state: ONLINE  
  scan: resilvered 576K in 0 days 00:00:00 with 0 errors on Sun Nov 15 00:20:48 2020  
config:  
  
        NAME        STATE     READ WRITE CKSUM  
        mypool      ONLINE       0     0     0  
          mirror-0  ONLINE       0     0     0  
            ada2p1  ONLINE       0     0     0  
            ada1p1  ONLINE       0     0     0  
  
errors: No known data errors