RAID-10 Configuration on Linux – Everything You Need to Know

In this article, we’ll cover everything you need to know about RAID-10 configuration on Linux. As an experienced professional in the field, I understand the importance of RAID-10, which combines the benefits of performance and data redundancy, making it an ideal choice for users seeking high-speed storage with reliable data protection. I will walk you through the process of setting up a RAID-10 array using the mdadm tool and provide tips for monitoring and maintenance. Whether you’re a beginner or have experience with Linux, this guide will help you configure and manage RAID-10 for optimal results.

Introduction

First of all, I want you to ignore everything about servers, server vendors, and hardware RAID controllers. To be more general, everything “Enterprise.” In this article, I will show you how to configure a RAID-10 array, a type of RAID configuration that provides both data redundancy and performance benefits, on Linux (I will use Debian Linux) by leveraging the tools already provided by Linux (operating system). You can do that on any computer, provided it has at least four identical hard drives, and it doesn’t have to be server-specific.

Below, you’ll find what I consider essential FAQs that highlight RAID-10 on Linux advantages and benefits for your data management needs and why you should use it:

What is RAID-10, and how does it work on Linux?

RAID-10 combines the benefits of RAID-0 (striping) and RAID-1 (mirroring) to provide both performance and redundancy. In Linux, RAID-10 arrays are typically configured using the mdadm tool, which allows users to manage and monitor software-based RAID arrays effectively.

What are the benefits of using RAID-10 on Linux?

RAID-10 offers a balance between speed and data protection. Thanks to striping, it provides faster read/write speeds while ensuring data redundancy through mirroring. This makes it an excellent choice for databases, web servers, and other performance-critical Linux systems.

What hardware do I need to set up a RAID-10 array on Linux?

To set up a RAID-10 array on Linux, you need at least four identical or similar hard drives. While RAID-10 can be implemented on any Linux system, using drives of the same size and speed is recommended for optimal performance.

How do I monitor and maintain a RAID-10 array on Linux?

You can monitor your RAID-10 array using mdadm commands such as mdadm --detail /dev/mdX or cat /proc/mdstat. Regular checks, backups, and ensuring the health of individual drives are essential for long-term reliability.

Can I recover data if a drive fails in a RAID-10 array on Linux?

Yes, RAID-10 is designed to handle individual drive failures. If a single drive fails, you can replace it, and the array will automatically rebuild using the mirrored data. However, simultaneous failures of mirrored pairs can lead to data loss.

I have been using RAID-10 on Linux at work and on my custom-built home server, and it has always performed reliably. Now, let’s dive into the specifics of the setup.

Prerequisites

To get started, you’ll need a computer with Linux installed. I will be using Debian Linux as my primary system. If you’re interested in learning how to install and configure Debian on your computer, please look at my articles below, which provide detailed instructions:

Debian 11 Server – Minimal Installation Guide

Follow this guide for a Debian 11 Server minimal installation, providing a solid foundation for any server setup or project you want to build.

Read More!

Debian 11 Server – Initial Customization Guide

Discover introductory steps to streamline performance, security, and administration in our Debian Server Initial Customization guide.

Read More!

To successfully assemble the RAID-10 array, you will need at least four identical hard drives. I suggest you go a step further and use a separate single disk (or two identical disks in a RAID-1 array) for your operating system and five identical hard drives for the RAID-10 array (one disk will be used as a hot spare).

RAID-10 Usage Examples

I am using RAID-10 on one of my home servers, a versatile system that stores all my data and serves as a backup and recovery system. RAID-10, which combines mirroring and striping, is ideal for scenarios that require both high performance and fault tolerance. Here are a few examples where RAID-10’s versatility would be particularly useful:

• Database Servers
  • Use Case: High-speed transactional databases (e.g., MySQL, PostgreSQL).
  • Why RAID-10: Ensures quick read/write operations while providing redundancy to prevent data loss in case of disk failure.
• Web Servers Hosting High-Traffic Websites
  • Use Case: Websites or web applications that experience heavy read/write operations, like e-commerce platforms.
  • Why RAID-10: Offers fast data access to handle numerous simultaneous requests while maintaining uptime during drive failures.
• Virtualization Hosts
  • Use Case: Running virtual machines (VMs) for development, testing, or production environments.
  • Why RAID-10:The high IOPS demand of virtual machines is efficiently handled by RAID-10, with data redundancy safeguarding VMs from disk failure.
• Data Analytics Platforms
  • Use Case: Big data processing and analysis, like Hadoop clusters or Splunk deployments.
  • Why RAID-10: Combines fast sequential read/write speeds with data reliability, critical for analytics workloads.
• Backup and Recovery Systems
  • Use Case: Systems storing backups that require frequent restores or incremental updates.
  • Why RAID-10: Allows fast data recovery with minimal downtime while ensuring redundancy.
• High-Performance Computing (HPC) Systems
  • Use Case: Scientific simulations, weather modeling, or other compute-intensive tasks.
  • Why RAID-10: Supports the high-speed data access required for HPC workloads while ensuring reliability.

RAID-10 combines speed and redundancy, making it ideal for mission-critical systems where downtime or data loss is unacceptable.

RAID-10 Disk Types

When building my home server a couple of years ago, I opted for plain mechanical disks, which proved sufficient for my data storage/backup and recovery system. However, depending on your system’s use, you might find SSDs beneficial for their speed or even NVMEs for the ultimate performance and speed. Here’s a detailed breakdown of the advantages and disadvantages of using mechanical drives (HDDs), solid-state drives (SSDs), and PCIe drives (NVMe) for a RAID-10 array on Linux:

Using the table above, you can determine which disk type is most suitable for your specific use case. Here’s a brief summary of what I believe may be the best option for RAID-10 in Linux:

• For budget-conscious users: HDDs are suitable for archival or bulk storage needs.
• For balanced performance and cost: SSDs strike a good compromise for most workloads.
• For maximum performance: NVMe drives are ideal for database servers, virtualization, or other high-speed environments but come at a premium price.

Selecting the appropriate drives depends on your specific requirements, such as storage capacity, budget, and performance needs.

RAID-10 Configuration

Once the operating system is installed and all required hard disks are connected, you can start with the RAID-10 array assembly. I will presume all your disks are empty and unformatted. If not, you must first erase all the data from the disks you intend to use for your RAID-10 array. You can check my article on how to perform a safe erase of disk(s) on Linux below:

Secure Erase on Linux – Safely Wipe Drives for Data Security

Secure erase on Linux is crucial for data security. Learn the safest methods to wipe drives and protect sensitive information with this guide.

Read More!

Tools Required

First, let’s make sure you have all the required tools. The following CLI tools must be installed: lsscsi, parted, mdadm, smartctl, and mkfs. Optionally, if your disks are not empty, you will also need hdparm or dd. You can check the presence of the required commands by executing the following in the console:

for tool in lsscsi parted mdadm smartctl mkfs; do echo $tool; $tool --version; done

If the result for any of those tools is “command not found” or similar, you must install it. For instance, there is a good chance mdadm, and smartctl are missing, so execute the following to install them:

apt install mdadm smartmontools

Accepting the defaults for any questions the installer may ask is OK, but I suggest you read them and try to understand what is asked.

Disk Partitioning

You need to partition your drives properly before assembling the RAID-10 array. First, ensure that all the drives you plan to use are visible by executing the following command in the console:

lsscsi

The result should be similar to the one below. In general, make sure you have six disks present (or seven if the operating system is installed on a RAID-1 array, which is always a good practice):

[0:0:0:0]    disk    QEMU     QEMU HARDDISK    2.5+  /dev/sda
[0:0:0:1]    disk    QEMU     QEMU HARDDISK    2.5+  /dev/sdb
[0:0:0:2]    disk    QEMU     QEMU HARDDISK    2.5+  /dev/sdc
[0:0:0:3]    disk    QEMU     QEMU HARDDISK    2.5+  /dev/sdd
[0:0:0:4]    disk    QEMU     QEMU HARDDISK    2.5+  /dev/sde
[0:0:0:5]    disk    QEMU     QEMU HARDDISK    2.5+  /dev/sdf

In the example above, the operating system is installed on the /dev/sda disk. I will create one identical partition on each of the other drives (/dev/sdb to /dev/sdf), as I plan to use a RAID-10 array for data storage and backup/restore purposes. To accomplish this, please execute the following command in the console:

for disk in /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf; do echo $disk; parted -s -a optimal $disk mklabel gpt mkpart data ext4 2048s 100% set 1 raid on; done

As you can see, I have created an identical partition on each drive. Here are the specifics:

• Partition scheme: GUID Partition Table (GPT)
• Partition name: data
• Partition start: Sector 2048
• Partition end: End of the disk (100%)
• Partition flag: raid

With the partitions in place, you can now assemble a RAID-10 array. In the next chapter, we will see how.

RAID-10 Array Setup

RAID-10 configuration on Linux is relatively straightforward as long as you’ve completed all the necessary prerequisites. In the following example, I will demonstrate how to assemble a RAID-10 array using the mdadm command. The array we create will be a “type of “software RAID,” which means” no hardware RAID controller is involved; instead, the array is managed by the subsystems within the Linux kernel.

First, you need to check if there are any existing RAID arrays. To do so, please execute the following in the console:

cat /proc/mdstat

If there are no arrays present, the output should be like the one below:

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
unused devices: <none>

In case there are arrays present (i.e., if you installed the operating system on a separate RAID-1 array), the output will be similar to the one below:

Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] [multipath]
md0 : active raid1 sda3[0] sdb3[1]
      XXXXXXXXXX blocks [2/2] [UU]

md1 : active raid1 sda2[0] sdb2[1]
      XXXXXX blocks [2/2] [UU]

unused devices: <none>

This is very important because it will influence which array identifier (mdX) you use when creating a RAID-10 array. In the example above, you can notice that md0 and md1 are already active, so you must use md2 for your new RAID-10 array. In my example, the operating system is installed on /dev/sda, and I don’t have any active RAID arrays, so I will simply use md0:

echo /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 | xargs mdadm --create /dev/md0 -c512 -l10 -n4 --assume-clean

The RAID-10 array should assemble instantly, outputting the following result:

mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.

To summarize, I used the parameters which, in my opinion, are suited for RAID-10 array no matter the disk type:

• RAID array identifier: md0
• Chunk size (-c): 512
• Raid array type (-l): 10
• Number of Devices: 4
• Additional options: –assume-clean

Now, while the RAID identifier, type, and number of device parameters are straightforward, there might be a discussion about chunk size and the –assume-clean option.

The optimal chunk size can vary based on the type of disks you are using, and it often necessitates additional testing. In brief, you can experiment with different chunk sizes while conducting read-and-write operations to identify the best value for your setup. Additionally, if you are utilizing more than four disks in your RAID-10 array, it’s advisable to adjust the chunk size accordingly. For instance, if you create a RAID-10 array with eight disks, I recommend setting the chunk size to 1024. The chunk size is also a crucial factor for the file system, which I will explain later in this guide.

Considering the advancements in SSD and NVMe development, one might assume the erase block on those devices is large enough that using large chunk sizes like -c1024 makes no sense. Thus, a “default” value of -c512 is something that may be considered as a satisfy-all value.

Regarding the –assume-clean option, I advise using it only when you have brand-new (or refurbished) disks. Otherwise, omit the option altogether.

File System & Mount Point

Before you can begin using your RAID-10 array, you need to create a file system and mount it at a designated location. In this example, I will use the ext4 file system and mount my array to the directory /mnt/storage. To create an ext4 file system, you may utilize an online tool here to assist with the file system parameters. One important detail to know is the chunk size; in my example, I used 512 KiB. Therefore, for a RAID-10 array consisting of four disks and a chunk size of 512 KiB, you can run the following command in the console to create the ext4 file system:

mkdir /mnt/storage
mkfs.ext4 -b 4096 -E stride=128,stripe-width=256 /dev/md0

Once the file system is created, you can define the mount point. I suggest immediately adding the mount definition to the /etc/fstab file. To do so, open the /etc/fstab file in the text editor and add the following line:

/dev/md0    /mnt/storage    ext4    defaults,errors=remount-ro,nodelalloc,data=ordered,discard  0       0

Save and close the file and execute the following in the console:

systemctl daemon-reload
mount /mnt/storage

To check if everything is OK, you can execute the mount command in the console:

mount
...
/dev/md0 on /mnt/storage type ext4 (rw,relatime,discard,nodelalloc,errors=remount-ro,stripe=256,data=ordered)

And that is it. Congratulations! You have successfully configured the RAID-10 array with the ext4 file system and mounted it to a desired location.

Hot Spare Disk

While not mandatory, this RAID-10 configuration guide would not be complete without me showing you how to add a hot spare disk. Having a hot spare disk will spare you the trouble of acting immediately if your RAID-10 array becomes degraded (i.e. if one disk fails). The hot spare disk will automatically “jump in” in case of failure, and your array will also automatically start the rebuild process without you having to intervene.

Adding a hot spare disk is straightforward. In the example above, I already have a disk ready (/dev/sdf1) and partitioned (this is mandatory!). To add it, the following must be executed in the console:

mdadm --manage /dev/md0 --add-spare /dev/sdf1

And there you have it! With minimal effort, you can reduce one worry from your mind. It’s important to note that the hot spare will take over automatically in the event of a disk failure. However, you will still need to replace the failed disk and set the new one as a hot spare, but you won’t have to do this immediately. That sounds great, doesn’t it?

RAID-10 Maintenance

This section will guide you on monitoring your RAID-10 array. I will also explain how to configure alerts so that you are notified if your array becomes degraded or if there are issues with space or inode (the number of files). Whenever a problem occurs, you will receive an email notification, and I will show you how to set this up.

Monitoring

Monitoring a RAID-10 array in Linux ensures optimal performance and data integrity. Regular monitoring lets you detect issues such as degraded arrays, failed disks, or performance bottlenecks before they lead to critical failures. It may sound complicated, but trust me, it’s not.

I mentioned above that the Linux kernel controls our RAID-10 array. The Linux kernel stores information about system hardware and processes in the /proc directory, a unique directory known as a virtual file system. It is mounted in RAM, and each time you stop or restart your system, the files within it are flushed and regenerated during the next boot. The state of our array is saved in one such file: mdstat. To observe it, you can execute the following in the console:

cat /proc/mdstat

The following output will be displayed:

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid10 sdf1[4](S) sde1[3] sdd1[2] sdc1[1] sdb1[0]
      20953088 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]

unused devices: <none>

You can observe that our RAID-10 is active, the sdf1 device is marked as spare (S), and there are four devices in the array (sdb1, sdc1, sde1, and sde1), and they are all active/up (U). It’s just the way we configured it. To find out even more details, you can use the mdadm command with –detail option:

mdadm --detail /dev/md0

The following output will be displayed:

/dev/md0:
           Version : 1.2
     Creation Time : Fri Jan  3 14:12:14 2025
        Raid Level : raid10
        Array Size : 20953088 (19.98 GiB 21.46 GB)
     Used Dev Size : 10476544 (9.99 GiB 10.73 GB)
      Raid Devices : 4
     Total Devices : 5
       Persistence : Superblock is persistent

       Update Time : Fri Jan  3 16:11:47 2025
             State : clean
    Active Devices : 4
   Working Devices : 5
    Failed Devices : 0
     Spare Devices : 1

            Layout : near=2
        Chunk Size : 512K

Consistency Policy : resync

              Name : debian-bookworm-test:0  (local to host debian-bookworm-test)
              UUID : 68b63a8f:59e523f9:c9019d48:21e06a78
            Events : 3

    Number   Major   Minor   RaidDevice State
       0       8       17        0      active sync set-A   /dev/sdb1
       1       8       33        1      active sync set-B   /dev/sdc1
       2       8       49        2      active sync set-A   /dev/sdd1
       3       8       65        3      active sync set-B   /dev/sde1

       4       8       81        -      spare   /dev/sdf1

Everything in the output above should be relatively straightforward, but the “Used Dev Size” value might be a bit confusing. This value represents the size of the smallest device (disk drive) used in the array. In my example, I utilized four identical drives, each with a capacity of 10GB, which is the value reflected by “Used Dev Size.”

In addition to monitoring the array itself, you can also check the space and inode (number of files) usage. To do so, you can use df command to check disk space:

df -h /dev/md0
...
Filesystem      Size  Used Avail Use% Mounted on
/dev/md0         20G   24K   19G   1% /mnt/storage

And inode usage:

df -i /dev/md0
...
Filesystem      Inodes IUsed   IFree IUse% Mounted on
/dev/md0       1310720    11 1310709    1% /mnt/storage

This summarizes the monitoring process, which is quite straightforward. If you know of more advanced techniques for monitoring RAID arrays, please share in the comments.

Alerting

Alerting for a RAID-10 array in Linux is critical to maintaining data availability and system reliability. Configuring tools like mdadm to send email notifications or system alerts ensures you’re immediately informed of issues such as disk failures, degraded arrays, or rebuilding events. Timely alerts allow you to take swift action, minimizing downtime and preventing data loss.

Mdadm has a “built-in” monitoring service that will send email alerts to a system root email (/var/mail/root) by default. To check if the service is working, you can execute the following in the console:

ps aux | grep "[m]dadm --monitor"

If the output is empty, you can start the service manually:

mdadm --monitor --scan --daemonise

You can add this command to your system’s startup scripts or use a systemd service to ensure it runs on boot. This works out of the box on Debian (and possibly other distributions), eliminating the need to add any additional startup scripts. You can simply check the status of the monitor service to verify this:

[10:24:57:root@debian-bookworm-test:~]# systemctl status mdmonitor.service
...
● mdmonitor.service - MD array monitor
     Loaded: loaded (/lib/systemd/system/mdmonitor.service; static)
     Active: active (running) since Fri 2025-01-03 14:12:14 CET; 20h ago
       Docs: man:mdadm(8)
   Main PID: 1539 (mdadm)
      Tasks: 1 (limit: 4637)
     Memory: 344.0K
        CPU: 197ms
     CGroup: /system.slice/mdmonitor.service
             └─1539 /sbin/mdadm --monitor --scan

Jan 03 14:12:14 debian-bookworm-test systemd[1]: Started mdmonitor.service - MD array monitor.

This service will automatically send an email alert to the system root user (located at /var/mail/root) whenever there is a problem with the RAID array. If you would like to change the recipient’s email address, you can do so by editing the “/etc/mdadm/mdadm.conf” file. Simply set the MAILADDR directive to your preferred email address. Please note that this feature will require a mail transfer agent (MTA) to function correctly. We have an article available to assist you with this setup:

Send Mail – Gmail Relay Setup on Debian Linux

Effortlessly send mail from your Debian Linux server using Gmail as a reliable relay. Follow our step-by-step guide for DMA configuration.

Read More!

Unfortunately, there are no “native” services or mechanisms in Linux (that I am aware of) that would send alerts about space and inode usage. But that doesn’t mean it is impossible. All we need is a simple bash script. You can create one by opening a new file, let’s say “/usr/local/sbin/raid_usage_monitor.sh” and paste in the following content:

#!/bin/bash

# Set the RAID device or mount point
RAID_DEVICE="/dev/md0"
EMAIL="root"

# Function to check usage
check_usage() {
    local type="$1" # "space" or "inode"
    local usage

    if [ "$type" == "space" ]; then
        usage=$(df -h "$RAID_DEVICE" | awk 'NR==2 {gsub("%",""); print $5}')
    elif [ "$type" == "inode" ]; then
        usage=$(df -i "$RAID_DEVICE" | awk 'NR==2 {gsub("%",""); print $5}')
    fi

    echo "$usage"
}

# Send email alert
send_alert() {
    local level="$1"
    local type="$2"
    local usage="$3"

    echo -e "Subject: [$level ALERT] RAID $type Usage on $RAID_DEVICE\n\n\
The $type usage on $RAID_DEVICE has reached $usage%. Please check the system." \
    | sendmail "$EMAIL"
}

# Check space usage
space_usage=$(check_usage "space")
if [ "$space_usage" -ge 95 ]; then
    send_alert "CRITICAL" "disk space" "$space_usage"
elif [ "$space_usage" -ge 85 ]; then
    send_alert "WARNING" "disk space" "$space_usage"
fi

# Check inode usage
inode_usage=$(check_usage "inode")
if [ "$inode_usage" -ge 95 ]; then
    send_alert "CRITICAL" "inode" "$inode_usage"
elif [ "$inode_usage" -ge 85 ]; then
    send_alert "WARNING" "inode" "$inode_usage"
fi

This script will send a warning if space or inode usage reaches 85% and a critical warning when it reaches 95%. Please adjust RAID_DEVICE to match the array. Also, if you followed our Send Mail – Gmail Relay Setup on Debian Linux article, it is OK to leave root for EMAIL. You must make the script executable, and to do so, please execute the following in the console:

chmod +x /usr/local/sbin/raid_usage_monitor.sh

To make the script run automatically, you must define a cron job. To do so, first execute the following in the console:

crontab -e

Past the following line at the bottom of the file:

0 3 * * * /usr/local/sbin/raid_usage_monitor.sh

Save the file and close it. The script will now execute daily at 3 AM, and you will receive an alert email if there is a warning or critical warning regarding space or inode usage.

Videos

We’re thrilled to announce that this article on RAID-10 configuration on Linux is the first on our website to feature related videos! Dive deeper into the setup process and concepts with our carefully curated video section, designed to complement the guide and make learning more engaging. We’re excited to bring this new dimension to our content and hope it enhances your experience. Check them out below!

We have a YouTube video and a special video on Asciinema that allows you to pause the video and copy/paste the commands from inside the video. How great is that!

RAID-10 Configuration on Asciinema

RAID-10 Configuration on YouTube

Conclusion

RAID-10 delivers an outstanding combination of performance, redundancy, and reliability, making it the perfect solution for your critical workloads. This guide equips you with the essential steps required to configure your RAID-10 array, monitor its health, and set up alerts, ensuring that your setup is both efficient and secure on Linux. By leveraging the strengths of different drive types, recognizing optimal use cases, and establishing proactive monitoring, you can harness the full potential of your RAID-10 array and protect your valuable data. Embrace the power of RAID-10 for seamless computing!