Use RAID & Pre-Tested Hard Drives to Improve Your Storage System
Table of Contents
1.) What is RAID
2.) Standard RAID Levels
3.) Nested RAID Levels
4.) The Right RAID for You
5.) Best RAID Levels for SMB
6.) Rebuilding Your RAID Array
7) Pre-Configured RAID and Pre-Tested Drives
8.) Limitations of RAID
9.) HDD vs SSD RAID
10.) Conclusion
At some point, a drive in one of your servers will fail. Without RAID as a safety net, data on a failed drive would have to be painstakingly restored from backups, eating up a great deal of your time and energy. However, if you have your drives configured into a robust RAID array, you can simply replace a failed drive and rewrite the missing data from the rest of the drives onto the replacement drive, allowing you to survive the drive failure without needing to navigate the complexity of recovering data from backups.
RAID (commonly the abbreviation of Redundant Array of Independent Disks) is a data storage technology that uses redundancy to ensure server availability and often increased drive performance. Redundancy achieved with RAID ensures allows for fault tolerance and recovery from drive failure without service disruption, thereby increasing data security.
Before discussing RAID, it is important to note that RAID is not backup, and it should never be treated as such. Buffalo recommends utilizing as many layers of data protection as possible to safeguard you from the devastating consequences of data loss.
1.) What Is RAID
On the most basic level, RAID is a data storage virtualization structure that combines multiple physical disk drives into a single logical unit. RAID is often used to increase the availability and (in some cases) performance of the data storage system.
A typical RAID array often contains one or more of the three following elements: striping, mirroring, and parity.
Striping
Data striping refers to writing data to separate drives. This is often used to improve performance. Writing data to separate drives minimizes read and write times, thus improving I/O performance.
Mirroring
Data mirroring refers to taking data and replicating it over multiple drives on the same array. In the event of a drive failure, you avoid total data loss as the full data set is available on another drive.
Parity
Parity provides fault tolerance by examining the data on two drives and storing the results on a third. When one drive fails, the data from the failed drive can then be recovered by working backwards from the other available drives, preventing data loss.
When a failed drive is replaced, the lost data is rebuilt from the remaining drives. RAID uses the space of one drive for a parity stripe, therefore multiple parity stripes require the use of the space equivalent of one drive per stripe.
2.) Standard RAID Levels
RAID is configured in different levels, which are basic configurations that employ the aforementioned striping, mirroring, and parity to create reliable data storage.
Standard RAID Levels
A standard RAID level will require a minimum number of drives to be configured. Due to advancements in data storage, certain RAID levels (such as RAID 2, RAID 3, and RAID 4) are theoretically possible but are considered obsolete and therefore no longer in use.
RAID 0 (Data Striping)
In a RAID 0 array, all the drives are combined into one logical unit. A RAID 0 array stripes data across all the drives, with each drive holding a portion of a file. This leads to increased read/write performance as RAID 0 offers almost the same read/write speed as the number of drives combined. For example, you can integrate two 3TB drives that read/write at 100MB/s into a 6TB RAID 0 array that can read/write data at almost 200MB/s.
RAID 0 is all about capacity and performance, but the lack of parity means no data protection. A single drive failure will result in you losing all of the data on the entire array. RAID 0 also does not scale well: the more drives you have in RAID 0, the more likely you are to lose all of your data. As such, RAID 0 is generally not recommended unless being implemented in specialized environments. Essentially, any environment where data can be quickly copied from the source should a drive fail can use RAID 0 to enjoy good read/write performance and offset the complete lack of data redundancy.
Minimum Number of Drives: 2
Capacity Utilization: 100%
Pros: Increased read/write performance, full utilization capacity, easy to implement
Cons: No redundancy
Recommended for: Non-critical, high-performance data environments such as testing environments
RAID 1 (Data Mirroring)
In a RAID 1 array, each drive contains the same data. A RAID 1 array will display one set of data as a logical unit, with the data mirrored exactly on all the other drives in the array. RAID 1 can offer increased read speed in some cases as data can be quickly accessed across multiple drives, but offers no write speed increases.
RAID 1 is all about redundancy. If a drive fails, you have an exact copy on another drive, so there is no need to restore data or rebuild the array, which can save you valuable time.
Minimum Number of Drives: 2
Capacity Utilization: 50%
Pros: Increased read performance, easy to implement, redundancy
Cons: Only half of capacity available, higher cost to achieve desired capacity
Recommended for: Data environments where availability is critical
RAID 5 (Data Striping + Parity)
In a RAID 5 array, parity is striped across all drives to help balance read/write access all drives. A RAID 5 array requires a minimum of three drives, and the utilization capacity is equal to the total number of drives in the array minus one. For example, a RAID 5 array with three drives means 66% capacity, while a RAID 5 array with four drives means 75% capacity, and so on. A RAID 5 array can offer increased read performance as in RAID 0, but typically much slower write performance as the system must write both the data block and parity data.
RAID 5 offers a good balance of capacity and redundancy. If a drive fails, the system can utilize parity to rebuild the lost data onto the replacement drive. However, two or more drives failing will result in total data loss on the array.
Minimum Number of Drives: 3
Capacity Utilization: 56-90% (depends on the size of the array)
Pros: Parity redundancy, slightly increased read performance
Cons: Lower write performance, longer rebuild times
Recommended for: Data environments that require fault tolerance, such as file storage or application servers
RAID 6 (Data Striping + Dual Parity)
A RAID 6 array essentially uses the same architecture as a RAID 5 array as the parity is striped across all drives to help balance read/write access all drives. RAID 6 uses two separate parity stripes. A RAID 6 array requires a minimum of four drives, and the utilization capacity is equal to the total number of drives in the array minus two. Like RAID 5, a RAID 6 array can offer increased read performance, but will have extremely slow write performance since the system must write the data block and then twice the parity data.
A RAID 6 array trades extra capacity for additional fault tolerance. A RAID 6 array can withstand two drive failures in the array, and is thus better suited for larger RAID groups to offset the lost capacity.
Minimum Number of Drives: 4
Capacity Utilization: 50-88% (depends on the size of the array)
Pros: Dual parity for additional fault tolerance, slightly increased read performance.
Cons: Much lower write performance, much longer rebuild times, higher cost.
Recommended for: Application servers for critical data with larger drives to increased storage.
3.) Nested RAID Levels
A nested RAID level is a RAID configuration that combines two or more standard RAID levels for additional performance or redundancy.
RAID 10 (Data Striping + Data Mirroring)
RAID 10 (sometimes referred to as RAID 1+0) combines RAID 1 and RAID 0 to offer multiple sets of mirrors striped together. RAID 10 offers very good read/write performance with redundancy and no parity calculations. However, RAID 10 tends to be more expensive as it requires more drives.
Minimum Number of Drives: 4
Capacity Utilization: 50%
Pros: Very high read/write performance, built-in fault tolerance with mirroring
Cons: Only half of capacity available, higher cost to achieve desired capacity
Recommended for: Servers that require both high write performance and fault tolerance, such as databases
RAID 50 (Data Striping + Parity)
RAID 50 (sometimes referred to as RAID 5+0) combines multiple RAID 5 sets with RAID 0. A RAID 50 array enjoys the parity of RAID 5 for redundancy while data striping allows more drives to be incorporated into the array for faster performance and shorter rebuild times.
RAID 50 is often used in large data environments where more drives can be added to the array to offset the capacity loss for parity. Similar to RAID 10, a RAID 50 array is optimal with a large number of drives, increasing its cost.
Minimum Number of Drives: 6
Capacity Utilization: 67-94% (depends on the size of the array)
Pros: Very high read performance, parity provides fault tolerance
Cons: Requires a lot of drives to assemble, increased cost to offset parity
Recommended for: High capacity, high performance data environments
RAID 60 (Data Striping + Dual Parity)
RAID 60 (sometimes referred to as RAID 6+0) combines multiple RAID 6 sets with RAID 0. A RAID 60 array requires a minimum of eight drives, and the utilization capacity is equal to the total number of drives in the array minus two. Like RAID 6, dual parity allows the failure of two drives in each RAID 6 set while striping increases capacity and performance.
Like RAID 50, a RAID 60 configuration should only be used with configurations of more than sixteen drives to offset the loss of capacity due to dual parity. RAID 60 is similar to RAID 50 but offers more redundancy, making it good for very large capacity servers.
Minimum Number of Drives: 8
Capacity Utilization: 50-88% (depends on the size of the array)
Pros: Dual parity provides additional fault tolerance, very high read performance
Cons: Requires a lot of drives to assemble, very high cost
Recommended for: High capacity, high performance data environments that require fault tolerance, such as video surveillance servers
4.) The Right RAID for You
Each RAID level has its strengths and weaknesses, and building the perfect RAID array for your needs will require you to carefully evaluate your data usage environment and budget. RAID is complex and many factors influence the final results of your array. Typically, the optimal RAID array for you will depend on the following four factors: performance, capacity, availability, and cost.
Performance
Raw performance (in terms of read/write operations) can be impacted by a variety of factors. Some RAID configurations require far more computational overhead—namely, any RAID levels that offer parity typically heavier processing in order to handle write operations, which will introduce latency. Other factors include system capabilities (software RAID) and implementation. Poor implementation can easily lead to less-than-optimal IOPS performance across the array.
Capacity
Drive capacity in a RAID array is often a direct trade-off with redundancy. Any RAID level that uses mirroring or parity for redundancy will sacrifice a certain portion of drive capacity. And although parity can be offset by increasing the number of drives, that will invariably increase cost as well.
Availability (Redundancy)
Availability refers to uptime and the fault tolerance of your RAID array. Implementing a parity RAID level allows you to afford the occasional drive failure in the array without the server going down, although it will drastically affect both usable capacity and write performance.
Cost
Cost is important as you will need to balance both the available features of your RAID array with the number of drives you use in your array. Expanding your RAID array means you will need to purchase a minimum number of drives to sustain the same RAID level, so cost is a factor that cannot be diminished.
As anything, focusing on one factor will typically come at the expense of another. A RAID 0 array will offer blazing fast read and write performance and 100% capacity utilization, but even one drive failing will wipe out all data on the array. It is generally agreed upon that the benefits of fast read/write access are far outweighed by the threat of losing all data in the event of a drive failure. On the flip side, RAID 6 offers maximized fault tolerance but will often have less-than-ideal writing speed, capacity utilization, and cost.
When it comes to RAID, there is no one-size-fits-all approach. Since each RAID level has its pros and cons, Buffalo recommends utilizing our RAID calculator to try out possible configurations and see which ones might meet your performance and capacity requirements.
5.) Best RAID Levels for SMB
For SMB, the trade-off between the various factors above becomes more apparent. Since most SMB tend to be working with a limited IT budget, it is paramount to avoid waste and inefficiencies when building a RAID array for your business. For example, RAID 0 and 1 both have the lowest cost of entry (two drives), but are typically not recommended for SMB. RAID 0’s speed and full capacity utilization can be a good option for processes that are not mission-critical, but its complete lack of redundancy means it’s extremely high risk. Compared to RAID 0, RAID 1 will offer redundancy, but at the cost of half the capacity. RAID 1 is typically only recommended for environments where a large capacity is not required, and the data needs to be fully recoverable in case there is a drive failure. Some common use cases of RAID 1 might be accounting systems or smaller environments that process mission-critical data.
Instead, RAID 5 or RAID 10 tend to offer the most benefits for SMB as they offer the best balance between performance, redundancy, and cost. RAID 5 is a popular configuration because it offers fault tolerance and a greater utilization capacity than RAID 1 and RAID 10. RAID 5 is a good fit for environments that require good read performance with parity such as fileservers and general storage or backup servers. However, RAID 5 is not suitable for high-write environments such as databases due to its generally slow write performance.
RAID 10 offers both fast read/write performance and data redundancy, but it provides only half capacity utilization and costs far more to implement to expand compared to RAID 5. A RAID 10 array is best suited for larger data environments where higher budgets can allow for higher performance. Without the need for parity, a RAID 10 array will also offer significantly faster rebuild times than RAID 5 should a drive fails—data can simply be copied over from other drives instead of having to rebuild the entire array.
6.) Rebuilding Your RAID Array
Rebuilding a RAID is necessary when a drive in the array stops working. Even if the array itself is still online due to parity, you should still replace the drive as soon as possible. A drive that goes offline means that the workload on the other drives increases. This puts the array on a slippery slope where errors are more likely to occur on the remaining drives, and if your entire array goes down data recovery may become impossible. It is always a good idea to know how and when to rebuild your RAID array when necessary.
Hot Spares
Hot spares are essentially standby drives in a RAID array. They are fully-functional blank drives with no data on them. In the course of a RAID array, if a drive fails, the hot spare drive is automatically substituted for the failed drive without requiring you to physically replace the drive. The RAID controller will automatically reconstruct data from the failed drive to the spare drive, allowing the array to continue operations. This action is typically referred to as “hot swapping”. Only certain RAID levels support hot spares—most commonly RAID 5 and RAID 6.
Typically, the hot spare can replace any drive in the array with the same capacity or smaller, and the hot spare should be of the same drive type (HDD or SSD) as the other drives in the array. Many modern NAS devices that offer RAID configurations will also feature the hot spare feature so that if a drive fails, it can be quickly replaced without needing to power down the device first.
RAID Maintenance
To ensure your RAID array continues to run without issues, it is important to take a few simple steps to make sure your drives remain as healthy as possible. An unchecked drive error can easily balloon into bad sectors and cause your drive to fail.
Run Regular RAID Checks
To stay up-to-date on the health of your RAID array, it is recommended to set up and run a RAID scan at regular intervals to check for bad sectors that might be on the drives of the array. Depending on the size of your array and how heavily it is used, you can run a scan once a week to once a month. Depending on the RAID controller and system, you can also schedule to the scan to run automatically. Detecting bad sectors early will go a long way in helping you avoid drive failures.
Monitor System Logs
Just as you should regularly run RAID scans, you should also monitor system event logs and alerts to detect possible issues as quickly as possible. Should you detect any warnings or errors, you should take precautions and resolve any issues sooner than later to preserve your hard drives.
Use Hot Spares
If possible, you should always have a hot spare available in your array as long as it is supported. Having a spare drive saves you the time and effort of having to bring your array offline in order to replace a failed drive. Instead, the hot spare will immediately replace the failed drive and you can then replace the hot spare at your convenience.
RAID Rebuild Safety
The general rule of RAID rebuilds is that the more drives in the array, and the larger the hard drives in the array, the longer the rebuild time will be. A RAID rebuild process can be complicated and time-consuming, with many possibilities of having something go wrong and result in data loss.
Until the data is fully restored, you must be extremely careful in using the array. You do not want to accidentally make changes to your files that can lead to overwriting data on a drive with errors, as it might render the data unrecoverable. If you can back up the data before proceeding, definitely do so to avoid taking chances. And if physically replacing failed drives, remove and replace only one drive at a time, as disrupting the sequence of the drives can lead to an array error.
One of the most important aspects to be aware of is the quality of the drive. An old drive may already have bad sectors or other errors on the drive itself. If you rebuild a RAID array using an older drive that has not been tested, you run the risk of the entire array crashing once the rebuild process is finished. And depending on the drive storage technology, you may consider not using certain drives at all if using them will ensure RAID rebuild failures.
If you cannot safely access or have backups of the data stored on your degraded RAID, or if you are not confident in rebuilding the array, do not attempt to rebuild the array. Instead, you should contact a trustworthy data recovery service to avoid risking losing your critical data. If you are setting up a RAID array for the first time, and you have the resources to spare, you should find out how long rebuilding your array will require so you can be ready if you do need to rebuild it in the future.
7.) Pre-Configured RAID and Pre-Tested Drives
Many modern NAS manufacturers offer pre-configured RAID out of the box. While some businesses would prefer to build their own arrays to control their own data storage, there are various benefits of pre-configured RAID.
Obviously, pre-configured RAID can save you the time and effort of having to build your own RAID array. A RAID build, especially a non-RAID 0 or RAID 1 array, can be complex and time consuming, and you should do extensive testing of the drives and the array to make sure there are no faults or errors before you start. On the other hand, NAS devices are manufactured to meet rigorous uptime demands. Once a NAS server is deployed and running, you might have very few opportunities in the future to take if offline for detailed testing.
Before building your own RAID array, you should also consider the drives being used. In short, you should only purchase hard drives from trusted suppliers. There are suppliers out there who do little in quality testing before selling (or re-selling) low quality hard drives. These may be drives that are manufacturer rejects, or pre-used hard drives that had the drive memory re-configured to change the firmware settings. These pre-used drives may display and behave as if they have a larger capacity than they actually do. Muddled drive capacity can be a huge issue, as if you go beyond a drive’s actual capacity without being aware, data newly-written to the drive may either overwrite existing data or simply not be written to the drive at all. It may seem that the drive is operating as normal, but when you try to access phantom files or folders that do not actually contain the labeled content, you will find your critical data lost and forever irretrievable.
While you can perform your own tests with hard drives, it can be very time-consuming. You can download and run specific drive testing utilities on your system to test your hard drives, or use existing diagnostic tools to check the drives for errors before using them. Most modern hard drives are equipped with S.M.A.R.T. (Self-Monitoring and Reporting Technology). S.M.A.R.T. is used to detect and repair drive issues such as bad sectors, lost clusters, directory errors, etc. Making sure that your hard drive vendor pre-tests their drives before you purchase them can go a long way to prevent drive issues from appearing in the future, all the while saving you valuable time and effort.
Even though hardware failures can and will occur eventually after extensive usage, you can still give yourself the best chance to avoid drive failures with good pre-tested drives and eliminate hassle factors.
8.) Limitations of RAID
Although RAID is designed to help, there are a few things to keep in mind on what RAID does not do:
RAID Does NOT Prevent Drive Failure
Although RAID is helpful in protecting you against a drive failure, RAID does not prevent drive failure. RAID will not be able to prevent data corruption, human error, cyberthreats, or other events that might cause drive failure. Although RAID can help minimize availability issues in the event a drive fails, having a RAID array in place will not effect 100% system availability.
RAID Does NOT Replace Backups
As mentioned above, RAID does not prevent drive failure. Therefore, RAID should not be used in lieu of backups—nothing can replace a well-planned and well-executed backup plan. Despite your best efforts, drive failures can still occur. You should still have backups in place to offset the possible repercussions of data loss. If you do not have backups in place, setting up a RAID array will be of little use for data protection.
RAID Does NOT Offset Storage Planning
Having a RAID array in place does not necessarily mean you can freely increase the size of the array. For most RAID levels, you cannot simply add another drive to the array should you require more storage. Instead, you will most likely end up rebuilding the array from scratch. And if you want to keep your current RAID level, you will need to purchase a minimum number of drives. For example, if you wish to expand an existing RAID 10 array, you will need to purchase at least four new drives to add two mirrored sets. If you wish to explore more flexible storage expansion options, iSCSI and SAN may be a better option.
9.) HDD vs SSD RAID
With SSD prices falling in recent years, SSD have been taking over HDD in some data storage setups. Even though most RAID arrays utilize HDD for the cost efficiency, many people are invariably wondering: HDD RAID or SDD RAID?
As hard drives go, SSDs tend to be less prone to mechanical drive failure as they have no moving parts and generate less heat. However, this does not mean that drive failures do not occur for SSDs—although rare, they still occur, and data stored on an SSD can still be lost. To compare the two drive types in RAID, it boils down to two crucial factors: performance and cost. Other aspects such as drive reliability typically do not factor into an HDD RAID vs SSD RAID. Learn more about SSD and NAS here.
In terms of raw performance, SSD RAID will always offer better read/write performance than HDD RAID. However, this performance increase can depend on a few factors, such as system configuration and types of data usage. For example, if using a software RAID controller, read/write speed may end up being bottlenecked by other system components, such as the processing speed of your computer. For smaller data environments, there may not be enough system resources to create a notable increase in performance.
The biggest limitation of SSD RAID still comes down to cost. SD prices have been falling over the past few years, but pound for pound SSD will still cost more per gigabyte than their HDD counterparts. When utilizing SSD in a non-RAID 0 array, you will lose additional capacity to make room for redundancy, drastically increasing costs if you need the raw capacity. This difference in cost becomes even more apparent when you need to expand your SSD RAID array, rendering SSD RAID as not always the practical choice depending on your environment and usage.
10.) Conclusion
All in all, RAID is a substantial way to optimize your storage and increase your drive performance while building in fault tolerance to circumvent inevitable drive failure. RAID provides a safety net so you can weather the failure of one or more drives without data loss, and can be used in conjunction with a well-planned backup plan for multi-layer data security. You will need to closely examine your specific environment to find out what RAID level is right for you.
Buffalo’s NAS devices come with RAID pre-configured out of the box to save you the hassle of assembling your own RAID array. Our devices come with hard drives included, and are tested to ensure the highest quality enterprise-grade drives so you can trust us with your data. Buffalo prides itself in providing reliable and easy-to-use data storage products along with personal customer service and 24/7 North America-based support. Responsible for many industry firsts and technology standards, Buffalo has four decades of networking and computer peripheral manufacturing and design experience. For more information about Buffalo Americas Inc. and its products, please visit http://www.BuffaloAmericas.com.