Exploring New Storage Options
My last post was focused on backups, and my work on this has uncovered a few more details which has caused me to expand the project into the core storage used for my main data as well.
Specifically, I’ve just converted (2) 4TB hard drives into a single RAID-0 striped storage volume. While most folks do this simply to get faster access to their data, and this was one of the reasons for my doing it, I also had another reason as well.
My photography / videography needs more space than could be afforded by the available space on the main 4TB drive I had within my system, even for the files I need relatively quick access to, so I had dedicated a full 6TB drive to “current and recent” projects. While this worked, it also left a lot of space on the 4TB drive unused. Essentially, wasted space.
By combining (2) 4TB drives into a single 8TB storage volume, I can put both my publishing data and the “current” photography data on the same volume and better utilize the space. Getting faster access to this data was just a bonus, which may potentially allow me to directly access common video files (i.e. – video backgrounds for green screen work) in my video editing without having to first move them to the faster SSD drive, which I’m using as a “workbench” drive.
Since I keep good automated backups on a daily schedule, there’s not much risk from the RAID-0 arrangement. Theoretically, I could change the backup schedule to hourly (or even less if I wanted to go crazy), and there would be much less risk.
As I write this, the data is being copied back over from a new 8TB external drive I’ll use for backups.
While researching the RAID options again, I ran across a piece of software called FlexRAID, which looks really interesting. Specifically, it allows the use of cheap consumer drives in a RAID array, with the added benefit of being able to add new drives at any time. Not as fast as a true RAID system, but much more flexible. I think I’ll use this to consolidate my backup drives together and give them some extra protection.
Once this project is done, I’ll have (1) 8TB storage volume inside my computer, (4) 6TB drives in an external case, merged into a single 24TB volume (probably striped in RAID-0 for extra speed), and another external case with several 8TB drives (currently, 3) consolidated into a FlexRAID setup, used to store backups of everything else. This is in addition to the OS drive, which is a 480GB SSD, and my “workbench” drive — a 1TB SSD.
Adding another 8TB drive to the backup volume wouldn’t cost too much, so I might go ahead and just do it to fill out the system. That would give me a full 24TB of backed-up storage (using a RAID-5 type arrangement) for my backups. Essentially, a 3rd level of data protection.
Should be several years before I need to add more storage, wouldn’t you think?
Since posting the above, I discovered that trying to use my external USB enclosure to stripe data in a RAID-0 array isn’t worth the effort. I’ve also done a little more research.
While FlexRAID seems to be a good and stable option (I spent a few days testing it, and it seems to work fine), I’m concerned that the developer is often unreachable, and there is no option for other support if needed. There are also some technical things which make it less than simple to use, and there’s not a lot of good documentation available. I certainly wouldn’t want to update the software, find that it doesn’t work right, and then not be able to ask for help.
So, I looked for similar options (by doing an online search for “alternatives to flexraid” — without the quotes) and found DrivePool, which also looks interesting. Seems to be more stable, and in some ways more flexible, although it does not use any type of RAID parity for data recovery, apparently due to speed of use, as well as simplicity of operation. DrivePool does support file duplication on multiple hard drives, which is a type of backup. There are many people who use DrivePool and FlexRAID together in a single system, so it’s possible to get the best of both.
When I consider that my current backup process is essentially creating an extra copy of my data, using a system like DrivePool that creates duplicate files isn’t that different. Duplication is quicker, but the backup CAN provide more than 1 copy of data that changes more often, allowing me to access a version before the most recent edits were done. With a duplication system, I’d have to save newly edited files with a new name in order to have access to previous versions. Not a big problem when you have enough drive space, but would add to the time involved in cleaning up older, and unneeded, versions.
DrivePool also has an option to create as many copies of certain folders/files as you define, so I could easily set up my commercial work to be better protected than the rest of my data. Kind of like having a backup of your backup, which I’ve been doing with some of my data. The other advantage of this is that when I need to access a duplicated file, DrivePool can read all copies simultaneously for faster access. I’m sure this will come in handy sometime.
Overall, it has been many years since I’ve needed to recover any data from a backup. Since my primary goal in this project has been to establish a “set and forget” system, I think DrivePool may be the best choice for me.
I’ll test this out for a few days and see how it performs.