I have the chance to set up my system again (better 8TB drives x 4) and add an extra two 8GB for RAM usage over the 2x4GB I have. I’ve noticed that my limitation is more CPU speed than RAM, which 8GB seems enough but I’ll get the 16GB if needed. Having a 453D, I cannot install Quts Hero and I’m told that it won’t be powerful enough anyway.
Ext4 > RAID-5?: Less overhead. Able to recover data from other volumes if 1+ drives fail and don’t need to chance rebuild on a new drive that will cost $$$$ and take a long time.
Thick v. Thin: I don’t need to have drive space flexibility for pools as it will be set to the max with snapshots set in advance that I should not need to adjust. (Willing to hear thoughts on this.)
My Goal: Store my personal digital photos and videos and use QuMagie as a Google Photos alternative. I do not need drive redundancy for uptime. I will keep a crc32 hash file for each photo and consider doing par32 archives/zip or rar archives of folders for backup purposes and data bad file restoration. Having a separate backup with the same crc32/zip/rar mirror allows me to address corrupt files if/when I find them.
One additional concern: I can see having about 100,000 photos and wonder whether performance will be a serious problem regardless of the RAM installed. I’m not sure how much of a difference it will make anyway since QuMagie will be the only app I will likely be running.
Actually - I think you helped answer this question earlier. Snapshots are supposed to address this issue by running them periodically. But originally I set up the NAS with RAID-5 since this was the ‘recommended’ set up when I began. Quite frankly, I’m not sure what benefit it provides other than, theoretically, being able to rebuild the drive (which will likley take a long time, perhaps a day) v. taking a periodic backup. The latter can work if you don’t add files often, as i will not. So now I’m rethinking using Ext 4 with snapshots v. RAID-5.
I just did a deeper dive into understanding what each does and now I get it. RAID-5 checks against the entire disk and doesn’t actually do file integrity checks against a checksum. RAID is the bottom layer - in a way it’s meant to create just one huge disk system that can then be rebuilt. If one disk fails it needs to be rebuilt for the system to work again. You can have layers that reside on top for a file system too, but the ability to read the ext4 is lost with the RAID as the bottom layer as it’s striped across various disks, etc.
Without RAID you can replace a drive by copy data from a backup onto that drive (if you have it identical) and just pop in the drive without a rebuild needed. That should be ideal as my data will be copied across 1-2 backups as I add additional photos to the main nas. All I actually need are separate disks that have file checks for data integrity for each file, not for the entire pool that is the sum of all the disks.
Filesystem scrubbing can check for file integrity. There isn’t any actually done with qts the way I hoped (with checksum) and I’ll stick with that to cross bridges when I get there. Will need to look into what scrubbing does on qts but I recalled it wasn’t available when I set up the first time.
So this will be a bit of a pain to reinstall but I don’t have much on there at all. I can actually install ext4 and not worry about all data being lost if I can’t rebuild. I can use snapshots and also take crc32 checksums as I do for file integrity checks on both the NAS and backup. (I try to keep 3 copies of the data.)
Thin is fine as this data set is not expanding beyond 4x8TB (and certainly not at current prices.) Thanks for the return to discussion.
RAID5 provides hardware redundancy for your disks. The whole point of RAID5 is spreading the risk of a single drive failing across multiple drives. So if a drive fails you can replace it with another one w/o losing your data. Without a RAID5 configuration if a drive fails, you basically lost your data.
There are also speed advantages to RAID arrays as well. And RAID gives you a much larger storage space than a single drive will give you. But RAID0 (basically just adding drives together) is very risky as yeah, you get a lot of storage space, but if one drive goes, you lose everything on all drives. What will take longer? Restoring a backup or rebuilding a raid while you can still access your data.
QTS does have a scrubbing option. So you can do that periodically.
How you want to configure things is up to you and your risk tolerance.
But don’t rely on RAID as a “backup.” It is not that.
Snapshots are great but they aren’t really a backup unless that are stored on some other drive. They will be useless if they are on your drive and the drive fails. Where they are useful is in individual file recovery. Say you accidentally delete or modify a file in a way you don’t want. Well, with snapshots you can go back in time and recover a previous version of that file.