Thanks for the extra effort to assist. I wasnāt quite sure how QuMagie prefers photo image folders because the dates of the photos can be unclear. For example, sometimes photos have EXIF data and at others they donāt. Same goes for correct create dates, which you have to hope is embedded into the file name if all the others are absent. I recall that QuMagie can also use date based hierarchies, e.g.
photos/2010/01/John Doe Birthday
photos/2010/01/Richard Roe Anniversary Party
photos/2010/02/Grateful Dead Concert
so that photos in these buckets are identified as being within January 2010 and February 2010 if all else fails. This may be too much of an effort as so many sets span several days and can go across months, etc. I may just stick with year and events and only split up those that cross the year mark.
I wrote QNAP about this and got a ridiculously great email back with setup recommendations and references to the website so it looks like this shouldnāt be too great of a problem, including confirmation of using the single pool and RAID 5 setup which seems most popular and logical if using RAID. The other aspect I didnāt consider was the storage of keywords and AI assisted search, which Iām told is stored in the NVME cache as that would make the most sense for quick retreival and search to identify images. Glad I maxed out at the time when all of this was affordable.
Backups wonāt help on bit rots. Without file checksums, you canāt tell which copy is valid unless one of them just doesnāt load. ZFS has data checksum and refuses to load the blocks when checksums mismatch, which at least reminds you to check your backup when bit rot happens. Many multmedia files still can be loaded after bit rot, unless the rot is on the metadata part.
I once got a brand new QNAP with memory issue. Using QTS, it just runs and crashes randomly. After switching to QuTS Hero, the checksum mimsmatch blocked me opening a file on my RAID 1, indicating that a same error occured on both disks and reminding me to check its memory.
QTS does have scrubbing but thatās on block device level rather than filesystem level (and the behavior when a mismatch is found is unknown). Also QTS doesnāt support scrubbing on RAID 1.
Iāve seen issues with QuMagie getting old pics assigned with new dates. Iām not sure what it was about these handful of photos (bad EXIF data so it used the date when the file was copied over to the NAS?). But everything else I have going back over 20 years is great.
If you back up a file that has bit rot you are correct. But the idea in my mind was that an archived file that was backed up before the bit rot on another piece of media is likely still good.
Thank you for your post and agreed. The assumption is that I will have assembled a set that is complete from all my files and validated.
Iām considering creating a crc32 or md5 hash for each file or kept as mentioned with a catalogued checksum that will identify when a file has been corrupted. Iāve done this for other media, such as a video library, and keep 2 additional backups in addition to the live set, which rarely changes and usually only has new files added.
Bit rot rarely happens but bad sectors do happen as do bad writes on occasion (such as when there are unexpected power outages) and that is where it gets problematic. For backups, many have suggested using WinRAR for datasets and having them as both with built in hashing to validate and also for rebuilding due to corruption. The alternative is to use ZIP with PAR files. While it is a laborious process, if Iāve got 500 folders of photos and videos from different years, it might be a feasible project to do over several months time.
RAID 1 is never a proper backup solution IMHO. The moment one copy fails so will the other. What you describe on QTS is another very good point and hence Iām left with the above types of validation and recovery approaches.