Thanks for the extra effort to assist. I wasn’t quite sure how QuMagie prefers photo image folders because the dates of the photos can be unclear. For example, sometimes photos have EXIF data and at others they don’t. Same goes for correct create dates, which you have to hope is embedded into the file name if all the others are absent. I recall that QuMagie can also use date based hierarchies, e.g.
photos/2010/01/John Doe Birthday
photos/2010/01/Richard Roe Anniversary Party
photos/2010/02/Grateful Dead Concert
so that photos in these buckets are identified as being within January 2010 and February 2010 if all else fails. This may be too much of an effort as so many sets span several days and can go across months, etc. I may just stick with year and events and only split up those that cross the year mark.
I wrote QNAP about this and got a ridiculously great email back with setup recommendations and references to the website so it looks like this shouldn’t be too great of a problem, including confirmation of using the single pool and RAID 5 setup which seems most popular and logical if using RAID. The other aspect I didn’t consider was the storage of keywords and AI assisted search, which I’m told is stored in the NVME cache as that would make the most sense for quick retreival and search to identify images. Glad I maxed out at the time when all of this was affordable.
Thanks again and very much appreciated.
Backups won’t help on bit rots. Without file checksums, you can’t tell which copy is valid unless one of them just doesn’t load. ZFS has data checksum and refuses to load the blocks when checksums mismatch, which at least reminds you to check your backup when bit rot happens. Many multmedia files still can be loaded after bit rot, unless the rot is on the metadata part.
I once got a brand new QNAP with memory issue. Using QTS, it just runs and crashes randomly. After switching to QuTS Hero, the checksum mimsmatch blocked me opening a file on my RAID 1, indicating that a same error occured on both disks and reminding me to check its memory.
QTS does have scrubbing but that’s on block device level rather than filesystem level (and the behavior when a mismatch is found is unknown). Also QTS doesn’t support scrubbing on RAID 1.
I’ve seen issues with QuMagie getting old pics assigned with new dates. I’m not sure what it was about these handful of photos (bad EXIF data so it used the date when the file was copied over to the NAS?). But everything else I have going back over 20 years is great.
If you back up a file that has bit rot you are correct. But the idea in my mind was that an archived file that was backed up before the bit rot on another piece of media is likely still good.
But I get your point. It is a good one…
Thank you for your post and agreed. The assumption is that I will have assembled a set that is complete from all my files and validated.
I’m considering creating a crc32 or md5 hash for each file or kept as mentioned with a catalogued checksum that will identify when a file has been corrupted. I’ve done this for other media, such as a video library, and keep 2 additional backups in addition to the live set, which rarely changes and usually only has new files added.
Bit rot rarely happens but bad sectors do happen as do bad writes on occasion (such as when there are unexpected power outages) and that is where it gets problematic. For backups, many have suggested using WinRAR for datasets and having them as both with built in hashing to validate and also for rebuilding due to corruption. The alternative is to use ZIP with PAR files. While it is a laborious process, if I’ve got 500 folders of photos and videos from different years, it might be a feasible project to do over several months time.
RAID 1 is never a proper backup solution IMHO. The moment one copy fails so will the other. What you describe on QTS is another very good point and hence I’m left with the above types of validation and recovery approaches.
Thanks for this earlier post. It makes me think that toiling to separate media by year and month may not make much of a difference (and sometimes I may not even know based on not having sufficient exif data, only knowing the year. For photos that I do know the date, it appears they are burned into the file name anyway - for example the Google Pixel names photos PXL_20250702_ so that it is YYYYMMDD and that can be inferred by Qumagie, I hope.
I have been separating files by quarters (Q1, 2, 3, 4) initially to prevent having 20,000+ files in a folder but I’m finding that I have, at most, about 10-15,000 JPGs. (Another 15,000 would be the crc32 txt files which will be ignored or I could move them to a separate CRC folder.)
What I will do is follow the above convention and have Year/Source/ and, if I have the trips separated, will include a month and description in the subfolder. Year/SonyRX100/January Bahamas Vacation/ and general would be Year/SonyRX100/General/
