I have copied 3x the same 100GB file (just renamed it file1, file2, file3) to a shared folder on ZFS with Inline Compression = turned ON and Deduplication= turned OFF.
=> Yet, the shared folder only shows ≈ 100GB occupied despite that I have actually copied 3 x 100GB. How is this possible when Deduplication is turned OFF?
Verification Test
In a second test, I have copied 3x the same 100GB file (just renamed it file1, file2, file3) to another shared folder on ZFS, this time both Inline Compression = turned OFF and Deduplication= turned OFF as well.
=> Now, indeed the shared folder shows ≈ 300GB occupied.
The apparent storage savings you are seeing are entirely due to ZFS inline compression, which reduces the actual disk space usage for each file copy independently.
Since all 3 files are in fact the same you get good compression savings on two of them.
now if you try 3 different 100gb files that have different data content, you’ll see a different file compression ratio for sure.
Hi Simon, thanks for replying so quickly. Today I have further investigated and verified your point pertaining to ZFS LZ4 inline compression.
The 100GB file is a disk-image (.dmg image) which I created solely for testing purpose, so the disk-image actually contains no data, thus 0x00000000 (zeros) ad nauseam.
So, for testing purpose I zipped this .dmg image directly on macOS and indeed, the file was shrinking from 100GB to 100MB ! By the way the zipping took almost 5 minutes on a MacStudio with M2 MAX processor, 32GB of high performance shared memory and Apple soldered SSD.
Now, with LZ4 inline compression from ZFS, obviously LZ4 is able to reach the same or even better compression and on top at a much, much faster pace. In the case of the zipped .dmg file which was down to 100MB from 100GB, it has further compressed it from 100MB to less than 2MB !
So that’s why I could see next to no storage space consumption/change in the QuTS hero Storage & Snapshots Control Panel when I copied the same 100GB file three times or four times to the QNAP shared folder with ZFS inline compression enabled. Because LZ4 each time compressed the file down from 100GG to less than a mere 2MB.
Hence, your absolutely right: The reason why I did not see any data increase on the shared folder despite copying hundreds of gigabytes to the share, solely is because LZ4 was able to compress this data at a level of 99.9%
In contrast, when copying the same 100GB file three or four times to the share with inline compression disabled, each time indeed an incremental 100GB of the shared folder/storage capacity were consumend.