ZFS Speed problems ?

[Lewis]

Hi,

I need help to understand how does ZFS work on QNAP 'coz it seems i missed some crucial info (I apology if this is somewhat general knowledge that i haven’t know) so please bare with me.

Here is what i did with my QNAP TS-h2477AUX NAS.

At first i had only 6 * 18TB drives (Toshiba MG09) so i made pool/volume with those 6 drives as RAID 6 (4+2 for parity 'coz i need a lot off space so i can’t go with RAID 10, too much "lost space with already too expensive TB/$ in these days) and started copying files from my Synology sicne i have a lot of data (and lot of small files/images which normaly tahe time to transfer) and i started few days ago.

After few days I got another 3 drives (same 18TB Toshiba MG09) so i can cover up same 108TB volume like i have in my current Synology NAS.
Then i Expanded my ZFS pool with those 3 new HDDs added in one go (i took about 15h to calculate/expand but it worked overnight and it was ready tomorrow) and i “gained” roughly 40TB more of space which is good/expected :).

So now my pool/volume contains 7+2 (RAID 6) and all seems to be working fine until i try to test speed and get surprised by slowness.

I use Black Magic Design Disk speed tool to see how fast i can pull data from NAS since i EDIT 4K videos from NAS in Davinci Resolve and to my surprise speed of QNAP (i have both 10Gbit LAN connections plugged into my 10Gbit switch and SMB multichannel is Auto/ON) is roughly 290MB Read and 450MB Write ?

I did expect much higher speeds (since my Synology with 8 drives (4*18TB + 4*22TB in SHR-2 (Basically Synology Raid 6 with mixed drive sizes) gives me 850MB Read and 900MB Write over same PC/switch combo where QNAP is also connected to.

So i wonder how is QNAP that much slower with 8x times stronger CPU and more RAM + apparently much better File System (ZFS vs BTRFS) ?

What did i do wrong?
Maybe something is wrong in my setup but this is way way too slow speeds.

I’ve checked online and among all the “AI nonsense” what Google spits out nowadays something might even be true. It says that RAID 6 with odd number of drives (9) might be the problem here so i should either go to 8 drives or buy 10th drive to gain back speed?

Any truth in that ?

At some point it even says that i need to delete my volume and start from scratch with all disks inside due some “quirks” of how ZFS works but that has no sense to me 'coz then POOL expansion is basically useless if i can’t expand pool without rebuilding my Volume from scratch every time ? That surely has no sense - right ?

Please advise/help..

Thanks

[dolbyman]

adding disks to a zfs vdev is relatively new

did you do a speedtest before expanding ?

[Lewis]

Sadly i did only last week when i was testing with 5x*8TB drives and i had results around 450/500 in RAID 6 which i assumed is OK-ish/normal due being only 3+2 setup (2 for parity). I was happy with 3 drives hitting 500mb and expected more when i add more disks. That was before my 18TBs arrived which i use now.

volume 1 - 2xNVMe 2TB drives for System/Apps - RAID 1
volume 2 - 9x18TB MG09 Toshiba’s for data/video/photo - RAID 6
volume 3 - 2x8TB for Surveillance only - RAID 1

So sadly i have no test for volume 2 prior expansion (when it was with 6 disks) but these speeds are very low

Now it’s even worse hitting 90/180.
My pool still have 60+TB free space.

Here is screen grab

[Becker2020]

@Lewis

What did i do wrong?

Hello,

in a other Post you say “over-provisioning … not needed for HDDs.”
Have you change it ?
Storage pool over-provisioning | QuTS hero h6.0.x

And how is your ARC-RAM setting ?
Storage global settings | QuTS hero h6.0.x

[Lewis]

Hey @Becker2020 - that’s good ide i’ll see does bringing back “over-provisioning” help that. Although accordign to all info i’ve read that helps only/if space is filled up 90%+ whihch is not my case since i have more than 50% free space on my pool currently, but i’ll try.

As for ARC-RAM i havent thouched that, it’s set as is by default:

here is screen-grab of those

And BTW here is my Snyology speed result:

EDIT: I just tired Pool over-provisioning to 5% (more than 5TB of my pool space) and i see no difference in speed. It oscillates from 80-120MB/s Read and 150-170MB/s Write on multiple tests/tries.

[Becker2020]

Then your nas have another problem