I have a Qnap TVS H1688X with a qnap Intel E810 100Gbe card and 10TB (WD Gold) x 12 in raid 5 Stroage pool 2 plus 2x1TB Samsung 990 Pro NVME in Raid 1 for a storage pool 1, this is connected to my qnap switch M7308R-4X via 100gbe DAC cable.
The Intel based windows 11 enterprise PC also has Intel E810 25Gbe card which has Dual fibre cable going to the switch with SMB Multichannel enabled.
When I transfer from the PC to the Qnap I only get on average between 1.1Gbps to 1.5Gbps transfering large multigig files. I thought I would be getting over 2Gbps.
FEC is disabled on the switch and nics
I tried the qnap with a 100gbe to 25Gbe adaptor and still the same.
What I wanted or know is this the best I can achieve? As on 10gbe I could get around 1.1Gbps on this qnap, not seeing any huge benefits for the cost occured.
Am I missing a setting or something?
All the devices have the latest firmware/software updates.
just corrected the thread, I thought I had cache drives but i set it up as Storage Pool for the NVME in Raid 1 for the System and not setup as as cache drives.
No, but not sure if I am missing a setting or configured it right.
On your 10Gbe are you hitting 1.1Gbps -1.2Gbps?
just noticed my Qnap is still on h5.2.9.3410 Build 20260214 (no updates on online firmware update) but there is a newer firmware i can see from the website which is the h5.3.3.3424 Build 20260305
The MTU I changed to 1500 after i was having issues on my PC’s AV (Eset) if i set it to 9000 MTU.
i got enterprise as some features are only enabled in this version or possibly the workstation version of Windows 11 (not Pro) eg SMB Multi Channel and additional options within the NIC interface
You really didn’t need to do that. h5.3 is really meant for those users who have another identical NAS operating as a fallback/redundant device. There are some other changes in 5.3 as well that you really may or may not want.
12 SATA drives won’t do 2200 MB/sec. When you have a 25G card on your computer (I am using the Sonnet Twin 25G, which is a Mellanox CX6 card with a thunderbolt connection to the Mac) - and connect this to a 24 bay TS-h3087XU-RP, or TS-h2477AXU-RP, or an all flash drive QNAP like a TS-h1290FX, or TS-h2490FU - then you will see 2200 MB/sec over 25G (as opposed to 1000 - 1100 MB/sec in a 10G system). While I have not personally tried this - if you were to get the new TVS-AIh1688ATX, and put four expensive U.2 NVMe drives in the 4 SSD slots, then you would probably see 2200 MB/sec using the U.2 drives. And if you just want to cry, but do a little experiment for yourself (since you spent the money for all of this stuff) - go purchase a QNAP QM2-4P-384 card, (it’s about $200) and put four M.2 NVMe drives on that card, and stick it in one of the PCIe slots of your TVS-h1688X. This will dramatically outperform your twelve 7200 RPM SATA drives.
You are not seeing any huge benefits from the cost occured - because your 12 drive QNAP cannot achieve the total aggregate bandwidth that you want for 25G (or 100G) speeds. This ain’t rocket science. Don’t believe me - try the QM2 card with the four M.2 drives in your TVS-h1688X, and you will see quickly.
Thanks Bob, I just wanted the maximum performance from the hardware I have.
According to the below I should be able to get more than 1.1Gbps and hence why is I see speeds varying, otherwise I would just see the top end speed only.
I did create another storage pool on my nvme to see the maximum and hit a short burst speed of 2.5Gbps.
On my Windows 11 Enterprise I wanted to use RDMA which I can see the option RoCEv2 on device manager of the Intel E810 nic but when I check via powershell it has it as false.
I have installed the DCB and SMB Direct feature on my windows 11 enterprise machine and the qnap switch has pfc enabled.
On the qnap I cannot see any options for enabling RDMA or SMB Direct? Not sure if this is only available on the flash based qnap models.
I just played with the NASCompares calculator. I don’t care what it shows - this is not accurate. I build a ton of systems - with 4 drives, 6 drives, 8 drives, 12 drives, 16 drives, 24 drives, and then the flash systems using 12 and 24 U.2 NVMe drives. The little calculator shows 800 MB/sec with only 4 drives. That is complete nonsense in the real world. I don’t care if it’s QNAP or Synology, or another manufacturer. Over a 10G network using a QNAP 10G card, and a Mac with a 10G NIC (typically a Marvell Aquantia AQC-107 or 113) - you CANNOT get that with only 4 drives. As I said in the last post - if you want to see 2200 MB/sec over your 25G network connection, get a QNAP QM2-4P-384 card, put four M.2 drives on there, and try that. This will give you what you want. - Bob Zelin
Thanks bob, I know you build a ton of systems and have followed you comments from the old qnap forum and reddit for years.
Can I ask which do you think is better the mellanox connectx 25gbps cards or the Intel E810 series for Windows based machines? It will be gone to know your opinion.
well - I have NO experience with the Intel E810 cards, and I have seen all the complaining on the Ubiquiti forums about this series of Intel cards when it came to FEC stuff - So I have always stuck with Mellanox cards.
The QNAP 25G cards are the QXG25G2SF-CX6 which is a Mellanox CX6 card, and the QNAP
QXG25G2SF-E810 which is the Intel E810 card. But just because you put in a CX6 card, does not mean that 12 SATA drives are going to give you 2200 MB/sec. I am strongly advising you to simply try the QNAP QM2-4P-384 card with four matching M.2 NVMe drives as a single storage pool - and now check your 25G speeds.
