QuTS Hero is not available for download for the TS-877XU-RP on the official QNAP homepage. However, there are posts online, including on Reddit, which suggest that the TS-877-XU-RP (at least the rack version) is compatible with QuTS Hero. Is it possible to install QuTS Hero on a TS-877-XU-RP?
Looking at the download page for the TS-877XU-RP at Download Center | QNAP (US) does not show Hero as available for this model. It may just be too old.
The AMD Ryzen™ 5 2600 in your model should be more than sufficient to support hero.
I’m not sure why they don’t support it. Maybe @SteveKo can provide some insight. Or perhaps open a support ticket and ask.