when using a standard CPU like E5-2698 v4
That is not standard, that is outdated. That is a 2016 CPU. Given a 5 year replacement cycle - that should have been retired in 2021. Hence, you know, it is slow compared to a modern CPU. It is also a (for that time) high Core count cpu, which means bad per core performance. If you need high per core performance, you want speed optimized cores, not something with as many cores as possible. 2016 was before the core craze started by AMD and back then that was really a lot of cores.
Consequently, I am curious if it is feasible to request CHR to offload the taxing burden to a chip or specialized hardware part in the CPU or NIC.
Does anyone have any advice on how to do this?
Yeah, try it. In particular because hardware offload of that requires hardware support. Support which, funny enough, is NOT THERE. It is easy to talk of "specialized hardware in the CPU" when the CPU has no specialized hardware for that. It is even funnier to talk about that on a NIC - when access to the NIC is not through NIC specific drivers but via a hypervisor. Even IF a NIC has that, it is not exposed.
So, the question is moot as there is no magic "make this in hardware" switch when the hardware is not there.
If you need / want hardware offloading, look at the new high end mikrotik products - though I fear VPN is not something to be offloaded (except IPCSEC). And that on few products. The CHR is generally meant to allow higher core counts and - well - running it in the cloud. Hard to put hardware to i.e. Microsoft Azure - so a CHR is VERY wellcome.
So, that simply is not an option to start with given you talk about CHR. Getting a better CPU in is the only option. There are decent RYZEN based Servers (including server level motherboards) out now - you want to go as modern as possible with an as high core performance as possible. Makes me wonder about the 7800X3D.... see, gaming and single core performance are (still) pretty much the same. And as I said, you can get server grade motherboards for that.