CHR + KVM problem - low upload.

RouterOS Virtualization
1

Hello all. I have CHR (6.45.9 lts) installed on qemu/KVM. I have 100/100 mbits connection, but real upload is about only 5-7 mbits. I have two nics - one connected to isp:

and second nic connected to openvswitch bridge. I have tested it with bandwith test. Local interface is ok - 150/150mbps. External - only 100/7mbps. With direct connected notebook to isp all is ok - 100/100mbps.
When i do system/reboot - and test bandwidth - couple of minutes all is ok (100/100), but after it upload drops to 7mbits. CPU load on CHR and hypervisor is ok during all tests.
Also i tried different NIC types (virtio, rtl8139) - no luck.
Any ideas an suggestions how to fix this problem? Thank you in advance.

2
  1. What is the CPU usage?
  2. How many cores do you have assigned to the CHR?
  3. Is Fast Track Enabled?
  4. Have you disabled HyperThreading in the Bios and then set for max performance?
    if you are using hyperthreading - set the hypervisor - (if you can) to use delayed_ack
  5. Have you done a UDP Test to 127.0.0.1? (i know that sounds odd) but doing this you should be able to produce a good amount of traffic
  6. Drive wise - is this using local or Network Storage?
  7. does the hypervisor have additional vm’s running - might suggest keeping at least 1 cpu and some ram available for the hypervisor itself (dont know your settings so just a suggestion)
  8. Depending upon the hypervisor - set CPU Affinity (basically you set which core gets reserved to which physical cpu) this can do wonders

We can explore more - but these are the basics I start with

3

Thnak you very much for your reply.

  1. CPU usage on host machine during test max 20%. CPU usage on CHR - 1st core about 50%, second core 20%
  2. 2 cores (Intel(R) Xeon(R) CPU E3-1230 v6 @ 3.50GHz)
  3. Fast Track not supported on CHR
  4. Hyperthreading enabled.
  5. not tried yet
  6. local storage - ssd
  7. all other VMs shut down during tests
    8 Did not do that - but tested CHR with 4 assigned to vm cores and right after reboot i see 5-6mbps. With 2 cores upload was normal at least 5 mins. Strange results for me.
4

Enable multi queue on the virtio-net card
And make that equal to the number of cores u have in the VM

5

I do not use virtio. Because when i use it there is a problem in status - autonegotitation failed.

BTW - today i rebuilt system including qemu with python 3.7 default target instead of python 3.6 and upload now is 100mbps.

P.S. - not fiexed yet. Upload again dropped to 6-7mbps

6

What make are your physical NICs?

There are severe issues with some Intel NICs with some Linux Kernels. Specifically Intel i218/i219 using the e1000 driver in kernels from 4.15

https://github.com/torvalds/linux/commit/b10effb92e272051dd1ec0d7be56bf9ca85ab927

you can turn off TCP segmentation offloading via the following command on the hypervisor.

ethtool -K <interface> tso off gso off
7

Very strange behavior. Will try switch later from gentoo to centos 8.

Now all is ok 100/100.