Community discussions

MikroTik App
 
wpeople
Member
Member
Topic Author
Posts: 378
Joined: Sat May 26, 2007 6:36 pm

RouterOS as Xen guest with PCI passthru

Sun Dec 01, 2013 9:45 pm

Hello,
i'm interested if any of You using RouterOS as Xen DomU (guest).
My experience is max bandwidth test is around 100mbps with bridged network.
If I give the whole PCI card with PCI passthru to the DomU, it can reach almost the max speed of gigabit.

My problem is: i've installed Debian7 onto a new (HP DL380g8) server with Xen4.1
this has a known bug with PCI passthru (maybe with the scripts?) so i've upgraded Xen4.1 to Xen4.3 from Debian unstable.

Mikrotik see both ports of the NIC (the card is a HP NC360T) - in system/resource/pci
I have no idea why, but in interfaces, only the bridged NIC appears (the PCI passed not visible)
/system resource pci print detail
0 device="00:06.0" name="82571EB Gigabit Ethernet Controller (rev: 6)"
vendor="Intel Corporation" category="Ethernet controller" vendor-id="0x8086"
device-id="0x105e" irq=5 memory=0xF3080000-0xF309FFFF,0xF30A0000-0xF30BFFFF
io=0xC160-0xC17F

1 device="00:05.0" name="82571EB Gigabit Ethernet Controller (rev: 6)"
vendor="Intel Corporation" category="Ethernet controller" vendor-id="0x8086"
device-id="0x105e" irq=10 memory=0xF3020000-0xF303FFFF,0xF3040000-0xF305FFFF
io=0xC140-0xC15F

2 device="00:04.0" name="82540EM Gigabit Ethernet Controller (rev: 3)"
vendor="Intel Corporation" category="Ethernet controller" vendor-id="0x8086"
device-id="0x100e" irq=5 memory=0xF3000000-0xF301FFFF io=0xC100-0xC13F
(the last one is the bridged NIC)

i've tried to enable or disable the following options:
pci_power_mgmt=1
pci_msitranslate=0
xen_platform_pci=1

non of them helped


I have another server with Debian6 (Xen4.0) what works OK - however outdated.
Last edited by wpeople on Tue Dec 03, 2013 11:41 am, edited 1 time in total.
 
wpeople
Member
Member
Topic Author
Posts: 378
Joined: Sat May 26, 2007 6:36 pm

Re: RouterOS as Xen guest with PCI passthru

Tue Dec 03, 2013 11:39 am

system resource print detail output of the same card in different (native) installation:
1 device="02:00.1" name="82571EB Gigabit Ethernet Controller (rev: 6)" vendor="Intel Corporation" category="Ethernet controller" vendor-id="0x8086"
device-id="0x105e" irq=11 memory=0xFE880000-0xFE89FFFF,0xFE860000-0xFE87FFFF io=0xE400-0xE41F

2 device="02:00.0" name="82571EB Gigabit Ethernet Controller (rev: 6)" vendor="Intel Corporation" category="Ethernet controller" vendor-id="0x8086"
device-id="0x105e" irq=10 memory=0xFE8E0000-0xFE8FFFFF,0xFE8C0000-0xFE8DFFFF io=0xE800-0xE81F
 
wpeople
Member
Member
Topic Author
Posts: 378
Joined: Sat May 26, 2007 6:36 pm

Re: RouterOS as Xen guest with PCI passthru

Thu Dec 05, 2013 4:35 pm

I have a working PCI passthru under Xen with Debian6 (squeeze), Xen4.0, HP DL380G6 onboard Broadcom 2xDualport
since the mobo has two dualport NICs integrated, i can't passthru only one port, it's a must to use the whole second card.
(since DL380g8 has a quad port NIC integrated, i can use only the whole NIC for virtualisation purposes, that's why i added a second -dualport- server NIC)

the server is facing to a RB1100AH. If i make a btest from RB1100AH to the virtualised RouterOS, it can make
TCP: RB1100<-PC 240mbps
RB1100->PC 900mbps
if i raise the MTU to 9000bytes
240mbps increases to 290mbps
and 900mbps increases to 980mbps

UDP both: 900mbps/970mbps

it seems to me, there is a bottleneck (in interface queueing?) somewhere.
Because the NIC fully passed to Guest OS, i have no control over it at all in virtualisation enviroment (like disabling TCP offload, etc)

ps: the tests above made with ROS5.18. I've upgraded to 6.7 and i've lost an interface - what's still visible in the PCI list! thruput not raised, CPU usage not decreased.

edit: it seems, the RB1100AH cannot deal with btest-receiving more than 250mbps. if i replace the RB1100AH with a HP Microserver (with ROSx86) the NIC with PCI-passthru
can do 950mbps in any direction or 940/730 in both directions. (tcp)

Who is online

Users browsing this forum: No registered users and 8 guests