Community discussions

 
User avatar
jaxx
just joined
Topic Author
Posts: 11
Joined: Thu Jun 25, 2009 5:16 pm
Location: Paris, France
Contact:

ROS on KVM on Hosted server issue (Dedibox / Online.net)

Fri Mar 02, 2012 4:11 pm

Hello all,

I'm facing an interesting issue having installed ROS on a VM on a hosted server in France.
The supplier, Online.net (previously named Dedibox) supplies great servers at bargain prices (I'm renting a quad-core xeon with plenty of ram, unmetered gigabit, and hw raid for 50€/month, no rush, it's limited to french customers)

They supply up to 10 extra IPs per phyisical server (not on the same range), that you can push around your different servers within the same account and datacenter, either as a virtual nic (ethX:Y), or better, for a VM, for which they give you a MAC address associated to the IP and server (to prevent spoofing for instance).

Using classic Linux based VMs, it's pretty straightforward despite a slight difference: On the VM, You must use the host servers Gateway as a gateway, your given IP is to be defined as a /32:

For example, on one of my machines:
Host has 88.190.36.xxx/24, GW is 88.190.36.1

On the VM:
A Debian-like /etc/network/interfaces would look like this:
# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface (private lan within kvm)
auto eth0
iface eth0 inet static
        address 192.168.122.35
        netmask 255.255.255.0

auto eth1
iface eth1 inet static
        # notice it's a /32
        address 88.190.210.yyy
        netmask 255.255.255.255
        up route add -host 88.190.36.1 dev eth1
        up route add default gw 88.190.36.1 dev eth1
        # dns-* options are implemented by the resolvconf package, if installed
        dns-nameservers 8.8.8.8
        dns-search zzz.jaxx.org
(pardon the dns setting, it's a lame test :-) )
This works perfectly well, pingable from inside and outside the datacenters networks.
Though I admit using an IP that's not part of the interfaces range is mind twisting, it works on every OS tried until today

I believe transposing this conf to ROS would end up a bit this way:
/ip address
add address=192.168.122.2/24 disabled=no interface=ether1 network=192.168.122.0
add address=88.190.210.yyy/32 disabled=no interface=ether2 network=88.190.210.yyy
/ip route
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=88.190.36.1 scope=30 target-scope=10
add disabled=no distance=1 dst-address=88.190.36.1/32 gateway=ether2 scope=30 target-scope=10
Problem is:
Whatever I try as gateway value for the default route [ 88.190.36.1 | ether2 | 88.190.36.1%ether2 ]. It doesn't work.
It remains unreachable (or reachable with "ether2" but nothing goes through)
[jaxx@MikroTik] > /ip route print detail
Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit
 0   S  dst-address=0.0.0.0/0 gateway=88.190.36.1%ether2 gateway-status=88.190.36.1 unreachable distance=1 scope=30 target-scope=10
 1 A S  dst-address=88.190.36.1/32 gateway=ether2 gateway-status=ether2 reachable distance=1 scope=30 target-scope=10
 2 ADC  dst-address=88.190.210.yyy/32 pref-src=88.190.210.yyy gateway=ether2 gateway-status=ether2 reachable distance=0 scope=10
 3 ADC  dst-address=192.168.122.0/24 pref-src=192.168.122.2 gateway=ether1 gateway-status=ether1 reachable distance=0 scope=10
Though, a ping to 88.190.36.1 would always work (and I tried ping and arp values for the check method on the default route with no succes, not even a proxy-arp on ether2)

The only case I got something to work was, if I enlarged the adress from a /32 to a /11, the IP range of the provider (which could though lead to noise traffic), I could then reach and be reached from outside, but curiously not from other servers in the range. But again, /11 "is bad!" anyways.

And, I can sniff some traffic, including incoming ICMP requests when I ping the ROS VM, but the VM won't answer a single thing.

Any issues with /32 IPs on interfaces and sticking routes to it ?

We're a a handful of people already who'd like to get this working, the last resort would be having 1:1 NATs on the host (which works), but losing some functionality, elegance, and might even pick a few issues at the same time. But again, It should work, there's something we don't get (and there are better network engineers than me who've tried)

Mikrotik team : I wouldn't mind lending an access to a linux VM (which shares a LAN access to the ROS VM) to give it look.

Thanks in advance for any tips :-)

FYI: Debian Host, installed Archipel Orchestrator (VMs on qemu-kvm, RouterOS with virtio interfaces, LAN works, and sniffing sees traffic, so I doubt it's the VM system anyways)
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6283
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: ROS on KVM on Hosted server issue (Dedibox / Online.net)

Mon Mar 05, 2012 1:00 pm

if actual network is /11 you have to use that for network to work.
 
User avatar
jaxx
just joined
Topic Author
Posts: 11
Joined: Thu Jun 25, 2009 5:16 pm
Location: Paris, France
Contact:

Re: ROS on KVM on Hosted server issue (Dedibox / Online.net)

Mon Mar 05, 2012 1:34 pm

if actual network is /11 you have to use that for network to work.
Hi janisk

It's actually not supposed to be, it was the only trick to get it to work (and still, only from/to outside of the network, and the actual datacenter is on a /16), since bringing the iface up will create a dynamic route for the /11, il believes it can communicate directly to the whole range, when actually, the only device it can communicate through and receive from is the host servers gateway .36.1... preventing it from being reachable from inside the range.

The thing I don't get is that using a /32 on the ip, adding a static route through ether2 to .36.1 gets that destination pingable.
But putting .36.1 as the gateway for 0.0.0.0/0, status remains unreachable...
Any way to force it ?

It perfectly works on any family of Linux (i'm not a BSD freak, never tried) and Windows.
And it's pretty straightforward to be honest (ok, mind twisting at some extent)

Thanks for any trick you might imagine :-)

JB./.
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6283
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: ROS on KVM on Hosted server issue (Dedibox / Online.net)

Mon Mar 05, 2012 4:23 pm

for /32 addresses you have to set precisely other end IP address as a network, and subnets in that case does not really matter anymore. here is example:

Router1:
/ip address add address=10.0.0.1/32 network=10.99.88.77 interface=ether1
Rotuer2:
/ip address add address=10.99.88.77/32 network=10.0.0.1 interface=ether1
now, you have p2p addressing, check if other end supprots /32 addressing or else you will have to go with /30 where you waste a lot of addresses (if that is of a concern)
 
User avatar
jaxx
just joined
Topic Author
Posts: 11
Joined: Thu Jun 25, 2009 5:16 pm
Location: Paris, France
Contact:

Re: ROS on KVM on Hosted server issue (Dedibox / Online.net)

Mon Mar 05, 2012 4:35 pm

for /32 addresses you have to set precisely other end IP address as a network

w00t !

Now that works, seems I can reach everywhere using the /32, the gateway for network (and not forgeting the default route)
/ip address add address=88.190.210.xxx/32 disabled=no interface=ether2 network=88.190.36.1
/ip route add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=88.190.36.1 scope=10 target-scope=10
Thank you janisk !
 
kobuki
Member Candidate
Member Candidate
Posts: 148
Joined: Sat Apr 02, 2011 5:59 pm

Re: ROS on KVM on Hosted server issue (Dedibox / Online.net)

Fri Mar 23, 2012 4:31 pm

I'm sorry if it looks as if I wanted to steal this thread, in that case I'm going to open a new one, but my question seems relevant.

I'm also thinking of running a MikroTik ROS as a KVM appliance as a cheap and handy alternative to a separate racked product. It's in fact installed and running fine in a KVM VPS, using IDE as HDD and Virtio for the emulated network adapters. Seems fine, did a few tests, but overall I'm wondering if it's worth the efforts. What are your findings about this kind of setup? Is it performant enough? Is it stable, does it have a habit of showing a weakness that would prevent it from running this way? Security considerations? Primary use would be for basic routing, < 100 NAT rules, some mangling and around a dozen queues on a similar amount of VLANs. Expected BW is in the 150-250 Mbps range. Also to the MikroTik devs: what's your stance in this scenario for ROS?
 
User avatar
janisk
MikroTik Support
MikroTik Support
Posts: 6283
Joined: Tue Feb 14, 2006 9:46 am
Location: Riga, Latvia

Re: ROS on KVM on Hosted server issue (Dedibox / Online.net)

Mon Mar 26, 2012 9:05 am

the problem about your question is - it varies from site to site installation and load differs for different traffic even if amounts are the same. So best course of action - you can do 24hour uptime test to see how much it can pass. So set up the guest and run some bandwidth tests through it. or iperf, or any other simulated traffic)
 
kobuki
Member Candidate
Member Candidate
Posts: 148
Joined: Sat Apr 02, 2011 5:59 pm

Re: ROS on KVM on Hosted server issue (Dedibox / Online.net)

Mon Mar 26, 2012 4:33 pm

Yeah, thanks, that was my plan anyway. Use a ROS installation as gateway/shaper for some of the VPSes and do some testing. If it goes well, I can consider switching the entire HW node. I'll report some of my findings if anyone is interested. Unfortunately not much info is available in this subject (although there are already a few VPS hosting sites offering preinstalled RouterOS as well).

Who is online

Users browsing this forum: No registered users and 2 guests