Page 1 of 1

Xen won't go

Posted: Thu May 21, 2009 1:09 am
by tombrdfrd66
I'm no expert in Virtual Machining but have tried to understand and follow the Wiki on Mikrotik and Xen, setting up on a test 386 machine, as I can certainly comprehend its potential uses.

But I can't get it to go.

The machine=

/system resource> print
uptime: 43m
version: "3.23"
free-memory: 321336kB
total-memory: 353040kB
cpu: "Pentium"
cpu-count: 1
cpu-frequency: 498MHz
cpu-load: 1
free-hdd-space: 1850696kB
total-hdd-space: 4143716kB
write-sect-since-reboot: 1914
write-sect-total: 1914
architecture-name: "x86"
board-name: "x86"
[admin@Elaine] /system resource>


Xen =

[admin@Elaine] /xen print
Flags: X - disabled, C - configuration-changed
# NAME MEMORY WEIGHT STATE
0 Centos 256MiB 256 failed-to-setup-vm

[admin@Elaine] /xen> export
# may/21/2008 09:54:27 by RouterOS 3.23
# software id = xxxxxxxx
#
/xen
add comment="" console-telnet-port=0 cpu-count=1 disabled=no disk-images=\
hda:centos.img host-shutdown-action=shutdown initrd=centos.initrd.rgz \
kernel=vmlinuz-2.6.18-53.el5xen kernel-cmdline="\"ro root=/dev/hda1\"" \
memory=256MiB name=Centos weight=256
/xen global-settings
set memory-for-main=128MiB
[admin@Elaine] /xen> /file print
# NAME TYPE SIZE CREATION-TIME
0 CentOS-5.3-i38... .iso file 8701952 may/20/2008 18:47:31
1 vmlinuz-2.6.18... .el5xen file 2106541 may/20/2008 21:17:29
2 centos.initrd.rgz .rgz file 2244277 may/20/2008 21:17:35
3 centos.img .img file 2147483647 may/20/2008 21:34:37
[admin@Elaine] /file>


Any clues there?

Re: Xen won't go

Posted: Fri May 22, 2009 3:05 pm
by richinuk
Hi Tom,

Just looking at what you quoted:
/system resource> print
<...>
free-memory: 321336kB
total-memory: 353040kB
<...>
/xen global-settings
set memory-for-main=128MiB
You have the main memory at 128mb, but your /system resource print is showing 345mb. Have you rebooted the box since applying the "set memory-for-main" parameter? If not, you need to such that the main MT can allocate the 128mb leaving the rest for your VM - otherwise the VM can't allocate enough RAM to start. Also assuming this is the problem, you having 345mb (?) of RAM would not be sufficient for your 128mb system + 256mb VM.

Checking on mine, I have the 128mb system memory set and my /system resource print shows exactly 128mb assigned.

Rich

Re: Xen won't go

Posted: Sat May 23, 2009 3:10 am
by tombrdfrd66
Thanks Rich, but there's still something wrong. Even when I set main memory to 128MiB system resource still reports the full amount after a reboot - see below.

Xen is certainly running - it handles the boot process and reports that it starts RouterOS as dom0. It even creates the virtual interface to domU but can't start it. Something's wrong or missing but I've no clue what it is.


[admin@Elaine] /xen print
Flags: X - disabled, C - configuration-changed
# NAME MEMORY WEIGHT STATE
0 Centos 128MiB 128 failed-to-setup-vm
[admin@Elaine] > xen export
# may/23/2009 11:59:46 by RouterOS 3.23
# software id = xxxx-xxx
#
/xen
add comment="" console-telnet-port=0 cpu-count=1 disabled=no disk-images=hda:centos.img host-shutdown-action=shutdown \
initrd=centos.initrd.rgz kernel=vmlinuz-2.6.18-53.el5xen kernel-cmdline="ro root=/dev/hda1" memory=128MiB name=Centos \
weight=128
/xen global-settings
set memory-for-main=128MiB
/xen interface
add comment="" disabled=no dynamic-bridge=none dynamic-mac-address=02:4A:43:42:71:2F static-interface=vif1 type=static \
virtual-machine=Centos vm-mac-address=02:AB:45:15:30:82

[admin@Elaine] /system resource print
uptime: 1m40s
version: "3.23"
free-memory: 320540kB
total-memory: 353040kB
cpu: "Pentium"
cpu-count: 1
cpu-frequency: 498MHz
cpu-load: 3
free-hdd-space: 1849860kB
total-hdd-space: 4143716kB
write-sect-since-reboot: 4162
write-sect-total: 4162
architecture-name: "x86"
board-name: "x86"

Re: Xen won't go

Posted: Sat May 23, 2009 9:54 pm
by richinuk
Hmmm... this does seem a little odd. How much RAM does the machine have anyway? I'm a bit suspect that your "main" MT is still showing over 300mb of memory when you've set the limit at 128mb. When I first started playing with Xen I had a number of very similar issues - ie: VM fails to start, but log shows nothing helpful. The fix for me was to set the memory limit for main so that there's enough free memory to allocate to my Xen VM - but you've done that anyway.

I know this sounds a bit odd, but try setting the memory-for-main to another value, then back to 128mb and then reboot. I've had a couple of bizzar instances now where a setting hasn't taken effect, and then if I remove and re-apply the same config, it magically works!

FWIW, here's my config incase it helps. I personally can't see anything majorly different that is causing yours to fail - maybe the console port? Anyway, my MT PC has 512mb RAM total, my 1 VM has 256mb allocated, two virtual HDD images, and is based on the centos image from the MT Wiki pages.
[admin@HostServer1] > /xen pr d
Flags: X - disabled, C - configuration-changed
 0   name="Server" disk-images=hda:centos30gb-server.img,hdb:support-files.img
     initrd="centos.initrd.rgz" kernel="vmlinuz-2.6.18-53.el5xen"
     kernel-cmdline="" cpu-count=1 memory=256MiB weight=256
     console-telnet-port=64000 host-shutdown-action=shutdown state=running


[admin@HostServer1] > /xen exp
# may/23/2009 18:40:49 by RouterOS 3.22
# software id = 
#
/xen
add comment="" console-telnet-port=64000 cpu-count=1 disabled=no disk-images=\
    hda:centos30gb-server.img,hdb:support-files.img host-shutdown-action=\
    shutdown initrd=centos.initrd.rgz kernel=vmlinuz-2.6.18-53.el5xen \
    kernel-cmdline="" memory=256MiB name=Server weight=256
/xen global-settings
set memory-for-main=128MiB
/xen interface
add comment="" disabled=no dynamic-mac-address=02:CA:09:2E:40:92 \
    static-interface=vif1 type=static virtual-machine=Server vm-mac-address=\
    02:69:2F:C8:90:12


[admin@HostServer1] > /system resource pr
                   uptime: 1w2d4h17m2s
                  version: "3.22"
              free-memory: 105764kB
             total-memory: 131072kB
                      cpu: "Intel(R)"
                cpu-count: 2
            cpu-frequency: 2394MHz
                 cpu-load: 1
           free-hdd-space: 24351968kB
          total-hdd-space: 76924356kB
  write-sect-since-reboot: 21685962
         write-sect-total: 21685962
        architecture-name: "x86"
               board-name: "x86"

Re: Xen won't go

Posted: Sun May 24, 2009 12:42 am
by tombrdfrd66
Thanks Rich but still no go - I can't even get a ROS image to boot under Xen.

[admin@Elaine] > xen export
# may/24/2009 09:32:42 by RouterOS 3.23
# software id =
#
/xen
add comment="" console-telnet-port=0 cpu-count=1 disabled=yes disk-images=\
hda:centos.img host-shutdown-action=shutdown initrd=centos.initrd.rgz \
kernel=vmlinuz-2.6.18-53.el5xen kernel-cmdline="ro root=/dev/hda1" \
memory=128MiB name=Centos weight=128
add comment="" console-telnet-port=64000 cpu-count=1 disabled=no disk-images=\
hda:ros1.img host-shutdown-action=shutdown initrd="" kernel="" \
kernel-cmdline="" memory=128MiB name=ros1 weight=256
/xen global-settings
set memory-for-main=128MiB
/xen interface
add comment="" disabled=no dynamic-bridge=none dynamic-mac-address=\
02:4A:43:42:71:2F static-interface=vif1 type=static virtual-machine=\
Centos vm-mac-address=02:AB:45:15:30:82
[admin@Elaine] > system reso print
uptime: 4m47s
version: "3.23"
free-memory: 320280kB
total-memory: 353040kB
cpu: "Pentium"
cpu-count: 1
cpu-frequency: 498MHz
cpu-load: 1
free-hdd-space: 1784000kB
total-hdd-space: 4143716kB
write-sect-since-reboot: 7930
write-sect-total: 7930
architecture-name: "x86"
board-name: "x86"
[admin@Elaine] > xen print
Flags: X - disabled, C - configuration-changed
# NAME MEMORY WEIGHT STATE
0 X Centos 128MiB 128 disabled
1 ros1 128MiB 256 failed-to-setup-vm
[admin@Elaine] >



How do you get Xen to log itself? I've tried Xen> debug but if it does anything I've no idea where to find it.

I guess that, like many of the 'innovative' Mikrotik developments. I'll have to put it on the back-burner for a couple of years until they get around to writing a manual entry for it.

Re: Xen won't go

Posted: Mon May 25, 2009 12:50 pm
by hush
IMO, you did everything right, tombrdfrd66, but there's a problem with memory-for-main xen parameter in RouterOS versions 3.23 and 3.24 - seems like it's simply ignored.

I was using centos xen image with 3.22 and noticed it didn't work any more after upgrade to 3.23 (failed-to-setup-vm status). It was reporting full physical memory size for main and ignored any setting I entered for memory limit in xen parameters - and the same thing happens with 3.24, too. After downgrade back to 3.22, everything is back to normal.

So it's either waiting for a fix from MT or back to good ole 3.22... that is, if you need xen.

Re: Xen won't go

Posted: Mon May 25, 2009 3:35 pm
by richinuk
Thanks for posting that hush. I was about to upgrade my 3.22 Xen machines :) I think I'll hold off for now.

Whilst Xen is very useful (particularly if you're trying to save on datacentre hosting fees), it is still a bit buggy, and I believe MT are openly stating that Xen is in an "alpha" status right now.

For example, I'm currently facing an issue where all my Xen enabled machines are kernel panic'ing every 9 or so days requiring a power cycle to recover. Removing the Xen package has resolved the panic's. I have an open case with support, but just be warned that it is not really production stability right now.

Rich

Re: Xen won't go

Posted: Wed May 27, 2009 12:42 pm
by tombrdfrd66
And Rich, thanks for posting that fix to the virtual ethernet problem:

http://forum.mikrotik.com/viewtopic.php ... 95&start=0

I dropped back to v3.16 and had no problem getting CentOS to work under Xen, but I banged my head for two hours trying to get the virtual ethernet link up before finding your fix and, bingo, all up and running.

I suppose if it was easy anyone could do it, so we'd all be out of a job!!!

Re: Xen won't go

Posted: Wed May 27, 2009 12:46 pm
by normis
Now that metarouter also supports other OS images, you should try it with CentOS

Re: Xen won't go

Posted: Wed May 27, 2009 7:07 pm
by richinuk
Will MetaRouter run on x86 hardware, or just routerboard? I'm using x86 based servers in a co-host facility.

Is there any feature difference with Xen and MetaRouter?

Re: Xen won't go

Posted: Wed May 27, 2009 7:13 pm
by richinuk
Glad it worked for you Tom.

Actually, do you mind emailing me on rich (at) goode.me.uk. I wanted to discuss something offline with you (and PM's are disabled on the forum).

Cheers,
Rich

Re: Xen won't go

Posted: Fri Jun 05, 2009 3:55 am
by tombrdfrd66
Yay, guys. I got it going!

For other dumbos like I, the Xenised CentOS 5 on offer from MT via the Wiki installs with an IPtables firewall in place which blocks practically everything, which is why it took me many happy hours of headbanging to discover why I couldn't connect to Webmin having installed it.

Re: Xen won't go

Posted: Tue Jun 30, 2009 12:05 pm
by tombrdfrd66
there's a problem with memory-for-main xen parameter in RouterOS versions 3.23 and 3.24 - seems like it's simply ignored.
Doesn't seem to be fixed in v3.25 either.

Re: Xen won't go

Posted: Fri Jul 03, 2009 8:17 am
by dssmiktik
there's a problem with memory-for-main xen parameter in RouterOS versions 3.23 and 3.24 - seems like it's simply ignored.
Doesn't seem to be fixed in v3.25 either.
No go for me either CentOS image from mikrotik wiki; x86 Level 4 (tested 3.24 and 3.25) with Xen.

Re: Xen won't go

Posted: Wed Jul 08, 2009 5:04 pm
by hush
Well, seems like 3.25 was a dud... let's just hope the "memory-for-main" problem gets addressed in 3.26.

While at it, it would be nice to have some faster way of creating empty disk images for xen. As it is, it took me almost an hour to create a measly 5GB image using "Make RouterOS image" option, just to delete its contents right away under Xen/CentOS so I can use it as a SMB share...

Re: Xen won't go

Posted: Wed Jul 08, 2009 7:59 pm
by dssmiktik
Well, seems like 3.25 was a dud... let's just hope the "memory-for-main" problem gets addressed in 3.26.

While at it, it would be nice to have some faster way of creating empty disk images for xen. As it is, it took me almost an hour to create a measly 5GB image using "Make RouterOS image" option, just to delete its contents right away under Xen/CentOS so I can use it as a SMB share...
Thoughts exactly here too.

Again, I'm sorry but Mikrotik why do these issues break when they used to work? If Xen ran but crash or something I'd understand, it's experimental, but something stupid like main-memory not even being acknowledged by RouterOS seems more like a RouterOS issue, not Xen.

If there's anything I'd like more to see from RouterOS it's not so much features as just a damn stable and high level of Quality Assurance put into each and every RouterOS version (starting with 3.26 :D )

Re: Xen won't go

Posted: Tue Jul 14, 2009 6:55 pm
by JJCinAZ
Seems broken in 3.26 also. It will not obey the main memory parameter, thus leaving no memory for VM's.

Re: Xen won't go

Posted: Wed Jul 15, 2009 5:36 am
by dssmiktik
Seems broken in 3.26 also. It will not obey the main memory parameter, thus leaving no memory for VM's.
Installing 3.26 on bare bones to test Xen.

Does Xen require hardware virtualization in BIOS? My original system did not have hardware virtualization in BIOS.

Re: Xen won't go

Posted: Thu Jul 30, 2009 8:20 pm
by JJCinAZ
Does Xen require hardware virtualization in BIOS? My original system did not have hardware virtualization in BIOS.
Not that I know of. Did you get it working at all?

Re: Xen won't go

Posted: Thu Jul 30, 2009 9:02 pm
by dssmiktik
No go on 3.26 or 3.27. Tried KVM on 3.27 and couldn't get it working either for what it's worth.

Re: Xen won't go

Posted: Sat Aug 01, 2009 2:29 pm
by tombrdfrd66
I've given up on Mikrotik's attempts at Xen - RouterOS seems quite happy as a guest OS in a VirtualBox on Ubuntu Server 9.04 which is good enough for me. I agree with mssmiktik - even if MT can get something working in one release there seems a good chance they'll break it in a later one.

Re: Xen won't go

Posted: Sat Aug 01, 2009 9:37 pm
by dssmiktik
tombrdfrd66,

How is Mikrotik running in a VM? What do you primarily use it for? I'm thinking maybe it would be great in a VM as web-proxy, hotspot, and maybe VPN links, but not so well as a core router.

Thanks

Re: Xen won't go

Posted: Sun Aug 02, 2009 2:03 am
by tombrdfrd66
I find it works very well in a VirtualBox. In a VMWare virtual machine I found it froze up after about 36-hours for a reason I never determined. In both cases it is/was with Ubuntu Server 9.04 as host OS.

I use it at the Internet gateway of a small network. RouterOS terminates a dozen internal l2tp tunnels and handles all the firewalling and routing, and can talk directly to FreeRADIUS +MySQL for AAA on the same box. I can also have Apache, DNS, mail, web-proxy and etc. servers running on the Ubuntu and even offer network storage to users on spare HDD capacity - all on the one machine.

Re: Xen won't go

Posted: Thu Aug 06, 2009 1:35 pm
by wdca
MT 3.27 level 4 x86
MT is on DOM flash 245MB and the centos is on CF8GB
 0   name="centos" disk-images=hdd:secondary-slave/centos.img 
     initrd="secondary-slave/centos.initrd.rgz" 
     kernel="secondary-slave/vmlinuz-2.6.18-53.el5xen" 
     kernel-cmdline="root=/dev/hdd1" cpu-count=1 memory=200MiB weight=199 
     console-telnet-port=33433 host-shutdown-action=shutdown 
     state=failed-to-setup-vm
The same problem as above ?
How to fix it? Is the memory problem still present in 3.27?

Re: Xen won't go

Posted: Sun Aug 23, 2009 10:10 pm
by dorijan
I tested on 3.28 and still there is a problem...
/xen
add comment="" console-telnet-port=6000 cpu-count=1 disabled=no disk-images=\
    hda:centos.img host-shutdown-action=shutdown initrd=centos.initrd.rgz \
    kernel=vmlinuz-2.6.18-53.el5xen kernel-cmdline="ro root=/dev/hda1" \
    memory=128MiB name=cent weight=256
/xen global-settings
set memory-for-main=150MiB
/xen interface
add comment="" disabled=no dynamic-bridge=bridge1 dynamic-mac-address=\
    02:3D:FD:45:9C:38 type=dynamic virtual-machine=cent vm-mac-address=\
    02:2A:69:B2:E3:4C
0 cent 128MiB 256 failed-to-setup-vm

Re: Xen won't go

Posted: Mon Aug 24, 2009 12:06 pm
by wdca
Please mikrotik fix XEN!!!!

Re: Xen won't go

Posted: Mon Aug 24, 2009 10:48 pm
by dssmiktik
I have a problem with 3.28 x86 Xen package also. Same problem as above.

Re: Xen won't go

Posted: Wed Sep 09, 2009 11:08 am
by thavinci
Same problem here....

Re: Xen won't go

Posted: Mon Sep 21, 2009 11:12 pm
by dorijan
I tested with version 3.30 and still not working :((((
mt team pleeease fix this

Re: Xen won't go

Posted: Sat Oct 10, 2009 2:35 am
by kez
Please, Mikrotik Team. Fixes the Xen package.
Like said, the parameter memory-for-main doesn't works in > 3.22 versions.
In "real" Linux, I found a solution for this problem:
http://systembash.com/content/memory-al ... erver-fix/

Please, we need an official word :(