Xen won't go

I’m no expert in Virtual Machining but have tried to understand and follow the Wiki on Mikrotik and Xen, setting up on a test 386 machine, as I can certainly comprehend its potential uses.

But I can’t get it to go.

The machine=

/system resource> print
uptime: 43m
version: “3.23”
free-memory: 321336kB
total-memory: 353040kB
cpu: “Pentium”
cpu-count: 1
cpu-frequency: 498MHz
cpu-load: 1
free-hdd-space: 1850696kB
total-hdd-space: 4143716kB
write-sect-since-reboot: 1914
write-sect-total: 1914
architecture-name: “x86”
board-name: “x86”
[admin@Elaine] /system resource>

Xen =

[admin@Elaine] /xen print
Flags: X - disabled, C - configuration-changed

NAME MEMORY WEIGHT STATE

0 Centos 256MiB 256 failed-to-setup-vm

[admin@Elaine] /xen> export

may/21/2008 09:54:27 by RouterOS 3.23

software id = xxxxxxxx

/xen
add comment=“” console-telnet-port=0 cpu-count=1 disabled=no disk-images=
hda:centos.img host-shutdown-action=shutdown initrd=centos.initrd.rgz
kernel=vmlinuz-2.6.18-53.el5xen kernel-cmdline=“"ro root=/dev/hda1"”
memory=256MiB name=Centos weight=256
/xen global-settings
set memory-for-main=128MiB
[admin@Elaine] /xen> /file print

NAME TYPE SIZE CREATION-TIME

0 CentOS-5.3-i38… .iso file 8701952 may/20/2008 18:47:31
1 vmlinuz-2.6.18… .el5xen file 2106541 may/20/2008 21:17:29
2 centos.initrd.rgz .rgz file 2244277 may/20/2008 21:17:35
3 centos.img .img file 2147483647 may/20/2008 21:34:37
[admin@Elaine] /file>

Any clues there?

Hi Tom,

Just looking at what you quoted:

/system resource> print
<…>
free-memory: 321336kB
total-memory: 353040kB
<…>



/xen global-settings
set memory-for-main=128MiB

You have the main memory at 128mb, but your /system resource print is showing 345mb. Have you rebooted the box since applying the “set memory-for-main” parameter? If not, you need to such that the main MT can allocate the 128mb leaving the rest for your VM - otherwise the VM can’t allocate enough RAM to start. Also assuming this is the problem, you having 345mb (?) of RAM would not be sufficient for your 128mb system + 256mb VM.

Checking on mine, I have the 128mb system memory set and my /system resource print shows exactly 128mb assigned.

Rich

Thanks Rich, but there’s still something wrong. Even when I set main memory to 128MiB system resource still reports the full amount after a reboot - see below.

Xen is certainly running - it handles the boot process and reports that it starts RouterOS as dom0. It even creates the virtual interface to domU but can’t start it. Something’s wrong or missing but I’ve no clue what it is.
[admin@Elaine] /xen print
Flags: X - disabled, C - configuration-changed

NAME MEMORY WEIGHT STATE

0 Centos 128MiB 128 failed-to-setup-vm
[admin@Elaine] > xen export

may/23/2009 11:59:46 by RouterOS 3.23

software id = xxxx-xxx

/xen
add comment=“” console-telnet-port=0 cpu-count=1 disabled=no disk-images=hda:centos.img host-shutdown-action=shutdown
initrd=centos.initrd.rgz kernel=vmlinuz-2.6.18-53.el5xen kernel-cmdline=“ro root=/dev/hda1” memory=128MiB name=Centos
weight=128
/xen global-settings
set memory-for-main=128MiB
/xen interface
add comment=“” disabled=no dynamic-bridge=none dynamic-mac-address=02:4A:43:42:71:2F static-interface=vif1 type=static
virtual-machine=Centos vm-mac-address=02:AB:45:15:30:82

[admin@Elaine] /system resource print
uptime: 1m40s
version: “3.23”
free-memory: 320540kB
total-memory: 353040kB
cpu: “Pentium”
cpu-count: 1
cpu-frequency: 498MHz
cpu-load: 3
free-hdd-space: 1849860kB
total-hdd-space: 4143716kB
write-sect-since-reboot: 4162
write-sect-total: 4162
architecture-name: “x86”
board-name: “x86”

Hmmm… this does seem a little odd. How much RAM does the machine have anyway? I’m a bit suspect that your “main” MT is still showing over 300mb of memory when you’ve set the limit at 128mb. When I first started playing with Xen I had a number of very similar issues - ie: VM fails to start, but log shows nothing helpful. The fix for me was to set the memory limit for main so that there’s enough free memory to allocate to my Xen VM - but you’ve done that anyway.

I know this sounds a bit odd, but try setting the memory-for-main to another value, then back to 128mb and then reboot. I’ve had a couple of bizzar instances now where a setting hasn’t taken effect, and then if I remove and re-apply the same config, it magically works!

FWIW, here’s my config incase it helps. I personally can’t see anything majorly different that is causing yours to fail - maybe the console port? Anyway, my MT PC has 512mb RAM total, my 1 VM has 256mb allocated, two virtual HDD images, and is based on the centos image from the MT Wiki pages.

[admin@HostServer1] > /xen pr d
Flags: X - disabled, C - configuration-changed
 0   name="Server" disk-images=hda:centos30gb-server.img,hdb:support-files.img
     initrd="centos.initrd.rgz" kernel="vmlinuz-2.6.18-53.el5xen"
     kernel-cmdline="" cpu-count=1 memory=256MiB weight=256
     console-telnet-port=64000 host-shutdown-action=shutdown state=running


[admin@HostServer1] > /xen exp
# may/23/2009 18:40:49 by RouterOS 3.22
# software id = 
#
/xen
add comment="" console-telnet-port=64000 cpu-count=1 disabled=no disk-images=\
    hda:centos30gb-server.img,hdb:support-files.img host-shutdown-action=\
    shutdown initrd=centos.initrd.rgz kernel=vmlinuz-2.6.18-53.el5xen \
    kernel-cmdline="" memory=256MiB name=Server weight=256
/xen global-settings
set memory-for-main=128MiB
/xen interface
add comment="" disabled=no dynamic-mac-address=02:CA:09:2E:40:92 \
    static-interface=vif1 type=static virtual-machine=Server vm-mac-address=\
    02:69:2F:C8:90:12


[admin@HostServer1] > /system resource pr
                   uptime: 1w2d4h17m2s
                  version: "3.22"
              free-memory: 105764kB
             total-memory: 131072kB
                      cpu: "Intel(R)"
                cpu-count: 2
            cpu-frequency: 2394MHz
                 cpu-load: 1
           free-hdd-space: 24351968kB
          total-hdd-space: 76924356kB
  write-sect-since-reboot: 21685962
         write-sect-total: 21685962
        architecture-name: "x86"
               board-name: "x86"

Thanks Rich but still no go - I can’t even get a ROS image to boot under Xen.

[admin@Elaine] > xen export

may/24/2009 09:32:42 by RouterOS 3.23

software id =

/xen
add comment=“” console-telnet-port=0 cpu-count=1 disabled=yes disk-images=
hda:centos.img host-shutdown-action=shutdown initrd=centos.initrd.rgz
kernel=vmlinuz-2.6.18-53.el5xen kernel-cmdline=“ro root=/dev/hda1”
memory=128MiB name=Centos weight=128
add comment=“” console-telnet-port=64000 cpu-count=1 disabled=no disk-images=
hda:ros1.img host-shutdown-action=shutdown initrd=“” kernel=“”
kernel-cmdline=“” memory=128MiB name=ros1 weight=256
/xen global-settings
set memory-for-main=128MiB
/xen interface
add comment=“” disabled=no dynamic-bridge=none dynamic-mac-address=
02:4A:43:42:71:2F static-interface=vif1 type=static virtual-machine=
Centos vm-mac-address=02:AB:45:15:30:82
[admin@Elaine] > system reso print
uptime: 4m47s
version: “3.23”
free-memory: 320280kB
total-memory: 353040kB
cpu: “Pentium”
cpu-count: 1
cpu-frequency: 498MHz
cpu-load: 1
free-hdd-space: 1784000kB
total-hdd-space: 4143716kB
write-sect-since-reboot: 7930
write-sect-total: 7930
architecture-name: “x86”
board-name: “x86”
[admin@Elaine] > xen print
Flags: X - disabled, C - configuration-changed

NAME MEMORY WEIGHT STATE

0 X Centos 128MiB 128 disabled
1 ros1 128MiB 256 failed-to-setup-vm
[admin@Elaine] >



How do you get Xen to log itself? I’ve tried Xen> debug but if it does anything I’ve no idea where to find it.

I guess that, like many of the ‘innovative’ Mikrotik developments. I’ll have to put it on the back-burner for a couple of years until they get around to writing a manual entry for it.

IMO, you did everything right, tombrdfrd66, but there’s a problem with memory-for-main xen parameter in RouterOS versions 3.23 and 3.24 - seems like it’s simply ignored.

I was using centos xen image with 3.22 and noticed it didn’t work any more after upgrade to 3.23 (failed-to-setup-vm status). It was reporting full physical memory size for main and ignored any setting I entered for memory limit in xen parameters - and the same thing happens with 3.24, too. After downgrade back to 3.22, everything is back to normal.

So it’s either waiting for a fix from MT or back to good ole 3.22… that is, if you need xen.

Thanks for posting that hush. I was about to upgrade my 3.22 Xen machines :slight_smile: I think I’ll hold off for now.

Whilst Xen is very useful (particularly if you’re trying to save on datacentre hosting fees), it is still a bit buggy, and I believe MT are openly stating that Xen is in an “alpha” status right now.

For example, I’m currently facing an issue where all my Xen enabled machines are kernel panic’ing every 9 or so days requiring a power cycle to recover. Removing the Xen package has resolved the panic’s. I have an open case with support, but just be warned that it is not really production stability right now.

Rich

And Rich, thanks for posting that fix to the virtual ethernet problem:

http://forum.mikrotik.com/viewtopic.php?f=2&t=29895&start=0

I dropped back to v3.16 and had no problem getting CentOS to work under Xen, but I banged my head for two hours trying to get the virtual ethernet link up before finding your fix and, bingo, all up and running.

I suppose if it was easy anyone could do it, so we’d all be out of a job!!!

Now that metarouter also supports other OS images, you should try it with CentOS

Will MetaRouter run on x86 hardware, or just routerboard? I’m using x86 based servers in a co-host facility.

Is there any feature difference with Xen and MetaRouter?

Glad it worked for you Tom.

Actually, do you mind emailing me on rich (at) goode.me.uk. I wanted to discuss something offline with you (and PM’s are disabled on the forum).

Cheers,
Rich

Yay, guys. I got it going!

For other dumbos like I, the Xenised CentOS 5 on offer from MT via the Wiki installs with an IPtables firewall in place which blocks practically everything, which is why it took me many happy hours of headbanging to discover why I couldn’t connect to Webmin having installed it.

Doesn’t seem to be fixed in v3.25 either.

No go for me either CentOS image from mikrotik wiki; x86 Level 4 (tested 3.24 and 3.25) with Xen.

Well, seems like 3.25 was a dud… let’s just hope the “memory-for-main” problem gets addressed in 3.26.

While at it, it would be nice to have some faster way of creating empty disk images for xen. As it is, it took me almost an hour to create a measly 5GB image using “Make RouterOS image” option, just to delete its contents right away under Xen/CentOS so I can use it as a SMB share…

Thoughts exactly here too.

Again, I’m sorry but Mikrotik why do these issues break when they used to work? If Xen ran but crash or something I’d understand, it’s experimental, but something stupid like main-memory not even being acknowledged by RouterOS seems more like a RouterOS issue, not Xen.

If there’s anything I’d like more to see from RouterOS it’s not so much features as just a damn stable and high level of Quality Assurance put into each and every RouterOS version (starting with 3.26 :smiley: )

Seems broken in 3.26 also. It will not obey the main memory parameter, thus leaving no memory for VM’s.

Installing 3.26 on bare bones to test Xen.

Does Xen require hardware virtualization in BIOS? My original system did not have hardware virtualization in BIOS.

Does Xen require hardware virtualization in BIOS? My original system did not have hardware virtualization in BIOS.

Not that I know of. Did you get it working at all?

No go on 3.26 or 3.27. Tried KVM on 3.27 and couldn’t get it working either for what it’s worth.