CHR should run containers, docker or whatever.

Thats great
+1

Please clarify how you would use it and why

Just leave it here: https://www.youtube.com/watch?v=PivpCKEiQOQ

So true! snowflake feature requests should be discarded feature requests should be related to core functions and perhaps limited to things that pertain to routing
I would rather have stability vs Pandora's Box.
Stability would be having the least amount of things to accomplish the desired task.
Its like adding a trailer brake controller to your Porsche. Sure it can be done. But only an idiot would ever want it.

I would really like to get proper MPLS MTU handling in ESX viewtopic.php?f=14&t=122446&p=620492#p606682
TSO and LRO need to be switched off in the VMXNET3 driver…

We see that the CHR is getting quite popular, so we would like to ask everyone, what new features would you like to see in the future? List your top requests for Cloud Hosted Router (CHR) and virtual machines that we do not have yet.

I would like to see a CHR ISO image.
I would like the ability to directly install a CHR ISO onto a server (skipping & not using a hypervisor). Thus a CHR could have direct access to all hardware devices and eliminate the need for a hypervisor. This could dramatically speed up a CHR because there would be no top-layer hypervisor consuming precious CPU built-in cache and other hardware resources. Such a system could possibly be 50 percent to 2,000 percent faster than using a hypervisor running a virtual CHR. Keep all of the wireless packages in the CHR ISO. Add support for 40-gig interfaces. With such a system, it could directly compete against most of the other high-end carrier-class non-Mikrotik routers , because the CHR (now a PHR Physical Hosted Router) would have the ability to use 100 percent of all physical Xeon processor cores and all Xeon built-in CPU cache. A mini-motherboard directly running CHR with high-end high-throughput wireless cards could directly replace most Mikrotik WDS links where both Mikrotiks average 10 to 60 percent load - something like this might be 1 percent cpu load. ((( The greater the CPU load , the greater the propagation time to get through a system & the slower things run))). A very fast high end Xeon directly running a CHR without a hypervisor just might be able to route at full network speed for most interfaces (approaching the speed of an ASIC chipset in a switch chip). And if the CHR system is small enough, then possibly 100 percent of the CHR could be run in high-speed Xeon CPU cache - wow )))

Also - I suspect a CHR ISO directly running on a very high-end Xeon set of processors is what Mikrotik needs to be able to sustain L3 routing at network speeds with 1 to 4 40-gig network cards. Currently, I have never heard of anybody running full L3 sustained network speeds on anything beyond L3 10-gig interfaces to anything external on a hypervisor hosting a CHR..

Also , the base system should only include the most basic system packages. Any additional package should be a package add. Also, default should be all non-necessary system/IP services disabled and any power management disabled. The smaller it is, the more likely almost everything will fit into and run entirely in built-in CPU cache.

And - with such a system, it might be nice to have an optional shell-package which supports a direct shell access to the underlying **ux system.

I would like to see a VMware Paravirtual SCSI device supported. (For those who use a hypervisor).

I would like to see a CHR ISO image.
I would like the ability to directly install a CHR ISO onto a server (skipping & not using a hypervisor). Thus a CHR could have direct access to all hardware devices and eliminate the need for a hypervisor.

How is this different from the normal x86 install that has literally been an option for over a decade? Download: https://mikrotik.com/download (X86 section, right after ARM - the CD image is literally an ISO as you requested) and license info: https://wiki.mikrotik.com/wiki/Manual:L ... nse_Levels (Non-CHR X86 licenses are feature-based, not port-speed-based). CHR was introduced because the basic X86 version assumed it was running on bare metal, and wasn't properly suited for being installed as a VM.

I would like to see a CHR ISO image.
I would like the ability to directly install a CHR ISO onto a server (skipping & not using a hypervisor). Thus a CHR could have direct access to all hardware devices and eliminate the need for a hypervisor.

How is this different from the normal x86 install that has literally been an option for over a decade? Download: https://mikrotik.com/download (X86 section, right after ARM - the CD image is literally an ISO as you requested) and license info: https://wiki.mikrotik.com/wiki/Manual:L ... nse_Levels (Non-CHR X86 licenses are feature-based, not port-speed-based). CHR was introduced because the basic X86 version assumed it was running on bare metal, and wasn't properly suited for being installed as a VM.

Fist - x86 is a 32-bit system
Second - x86 does not have any paravirtual support
Third - From my hand-on real-world experience using x86 as the 207.32.195.2 public btest server, I would experience a full x86 lockup sometimes 4 times a day. Now that I am using a CHR 64-bit system with some paravirtual device support, it has NEVER locked up.
Fourth - a 64-bit system directly supports much much larger access to big memory.
Fifth - I have no idea if a x86 32-bit ROS system could even hold a full ipv4 & ipv6 bgp routing table without problems - let alone run at 5-gig plus sustained speed without locking up many times a day.
Sixth - 64-Bit operating systems also support 64-bit access to memory (faster)
Seventh - 64-Bit systems also can also use more efficient 64-bit compilers and use 64-bit instructions (faster again)
Eighth - I believe 64 bit systems also have additional registers (more registers than 32-bit systems) (again faster again)
Ninth - a fully 64-bit compiled system can process more instructions (some new 64-bit instructions) which can result in fewer CPU clock cycles need to perform a function. (((64-bit hardware can easily out-run 32-bit hardware with both CPUs clocked at the same CPU clock speed.
Tenth - newer hardware devices (40-gig network cards for example) usually come with **ux optimum 64-bit drivers. (again - again faster again).
Eleventh - a 32-bit system running 32-bit code can use twice the CPU/memory read/write clock cycles as a 64-bit system running 64-bit code. (again - lots of these - faster again)

So I ask - what good is a system (possibly 32-bit) on a multi-many core system if the clock speeds of the CPU cores is sllooowww and has little or nooo built-in CPU cache. In 5 years, we could have 10-Ghz Xeon CPUs 25-+ core with multi-gig CPU cache per core. It will happen sooner than you think. My first computer was a 4-bit PPS04 running at 300 Hz (then upgraded to the 8008 - then the 8086 and now the 64-Bit Xeon CPUs - my how things get faster.
The first hyper-visor I ever experienced back in the late 80s/early-90s took 3 days to boot DOS. So , even though hyper-visors are faster - a no-hyper-visor system is the fastest least overhead method to make something run incredibly fast. Thus a CHR ISO could kick start a 40-gig L3 full-network-speed routing platform.

North Idaho Tom Jones

Fist - x86 is a 32-bit system

I was not aware that the X86 downloads were 32 bit only. Every one of your other points is based on this issue, so it really sounds like what you are asking for is an X86 install that is 64-bit. I understand at this point it may be easier to base this off of the CHR code, but this isn't a feature request for CHR itself. Also, given the "raw image" that is available, can't you just transfer, e.g. with dd, this to a hard drive? ethernet driver support might be a bit limited based on whatever is in the kernel, but at least this way you'd have a 64 bit OS on bare metal?

Fist - x86 is a 32-bit system

I was not aware that the X86 downloads were 32 bit only. Every one of your other points is based on this issue, so it really sounds like what you are asking for is an X86 install that is 64-bit. I understand at this point it may be easier to base this off of the CHR code, but this isn't a feature request for CHR itself. Also, given the "raw image" that is available, can't you just transfer, e.g. with dd, this to a hard drive? ethernet driver support might be a bit limited based on whatever is in the kernel, but at least this way you'd have a 64 bit OS on bare metal?

Click
Yea - I've been wanting to transfer a virtual CHR hdd system to a physical system and dump the hyper-visor and try running it on bare metal. I think it would work. But - I don't know about 10-gig network card driver support ... ??? However, 1-Gig Intel E1000 series network card might be supported.

I would agree that a x86_64 ISO would be useful. I utilize several CHR and I do like them. The license check in I do find annoying. Kinda has a Microsoft feel.

The Elephant in the room is why there is not a CCR that has the balls to handle the more demanding tasks quickly and reliably. I don't think it matters if they have 400 cores if the frequency is only 1GHz or 1.2GHz

Perhaps if they could offload BGP to a FPGA or something. I would imagine that if Tilera GX came in 3GHz multicore we would not be complaining as much and CHR and x86_64 would not be that huge of an issue.

I would agree that a x86_64 ISO would be useful. I utilize several CHR and I do like them. The license check in I do find annoying. Kinda has a Microsoft feel.

The Elephant in the room is why there is not a CCR that has the balls to handle the more demanding tasks quickly and reliably. I don't think it matters if they have 400 cores if the frequency is only 1GHz or 1.2GHz

Perhaps if they could offload BGP to a FPGA or something. I would imagine that if Tilera GX came in 3GHz multicore we would not be complaining as much and CHR and x86_64 would not be that huge of an issue.

Show me any Tilera / ROS system that can btest udp to 127.0.0.1 and get faster than 20 gig (or even half of that or even half of half of that) - there is no such thing. My ol clunker several years old CHR on a Xeon ESXi system can hold 19 gig all day long. Taking it to newer CPUs and running bare metal should hit 30+ gig on a btest to 127,0,0,1

Yea - I've been wanting to transfer a virtual CHR hdd system to a physical system and dump the hyper-visor and try running it on bare metal. I think it would work.

you may just convert your x86 installation to x86_64
viewtopic.php?p=587018#p587018

Second - x86 does not have any paravirtual support

What do you need it for on a bare metal?

Sixth - 64-Bit operating systems also support 64-bit access to memory (faster)
Seventh - 64-Bit systems also can also use more efficient 64-bit compilers and use 64-bit instructions (faster again)
Eighth - I believe 64 bit systems also have additional registers (more registers than 32-bit systems) (again faster again)
Ninth - a fully 64-bit compiled system can process more instructions (some new 64-bit instructions) which can result in fewer CPU clock cycles need to perform a function. (((64-bit hardware can easily out-run 32-bit hardware with both CPUs clocked at the same CPU clock speed.

It's not that simple in reality. Being that straightforward shows to me that you do not really understand how this 32bit vs 64bit stuff works.

Please note that I do not argue that bare metal x86_64 ROS is a good thing to have. It would definitely have a lot of advantages. I'm just pointing out that your arguments are somewhat inaccurate.

Eleventh - a 32-bit system running 32-bit code can use twice the CPU/memory read/write clock cycles as a 64-bit system running 64-bit code. (again - lots of these - faster again)

WAT?

Show me any Tilera / ROS system that can btest udp to 127.0.0.1 and get faster than 20 gig (or even half of that or even half of half of that) - there is no such thing.

I'd take it if your are only looking for something to run btest on. Comparing how fast btest is on a given hardware is just that- comparing the btest performance. Btest is known to be (a) single-threaded and (b) highly inefficient (even Mikrotik stuff have admitted that here multiple times). Result of such comparison will have no direct indication of how fast the other tasks run on the same hardware.

PS. There are lots of threads on this forum where the x86_64 version of RouterOS is requested, and where pros & cons are discussed. Let's keep this thread CHR-related. Thanks.

Edit: Further x86 vs x86_64 RouterOS discussion is here: viewtopic.php?f=2&t=126117

We have several places where we run CHR on AWS, both as central location for management/vpn-termination/dude and to use RouterOS for VPN-tunnels instead of having to struggle with the AWS VPN service.
To take this concept further, we would really be helped by the ability to install CHR on KVM and Google Cloud as well.

Google cloud requires features that are not possible to back-port to the version of the kernel that is used in the CHR. So unfortunately at the moment it is not possible to run the CHR on the Google cloud.

We have several places where we run CHR on AWS, both as central location for management/vpn-termination/dude and to use RouterOS for VPN-tunnels instead of having to struggle with the AWS VPN service.
To take this concept further, we would really be helped by the ability to install CHR on KVM and Google Cloud as well.

Please support virtio SCSI storage controllers, see viewtopic.php?f=15&t=120413

Make CHR to run on barebone switches. http://www.edge-core.com/productsInfo.p ... 143&id=264

I second that for sure!!!

First, it is not easy to use disk file. VPS provider often provide some linux distribution only.
We have a script can install RouterOS CHR on a VPS current running Ubuntu but there are still some problem.
I think we should have a installer can run on a lot of linux distribution.

Second, I think it's necessary to support more platform because VPS provider using variety of VM config that CHR can not run on.
Currently i have some VM provided by Bandwagon,which use a disk driver so CHR can not run on it.

Please support virtio SCSI storage controllers, see viewtopic.php?f=15&t=120413

yes this is important.

Currently i have some VM provided by Bandwagon,which use a disk driver so CHR can not run on it.

I guess it will always be possible to craft some environment in which a binary-only distribution cannot run.
The question is if it is worth the trouble to cater for that, or one just has to wait until all the "incompatible" and "difficult" environments just vanish from the market because customers no longer want to put up with those difficulties.

Currently i have some VM provided by Bandwagon,which use a disk driver so CHR can not run on it.

I guess it will always be possible to craft some environment in which a binary-only distribution cannot run.
The question is if it is worth the trouble to cater for that, or one just has to wait until all the "incompatible" and "difficult" environments just vanish from the market because customers no longer want to put up with those difficulties.

There is hugge difference between "not working, becouse of difficult environment" and "not working, becouse of lack of one of the most important, mainline drivers". Look at similar problem at VyOS: https://phabricator.vyos.net/T389

Difficulty level Easy (less than an hour)

It's not only my whim to run CHR on KVM with SCSI-virtio. Look at the problem like this: KVM VPS hosting is cheapest full virtualisation on the market. If I want to provide homogenicall network environment with Routerboard devices It's natural to use CHR. If I can't use it becouse using obslate mode (driver) of Virtio storage (look closely at: https://pve.proxmox.com/wiki/Paravirtua ... or_Windows ) everything is going to be more and more complicated (more expensive VPS with vmware or diffrent OS).


I tell it once more: Nowdays it's absolutly crucial to have support of VirtIO SCSI in CHR, please MT developers, make it work (please, please, please)

Edit:
Fixed in >= 6.42rc52

I want to see the following features:

- SD-WAN Overlay (Group multiple physical circuits into a "logical" one) using an overlay network (i.e. multiple IPSec or SSL tunnels on each physical circuits)
- Zero touch provisioning for SD-WAN interconnection
- Central controller to centrally manage all the RouterOS SD-WAN routers
- Central controller to define policies to be pushed for QoS, load balance per application layer and automatic re-routing based on policies.
- Traffic analysis / reporting

I doubt you will see such features. Would it be cool? Absolutely! I would imagine you will grow old and die before you see the requested features

I made a Mikrotik based SD-WAN solution for a client that works quite well. I spent a month in the lab testing various scenarios to get the desired function. I think Pepwave & Mushroom are some possibilities if you are looking for manufactures that currently support SD-WAN solutions.

scnr, a unifi controller

Please add Metarouter

Please add Metarouter

VM inside VM? Are you serious?

Inception hehe

Please add Metarouter

VM inside VM? Are you serious?

Servers have had hardware support for this for ages. It is actually not as bad as it first sounds, but as always it depends on the application.

ROS can act as hypervisor host. Download extra packages for CHR and install KVM package. Keep in mind that nested virt (vm-in-vm) is not supported.

+ VM Tools please.

Everyone is asking for Docker and other advanced things. How about we start with some very simple fundamental things like open-vm-tools...

Thanks!

Desired features:

• Graceful shutdown/reboot on XenServer
• XenMotion between hosts without freezing MikroTik router. (currently will cause MikroTik to lock up)

VMTools is a must!

Small web server, if MikroTik would want to give us something like that, would also make sense for any other RouterOS, not just CHR. And it's also true for pretty much anything else you can think of (any service and such).

CHR needs VM-specific stuff. VM tools being a nice example. Or all kinds of virtual drivers. And VMDK disks distributed as SCSI instead of IDE would also be nice.

+1 I think this should be a priority https://github.com/vmware/open-vm-tools

Another suggestion is if is possible to integrate storage over network (smb/nfs) mounts. In virtualization environment we can have both FB storage via esxi for example or other vm used as a storage 'proxy'. I'm not sure how complicated is to implement nfs/smb clients in routeros but is useful function for config backups / proxy storage (in other vm not over physical network .. form other side chr vm disk can be big but probably someone will find benefit in this for HW router board, no idea ) / and probably more

Desired features:

• Graceful shutdown/reboot on XenServer
• XenMotion between hosts without freezing MikroTik router. (currently will cause MikroTik to lock up)

+1

I see several requests across the forum for SD-WAN, CHR is a good candidate for http://wanproxy.org/ I just test it and look promising (at less for small network). Unfortunately total lack of documentation, however it is BSD licensed which should be ok to implement it in routeros. I'm sure routeros developers can make and they own tcp/wan optimization solution. The only stopper which I see is the cpu/ram usage for those types of solutions (limited in hw).. which we don't have in CHR (vm = cpu/ram+++++)

I doubt you will see such features. Would it be cool? Absolutely! I would imagine you will grow old and die before you see the requested features

I made a Mikrotik based SD-WAN solution for a client that works quite well. I spent a month in the lab testing various scenarios to get the desired function. I think Pepwave & Mushroom are some possibilities if you are looking for manufactures that currently support SD-WAN solutions.

Can you please share with us what you mean with 'I made a Mikrotik based SD-WAN'. Do you mean: A -> some SD-WAN -> Mikrotik router -> some connectivity -> Mikrotik Router -> some SD-WAN -> B . Or you mean some directly integrated solution with roureros / maybe meta router ?

I doubt you will see such features. Would it be cool? Absolutely! I would imagine you will grow old and die before you see the requested features

I made a Mikrotik based SD-WAN solution for a client that works quite well. I spent a month in the lab testing various scenarios to get the desired function. I think Pepwave & Mushroom are some possibilities if you are looking for manufactures that currently support SD-WAN solutions.

Can you please share with us what you mean with 'I made a Mikrotik based SD-WAN'. Do you mean: A -> some SD-WAN -> Mikrotik router -> some connectivity -> Mikrotik Router -> some SD-WAN -> B . Or you mean some directly integrated solution with roureros / maybe meta router ?

Client location has somewhat regular interruptions to their Internet. Currently we have a Coax, Fiber, LTE ISP connections. I form three tunnels back to my NOC (One per ISP) I allocated a /29 to them from my NOC and this works great. Their Internet connections are essentially just conduits to my NOC. They can lose two of the three Internet connections and they never notice. It wont even drop a VoIP call. Twas a bitch to build but it works well.

Good to see MikroTik are responsive.
Xen Tools and VM Tools are now available in 6.42RC. The "current" build of 6.42 can't come fast enough for me!

Good to see MikroTik are responsive.
Xen Tools and VM Tools are now available in 6.42RC. The "current" build of 6.42 can't come fast enough for me!

THANK YOU MIKROTIK!

MT Support
is so difficult to create an iso version of CHR ?
because some KVM like ramnode dont allow to use virtual disk only iso

Don't ask about CHR. I think that this product isn't interesting for MT developers as in the beginning of existence.
To bypass your problem, use ISO of your favourite live linux (without installation) and my guide:
viewtopic.php?t=120413

Keep in mind if your cloud provider (ramnode) use current version of KVM, they probably forcing to use VirtIO-SCSI.... and CHR currently doesn't supports it.
viewtopic.php?f=15&t=124905&start=100#p626094
I will be appreciated if you will test it.

Don't ask about CHR. I think that this product isn't interesting for MT developers as in the beginning of existence.
To bypass your problem, use ISO of your favourite live linux (without installation) and my guide:
viewtopic.php?t=120413

Keep in mind if your cloud provider (ramnode) use current version of KVM, they probably forcing to use VirtIO-SCSI.... and CHR currently doesn't supports it.
viewtopic.php?f=15&t=124905&start=100#p626094
I will be appreciated if you will test it.

Hi thank allot i will post a print on your topic for dont fload this topic

Re: CHR suggestions for new functionality

How about a Mikrotik 8 or 12 or 24 port Ethernet switch - but not switch like what you are used to - noooo - how about this instead ....

1) - A Mikrotik L2/L3 switch that is actually a PC card that fits into a PCI slot on in a server rack-mount computer.
2) - Some VMware ESXi drivers for the Mikrotik switch (and also for other hypervisor OS systems).

Hardware switches are always faster than software emulated switches.
It makes sense that by injecting a real physical hardware switch into a hypervisor system, you would gain I/O throughput speed.
It is not un-common for a hypervisor server system (VmWare ESXi or other) to be running 12 to 50+ virtual operating systems.
All hypervisors use CPU resource consuming software emulated switches. This is a real I/O throughput killer on a busy system.
By adding to or replacing the hypervisor switch with a real physical switch, all virtual machines can run at true physical port speeds while avoiding the I/O bottelnecks of a simulated/emulated layer 2 ethernet switch.

I would think one hardware engineer and one software engineer could easily kick out a working prototype.
I also think a Mikrotik product like this could be a world changing event for the way all server centers operate.

North Idaho Tom Jones

MT Support
is so difficult to create an iso version of CHR ?
because some KVM like ramnode dont allow to use virtual disk only iso

For easier installing we have introduced CHR in OVA format, it is available on the download page.

ISO is something to make a CD from. Why don't you simply migrate to a cheaper and more modern Cloud provider?

Linode can do it, Hetzner can do it. More powerful machines, SSD disks and cheaper price:

https://www.hetzner.com/cloud

Regarding Hetzner, CHR would need to be able to boot from Virtio SCSI. Not a good example normis.

Question about CHR ,
1st - it makes sense that ROS 32-Bit probably has everything compiled with a 32-bit compiler.
Now for my question : Is everything in CHR compiled with a 64-bit compiler ?
Or - are just some things in CHR compiled with a 64-bit compiler (such as the booted operating system and/or some programs that may need greater than 2 gig of direct memory access) ?

The reason I ask - and I might be wrong here - is because at one time when I did a CHR version downgrade using system-packages, I think I used a 32-bit ROS version and I think it worked - which makes me think that 32-Bit ROS and CHR might share some of the same 32-bit compiled code --- don't know - just asking.

If parts of CHR are 32-bit , then another question , would a 100 percent everything in CHR compiled with a 64-bit compiler possibly run any faster ?

Just a thought/question

also - it would be nice if x86 32-bit ROS could support paravirtual ethernet drivers - such as a vmxnet3 network card. I suspect this might be one of the reasons why I sometimes see x86 32-bit ROS lock-up under heavy network card I/O when using emulated network cards without any paravirtual driver support.

also - it would be really desireable to see an ISO boot to install CHR. Then CHR could run on bare metal boxes much much faster than in a hypervisor enviornment where it is not using 100 percent of all CPU/memory & I/O resources - and still be able to install in a hypervisor enviornment if wanted.

also - it would be nice if CHR could boot to a paravirtulized (driver optomized) hard disk. Paravirtual drivers are almost always faster and use less CPU/memory/interrumpts and deliver faster I/O throughput.

North Idaho Tom Jones

ISO is something to make a CD from. Why don't you simply migrate to a cheaper and more modern Cloud provider?

Linode can do it, Hetzner can do it. More powerful machines, SSD disks and cheaper price:

https://www.hetzner.com/cloud

Could you tell us please when is planned to add VirtIO-SCSI boot support?

Could you tell us please when is planned to add VirtIO-SCSI boot support?

The kernel used by RouterOS v6.x does not support it, so you can probably safely say: not until RouterOS v7.
That version could be released next week, next month, next year or next decade...

CHR would need to be able to boot from Virtio SCSI. Not a good example normis.

We'll see about that.

What's new in 6.42rc28 (2018-Feb-16 07:02):

*) chr - added "virtio-scsi" driver on KVM installations;
*) chr - added support for Hyper-V ballooning;
*) chr - added support for Hyper-V guest quiescing;
*) chr - added support for Hyper-V host-guest file transfer;
*) chr - added support for Hyper-V integration services;
*) chr - added support for Hyper-V static IP injection;
*) chr - added support for NIC hot-plug on VMware and Xen installations;
*) chr - fixed additional disk detaching on Xen installations;

We would like to receive your comments about these CHR features.

Hi!

Could you tell us please when is planned to add VirtIO-SCSI boot support?

Look at the release notes of ROS 6.42rc28 (2018-Feb-16 07:02):

*) chr - added "virtio-scsi" driver on KVM installations;

Is that what you are looking for?

Very nice. I can confirm, rc28 is able to boot in Hetzner cloud. Thank you!

Can we run CHR in Google Cloud now?

Could MPLS on vsphere/esxi be requested as new functionality?

Thu Jul 06, 2017 3:43 am

TSO, GSO and GRO need to be disabled also on guests, so you will have to wait for new CHR build.

viewtopic.php?t=122446
viewtopic.php?t=128396

Yes. We tested on 6.42rc30, instructions available on the wiki page - https://wiki.mikrotik.com/wiki/Manual:CHR_GCE

Can we run CHR in Google Cloud now?

these new features are packages that can be deactivated or disabled? These drivers can not cause conflicts with each other?

What's new in 6.42rc28 (2018-Feb-16 07:02):

*) chr - added "virtio-scsi" driver on KVM installations;
*) chr - added support for Hyper-V ballooning;
*) chr - added support for Hyper-V guest quiescing;
*) chr - added support for Hyper-V host-guest file transfer;
*) chr - added support for Hyper-V integration services;
*) chr - added support for Hyper-V static IP injection;
*) chr - added support for NIC hot-plug on VMware and Xen installations;
*) chr - fixed additional disk detaching on Xen installations;

We would like to receive your comments about these CHR features.