I think, it’s problem with conntrack on xen host. you should try to use your root router only for hosting other virtual routers, and then do all queueing on virtual machines
in my case, i’m using the ‘real’ router for only load balancing my 3 adsl line and use the virtual
for queueing and redirect to external squid proxy but my simple queue in virtual router still won’t works,
i’m even disabled & uninstall xen package on virtual one but the problem still occured