I solved a similar problem "reversing" the VRF, i.e. having only one on the LAN side and having the three interfaces in "main".
See here (mind you, not necessarily "right", but it works) to have an idea of the approach I am using:
Thread start:
Attempting to evolve from caveman's failover
Relevant posts are:
Attempting to evolve from caveman's failover - #36 by jaclaz
Attempting to evolve from caveman's failover - #37 by jaclaz
Cannot say how this approach might be adapted to your case (with the l2tp-vpn) but maybe it gives you some inspiration.