Okay just to confirm requirements are understood.
You have three Internet Links. ISP1, ISP2,ISP3
Normal traffic flow requires Bridge1 users go to through ISP1 and Bridge 2 users go through ISP2.
If ISP1 is down, all traffic using ISP1 should flow throught ISP2
If ISP2 is also down, all traffic should flow through ISP3
+++++++++++++++++++++++++++++++++++++++++++
What I dont get is why you use ports to segregate users,
What are the different groups of users, typically they are identified by subnets? How many?
Are there any subnets that span more than one port?
Why are you removing port9 from any traffic if ISP1 is down, do you simply mean to say they should not have any failover? ( as if ISP1 is not available subnets on port 9 should not have access to either ISP2 or ISP3? )
You have not described any possible servers on network??
A good diagram always helps clarify planning and communicate intentions…
I am not segregating users, from port 1 to port 8 I have VOIP phones
Ports 9 and 10 are connected on a Cisco 890 through the GE and FE ports.
That’s why it has to be on a different network, and that’s why you should disable port 9, in case ISP1 goes down, so that Cisco understands that it had a drop, and closes the VPN through port 10, and no longer through port 9.
I know it’s confusing, and it really doesn’t make sense, but as I said at the beginning, it’s what I need at the moment, and I’m breaking my head, but so far my attempts haven’t worked out very well
Isolating bridge1 traffic from bridge2 traffic is just a matter of firewall rules - it is enough to only allow traffic from the bridges that is routed via any WAN and drop the rest in the forward chain of /ip firewall filter.
How to implement a failover from ISP1 to ISP2 down to ISP3 has also been described a thousand times here; you just need another routing table that only contains a default route via ISP2 and a routing rule that tells the router to use that routing table for anything that has a source address matching the subnet of bridge2.
So the only “exotic” requirement is to disable ether9 (which is a member port of a bridge and the remaining ports of that bridge must not be affected) if the internet is unreachable via ISP1; you can do that using a script that tracks the state of the route via ISP1 and disables ether9 whenever the route becomes unavailable or, if you need that not even a single packet that comes in via ether9 ever passes via any other WAN than the ISP1 one, you can use /interface bridge filter to assign a packet-mark to packets ingressing via ether9 and then use an action=drop rule in chain forward of /ip firewall filter, placed before (above) the “accept established,related” one, to drop packets with that packet-mark. But the latter approach doesn’t inform the device connected to ether9 about the ISP1 outage, so you may want to use both approaches simultaneously.
If this doesn’t help enough, post a properly anonymized configuration to get the detailed configuration commands - of course along with the answers to the implicit question above.
Ah, with these usage requirements, I’d probably use a different approach, I’d just remove ether9 from bridge1 and attach to it yet another subnet for the Cisco’s GE uplink, and another routing table and a corresponding routing rule for it. So traffic coming in via ether9 would only ever use ISP1, and traffic coming in via ether10 would only ever use ISP2. Whether you need to infomr the Cisco about ISP1 outage by shutting down ether9 or not depends on how the Cisco behaves.
Both the keywords “VPN” and “VoIP” suggest that you will need a housekeeping script to restore the traffic after WANs recover from outages, as you most likely have NAT on all WANs.
That code is inconsistent in terms that you remove the ESP connections directly whereas for the time-based choice you use a :foreach; worse than that, the foreach alone doesn’t save you from the script stopping on first error (a connection disappearing spontaneously before you remove it), you need the :do … on-error construct to ignore such events. But that’s not the essence.
