I would like to make load balacing between the pppoe gateway on the rb3011 and a LTE gateway with static ip that resides on a trunk of the lan connected in wifi . Someone can explain to me if you can do it.
Currently in the second house I take advantage of the two gateways by configuring the routes directly on the laptop; in the first house I can’t.
I did as the manual says, but maybe I am wrong:
#pppoe-out1 on eth1
#ether5 interface connected wifi to house2 (I can ping on the gateway 192.168.8.1)
#bridge all interface rb3011
/ip address
add address=10.0.0.0/24 network=10.0.0.0 broadcast=10.0.0.1 interface=bridge
add address=192.168.8.2/24 network=192.168.8.0 broadcast=192.168.8.255 interface=ether5
/ip firewall mangle
add chain=input in-interface=pppoe-out1 action=mark-connection new-connection-mark=WAN1_conn
add chain=input in-interface=ether5 action=mark-connection new-connection-mark=WAN2_conn
add chain=output connection-mark=WAN1_conn action=mark-routing new-routing-mark=to_WAN1
add chain=output connection-mark=WAN2_conn action=mark-routing new-routing-mark=to_WAN2
add chain=prerouting dst-address=192.168.8.0/24 action=accept in-interface=ether5
add chain=prerouting dst-address-type=!local in-interface=bridge per-connection-classifier=both-addresses-and-ports:2/0 action=mark-connection new-connection-mark=WAN1_conn passthrough=yes
add chain=prerouting dst-address-type=!local in-interface=bridge per-connection-classifier=both-addresses-and-ports:2/1 action=mark-connection new-connection-mark=WAN2_conn passthrough=yes
add chain=prerouting connection-mark=WAN1_conn in-interface=bridge action=mark-routing new-routing-mark=to_WAN1
add chain=prerouting connection-mark=WAN2_conn in-interface=bridge action=mark-routing new-routing-mark=to_WAN2
/ip route
add dst-address=0.0.0.0/0 gateway=pppoe-out1 routing-mark=to_WAN1 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=192.168.8.1 routing-mark=to_WAN2 check-gateway=ping
/ip firewall nat
add chain=srcnat out-interface=pppoe-out1 action=masquerade
add chain=srcnat out-interface=ether5 action=masquerade
You have provided only configuration from one of the houses plus I don’t understand the remark
So in general, you have to configure both houses’ routers the same way as described here. The critical point is to avoid application of the load distribution rules on connections initiated by packets coming from the other house; these packets must always be sent out via local WAN.
I do not have routers on the second home is all a single lan , in the second home I added a LTE gateway that provides only a static ip.
In the second house, I can use both gateways with my laptop (win7)because I added the routes and gateways manually.
From the laptop of my first home I can’t ping to the gatewayLTE.
Sorry, I’ve missed that. So actually the only interface of the router in house 1 which hosts the 192.168.8.0/24 subnet is ether5, and when you connect a laptop to that router, the laptop gets its IP address from some other subnet associated to bridge, right? And when the laptop is connected in house1, those routes on it which it uses for load balancing when connected in house2 (I still don’t know how that should be working unless it is a linux laptop) are replaced by a default route to Mikrotik’s IP address associated to bridge1?
If the above is correct, what exactly does not work?
OK, I wasn’t sure so I wanted to avoid doubt.
That’s not a big surprise as from the perspective of the Mikrotik, packets from the laptop to the LTE gateway are forwarded ones, so the way your mangle rules are currently set, the PCC handling affects these packets too. I haven’t found a way how to check how the both-addresses-and-ports hash treats icmp packets - as there is no port in icmp, unless the hash includes the icmp id instead of one of the ports in the calculation, you may end up marking all packets for 192.168.8.1 coming from the address of your laptop for getting routed via pppoe-out1. You can check whether it is the case by swapping the current mapping of the remainder value to connection-mark in the two rules (i.e. by making packets with reminder 0 be connection-marked with WAN2_conn and packets with reminder 1 with WAN1_conn).
If you want to be able to access the LTE gateway and/or other devices in the 192.168.8.0/24 subnet from a laptop connected to the Mikrotik, you have to exclude this subnet from the PCC handling. You can do that by adding /ip route rule add action=lookup-only-in-table table=main dst-address=192.168.8.0/24 or by adding dst-address=!192.168.8.0/24 to the action=mark-routing or action=mark-connection rules’ conditions.
It’s not a problem, it’s the goal and it was clear, so I wanted to know what are the actual problems which prevent achieving of this goal 
So far it seems that it actually works as-is but you’ve chosen a method to test it which gave you misleading results because you were actually testing something else.
thanks sindy ,
now I try to make the changes that you have recommended .
I’ll post the results in a while (if it doesn’t stop everything…) don’t let me down, you’re my lifeline
I tried the changes but can’t get them to work at this point I’d be happy with a simple failover. Now my LAN is configured as originally without any PCC rules has a pppoe gateway, and a dhcp server managed by the RB3011 . In the second house there is always the gateway with only the static ip and the access to the whole LAN works normally.
I tried doing so:
/ip route add gateway=10.0.0.1 check-gateway=ping
/ip route add gateway=192.168.8.1 distance=2
but there is something in the firewall that prevents you from reaching the LTE gateway. 
Where exactly in the second house? On the laptop which lives there? I suppose that if its gateway is the 192.168.8.1, it uses the LTE box, and if the gateway is 192.168.8.2, it uses the PPPoE via the Mikrotik
This way, the route with distance=2 is only used if the other one (with default value of distance which is 1) is down; it can happen that the gateway of the higher priority route is pingable but you cannot get anywhere further through it so it is not a good approach. Plus, in your case, 10.0.0.1 is in the LAN subnet, so it cannot be a gateway; a gateway must be either the pppoe-out1 itself as an interface or, better if possible, the remote address indicated by the PPPoE server because in such case you can use the recursive next-hop search to track the link state all the way to the Internet.
This is what happens if you don’t post the complete configuration You’ve only posted part of the firewall configuration but it wasn’t clear from the posting, and in some other similar topics someone has stated he had no firewall at all and I have mixed the two together, concluding that firewall cannot be what prevents the load distribution in your case.
So try again with the complete output of /export hide-sensitive, obfuscating eventual public addresses if they appear there (hide-sensitive only removes passwords and alike)
I couldn’t get how you can configure two gateways in different subnets unless the two subnets are accessible via the same interface or there are two interfaces on the laptop, and the config export reveals you’ve really attached two subnets to the same L2 space
Doing so is not exactly “wrong” but it may complicate not only understanding but also making the firewall rules selective enough (as e.g. packets matching in-interface=bridge can come from any of the subnets).
I’ve originally built an impression that the 192.168.8.0/24 was attached to a dedicated interface (ether5) and that the devices in house2 had addresses from that network.
But to the topics, when you look at the rules in chain=forward of the /ip firewall filter, the last two explain why it doesn’t work:
add action=accept chain=forward comment=“accetta pacchetti in uscita sull’interfaccia pppoe -separazione vlan” out-interface=pppoe-out1
add action=drop chain=forward
All the rules before those two accept only packets belonging to already established connections and those coming in/out via ipsec tunnels, so any new connections can only establish via pppoe-out1.
So a rule accepting also packets with out-interface=bridge inserted before the action=drop one should be enough to make it possible to use the LTE as another uplink, so the result would be
/ip firewall filter
…
add action=accept chain=forward comment=“accetta pacchetti in uscita sull’interfaccia pppoe -separazione vlan” out-interface=pppoe-out1
add action=accept chain=forward comment=“accetta pacchetti che usano LTE come gateway” out-interface=bridge
add action=drop chain=forward
But another problem remains - it may not be possible to tell the LTE box that to deliver packets to 10.0.0.0/24, it has to use 192.168.8.2 as a gateway. So you’ve used an action=masquerade rule saying that whatever is sent out via ether5 should be src-nat’ed to the IP address associated to ether5 which is in the LAN subnet of the LTE box so no route to 10.0.0.0/24 is necessary on that box. However, no IP address at all is associated to ether5; it is a member interface of a bridge interface named bridge, and that interface has two distinct IP addresses attached to it so the action=masquerade rule could easily choose a wrong one if only modified to use out-interface=bridge.
It is not possible to have an IP configuration attached to a physical interface and at the same make that same interface a member port of a bridge.
So we again have to find a workaround for the two subnets sharing the same L2 space. In /ip firewall nat, replace the
add chain=srcnat out-interface=ether5 action=masquerade
by
add chain=srcnat out-interface=bridge dst-address=!10.0.0.0/24 src-address=10.0.0.0/24 action=src-nat to-addresses=192.168.8.2
sindy you are a great wizard!!! it works wonderfully! I would never have arrived without your help. thanks again!! 
failover also works perfectly, now I try to rewrite the CPP rules but I think and hope not to find problems now the most complex stage is over.
Please add in-interface=bridge to the rule action=accept chain=forward comment=“accetta pacchetti che usano LTE come gateway” out-interface=bridge to improve security. Your setup is so unusual that I haven’t realized immediately that this condition should be there. It’s not a big deal that it was missing for a while as there’s NAT on the pppoe-out1, otherwise it would have been a serious security hole.
This morning I was convinced I could finally get the load balancing instead as soon as I add the rules below the lan freezes.
I thought that having a gateway on the bridge, it was enough to replace in-interface=ether5 with src-address=192.168.8.1 instead the issue is more complex.
/ip firewall mangle
add chain=input in-interface=pppoe-out1 action=mark-connection new-connection-mark=WAN1_conn
add chain=input src-address=192.168.8.1 action=mark-connection new-connection-mark=WAN2_conn
add chain=output connection-mark=WAN1_conn action=mark-routing new-routing-mark=to_WAN1
add chain=output connection-mark=WAN2_conn action=mark-routing new-routing-mark=to_WAN2
add chain=prerouting dst-address=192.168.8.0/24 action=accept src-address=192.168.8.1
add chain=prerouting dst-address-type=!local in-interface=bridge per-connection-classifier=both-addresses-and-ports:2/0 action=mark-connection new-connection-mark=WAN1_conn passthrough=yes
add chain=prerouting dst-address-type=!local in-interface=bridge per-connection-classifier=both-addresses-and-ports:2/1 action=mark-connection new-connection-mark=WAN2_conn passthrough=yes
add chain=prerouting connection-mark=WAN1_conn in-interface=bridge action=mark-routing new-routing-mark=to_WAN1
add chain=prerouting connection-mark=WAN2_conn in-interface=bridge action=mark-routing new-routing-mark=to_WAN2
/ip route
add dst-address=0.0.0.0/0 gateway=pppoe-out1 routing-mark=to_WAN1 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=192.168.8.1 routing-mark=to_WAN2 check-gateway=ping
/ip firewall nat
add chain=srcnat dst-address=192.168.8.1 action=masquerade
Also, can you, via DHCP server, force the use of the LTE gateway to a certain host?
The reason is again the fact that you use the unusual network setup with two subnets on the same interface but your firewall rules are copied from a setup which relies on interface name as an unambiguous information about packet source. So what actually happens is that you assign a routing-mark also to packets coming from the LTE gateway and destined to your LAN devices, which makes them use the default route with that routing-mark because the routing-mark wins over the fact that the destination address is in a locally present subnet.
So using @Sob’s approach here because your existing configuration is already complex enough, add the following to your configuration:
/ip route rule add dst-address=10.0.0.0/24 action=lookup-only-in-table table=main
This routing rule will override any routing mark assigned and make sure that packets for devices in local LAN will be routed properly, using the dynamic routes for directly connected subnets which are only added to the default routing table (main).
Also your PCC rules rely on the wrong assumption that whatever comes from bridge is coming from the LAN subnet. Your action=mark-connection rules rewrite already assigned connection-marks with new ones; this doesn’t matter too much for packets sent from the LAN subnet as the PCC gives the same results for all packets belonging to the same direction of the same connection, but it is a disaster for packets coming from the LTE where the roles of src and dst address and port are swapped so the PCC matches are different. One way to avoid this is to only assign connection-marks to packets belonging to connections which don’t have any yet. To do so, it is enough to add connection-mark=no-mark condition to the action=mark-connection rules.
You are another victim of a common misconception. The packets which come from the internet via the LTE gateway do not have the LTE gateway’s IP address as source one. They keep the actual IP address of the remote sender. So to identify such packets, you have to use a different set of conditions in that rule, such as in-interface=bridge src-address=!10.0.0.0/24.
The IP address of a gateway configured for a route is never used in packet’s header. It is only used to determine the MAC address of the gateway device so that the packet with the original destination IP address unchanged could be packed into an L2 frame and sent to that MAC address.
So all in all the following modifications are required:
/ip firewall mangle
add chain=input in-interface=pppoe-out1 connection-mark=no-mark action=mark-connection new-connection-mark=WAN1_conn
add chain=input src-address=192.168.8.1 in-interface=bridge src-address=!10.0.0.0/24 connection-mark=no-mark action=mark-connection new-connection-mark=WAN2_conn
add chain=output connection-mark=WAN1_conn action=mark-routing new-routing-mark=to_WAN1
add chain=output connection-mark=WAN2_conn action=mark-routing new-routing-mark=to_WAN2
add chain=prerouting dst-address=192.168.8.0/24 action=accept src-address=192.168.8.1
add chain=prerouting dst-address-type=!local in-interface=bridge per-connection-classifier=both-addresses-and-ports:2/0 connection-mark=no-mark action=mark-connection new-connection-mark=WAN1_conn passthrough=yes
add chain=prerouting dst-address-type=!local in-interface=bridge per-connection-classifier=both-addresses-and-ports:2/1 connection-mark=no-mark action=mark-connection new-connection-mark=WAN2_conn passthrough=yes
add chain=prerouting connection-mark=WAN1_conn in-interface=bridge action=mark-routing new-routing-mark=to_WAN1
add chain=prerouting connection-mark=WAN2_conn in-interface=bridge action=mark-routing new-routing-mark=to_WAN2
/ip route
add dst-address=0.0.0.0/0 gateway=pppoe-out1 routing-mark=to_WAN1 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=192.168.8.1 routing-mark=to_WAN2 check-gateway=ping
/ip firewall nat
add chain=srcnat dst-address=192.168.8.1 action=masquerade
You can use DHCP to force a specific configuration to a specific host, but it is not possible to force a gateway from some subnet to a device which doesn’t have an IP address from that same subnet. So a device which has only an address from 10.0.0.0/24 cannot be given a gateway from 192.168.8.0/24.
What you probably actually want (do you remember the sticker “I wanted a Lamborghini, but I was unable to pronounce it?”) is that for a particular device, the PCC rules would be overridden and connections of that device would always be handled using the to-WAN2 table. So do exactly that - create a static DHCP lease for that device’s MAC address, causing it to always get the same IP address (say, 10.0.0.10) while the other settings like default gateway, DNS etc. remain the same like for all the other devices, and insert the following rule just before the PCC rules:
add chain=prerouting dst-address-type=!local in-interface=bridge src-address=10.0.0.10 connection-mark=no-mark action=mark-connection new connection-mark=WAN2_conn passthrough=yes
you must srcnat traffic going to gateways, it’s easy for gateways with master interface, but for gateways inside the bridge you must srcnat to router ip address inside the bridge (usually 192.168.88.1) by checking the connection mark, so the internal gateway respond to router instead of directly to the client.
sindy,
as soon as I enter the /ip route rules the lan freezes.
/ip route
add dst-address=0.0.0.0/0 gateway=pppoe-out1 routing-mark=to_WAN1 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=192.168.8.1 routing-mark=to_WAN2 check-gateway=ping
alli,
I do not understand what you mean by,you can explain better?