NEW USER PATHWAY TO SUCCESS [ if you find a way to run into problems despite the guides, let me know where to improve ]
A. Configuring Off Bridge
B. Firewall
C. VLAN Filtering
D. AP/Switch RoS Setup
E. Port Forwarding (destination nat)
F. Wireguard
G. Backup/Export Config
H. NetInstall RoS
I. IP Route Multi-WAN
J. Directing Users out A Specific WAN - (no mangling required)
K. Directing Firewall Address List out A Specific WAN (with mangling)
l. Multi-WAN Input To Servers (with mangling)
M. First Access using WINBOX
N. Zerotier
O. Load Balancing PCC
P. Switch Chip Vlans
Z. Super Useful Scripts

HAVING ISSUES AND NEED HELP READ THIS FIRST - viewtopic.php?p=908118

JUST STARTING TO CONFIG YOUR MIKROTIK DEVICE OR SIMPLY BROWSING - SEE BELOW

The first thing a new user should do is go straight to Para L. and watch the linked videos. If you have a grasp on the basics of winbox then the following order of help topics is designed to get you up and running safely and efficiently (A & B), and upon success one can venture into more interesting configurations that include vlans and using MT devices for other than routing (C & D) and finally, when ready, to start to tackle slightly more advanced topics/requirements (E & F) such as remotely and securely connecting to your device.

A. CONFIGURING (off) BRIDGE - viewtopic.php?t=181718 (all MT devices)

B. FIREWALL - viewtopic.php?t=180838

C. VLAN FILTERING - viewtopic.php?t=143620 and relationship between bridge and vlan filtering - viewtopic.php?f=2&t=173692
***** Did you Turn ON VLAN filtering after configuring /ip bridge port and /ip bridge vlan settings *****

Note: MT help topics have vastly improved on this topic and can be found at:
https://help.mikrotik.com/docs/display/ ... NFiltering
https://help.mikrotik.com/docs/display/ ... VLAN+Table

VLANS & HYBRID PORTS

Rule of Thumb: A Port (any port) can ONLY HAVE ONE "1" untagged vlan.

-An access port can only have one untagged vlan.
-A hybrid port can only have one untagged vlan (and as many tagged vlans as required - and obviously not tagged on the same vlan as untagged).
-A trunk port can only have one or more tagged vlans.

(example)
....................... ...........................

D. AP SWITCH SETUP (using any RoS device as an AP/Switch) - viewtopic.php?t=182276

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

E. PORT FORWARDING - viewtopic.php?t=179343

F. WIREGUARD - viewtopic.php?p=906311

G. BACKUP/EXPORT CONFIG - https://help.mikrotik.com/docs/display/ROS/Backup / https://help.mikrotik.com/docs/pages/vi ... Id=2555971
(courtesy of mkx)
The most direct method is "BACKUP" - a binary file, created by the admin. This file is really only intended to restore a configuration on the very same device, preferably running the same version of ROS. If restored to another ROS version, there's a built-in configuration converter which mostly does a good job (but not always). If restored to another device of same type, MAC addresses are overwritten. Restoring it to a device of different model doesn't work (either creates random results or fails altogether).
Backup Function ---> Select Files in winbox.

Then next and most common method is called "EXPORT". This is an admin created text export of the current configuration. This text file shows commands needed to configure the device to its current state. This export lacks some items that make complete setup: users, passwords, certificates, SSH keys, etc. The benefit of using it is that it's text file, so it's pretty easy to apply it to a different device ... one has to adjust commands and that's it. It can be applied verbatim to a device, but a devices configuration needs to be completely blank beforehand.
Export Function ---> Select New Terminal in winbox. and use the apppropriate syntax.

Export Syntax For Admin
/export verbose file=anyname { shows every possible entry - rarely used }
/export show-sensitive file=anyname { useful for most admin work and NOT to be posted }
Export Syntax For Forum (public viewing)
/export hide-sensitive file=anyname { for version 6 }
/export file=anyname { for version 7 }

Note1:The EXPORT format is the preferred communication tool for forum members when discussing problems (screenshots are requested if needed).
Caution: One should not expose public IPs or public gateway IPs in any posted Export file or screenshot!
Note2: Export is best viewed using NOTEPAD++


----> Where to find DEFAULT CONFIG - /system default-configuration print

H. NETINSTALL - should be used if any security concerns arise OR if your firmware version seems to be acting strangely or the firmware version is really old!!

Another way of getting your unit(s) up to date is to netinstall devices. I would say it's preferred because it also installs recent default configuration while upgrades convert running configuration. A few versions between your running ROS and current one brought some big changes and it is possible for upgrade process to fail to convert old config to new one. In addition, new default config makes much more secure setup in certain aspects. In other words, if you are many firmware versions behind, netinstall is the least painless way to get to the latest config and is more secure.

Sage tips/advice on NetInstall if having difficulties......
a. Regarding Etherboot for all devices, the most error-free method is - to press the reset button, keep it pressed, power on the device, and wait until the device shows up in the NetInstall window, then release the button. This holds true for the Audience as well.
b. Try to have a direct cable connection between computer and device (laptop to device for example).
c. A trick which helps for some is to only use a dumb switch in between, nothing else connected.
d. Also on THE COMPUTER being used to run netinstall, there may be issues if there are multiple NETWORK interfaces. If experiencing difficulties on the PC, it is best to ensure all other NETWORK interfaces are turned off.
e. Enable NETBIOS, over TCP/IP ( active vice automatic or disabled )!

User Comment: In my experience, Netinstall often fails to detect devices after the ethernet link to the MT device goes down and up again while netinstall is running.
Either place a dumb switch between the PC and the MT box to netinstall, so the PC ethernet link stays up during reboot while entering Etherboot.
Or first put the MT box in Etherboot mode, wait for the link to come up and start Netinstall after the link is up.

General Refs:
https://help.mikrotik.com/docs/display/ROS/Netinstall
https://help.mikrotik.com/docs/pages/vi ... Id=1409054
https://wiki.mikrotik.com/wiki/Manual:E ... set_button
RESET VIDEO FROM MT - https://www.youtube.com/watch?v=6Unz92rABs8


I. IP ROUTE - Multi-WAN { rp-filter=loose NOT strict }

There is much confusion on IP routes for the new beginner in part because we dont know how it got there in the first place. Mikrotik default settings in IP DHCP settings, typically for ether1, include two items of interest. Use ISP "PEER" DNS (check box selected) and use ISP Route ( Add Default Route is selected to "yes" ). Thus an IP Route is already provided (the default route) if the admin selects YES. One can uncheck this selection and enter manual routes which is usually the case once you get past one ISP provider. This section will focus on using manual routes which means ensuring the selection for Add Default Route is set to "no". The manual routes below will work regardless of firmware (6/7) chosen.

DUAL WAN - FAILOVER ( basic )
/ip route
add check-gateway=ping comment=Primary ISP distance=5 dst-address=0.0.0.0/0 gateway=Primary-gatewayIP
add comment=SecondaryISP distance=10 dst-address=0.0.0.0/0 gateway=Secondary-gatewayIP

The syntax check-gateway=ping, tells the router to ping the gateway IP every 10 seconds or so. In this example, the router will still switch to ISP2, if ISP1 is not available, and the router will keep checking to see if WAN1 becomes available again, and if it does will switch router internet traffic back to ISP1 due to the fact that the shorter distance of ISP1 will also point the router to selecting ISP1 if both are available.

Recursive Routing
In its simplest form/def'n - A recursive route is simply one that points to another route in your routing table instead of to a directly connected link.
Recursive routing occurs when a route either static route or dynamically learned has a next-hop that is not directly connected to the local router. This second lookup in the routing table is called recursion. A recursive route in general is one that points to a next-hop which isn't directly connected to the router. A directly connected link would be something that has your ISP gateway IP in the route. An internal route would be one on the router itself, (like all the subnet or vlan routes - dst-address=vlansubnet gwy=vlan-name)

WHY RECURSIVE? --> Many experienced admins don't trust that the router to ISP connection as the best way to ensure connectivity to the WWW. The reason is that often the ISPs internal network connection can be up (modem to next hop within the ISP network) but the ISPs connection to the internet is down. Thus the router will think the gateway is working when nothing on the internet can be reached. Admins solve this issue with something called "Recursive Routing".

To accomplish this, the admin, by use of the DNS addreses as the gateways, verifies the WAN connection works (is live). We ping those addresses to establish if the connection works and then transform the connection to the ACTIVE state. In the examples we are using a common DNS provider IP address such as 1.0.0.1, and 9.9.9.9.

TWO RULES OF THUMB (scope & target scope):
First Rule. The resolving route (DIRECT - connected route) with dst-address TO the "real WWW IP (dns site)" and with local ISP gateway IP, has Target-Scope=X and the recursive route (INDIRECT - external route) with gateway IP VIA the "real work WWW gateway IP (dns site)" has Target-Scope=X+1. In other words, the farther one gets from the router, the TS increases by one.

Second Rule. Between the same two routes being compared, the Direct , connected route, with local ISP gateway IP (resolving route) has to have a SCOPE that is equal to or less than the TARGET SCOPE of the recursive route. In other words, the scope of the route must be equal or less than the target scope of the next farthest route.

FARTHEST ROUTE: SCOPE= (doesnt matter) / TARGET SCOPE=Y+2 (recursive route)
CLOSER ROUTE: SCOPE= Y+2 or less / TARGET SCOPE=Y+1 (recursive route)
CLOSEST ROUTE: SCOPE=Y+1 or less / TARGET SCOPE=Y (gateway=ISP, resolving route)
INTERNAL ROUTE: ( within router, scope is not used, no recursive action at all )

DUAL WAN - RECURSIVE
/ip route
add check-gateway=ping distance=3 dst-address=0.0.0.0/0 gateway=1.0.0.1 scope=10 target-scope=12
add distance=3 dst-address=1.0.0.1/32 gateway=PrimaryISP-gatewayIP scope=11 target-scope=11
add comment=SecondaryISP distance=10 dst-address=0.0.0.0/0 gateway=SecondaryISP-gatewayIP scope=10 target-scope=30

Note1: As you can see TS as you get closer to the router decreases by 1.
Note2: As you can see the scope of the next route is equal to or less than the TS of the preceding route.

USING TWO RECURSIVE ROUTES - FLAT
Additionally many admins also like to use at least two different public IPs to confirm connectivity (removes the rare but not rare enough occasion that DNS providers are not available). Thus as per the below example, a secondary DNS is checked to verify indeed that ISP1 is not available, when the first DNS checked is not working. If both DNS recursive routes show as not available, the router will select the Secondary WAN for all traffic. In the meanwhile the router will continue checking the availability of both DNS recursive routes for connectivity and will switch back to ISP1 if at least one of them is available.

/ip route
add check-gateway=ping distance=3 dst-address=0.0.0.0/0 gateway=1.0.0.1 scope=10 target-scope=12
add distance=3 dst-address=1.0.0.1/32 gateway=PrimaryISP-gatewayIP scope=10 target-scope=11
+++++++++++++++++++
add check-gateway=ping distance=4 dst-address=0.0.0.0/0 gateway=9.9.9.9 scope=10 target-scope=12
add distance=4 dst-address=9.9.9.9/32 gateway=PrimaryISP-gatewayIP scope=10 target-scope=11
+++++++++++++++++++
add comment=SecondaryISP distance=10 dst-address=0.0.0.0/0 gateway=SecondaryISP-gatewayIP scope=10 target-scope=30

Note: As you may see, its easiest just to set all the scopes the same and in this case to 10, which ensures the Second Rule from above is followed.

USING TWO RECURSIVES - NESTED
Finally, admins can accomplish the same effect as the "flat" approach above by layering or nesting the recursive routes with each other with the same outcome. This is a more complex setup but is deemed to be more efficient overall especially if using multiple routing tables (academic and not required for new users). It is included for those interested. The tricky part is that we will use an address that really doesnt exist to force the router to look for a routing, in this case randomly choose 10.10.10.10/32. Thus we create a top rule with no apparent path and then recursively look for a path through both DNS addresses which then are resolved by the direct route through the ISP gateway. So although it appears we are using three Other addresses in nested recursive routing we are still only using 2 KNOWN DNS addresses and one 'BOGUS" address. The bogus address is used to force the router to be recursive for both the KNOWN Dns addresses. In other words, the flat approach is Recursive to Resolving (two steps), nested is Recursive to Recursive to Resolving (three steps).

/ip route
dst-address=0.0.0.0/0 gateway=10.10.10.10 scope=10 target-scope=14
++++++++++++++++
add check-gateway=ping dst-address=10.10.10.10/32 gateway=9.9.9.9 scope=10 target-scope=13
add dst-address=9.9.9.9/32 gateway=PrimaryISP-gatewayIP scope=10 target-scope=12
+++++++++++++++
add check-gateway=ping dst-address=10.10.10.10/32 gateway=1.0.0.1 scope=10 target-scope=13
add dst-address=1.0.0.1/32 gateway=PrimaryISP-gatewayIP scope=10 target-scope=12
+++++++++++++++
add comment=SecondaryISP distance=10 dst-address=0.0.0.0/0 gateway=SecondaryISP-gatewayIP scope=10 target-scope=30

Note: All Primary routes have same distance, only the last Secondary route has a higher distance.


J. DIRECTING Users out Specific WAN { no mangling }

In multi-WAN setups, it is often that the admin needs to direct a whole subnet or one or more users out a specific WAN. This is typically applied when the users are supposed to go out a Primary WAN but a few users need to go out the Secondary WAN. The same approach by the way is used in Wireguard configs, when one has to force some users into the wireguard tunnel for internet (at a remote site) because the bulk of users locally go out the local WAN for internet. If you need a group of IPs and its too numerous for this method, check out the next paragraph which addresses ensuring an address list goes out a specific WAN.

GENERAL EXAMPLE
Version 6 approach
Solution: Consists of two steps: ONE - create an additional route for the secondary WAN (copy of the original one) and add a routing-mark!, and TWO - create an associated Routing Rule which will direct the User to that Routing Mark via its Table selection parameter.
[Step1]
dst-address=0.0.0.0/0 gwy=ISP1 distance=5 check-gateway=ping
dst-address=0.0.0.0/0 gwy=ISP2 distance=10
dst-address=0.0.0./0 gwy=ISP2 distance=10 routing-mark=useSecondary

[Step 2]
/ip routing rule
add src-address=SUBNET or IPaddress Action=Lookup-only-in-table table=useSecondary

Version 7 approach
Solution: Very similar to the Version 6 approach but we have to physically create the new table and the rest is the same.

[Additional Step]
/routing table fib add name=useSecondary

Note1: Action=Lookup-only-in-table means the router will NOT look for any other routing for said traffic in the main table, if the Secondary WAN is not available, and the traffic will be dropped.
Note2: If the admin wants the users to be able to access the Primary WAN, if the Secondary WAN is down, then use ACTION=Lookup.

MULTIPLE USERS
If you have more than one user, that needs to be 'redirected', then simply create MULTIPLE Routing Rules! . .........................

Note3 (see below) : Routing Rule order counts much like firewall rules etc. This may come into play when you have more complex requirements.

COMPLEX CASE: [ Three Subnets ]
THREE SUBNETS. Primary WAN is for subnet A and subnet B, but users on subnet C should go out Secondary WAN. In addition, one user on subnetC (the exception) should actually follow the same routing as subnets A&B. In terms of failover, users from subnet A and B should be able to use ISP2, if ISP1 is not available. However, users from subnet C should not have access to ISP1 if ISP2 is not available.

In terms of planning purposes, it should be clear that we are going to have to redirect Subnet C users to WAN2. In addition we are going to have redirect Exception User on Subnet C to mimick routing that users on Subnets A,B follow. Therefore we will have to create additional routes for both WANS, to be able to force users as needed. (subnet C to wan2, exception user on subnet C to wan1)

dst-address=0.0.0.0/0 gwy=ISP1 table=main distance=5 check-gateway=ping
dst-address=0.0.0.0/0 gwy=ISP2 table=main distance=10
dst-address=0.0.0.0/0 gwy=ISP1 table=main distance=5 check-gateway=ping table=usePrimary
dst-address=0.0.0./0 gwy=ISP2 distance=10 table=useSecondary

/routing table add name=usePrimary fib
/routing table add name=useSecondary fib

/routing rule add src-address=Exception IP action=lookup table=usePrimary { for exception user on subnet C }
/routing rule add src-address=subnetC action=lookup-only-in-table table=useSecondary { for subnet C }

Order is key: If we had the routing rules order reversed, the exception user would have been captured by that Routing Rule, since located in subnet C and forced to WAN2, thus we had to match that user to the correct routing rule, FIRST, to ensure traffic went to WAN1

SERVER EXAMPLE: [ Local User Access ]
It is thusly possible to direct a server out a specific WAN. Lets say we have a VOIP server 10.10.10.2 that has been directed out WAN2.

However a problem arises! The local LAN users should also be able to access the Server but with the rules set above, all responses to LAN users would be directed out to WAN2 and not back to the users. The solution is to create another route rule that identifies traffic being sent to the Server and informs the router which table such traffic should use (table main). The router, via table main, will select the most appropriate routing which will be the interface for the subnet the user sent the request from, and the returns will go back to the local user. ORDER IS IMPORTANT........... In this case ensure the more generic rule is stated first!!

/routing rule add dst-address=10.10.10.2/32 action=lookup-only-in-table table=main
/routing rule add src-address=10.10.10.2/32 action=lookup-only-in-table table=useWAN2

K. MULTI-WAN & Firewall Address List { with mangling }

In this type of scenario you EITHER have a group of internal/local IP addresses (could be something less than a subnet or bunches of IPs from different subnets) where route rules are just to onerous OR you have a bunch of specific destination addresses that when selected from ANY user, should go out a specific WAN. In these cases its typical to disable fasttrack as mangling and fasttrack are not compatible.

SOURCE ADDRESS BASED EXAMPLE
Assuming two separate PPPOE connections for a dual WAN scenario and a firewall address list called special-group, where this group needs to go out pppoe-2
..............

If you wish to keep fasttrack for the rest of the traffic, modify two copies of the regular established,related rule and put them in front of the fasttrack rule - one rule for originating traffic and one rule for returning traffic. The modification is the addition of the firewall address list!
add action=accept chain=forward connection-state=established,related src-address-list=special-group
add action=accept chain=forward connection-state=established,related dst-address-list=special-group

DESTINATION ADDRESS BASED EXAMPLE
Assuming two separate PPPOE connections for a dual WAN scenario and a firewall address list of destination addresses called external-group, where any traffic headed towards these destinations needs to go out pppoe-2.
If you wish to keep fastrack for the rest of the traffic repeat the process as shown in the previous example above.........
add action=accept chain=forward connection-state=established,related dst-address-list=external-users
add action=accept chain=forward connection-state=established,related src address-list=external-users


L. MULTI-WAN Input To Servers & Mangling { rp-filter=loose NOT strict / fastrack=disabled }

For discussion purposes we are describing a scenario where the Admin has one or more servers behind the router and they should be accessible from both WAN connections, regardless of which is Primary or which is Secondary. How the users reach the servers is up to the admin. Basically the idea is to capture the incoming traffic from external users and then ensure the traffic goes out the same WAN it came in on! (Thanks to Sob) the following script forms the basis for this V7 approach: ...................................
The only difference between V7 and V6, is that in V6 there is no separate table and one adds the routing-mark in the ip route itself! ..................................

Example 1: DUAL WAN FAILOVER
................................................... ......................
Note: The Table and Mangle creation are identical for all examples.

Example 2: DUAL WAN FAILOVER RECURSIVE
................. ....................

Example 3: DUAL WAN FAILOVER TWO RECURSIVE (flat)
.....................
Example 4: DUAL WAN FAILOVER TWO RECURSIVES (nested)
................................ .....................................................


M. First Time Access & WINBOX (must watch videos for the new user)
First Access - https://www.youtube.com/watch?v=Q9qwgKrw-0g
First Config - https://www.youtube.com/watch?v=Q9qwgKrw-0g
More on Winbox - https://www.youtube.com/watch?v=MKVJYxhmSkQ
Basic Firewall - https://www.youtube.com/watch?v=6boYA7xdjZY&t=1376s
Basic Routing - https://www.youtube.com/watch?v=FBknCFkH0J8&t=21s
Static Routes - https://www.youtube.com/watch?v=ZYeMuYBAVrQ&t=214s

N. ZEROTIER (courtesy of Amm0)
viewtopic.php?t=183424

O. LOAD BALANCE PCC
https://mum.mikrotik.com/presentations/US12/steve.pdf
https://mum.mikrotik.com/presentations/US12/tomas.pdf

P. SWITCH CHIP VLANS
https://help.mikrotik.com/docs/display/ ... +Switching
https://help.mikrotik.com/docs/display/ ... switchchip
https://www.youtube.com/watch?v=Rj9aPoyZOPo - Vlans using the Switch Chip
https://www.youtube.com/watch?v=rvQ6o4RfnoU - Configure Vlan on Switch Chip
https://www.youtube.com/watch?v=YLtGQAQ8iS0 - CRS3XX Step by Step

Z. SUPER USEFUL SCRIPTS
(i). Capture LOGGED IPs into Firewall Address list - viewtopic.php?t=148397
b. next
c. next
d. next
e. next

hates maybe... 1 + 1 = 2 . Period. Hate diff ways to do things or how things can be done in diff ways. Settle on ONE method.

IMO the path you created here is excellent especially for the newbie .... Tik should make this a STICKY plus Tik should consider a similar approach in their new and improved docs which are sorely lacking information of value.

hates maybe... 1 + 1 = 2 . Period. Hate diff ways to do things or how things can be done in diff ways. Settle on ONE method.

My point with maybe, is that the OP always finds ways to screw things up, they are the best testers!!!

[edit:nm]

anav,
I'm really enjoying your IP ROUTE - Multi-WAN section. I appreciate the updates and clarification. Its a lot of work to produce something like this, keeping things as simple as possible. Would like to see support added for how to allow Multi-WAN inbound support (ROS v6 & v7) too. This requires mangle based on what I've read. This simplest way while still supporting the Recursive option would be valuable.

Regarding WireGuard, something that I've been doing is putting a WireGuard server behind a MikroTik firewall. I've had to do this because I can not move to ROS v7 yet in production. Thus I had to find a way to support both a pure Linux based wireguard instance (I was building Ubuntu RPi servers for a while) and also a MikroTik functioning only as a wireguard endpoint. I really like using the MikroTik option. But how to do this took some trial and error. I would like to get that to you and see if you would like to incorporate it into your topic designs about WireGuard. If so, do you want the article here, or in another post?

It's simple, for each WAN do something like (v7):
For v6 it's almost the same, only you don't need to create routing table and route's routing-table parameter is named routing-mark.

I dont get the requirement multi-wan input, do you mean to ensure all traffic coming in on one WAN PORT is forced out the same WANPORT.

Sob needs to explain each line to me, not what it does but its purpose in the scheme of things........ besides syntax tell me 'the story'.

So far I see mark any new connection coming in on a particular WAN,
then he assigns a routing mark to that connection mark.
He does both these steps in prerouting, okay.

What I dont get is the last line??
It appears that he is marking outgoing traffic with the same naming convention for routing mark.........
Not sure how any of this helps...

IF I go out WAN2, the return traffic is going to come in on WAN2.

IF I come in on WAN1, going out on WANX will be predicated on routes.

+++++++++++++++++++++++++++++++
Hmm pcunite were you talking PRIMARY and FAILOVER, or PCC load balanced routes???

Unless I misunderstood the request, multi-WAN router with e.g. 1.2.3.4 on WAN1 and 2.3.4.5 on WAN2, internal webserver with forwarded ports 80 and 443, and it should be possible to reach it using both public addresses. So you mark new incoming connections based on WAN interface, and then mark routing for responses. The one in prerouting is for server (traffic from LAN), other in output is for router itself, e.g. if you'd run VPN server on it, which should be also accessible using both addresses (this can be also done using routing rules).

PCUNITE, I cover your topics for wireguard already.

Check out para's 7 & 8
viewtopic.php?p=906311#p906311

Unless I misunderstood the request, multi-WAN router with e.g. 1.2.3.4 on WAN1 and 2.3.4.5 on WAN2, internal webserver with forwarded ports 80 and 443, and it should be possible to reach it using both public addresses. So you mark new incoming connections based on WAN interface, and then mark routing for responses. The one in prerouting is for server (traffic from LAN), other in output is for router itself, e.g. if you'd run VPN server on it, which should be also accessible using both addresses (this can be also done using routing rules).

Hi Sob, so you are thinking primary and failover and where the external users should be able to access either WAN to get to server.
The right solution there is to give users the dyndns URL to the ACTIVE WANIP. So as MYNETNAME, if the primary ISP goes down the secondary ISP should take over be it the IP cloud or some other service.

In other words, I can paint at least 5 different scenarios and its up to pcunite to come clean on the requirements, not playing a chasing game on this one LOL

Two disadvantages:

1) DDNS change is not instant. There can be delay before router updates it, also on client side it takes some time before old cached record expires (not much, but it's not zero).
2) If there's long open connection (e.g. to VPN server) and it was made to backup WAN, you may not want to break it when primary comes back (although sometimes maybe you do, e.g. if backup has much worse parameters than primary, then it would be worth it to reconnect there).

Unless I misunderstood the request, multi-WAN router with e.g. 1.2.3.4 on WAN1 and 2.3.4.5 on WAN2, internal webserver with forwarded ports 80 and 443, and it should be possible to reach it using both public addresses.

I can paint at least 5 different scenarios and its up to pcunite to come clean on the requirements ...

Sob is correct. I'm asking about making connections to the router from multiple WAN inputs. There could be many reasons why someone would want this. I don't intend to challenge them. I'm asking about how to do it because this is an academic thread.

I get that but I need to understand the requirements to talk about the config. Saying I need multiwan inputs is bereft of detail............
PCUNITE, see new para K!

I get that, but I need to understand the requirements to talk about the config ... PCUNITE, see new paragraph K!

Excellent! I shall test this soon. So, you are correct, I need an example. I shall present one here:

Some customers don't actively use fail-over and they don't actively load balance, not traditionally speaking. Instead, they use ISP1 for all standard traffic in their environment and an ISP2 is used only for VoIP and media streaming services. Thus, they have two egress and ingress WAN providers. However, both WAN1 and WAN2 host servers (applications listening on ports) that need to be reached from the outside. Thus even when ISP1 is primary, secondary ISP providers are much in use and need external control, monitoring, and accessibility by admins and users alike.

I am of the same boat, I have some uses for WAN2 only, in that case route rules work fine, no mangling required.
Really the only reason to mangle if is you have incoming traffic on both wans to servers and you have a complex set of users using the different wans.

I'm attempting to configure the Multi-WAN Input To Servers (with mangling) option. I appeal for assistance as I'm unable to get this to work.

Ether1 is ISP1, Ether3 is ISP3, (Ether2 is not plugged in at the moment). While pinging ISP3's static IP, I can see ping attempts being counted in the following rule:

add action=mark-connection chain=prerouting connection-state=new disabled=no in-interface=ether3 new-connection-mark=WAN3_conn passthrough=yes

but they never make it to new-routing-mark:

add action=mark-routing chain=prerouting connection-mark=WAN3_conn disabled=no in-interface-list=LAN new-routing-mark=ISP3_route passthrough=yes

Here is my sanitized configuration. Anything stand out to you?

Your routes are not setup correctly! The gateway to the actual ISP is only used once per ISP set of routes!!
From this .......
......... To THIS
................. .............

Then looking at your secondary and tertiary fall back routes..............
From this
......... .................

To THis (had one too many routing-mark assigned on isp2 routes)
............ ..............

Finally get rid of these NOT required......
#Other
add check-gateway=ping comment=Host1 distance=1 gateway=8.8.8.8 scope=10 target-scope=12
add check-gateway=ping comment=Host2 distance=2 gateway=9.9.9.9 scope=10 target-scope=12

Final comment since you mangle ICMP traffic there is no guarantee that ping will work, based on the errors so far LOL. Suggest removing that mangling, but perhaps you have legit reasons for frigging with it

@pcunite: Multi WAN and rp-filter=strict are not friends.

@pcunite: Multi WAN and rp-filter=strict are not friends.

Oh my goodness. The rp-filter manual states: Warning: strict mode does NOT work with routing tables. Opps! Now that I've changed this, I can access internal servers now from both WAN ports! Woo-hoo! Adding input/output rules took care of ping. Very interesting. What is a work around for strict mode (RFC3704)?

This PDF states: "Underlying principle of rp_filter is to block outbound traffic if the IP does not belong to the subnet that resides on the LAN" Okay, so I need a list of what I consider valid? Other comments I see are: "if the interface is not the best reverse path the packet check will fail" Well, true I guess because it is going out a different interface from default. I think a firmware update should address this. These packets can be tracked.

As per my amended Guide, for multi-wan rp-filter=loose, and of course if mangling involved fastrack=disabled!
@pcunite just ensure that tcp tracking is turned on, AND uncheck the box for loose tracking - WE WANT strict tracking, which makes our invalid rule have more teeth.

Yes, common suggestion is to use rp-filter=loose, but according to the thread linked by @pcunite, if it takes into account default routes (that cover any possible address), it's useless for this kind of multi WAN configs.

Yes, common suggestion is to use rp-filter=loose, but according to the thread linked by @pcunite, if it takes into account default routes (that cover any possible address), it's useless for this kind of multi WAN configs.

You know very well that being obtuse is not helpful and will cause me to post again! ;-P
RP filter loose will NOT cause problems in multi-wan scenarios.
RP filter strict will very likely cause problems in multi-wan scenarios.

Hence the guidance I will continue to provide in multi-wan scenarios is to ensure RP-filter is set to loose.
Yes there may be some ways around it, but until someone like pcunite plays with it, tests it, finds the boundary conditions etc etc, then the guidance wont change. An article is not good enough.
Of course if Sindy provides the guidance, then I wont have to wait for pcunite.

It's not that rp-filter=loose would cause problems, only in this scenario it may not provide any benefits over rp-filter=no (default).

Certainly no EXTRA benefits, thats for sure but then also recommended tcp tracking strict (which makes invalid rule stronger).

Yes there may be some ways around it, but until someone like pcunite plays with it, tests it, finds the boundary conditions etc etc, then the guidance wont change. An article is not good enough. Of course if Sindy provides the guidance, then I wont have to wait for pcunite.

I know so very little. Very grateful to this forum for the help. I'm good at formatting documents! But I lack proper understanding. Anav is a master at tearing up configurations (editing), and mkx, sindy, Sob, Zacharias, pe1chl, and so many others (forgive me if I don't mention your name) have the actual expertise. Then a host of silent professionals too timid to speak what they know. Maybe they can't. Look at my QoS article that rots. I simply don't know enough to fix it.

It would be cool if we could form a website, and thereby crowdsourse the documentation and formatting the way we know is best, and then backport it into the wiki (assuming MikroTik would accept it). I don't care if someone uses our hard work as long as knowledge is shared. Silence is hindering MikroTik. People just need answers, not arrogance. So, imagine a monthly posting schedule to were we address every issue, but it is formatted on a website that makes it easy to learn and understand the concepts.

It took me an entire month to prove the VLAN article. It took the creation of a lab, testing, seeking help from strangers, stringing words together, and making images. But with the help of all of us, maybe we could create something truly special?

It would be cool if we could form a website, and thereby crowdsourse the documentation and formatting the way we know is best, and then backport it into the wiki (assuming MikroTik would accept it). I don't care if someone uses our hard work as long as knowledge is shared. Silence is hindering MikroTik. People just need answers, not arrogance. So, imagine a monthly posting schedule to were we address every issue, but it is formatted on a website that makes it easy to learn and understand the concepts.

Concur, a very good suggestion! Have had the same thoughts. There are so many good solutions found in the forums that should be gathered to a common place as "best practice" or something like it. Or maybe Sob/Sindy might want to write "the" book "RoS Best Practice"...

There are so many good solutions found in the forums that should be gathered to a common place as "best practice" or something like it. Or maybe Sob/Sindy might want to write "the" book "RoS Best Practice"...

I like, "MikroTik, the definitive guide"

Well, let's have a Zoom meeting. I invite sindy, mkx, anav, amm0, larsa, chupaka, pe1chl, sob, and zacharias. I invite anyone willing and capable in being apart of the formation of a new website and articles series that addresses the most pressing configuration needs the MikroTik community faces. These will ultimately be turned into a PDF and or a printed book. The weary MikroTik sojourner need no longer to despair. "MikroTik, the definitive guide. Best practices from the community. Volume 1"

Not as knowledgeable as the others but I can proof-read. Easier with time-zone differences too.

If I don't understand something right away, there is a possibility some more beefing might be needed

Hmmm, I am not sure I can add any academic knowledge to the endevour. I am as stated good at tearing things up so if you need paper shredded................
One thing is to think about the audience and intentions. I am trying to make things easier for beginners and helping them on a journey to understand what the config means, which is more important than getting it right the first time (copy and paste someone elses works doesnt get one very far as I can attest too)..

One thing is to think about the audience and intentions. I am trying to make things easier for beginners and helping them on a journey to understand what the config means, which is more important than getting it right the first time (copy and paste someone elses works doesnt get one very far as I can attest too)..

Well, the website would actually be for you. It will not replace what you do in the forums, but augment the "how". So, imagine that you use your vast expertise with endless bad configurations, and help us build proper ones that make sense. You will still be in the forums (the website is not a replacement for the official forums) and be linking to documents you helped create.

The links you have here are a mess (not your fault). Imagine linking to something that has good formatting and lengthy or short topics depending on the user you're working with. Now, I'm trying to sell you! Haha, but if it doesn't make sense, we don't have to do it.

You did a lot of linking to my VLAN article and I do appreciate it. But it was created for you and the community. But the community needs more than I can or even you can provide. It needs a series of answers offsite but that directs them back here as well.

Not as knowledgeable as the others but I can proof-read. Easier with time-zone differences too.

All of our past experiences bring something to the table. I'm involved with MikroTik mostly from dealing with phone systems. I'm extremely weak in other areas. When I need help, I'm lost really quick. I think a lot of people could use a website_to_PDF presentation that is enjoyable to read and leads you along many of the common concepts that get asked here alot.

It is such a waste for some of the forum's brightest minds to wrestle in the weeds while trying to simultaneously teach concepts in a forum with bad configurations and methodology over and over. I was needing help recently and I had to spend hours reading many random threads only to come up with an invalid answer. This is madness.

So, imagine an improved format of the vlan article. I've rightly received criticism on the syntax format I used because it doesn't mirror the export command. I was going for a balance between maximum shortest line + easy on the eyes + teaching a concept. But creating a version that closely mirrors export would help to prevent tripping up new users who are trying to edit the configuration. On the proposed website, I could show different styles because I could create efficient links, formatting, and versioning.

But I'm useless without the knowledge that comes from others here. I don't have the answers people need, I'm not good at working with bad configurations, yadda, yadda.

I am trying to make things easier for beginners and helping them on a journey to understand what the config means, which is more important than getting it right the first time (copy and pasting someone else's works doesn't get one very far as I can attest too)

Very true. The VLAN article in many ways has failed. Something better is needed and I don't think the forum itself is the answer. The amount of work you're doing is incredible and I wonder if you can sustain it. In my line of work we call it "customer journey". I do think more teaching is needed but alongside the syntax (and maybe with GUI examples). I don't know. Something needs to be different because I can't get the help I need. Is the forum truly for helping or contesting? I think we can help and it not be a great burden on everyone.

So, a website under the control of the forum's top editors (fifteen of us?) and we build, test, create, and promote configurations. We promote the knowledge here that was built, presented, and hosted elsewhere. We crowdsource the RouterOS user experience. Something MikroTik is unable or unwilling to do. I think I could get a corporate sponsor to pay for the hosting etc.

I have to confess something (semi-seriously), I'm in this forum to hide from work (not in a dishonest way, that I'd get paid for doing something else while being here, it's all my "free" time) and other responsibilities, procrastination is the popular term for it.

Related to that, I suffer from perfectionism. I can point out a mistake in someone's config, give them hints, or play with some config because it's fun, etc. It's easy, because there are no commitments or anything (although... I tend to not abandon threads and sometimes I get the feeling that I should have left some to someone else; but it doesn't happen very often). But it's completely different to create serious quality document, because such thing should ideally explain everything, and it adds up. It quickly turns into work, and... see the beginning. I have writing up some things (like @anav is doing) in my virtual TODO list for years, but somehow I can't get to it.

Also related, I hate to promise anything to anyone, because when I do and it doesn't go as planned, it doesn't help with everything else. So while I may provide some help or feedback, meaning that I'm not against it, rather don't expect too much from me.

Yes Please SOB, leave me the easy ones, and just take the hard ones!!

I have to confess something ...
<SNIP>
... it's completely different to create serious quality document ...

Great points. Yeah, sometimes I check out the forums to decompress too. And making a website with perfect configurations is a lot of effort! If you have a full time life, its just another thing to get done before bed. I get that. Well, its fun to hope we might get good documentation someday!

A question about Section J. DIRECTING Users out Specific WAN. Say I'm directing a specific IP address out WAN2 (10.2.2.2 is a VoIP server). That's great, however, the LAN can no longer access it. So, do I need a rule like this?

/ip route rule add action=lookup-only-in-table dst-address=10.2.2.0/24 table=main

I need something to ensure that LAN access to that IP which is forced out WAN2, is still accessible from other default LAN traffic that is default routed out WAN1. I think this would be an incredibly desirable feature.

Not sure what you mean?
Routing traffic out a specific WANIP, should not interfere with LAN to LAN traffic or VLAN to VLAN traffic which is more controlled by firewall rules.

The reason I can say this is because Vlan to Vlan traffic is more specific than 0.0.0.0/0 and thus the default routes (DAC) created by IP addresses and gateways of existing vlans or subnets will take precedence of the IP route out WAN1 or WAN2. The traffic should simply flow, if your firewall rules allow it.

Maybe I am overlooking something here, so dont quote me as being correct, its just a counter thought at the moment, trying to work through the logic!

Not sure what you mean?
Routing traffic out a specific WANIP, should not interfere with LAN to LAN traffic or VLAN to VLAN traffic which is more controlled by firewall rules.

I may have something mis-configured. Will test and report back.

Of course it can interfere. If you use routing rule saying that internal address X should use specific routing table, which contains only default route to internet, then even traffic from X to other local subnets will be sent to internet according to given routing table. And @pcunite's fix with another routing rule is correct.

Hmmm...
I so badly wanted to state Sob was wrong but after going around in circles...........

Ahhhhhhhhhhh I see your doing it by DESTINATION ADDRESS not by source address!!
Okay, now it makes sense.
Rather smart of you pcunite.............

Also the potential implications of forcing an IP out a specific WANIP cannot be overstated!!
But if you only had read my Para J, "SERVER EXAMPLE"

Ahhhhhhhhhhh I see your doing it by DESTINATION ADDRESS not by source address!!
Okay, now it makes sense.
Rather smart of you pcunite.............

Also the potential implications of forcing an IP out a specific WANIP cannot be overstated!!
But if you only had read my Para J, "SERVER EXAMPLE"

Haha, well the credit goes to Sob. I saw a post he made a while back. That is why collections of information are so helpful. I've spent hours trying to sort all this out. When I get everything done, maybe I'll produce a monster document on how to really setup a multi-wan scenario that covers what most people need. Lots of practical things people need to know. There is a lot to it.

Too difficult to cover every scenario but thats why the current explanation in the user article spans 3 paragraphs I, J, K
At some point in time, we can get together to revamp what is there into something more coherent........ both vlans and the other topics as you wish to progress.
We dont need the other trash talkers, sob mkx etc.... involved because they will be sure to correct all our faux pas, once such a revamp is done and thus we can expect to publish and then make the necessary changes that will come. Its like a running race when you pass the first place runner and are happily in the lead but your strategy involves knowing that others will be kicking by you and thus you have to mentally prepare to up the game shortly! Probably can look at doing this starting in July..... I expect that skype sessions will be in order.

Don't cover every scenario, just explain how it works. It's great about machines, they simply follow exact steps and rules, no moods or whims, tell them what to do and they will do it.

Btw, info about "DUAL WAN - Simple" is wrong. First, switch to ISP2 will happen only if ISP1 fails in certain way, e.g. if DHCP is not able to renew lease. Then the gateway won't be reachable, because client won't have address and subnet that contains gateway. In this case, router will use route to ISP2. But if there's long DHCP lease and client isn't renewing it yet, gateway can be dead as Dodo and router won't notice. That's what check-gateway option is for. Second, when ISP1's route with lower distance becomes available, router will switch to it immediately. In short, you want following "DUAL WAN - FAILOVER", because "DUAL WAN - Simple" is pretty useless.

Done, good clarification that the check ping has nothing to with driving router selection it merely informs the router the route is available and the distance is the driving factor in the example.