I have inherited a network that someone else setup without any documentation and things are not all working properly. To make things a little harder on me the network is an hour drive away and has poor cell service and poor internet so working from there is a pain. I have a complete second set of all the Mikrotik hardware used so I am setting up a mock-up of that network in my office to get it working and then I will take that onsite and replace what is there.

The network is setup in 3 buildings with fiber running between them. The fiber goes from the main building to building #2 then from there to building #3. Internet comes in at the main building.

Each building has a Mikrotik Routerboard (RB 201 1UiAS-RM) & a Mikrotik Cloud Router Switch (CRS226-24G-25+RM) all of which are running RouterOS v6.28. I have been using the RouterOS webpage for configuring and looking at the configs. However I was only able to grab the current configs from the Routerboards as the CRS's all had non-default passwords on them so I am trying to build those from scratch.


In that diagram where it says Users this will mostly be Phones, Tablets & Laptops. Where it says Servers will mostly be network connected devices like TVs and HVAC controls and stuff like that. There are not really any application servers or anything.

What I am looking to do is get each building to be able to talk to the other networks and get out to the internet.

I would prefer not to mess with the routerboard setups if possible as that stuff seems to be working and since I am new to RouterOS I don't want to break anything.


I have been able to get the DHCP working on each network but then they cannot get out to the internet. I am fine with starting from scratch with the CRS's if that is easiest. If anyone can give me some things to try I would appreciate it. I am new to ROS but I am somewhat familiar with Cisco IOS and most other networking equipment.


Thanks!

If you want some real help, you will need to post the RouterBoard configs. We have no idea how it is configured from your message. We just have a general idea which cable is plugged into which device. We don't even know which ports are used to make the connections. Your network diagram seems to leave quite a bit of potential performance on the table by connecting the buildings through the RB2011s rather than the CRS226's 10Gbps ports. You may have a need to use more than 1Gbps between the three buildings. In which case your layout wouldn't hurt you.

Whoever set them up may be using VLANs or other features to achieve specific goals.

Knowing that you have fiber and have 2 10Gbps fiber ports in each of the CRS226-24G-2S+RM devices, I would be using the SFP+ ports in the switches to connect the buildings together and would get rid of two of the RB2011UiAS-RM devices. Having the three RB2011s in the network layout you posted might be advantageous, vs one RB2011 at the main building, if the users in each building need to talk, mostly, to the servers in the same building and need 100 - 300 Mbps for that traffic. If most of the traffic goes to the Internet, and the Internet is poor enough you don't want to build the config in place, an RB2011 will be more than sufficient.


If you want the buildings in separate subnets, I would use VLANs on the Main building router and VLAN configuration of the switches at each location to have 3 to 6 LANs, all of which terminate at the main building router. The RB2011UiAS-RM can handle routing for about 100 - 300 Mbps of traffic, depending on the configuration. If that turns out to be too much work for the RB2011UiAS-RM, a CCR01009-8G-1S will cost around as much as three RB2011UiAS devices and give you a lot more horsepower. A CCR1009 with an S+ slot would allow it to talk to your LAN at 10Gbps, if more than 1 Gbps throughput would be useful in your situation.

I agree, unless you need the 10gb ports for specific endpoints at each location, use them to create high speed VLAN trunks between the buildings. If the RB2011s provide enough routing performance for your application, I would actually repurpose two of them and use VRRP at your primary site to provide routing redundancy. A CCR1009 would be a great replacement for the RB2011s and provide a lot more routing performance without being cost prohibitive.

What is the best way to post the configs for the RB's? I have the .backup files of them or do whatever to get it another way.

This is for a remote cabin and supporting buildings. The internet is pretty poor, it is Wireless Internet. I doubt there will ever be more than 20 people total at the site and most will not be on the internet as they will be hunting & fishing. They really cannot get anything better for internet in the area and slow internet will not be a problem as it is expected.

The Main building is a permanent residence that has no more than 4 people at it and only part of the year. The 2nd building is where the "guests" will be when they are not out on the property. And the 3rd building is a maint. building which will have a couple maint guys at it. So speed should not really be an issue.

I also do not want to change any of the equipment. I am basically wanting to get it working and then I will be handing it off to someone else. I want to make it work. Backup the configs (both on the routers and to a flash drive or two) and then show the guys how to restore the configs if needed.

Currently internet works to the main building and to building #2 and everything is fine. It does not work to the 3rd building and I think the person who was setting it up quit before that was finished. However since I cannot get into the CRS's I cannot even try to copy what they have setup in there so far.

I am ok for doing the Vlan part you talked about if that is a better way I just don't know how to do that. Like I said I am trying to learn all this as I go.

Thanks for the help.

Ok I think I found the proper way to export the configs. I have changed the outside IP addresses but left everything else the same.

Like I said before the Router #3 does not work so I don't know if anything is right in that one. Main & router #2 are currently working.

http://wiki.mikrotik.com/wiki/Manual:Co ... Management

Don't bother with "backup" files. Use the export. It's safer. The commands in the export will closely match the menu structure of Winbox. You can read the file and look at winbox to help you learn the commands.

The export is plain text. Edit out any sensitive information. Replace the first two octets of public IPs with letters. Be consistent. Do not edit RFC1918 type addresses. Paste the edited configs into Code blocks in here.

We sell wireless Internet access across 100s of square miles in hilly, wooded terrain. It takes four hours to drive the highways from one end of our territory to the other. We sell up to 20Mbps plans which work. We sell bigger pipes if customers are willing to pay for them. "Wireless" is not synonymous with "pretty poor".

One 2011 will have plenty of horsepower for your setup. With so few devices, the next question is, do you even want to divide up the LANs?

You may need to change your password for your noip account......

^^^ Hmm I missed that in there. I changed it. I don't think it is being used at all anyways.


I hope doing the attachments is ok as I posted that before I saw your reply.

I guess it doesn't matter to me about dividing the LANs. I just didn't want it to run out of DHCP addresses if there are a lot of devices or something.


My comment about the internet was just about what is available where this place is. What they are using is the only option other than satellite and it is pretty bad. There isn't even Cell service on the property unless you drive about a 1/2 down the road to a large hill. Then your lucky to be able to make a call.

It looks like someone set out to build a Rube Goldberg network, then said, "the heck with it", and bridged the whole thing without bothering to clean up the mess. https://www.rubegoldberg.com

I'd dump the bath water and not worry about if the baby is still in the tub. What a mess.

You're fastest fix is likely to connect Main Router ether1, or sfp1 to Main switch any port, Main switch fiber port to second switch fiber port, second switch fiber port to third switch fiber port, and turn off the second and third routers. You can likely default the configuration of all three switches without danger (especially on your test network). Give them management IP addresses starting from 192.168.x.10 - 192.168.x.12. Make sure they are configured to be pure switches.

Your main (and only router) will be 192.168.x.1. If you want to VRRP the spare routers for redundancy at the main location, you can put them in 192.168.x.3 and 192.168.x.4. But I'd probably just configure them as cold spares with duplicates of the configuration of the primary router so that if something goes wrong, whoever is on-site can unplug one and plug in the next spare. I don't imagine 20 minutes of downtime is going to cause significant issues for your setup. Once you understand what you're doing you can complicate things and add-in automatic failover.

You are probably better off to default the config of the main router, use quickset to get the IP range you want on it. Then add in port forwards under IP Firewall NAT. Possibly the server inside will end up with a different IP, so you may need to change the rules to take that into account. You may want to make the main LAN subnet match the existing server IP addresses, if they are all in the same subnet, just to avoid having to change static IPs on the servers.

Add the no-ip script if you're using it. Easier would be to enable the IP cloud functionality. At that point, everyone should be able to get Internet.

Make sure your admin user password is REALLY good. Do the same for any other system users. The no-ip password did not qualify as good, maybe decent.

I would add a rule to the input chain, just before the drop rule, to allow your remote IP address(es) to manage the router from offsite. If you have multiple addresses from which you may want to manage the router, you can put them all in an ip firewall address-list and use src-address-list in your input chain allow rule.

I would enable L2TP/IPsec VPN so you can directly attach to everything up there from offsite. Just checking the box in quickset and adding a REALLY good password, different from your MikroTik admin password, will probably be sufficient to configure the router. It's easy, so just do it. It will save your bacon some day.

I will guess that this is all probably about 30 - 90 minutes worth of config time for a newbie. :-) It's even less config time if you just let it use 192.168.88.1/24 on the internal LAN.

Great thanks for the reply. A few questions:

I am fine with defaulting everything and making it as simple as possible. I thought they were using way too much equipment as it is.

Do I need to do anything specific to make the connections with the fiber daisy-chaining the 3 switches? Or just connect them with the SFP ports and they will work after being defaulted?

Make sure they are configured to be pure switches.

How do I do this?

I would add a rule to the input chain, just before the drop rule, to allow your remote IP address(es) to manage the router from offsite. If you have multiple addresses from which you may want to manage the router, you can put them all in an ip firewall address-list and use src-address-list in your input chain allow rule.

I assume you mean so I can connect to the router from my office? If so then yes that may be a good idea. What does a rule for that look like?

I would enable L2TP/IPsec VPN so you can directly attach to everything up there from offsite. Just checking the box in quickset and adding a REALLY good password, different from your MikroTik admin password, will probably be sufficient to configure the router. It's easy, so just do it. It will save your bacon some day.

Is this also for administering the router? Or so I can get to other systems inside the LAN? I won't be needing to get to any other systems and most likely won't need to get into the routers later but it will be nice to have that option.




Thanks again for the help.

I tried to figure out what you said but it is not working.

I reset the RB and 3 Switches to defaults.

I gave the main router the IP address of 192.168.3.1 and setup the DHCP to give out 3.10-3.254. That works if I plug into the RB.

I then am going from ETH2 on the RB to ETH1 on the first switch.

On the Quick Set page for that switch I have set as 192.168.3.2, Mode Router (should this be Bridge?) I have a static IP of 192.168.3.2 with a gateway of 192.168.3.1 and a Local Network IP of 192.168.3.2.

If I plug into that 1st switch I get a proper IP address (192.168.3.254) and can still get into both the config pages for the router and the switch.

I then added a 2nd switch for the 2nd building by going from the SFP2 on the 1st Switch to the SFP1 on the 2nd. I plan on going from the SFP2 on 2nd to SFP1 on 3rd switch as that will be how the building fiber is ran.

When I setup the next switch basically the same way as the first except using 192.168.3.3 for the IP I do not get a DHCP address. Even if I manually set it to a 192.168.3.x address I cannot get to the 3.1 or 3.2 config pages.

Ok figured out that if I take out the Fiber cable and just jump from eth to eth then the DHCP and stuff works. Why won't it work through the SFP port? I have tried different transponders and cables. Do I need a straight through fiber cable? Is the problem that I am doing this in a lab without the actual fiber run between buildings and I am just using a fiber cable going from one SFP to the other? I looked at the cables I have and they are crossed over from one end to the other.



Edit - I found a cable that I could swap the ends around on and that still didn't help. I suspect that I am missing something else to allow it to connect with the SFP port as the second I plug a jumper back in connecting them via Ethernet cable everything works. My problem is that I need the SFP ports to work as the only connection between buildings is the fiber.

Unfortunately, MikroTik's default config is different for various hardware models. I don't have any of the CRS2xx series devices. I'm not sure what their default config is.

I would guess you want your switches in quickset "bridge" mode. If that doesn't immediately work, double click each interface and make sure they all use the same master port. Make sure the master-port is the only interface with the master-port setting set to none.

Are your SFP devices 1.25Gbps or 10Gbps? You may have to configure the 10Gbps SFP slots on the CRS to run at the 1.25Gbps speed for them to recognize and use the 1G SFPs.

Ok that worked. Mostly anyways. The transceivers I have are all 1.25G so switching the ports to that worked except this model of CRS the SFP+2 ports are 10G ONLY. So I assume I need new transceivers if I want to use those. Which I will need to use them to make the jump from 2nd to 3rd building.

I do not know what kind of cable is run between the buildings. Will it still work if the cable is only rated for 1G if I use the 10G transceivers? I understand that if that is the case that the speed would be capped at 1G but I am fine with that.

I think that might be why they were using the RBs in the mix so they only had to use the 1 SFP port on each of the CRSs.

That's a good possibility for why they had the extra rb2011s.

We can still keep it a flat network and use the rb2011s. Just turn the second and third rb2011s to bridge mode. Make sure the sfp interface uses, or *is* the master port used by the 5 gigabit copper ports. We don't want any "gateway" ports running independent of the switch. You can have the master port of the first 5 ports and the master port of the second five 10/100 ports in the same bridge if you want.

The fiber is likely to be fast enough for 10 gbps, but it's not guaranteed to be. Remember to watch our for single-mode vs multi-media fiber / optics compatibility if you decide to try 10 gbps SFP modules.

I seem to have everything working except for being able to connect via VPN to the mynetname.net address but searching online it looks like that is no longer available. I will look more into that later.

I ended up just using the RBs in the mix like they had it setup and basically will have a backup for each RB & CRS they can have setting on the shelf to swap in if needed.

Thanks again for all your help.

The mynetname.net option works for me. I just connected to a client's router on the other side of the country using their IP cloud address.

Make sure you've enabled IP cloud. Of course if you have a static IP out there, you can always just put that into your VPN client.

I'm glad you have it working. I hope the configuration is a bit simpler and easier to understand/manage now.