Community discussions

MUM Europe 2020
 
Techally
just joined
Topic Author
Posts: 6
Joined: Tue Aug 07, 2018 5:14 pm
Location: Sapin

Small WiSP (50 Clients, 2 Mountains, 60Ghz, Fiber 600/60) Needs Your Help!!!

Mon Dec 02, 2019 8:06 pm

Preface:
I inherited this network setup when a friend passed away, I realise that it is NOT the ideal setup but it is what it is and as we do upgrades we are trying to rectify bad setups but it is a LIVE (Production) network with approx. 50 clients.
TAG_Overview.jpg
The Network:
Starts at the “OFFICE” (all references to diagram) with a 600/60 fiber router from provider, this unit is in bridge mode fiber to eth1. Gigabit cable connected from this to TestAC WAN port which gets the External IP.

OFFICE:
TestAC – Currently RBD52G-5HacD2HnD – Router with static routes for network, DHCP Server on 192.168.40.0/24 – Firewall with NAT
TAGFiber – 960PGS – DHCP Server 10.99.66.0/24 – PPP Server for 10.99.26.0/24 Clients – Firewall with NAT – Client limiting with Packet Marks and Mangle Rules
TAG4 – SA5? Bridged Sector TAGFiber for 10.99.26.0/24 Clients
LHG60Master – RBLHGG-60ad Master unit (AP) Bridge

Mountain 1:
LHG60Slave – RBLHGG-60ad Slave unit (SLAVE) Bridge
TAGMASTER – PB-960PGS – DHCP Server 10.99.2.0/24 - – PPP Server for 10.99.25.0/24 Clients – Firewall with NAT – Client limiting with Packet Marks and Mangle Rules (15-20 Clients)
PBColin – PB-960PGS – Bridged to TAGMASTER 10.99.2.0/24 - – PPP Server for 10.99.29.0/24 Clients – Firewall with NAT – Client limiting with Packet Marks and Mangle Rules (5-10 Clients)
TAGEE_Link – SXT5AC? – Bridge to TAGMASTER 10.99.2.0/24 to EE_AC Mountain 2
TAG1 – RB922UAGS-5HPacD-NM & am-5ac21-60- Bridged Sector TAGEMASTER to Cients 10.99.29.0/24
TAG2 – RB922UAGS-5HPacD-NM & am-5ac21-60- Bridged Sector TAGEMASTER to Cients 10.99.25.0/24

Mountain 2:
EE_AC – SXT5AC? – Bridge EEPB to TAGMASTER 10.99.2.0/24 Mountain 1
EEPB – PB-960PGS – Bridged to TAGMASTER 10.99.2.0/24 - – PPP Server for 10.99.28.0/24 Clients – Firewall with NAT – Client limiting with Packet Marks and Mangle Rules (15-20 Clients)
TAG3 – SA5? - Bridged Sector EEPB to 10.99.28.0/24 Clients


Problems Occurring:
All units in the Office including TAG4 Sector and all 10.99.26.0/24 Clients are functioning normally. At all points to the sector we have Gigabit speeds and all clients have 20/4 packages functioning fine. BTEST near Gigabit to all units and plug a laptop in and furthest point and run a speedtest.net test 660/58.
LHG60 Link is not aligned perfectly (about 0.4deg off) but link speeds of 2.3Gbps are stable over 1.9Km. BTEST at Mountain 1 other LHG60_Slave 250-320Mbps Laptop with speedtest.net 280/56 Mbps. (Happy to provide Further info on these units as we would probably not have bought these based on OUR real world results)

TAG1 Clients 10.99.29.5 report random (we have done extensive monitoring and it does appear random) speed test of 4-12Mbps on speedtest.net and TV buffering.

TAG2 Clients 10.99.25.5 report occasional speed test of 14-17Mbps on speedtest.net and occasional TV buffering.

EEPB Actual speedtest.net across AC Link 128/49Mbps BTEST to TestAC 180/180 Mountian2

TAG3 Clients here experience daily problems (weirdly during monitoring worse between 4pm – 7pm, possible sunset reflection etc on sectors and AC Link?) 1.9-20/3Mbps.
When the problem is not there all clients are fine with speedtest.net of 20/4Mbps as per their package.

Monitoring:
When the problem is occurring for the clients we can look at all equipment and NO high CPU etc and BTEST results to all units in every combination achieve normal (expected) results.
NOTHING is maxed out at any point?

Upgrade & Questions:
We have purchased a CCR-1009-7G-1C-1S+PC to replace the first link in the chain TestAC 10.99.40.1
I am fairly sure at this point (most customers have various IPTV solutions) that its not necessarily a speed or resource issue but the number of packets at times coupled with BAD config with potential routing issues and broadcast flooding. The various units that are handling the Mangle rules and packet marking & client limiting have got 20-50 firewall rules and 50 Mangle rules each.

I am a qualified network engineer with years of small network experience single subnet type stuff. I only came to Mikrotik & The Dude with inheriting this network 3 yrs ago (love Mikrotik love RouterOS and love scripting).
I have watched a lot of youtube videos, read a hell of a lot and built a Physical test networks and a GNS3 Virtual simulations in attempts to improve the network and track down these issues. As we only plan to add approx. 25 more clients to the network MAX and not all of the clients are here all year round (holiday homes). Our plan was to simplify the network with most equipment as Bridges back to the new CCR at the office (head end I think is the correct term)

The CCR would run DHCP for 10.99.2.0/24 with various bits to help with exchanging into a live network (middle of the night work)
NEW DHCP Server 10.99.99.0/24 for Management Equipment VLAN99 (would replace the existing 10.99.2.0/24 address that equipment currently has)
New DHCP Server 10.99.26.0/24 for Clients VLAN24 (TAG4)
New DHCP Server 10.99.25.0/24 for Clients VLAN25 (TAG2)
New DHCP Server 10.99.29.0/24 for Clients VLAN29 (TAG1)
New DHCP Server 10.99.28.0/24 for Clients VLAN28 (TAG3)
NEW Firewall Filter rules (possibly in lists) to replace all Firewall rules on current equipment
New Mangle and limiting rules to replace the ones on the other equipment reflecting the various NEW VLAN setup

Q1: Firstly is VLAN the way to go?

Q2: Move everything to be handled by the NEW CCR ?

Q3: If Q1 is yes then this is where im falling down, in physical and GNS testing to do VLANS on different versions of RouterOS (not possible to have latest version on all equipment 6.32.5 is the lowest on all sectors)

I can get the VLAN on the CCR with DHCP server, next piece of equipment gets either 10.99.2.0/24 address unless specify VLAN id in which case gets the appropriate VLAN ip address. With a bridged unit that’s fine but my next piece of equipment is a Router TAGFiber 10.99.66.0/24 effectively WAN LAN. In these situations I cant get the VLANS to pass WAN to LAN so the next piece of equipment ends up with a 10.99.66.0/24 address from the old config despite specifying VLAN id’s?

Wow I know that was a lot of reading but ive tried to be as concise as possible (there is various other parts on this network like a 5Ghz backup for the 60Ghz that auto fails over etc but all of these other parts function normally and we are more than capable at supporting and managing the network).
We are only very small 2 person outfit with very limited budget but would really appreciate any helpful advice on how we should proceed and possible assistance.
You do not have the required permissions to view the files attached to this post.
 
paulct
Member
Member
Posts: 303
Joined: Fri Jul 12, 2013 5:38 pm

Re: Small WiSP (50 Clients, 2 Mountains, 60Ghz, Fiber 600/60) Needs Your Help!!!

Tue Dec 03, 2019 8:22 am

Are you running any form of IGP? Or is it all statically routed?
 
Techally
just joined
Topic Author
Posts: 6
Joined: Tue Aug 07, 2018 5:14 pm
Location: Sapin

Re: Small WiSP (50 Clients, 2 Mountains, 60Ghz, Fiber 600/60) Needs Your Help!!!

Tue Dec 03, 2019 10:51 am

All static routing at this point.
I looked at OSPF, MPLS, BGP all seem like complete over kill for a network of this size?
 
paulct
Member
Member
Posts: 303
Joined: Fri Jul 12, 2013 5:38 pm

Re: Small WiSP (50 Clients, 2 Mountains, 60Ghz, Fiber 600/60) Needs Your Help!!!

Tue Dec 03, 2019 2:28 pm

OSPF would make it simpler to manage in the long run.

But first off replace the DHCP / NAT router with a CCR. Also do you have any monitoring of your links to see where the issue lies? e.g snmp CPU / memory / throughput.
You can get a quick instance of observium running to give you a better idea where issues may lie.
 
Techally
just joined
Topic Author
Posts: 6
Joined: Tue Aug 07, 2018 5:14 pm
Location: Sapin

Re: Small WiSP (50 Clients, 2 Mountains, 60Ghz, Fiber 600/60) Needs Your Help!!!

Wed Dec 04, 2019 1:35 pm

Thanks for the comments.
From what ive read yes OSPF would make management easier but Ive zero experience with OSPF, BGP or MPLS so figured this would just add to the unknowns.
This network is very small (from my office I can see all transmitters and pretty much all clients)

So my questions were:
Q1: Firstly is VLAN the way to go?

Q2: Move everything to be handled by the NEW CCR ? (Everything, all DHCP and ALL Firewall/NAT from all devices to the new CCR)

Q3: If Q1 is yes then this is where im falling down, in physical and GNS testing to do VLANS on different versions of RouterOS from one router to another.
If VLAN is YES then is the advice to put everything else basically in bridge mode after the CCR?

As for monitoring I have THE DUDE setup and has live monitoring, Ive also used various ways to monitor specific equipment CPU, Memory and throughput but as far as I can tell None of the equipment ever seems to be maxed out on resources or bandwidth??

Who is online

Users browsing this forum: No registered users and 21 guests