Good day everyone, first time user here.

Setting up a new (US-based) location and not able to move most (any?) of my network equipment with me, so if someone could do me a favor and check over this and see if I'm missing something (other than...you know...most of my marbles) or have the wrong idea, etc.
(If this is the wrong area, let me know...just seemed like the best)


Basically this will be a 90% Mikrotik network setup.

Normally I'm a Cisco (commence boo and hiss) or Sonicwall (pardon while I boo and hiss) guy, so I want to make sure I am not missing anything.


Router
CCR1009-7G-1C-1S+
DSL Modem
Draytech Vigor 130 (gateway/DSL-modem-only mode)
Firewall
Custom-built pfSense
Switches

• Dell 5548P (main)
• CRS112-8P (WAP PoE)
• CRS328-24P-4S+RM (x2) (interconnect between buildings)

Access Points

• cAP ac (x3) (main & secondary building)
• hAP ac (x2) (secondary & tertiary building)
• wAP ac (x2-4) (outdoor)

CAT6A to Fiber conversion
Perle CM-1110-SFP (x4) (30x convertor modules from main building)
CCTV System
Lorex HDIP3232W

That's the basic portion of it. I've tried to keep it simple.

Basic notes:

• I hate most firewall systems, pfSense at least gives me some options.
• VPN access is important-ish. Using Open-VPN (cause other options either suck or are way too expensive) for about 3 users
• CCTV is near vital, the system used has been used in a similar location and works.
• Location is using ADSL2/VDSL (Centurylink?) with around 12 mbps down and 1.5 mbps up
• There will be 10+ subnets for this location

I think this is all I am going to need, anyone see anything additional needed?

Too much devices to do simple and easy work. Especially from the wan speed point of view.

If you would like to have ovpn udp support, forget the mikrotik as endpoint. Ovpn implementation supports tcp only.

Good day Jarda

Too much devices to do simple and easy work. Especially from the wan speed point of view.

A concern, yes.
The thought is that if either of the providers (2 at the moment and a new one or two should come in [at max 5 years from now]) get off their collective hindquarters and give the location some sort of 100 mbps (Fibre) like they should this makes a lot of sense at that point.
Even without, there are a lot of devices, but kind of have to have them. The separate firewall is to make setting it up easier (due to office foolishness and playing the "what route goes to where" game) since I can just export and import rules and then tune any that need to messed with. Most of the local servers are going to be doing an overnight sync to get requisite data from HQ, weekend full sync. Connection across 5 acres of land. A lot of devices bounding around.
I'm not really fond of it, but after telling the Cisco side of me to shut up it makes sense.

If you would like to have ovpn udp support, forget the mikrotik as endpoint. Ovpn implementation supports tcp only.

I know, but that's all I am counting on. TCP means they cannot complain about their connection going down without it recording it in the logs.
And, heavens, is THAT excuse line the one that one of the possible VPNers is going to try at the end.of.every.freaking.month.

Nothing against, I just warned you. Too many things mean too many points to configure something, to look at if whatever does not work, just useless complexity. Also buying a ccr for that use is really overkill, something like rb750gr3 wil be able to play its role even after your 4 ISPS start to provide you 150Mbit/s connectivity. With the near future expectations you would not need a separate router because the switches (like crs125 or crs328) are able to provide you enough routing/natting performance. And when they will be out of breath, you just put rb750gr3 in front of the switches and let the switches to do switching only. Another firewall (virtualised?) is also someting above you do not need. On the other hand, if you have virtualisation environment, you can run CHR as router side by side to your firewall and connect it to the real world thru the manageable switch. Without any other physical device.

Try to keep everything simple, it will reward you.

Nothing against, I just warned you. Too many things mean too many points to configure something, to look at if whatever does not work, just useless complexity. Also buying a ccr for that use is really overkill, something like rb750gr3 wil be able to play its role even after your 4 ISPS start to provide you 150Mbit/s connectivity. With the near future expectations you would not need a separate router because the switches (like crs125 or crs328) are able to provide you enough routing/natting performance. And when they will be out of breath, you just put rb750gr3 in front of the switches and let the switches to do switching only. Another firewall (virtualised?) is also someting above you do not need.

The CCR is there for three reasons.

• My original comparison point was the Cisco 45xx router with some stuff thrown in. That is when someone threw a fit because of the number of Cisco contracts they would have...so to other options I went.
• Evidently it was made a corporate thing somewhere somehow that all important network equipment had to be rackmounted and $300+. As such that took the hEX out of the running right off. I was given...unpleasant...looks when I mentioned the hEX.
• If/when the ISPs get their stuff together and start running decent speed to the location, I need the router able to handle different possible connections. One of the ISP likes to basically run straight SFP/SFP+ to the customer.

So, you can see why, dealing with stupid decisions all over the place, I chose the router I did.

As I said, I want the other firewall because at some point in the past someone, somewhere at this organization put in the most stupid rulesets and backwards routing ever. I have mitigated a bit of it, but the main office is still run by someone who thinks that system is just genius.
As such I have to follow their belief...since I cannot rebuild the entire system I am using their stupid firewall system. I don't really want to, but it is far, far, far easier than either rebuilding all of the system myself (then probably having to go around the world to set it up at other locations...) or figuring out how to get their asinine rules to fit in the Mikrotik. They want to increase a few things by sizable percentages it becomes something I would be willing to do...but they do not and, as such, goofy firewall will be purchased and installed (and likely virtualized).

On the other hand, if you have virtualisation environment, you can run CHR as router side by side to your firewall and connect it to the real world thru the manageable switch. Without any other physical device.

Try to keep everything simple, it will reward you.

CHR is able to do production level environment? Truly and completely?
Bless you Jarda, you may have just made my life substantially easier.

Ok. I understand your constraints, but under them the technical discussions are loosing their importance. At least I pointed at CHR that really works, and I would say sometimes even much better than any routerboard because you can dedicate really powerful cores with big cache which is much better to run routeros than many relatively weak cores. You need to try. The plus is that you can try CHR for free also with p1, p10 or unlimited license. Actually you can use any other routing operating system virtually if it fits your needs better.

I will admit I am new to Mikrotik and RouterOS but you are using a lot of Mikrotik devices with built in firewalls. Use those instead of adding in a separate firewall device. How big an area do these buildings cover? What are the buildings made of? It might be possible to do the entire things with one device, depending on the buildings. But even if you have to scatter AP all over the place, you seem to have a lot of extra pieces you may not need. A map with device locations and connections would be helpful to determine if you are missing pieces or if you are going way overboard.

Ok. I understand your constraints, but under them the technical discussions are loosing their importance. At least I pointed at CHR that really works, and I would say sometimes even much better than any routerboard because you can dedicate really powerful cores with big cache which is much better to run routeros than many relatively weak cores. You need to try. The plus is that you can try CHR for free also with p1, p10 or unlimited license. Actually you can use any other routing operating system virtually if it fits your needs better.

That loud continued thumping sound you may have heard earlier today was the sound of my fist/head/etc. repeatedly impacting my desk here. This organization is just slightly too dumb to refer to it as stupid and has informed me that all "network devices" are to be hardware only. Even though I have Cisco virtualized systems running most of their "satellite locations" just behind the wall here. (I deleted about 10-15 sentences of irritated foul-mouthing of the situation, company, etc. To make everyone's lives easier...let me simply state I am extraordinarily irritated)

So...hardware only I go. Now I have to run the CCR and watch my irritation mount. No CHR virtualization...I will be running VMware anyway for a lot of different systems...but alas.

Thank you Jarda, for putting up with this foolishness and helping me to try to find an answer. I hope to use some of it, but if the company decides to tell me I can't use VMware for a bunch of other things I may tell them to take a flying leap.

Let me ask for one simple item more from you...do you see anything I missed? I did not think of anything too terrible but, with how this month has gone, I figured you might see it.

for wifi i suggest to replace hAP AC with cap AC

for outdoor wifi i suggest replacing wAP ac with cAP AC + outdoor enclosure


keep in mind with cAP ac + 48 volt 1.5 amp power supply you can daisy chain 2 accesspoint powering by PoE

I will admit I am new to Mikrotik and RouterOS but you are using a lot of Mikrotik devices with built in firewalls. Use those instead of adding in a separate firewall device. How big an area do these buildings cover? What are the buildings made of? It might be possible to do the entire things with one device, depending on the buildings. But even if you have to scatter AP all over the place, you seem to have a lot of extra pieces you may not need. A map with device locations and connections would be helpful to determine if you are missing pieces or if you are going way overboard.

Hello Hoov,

This is the best that I have at the moment. Total land area is something like 12 hectares, with distance between buildings being not less than 300 or so metres (with exception of two). As far as I know (last update: 6 hours ago) there are to be 4-5 buildings total. The issue is that this site should basically be headquarters v.2.

Main Building (roughly)


Entire place (very roughly)


There's another building or two there, but at least two of them are really close and thus the distance does not require fibre.

The firewall issue is a combination of things. Personally I want a single device so if something is wrong with it I can point to a single device and not have to play with everything else. Plus as I've said before the rules are just about the most stupid/goofy things I've ever seen and I suspect they're going to create hell if I try them on anything further than the router/firewall itself.
And understand I have done work at 2 of the largest ISPs in the US, so I can somewhat refer to the routing/firewall rules as stupid with some authority...

This has been, bar none, the most frustrating build-out I have had to go about yet.

for wifi i suggest to replace hAP AC with cap AC

for outdoor wifi i suggest replacing wAP ac with cAP AC + outdoor enclosure


keep in mind with cAP ac + 48 volt 1.5 amp power supply you can daisy chain 2 accesspoint powering by PoE

The hAP AC can be switched out, no issue.

wAP AC for cAP AC...I'd have to see that subjected to some fairly serious weather before I'd be comfortable. If you've got something regarding that I would most certainly appreciate it.

I needed to hold tight when the earthquake initiated by your heavily punching head arrived to me around the globe.

Never mind. Wash the blood from your glasses in order not to see the world so pink and return back to the grey of daily life.

Just do what your employer wants. Buy a lot of hardware and pretend how important and complicated is your work.

I don't think you forget anything, just you would maybe need more accesspoints and smartly distribute them. Also take care of non overlapping channels, mutual interference, reduction of tx power and other common things that will help you to keep good coverage. Capsman might help you to have central management and overview of the actual situation, also the dude might be helpful for you. You can run both on the CCR or in CHR virtually.

Another question is why you think that the firewall should be rather inside than outside of the network.

I needed to hold tight when the earthquake initiated by your heavily punching head arrived to me around the globe.

Never mind. Wash the blood from your glasses in order not to see the world so pink and return back to the grey of daily life.

Just do what your employer wants. Buy a lot of hardware and pretend how important and complicated is your work.

I don't think you forget anything, just you would maybe need more accesspoints and smartly distribute them. Also take care of non overlapping channels, mutual interference, reduction of tx power and other common things that will help you to keep good coverage. Capsman might help you to have central management and overview of the actual situation, also the dude might be helpful for you. You can run both on the CCR or in CHR virtually.

That is rather the plan I am sticking to unless they either lose their minds on the server budget (possible, that hits next Thursday and they are going to be less than happy with the price) or a generous, extraordinarily wealthy, previously unknown family member tosses off and leaves me a sizable inheritance.

Number of access points, as of the last summary I got from our engineering department, should be a fairly stupid number. I do not have a final number yet because I really refuse to listen to hour long dialogues regarding what is "important" and what is not. Essentially we are looking at around 25 (minimum) wAPs..probably 50. A lot, yes, but with them wanting "good coverage" for the entire site of 12 hectacres...it makes sense. Perhaps not good sense, but sense.

Gonna be a headache to manage, between the vendor I have to use and the ridiculous build of CAPsMAN needed to keep everything in line (another reason why I really wanted to use CHR) and realization that a Dude build is likely in my future...

Running CAPsMAN and Dude on a CHR should/would make my life simpler...

Another question is why you think that the firewall should be rather inside than outside of the network.

Because that's how I drew it up.
Ideally, I would take the...thing...out of VM, throw it on a reasonably fast rack server and place it ahead of the router. Overbuilding a touch? Yes.

Should look something like this updated brief diagram (for my own sanity)
*Note* Diagram just shows main office build, does not include runs across location. Partially because it's too confusing.

You know. Everything is possible and everything could have its good reason. There is nothing I can advise more now. I keep fingers crossed for you and hope I can read how the situation evolves. Good luck.