ccr 1036 not getting more then 1G connection

Hardware MikroTik hardware general
1

Hello all

I need you help. I am currently using this ccr 1036 model connected to 3 isp. 4g, 512m and another 512m link. I am using the hotspots feature and using the simple queue for each user to rate limit each user. I have about 10000 users on wifi using this service. The problem is I cannot reach the output beyond 1G. I have without using the simple queue I can reach about 3G. Only think traffic passing the simple queue has this problem.

When I monitor the peak it has about 2200 queues and randomly some cpu out of the 36 core hitting 100% for less then 2 seconds and down again. Is there a limitation in the total number of simple queues.

Thanks
Julian

2

How many concurrent users do you have connected. can you export your queue types and advise which ones you are using. Are you using predominantly simple queues or queue trees. Your PCQ configuration in queue type will be important as you would need to adjust the total-limit for the number of concurrent users, if your limit is set to the standard 2000 for instance you would only be able to have 40 users concurrently on that queue type. If you are using simple queues once you have between 600 and 1000 individual queues you are going to start seeing performance issues and it would then be advisable to use queue tree and mangle rules. Alternatively you can have less simple queues but match more traffic.

3

iHello Dgnevans,

The simple queue is injected by the radius each time some one is connected. This is to ensure each client that connects only get 1M of bandwidth. Is there any other method I can set the rate limit instate of using the simple queue? I need to check if I can disable on the radius site to create the simple queue. Hope you can advise how can I have a more general rate limit per user instate of using simple queue.
One more thing about PCQ setting. Rate is how much bandwidth I want to allocate right. Then there is limit and total limit. I understand that total limit/limit = concurrent session. Can you explain what is the limit? And if the total limit is 2000 it take about 2M of Ram. So if for example I want total of 4000 concurrent by doing the math total limit should be 400,000 and limit is 50. So concurrent is 4000 users and would use up about 400M of Ram. Am I correct? Please help me explain what is the limit value for also ya.
Thanks

4

I would try adjust settings in pcq type before editing radius config. Have a look at this link we discussed some of your questions here let me know if you have more http://forum.mikrotik.com/t/pcq-advice/106691/1

5

Hello guys,.

Right now my setting is no simple queue but only queue tree. I have setting as below.

Queue tree
rate is 15M
limit 50 KiB
Total Limit 250000
burst rate 20M
burst time 10 second.

I have about 1500 concurrent users. Once I took out the simple queue customers has been happier. Just want to ask with this setting I would have 5000 concurrent users right. What will happen if I increase the limit 50KiB to another value? Also I notice at peak the CPU is also about 70 to 80%. Is my setting correct? Please advise.

Thanks

6

if you increase the value to higher than 50 it will reduce the number of concurrent users from 5000 down. I believe the cpu reduce should be lower than that with using queue trees. During a peak time use the profile tool to view which process is causing high cpu usage.

7

Hello all,

I just check the tool profile to see which one is using allot of CPU. It looks like the queueing uses allot of CPU. Attach here is he screenshot. I have 2 type of queue tree one is office wifi and the other one is free wifi. Free wifi is a very low limit and it is always hitting its limit 24 by 7. Please advise. Can the free wifi queue cause the cpu to go up?
Screen Shot 2017-04-02 at 20.43.40.png

8

can you post your queue config.

9

This is my queue tree config

name=“Total Download” parent=global packet-mark=“” limit-at=0 queue=default priority=4 max-limit=2500M burst-limit=0 burst-threshold=0 burst-time=0s bucket-size=0.1
name=“Office WiFi Priority4” parent=Total Download packet-mark=Office_WiFi limit-at=0 queue=pcq-download-default priority=4 max-limit=2400M burst-limit=0 burst-threshold=0 burst-time=20s bucket-size=0.1
name=“Free_WiFi” parent=Total Download packet-mark=Free_WiFi limit-at=0 queue=free_wifi_pcq priority=4 max-limit=250M burst-limit=0 burst-threshold=0 burst-time=0s bucket-size=1
name=“Total_upload” parent=global packet-mark=“” limit-at=0 queue=default priority=4 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s bucket-size=0.1
name=“Office_wifi_upload” parent=Total_upload packet-mark=Office_WiFi_Upload limit-at=0 queue=pcq-upload-default priority=4 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s bucket-size=0.1
name=“Free_Wifi_upload” parent=Total_upload packet-mark=Free_WiFi_upload limit-at=0 queue=free_wifi_pcq_upload priority=4 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s bucket-size=0.1


Below is queue type

name=“free_wifi_pcq” kind=pcq pcq-rate=10M pcq-limit=50KiB pcq-classifier=dst-address pcq-total-limit=10000KiB pcq-burst-rate=0 pcq-burst-threshold=0 pcq-burst-time=10s
pcq-src-address-mask=32 pcq-dst-address-mask=32 pcq-src-address6-mask=64 pcq-dst-address6-mask=64

name=“free_wifi_pcq_upload” kind=pcq pcq-rate=10M pcq-limit=50KiB pcq-classifier=src-address pcq-total-limit=10000KiB pcq-burst-rate=0 pcq-burst-threshold=0
pcq-burst-time=10s pcq-src-address-mask=32 pcq-dst-address-mask=32 pcq-src-address6-mask=64 pcq-dst-address6-mask=64

name=“pcq-upload-default” kind=pcq pcq-rate=15M pcq-limit=50KiB pcq-classifier=src-address pcq-total-limit=250000KiB pcq-burst-rate=0 pcq-burst-threshold=0
pcq-burst-time=10s pcq-src-address-mask=32 pcq-dst-address-mask=32 pcq-src-address6-mask=128 pcq-dst-address6-mask=128

name=“pcq-download-default” kind=pcq pcq-rate=15M pcq-limit=50KiB pcq-classifier=dst-address pcq-total-limit=250000KiB pcq-burst-rate=20M pcq-burst-threshold=0
pcq-burst-time=10s pcq-src-address-mask=32 pcq-dst-address-mask=32 pcq-src-address6-mask=128 pcq-dst-address6-mask=128

Is there anything I done wrong?

10

I cannot see any configuration errors. confirm what was the processor usage when you were using simple queues? dud you est your throughput with simple queues and you new queue types?

11

hey bro with simple queue u do see random CPU going 100% with this max is about 60%. I have already taken all my simple right now all traffic coming in are hitting on the queue tree. In the new queue tree I have given each office user 15M burstable to 20M for pct-download-default. And for free wifi its 10M peruser max to 250M only.

12

Queues will become CPU intensive as you get more clients and they become more complicated. The more connection tracking required will require more cpu and you will also require more ram. i have 300 plus users on a ccr1009 never see usage for queues go aboce 1 % second site with similar setup around 600 usrs rarely goes above 2 % 10 simple queues at each. Confirm which software version you running on your router.
You may need to look at having a core router handling your connections coming in and then splitting traffic between 2 routers from there to your guest network and office networks.

13

Hello bro,

The router version is 6.38.5. Have read your suggestion. So it might be the router cannot take the amount of concurrent users? Most of the time there will be about 1400 to 2500 users. I need this router to support up to 20000 concurrent users if possible hehe. Is there a higher Mikrotik router model u can recommend?

Also just want to make sure the configuration is correct? I can just use queue tree without the simple queue right?

Thanks
Yau

14

You can do an export of your config then we can see if there is anything that may be wrong or better ways to do it. I don think there is any reason to use simple queue and queue tree together.

15

Below is the config.

apr/03/2017 09:23:24 by RouterOS 6.38.5

software id = W746-N7NR

/interface bridge
add name=WAN
/interface ethernet
set [ find default-name=ether4 ] comment="ether4 - Firewall Content Filtering (Trunk)" name=Firewall
set [ find default-name=sfp-sfpplus2 ] comment="sfp-sfpplus2 - LAN (Trunk)" name=LAN speed=1Gbps
set [ find default-name=sfp-sfpplus1 ] comment="sfp-sfpplus1 - ISP 1" name=WAN-Out
/interface vlan
add interface=Firewall name="FW to MK" vlan-id=1001
add arp=reply-only interface=LAN name="Free WiFi" vlan-id=122
add arp=reply-only interface=LAN name="Office WiFi" vlan-id=121
add interface=Firewall name="MK to FW" vlan-id=1000
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip dhcp-server option
add code=138 name="EAP Adoption" value="'10.30.10.203'"
/ip hotspot profile
add hotspot-address=10.121.255.254 login-by=https,http-pap,mac-cookie name="Office WiFi" use-radius=yes
add hotspot-address=10.122.255.254 html-directory=freewifi login-by=https,http-pap name="Free WiFi" use-radius=yes
/ip hotspot
add disabled=no interface="Office WiFi" keepalive-timeout=5m login-timeout=5m name="hs-Office WiFi" profile="Office WiFi"
add disabled=no interface="Free WiFi" keepalive-timeout=5m login-timeout=5m name="hs-Free WiFi" profile="Free WiFi"
/ip hotspot user profile
set [ find default=yes ] keepalive-timeout=5m session-timeout=10m shared-users=10000
/ip ipsec proposal
set [ find default=yes ] enc-algorithms=aes-128-cbc
add auth-algorithms=sha256 enc-algorithms=aes-128-cbc name=IPSec_to_DC pfs-group=none
/ip pool
add name="Office WiFi" ranges=10.121.0.1-10.121.255.253
add name="Free WiFi" ranges=10.122.0.1-10.122.255.253
add name=MGMT ranges=10.1.5.160-10.1.5.170
/ip dhcp-server
add add-arp=yes address-pool="Office WiFi" authoritative=yes disabled=no interface="Office WiFi" lease-time=2d name="Office WiFi"
add add-arp=yes address-pool="Free WiFi" authoritative=yes disabled=no interface="Free WiFi" lease-time=2h name="Free WiFi"
add address-pool=MGMT authoritative=yes disabled=no interface=LAN lease-time=5m name=WEMA-MGMT
/queue type
set 4 kind=pcq pcq-burst-rate=15M pcq-classifier=dst-address pcq-limit=100KiB pcq-rate=10M pcq-total-limit=4000000KiB
add kind=pcq name=free_wifi-pcq_download pcq-classifier=dst-address pcq-dst-address6-mask=64 pcq-rate=6144k pcq-src-address6-mask=64 pcq-total-limit=10000KiB
add kind=pcq name=free_wifi-pcq_upload pcq-classifier=src-address pcq-dst-address6-mask=64 pcq-rate=6144k pcq-src-address6-mask=64 pcq-total-limit=10000KiB
set 7 pcq-rate=6144k pcq-total-limit=250000KiB
set 8 pcq-burst-rate=25M pcq-rate=20M pcq-total-limit=250000KiB
/queue simple
add name="Free wifi" queue=free_wifi-pcq_upload/free_wifi-pcq_download target="Free WiFi"
add disabled=yes name=Office_Wifi priority=4/4 queue=pcq-upload-default/pcq-download-default target="Office WiFi" total-queue=pcq-download-default
/queue tree
add max-limit=4G name="Total Download" parent=global queue=default
add max-limit=3500M name="Office WiFi Priority4" packet-mark=Office_WiFi parent="Total Download" priority=4 queue=pcq-download-default
add max-limit=250M name="Free WiFi Priority8" packet-mark=Free_WiFi parent="Total Download" queue=pcq-download-default
add name="Total Upload" parent=global priority=4 queue=default
add name=Office_Wifi_Upload packet-mark=Office_WiFi_Upload parent="Total Upload" priority=4 queue=pcq-upload-default
add name=Free_Wifi_Upload packet-mark=Free_WiFi_Upload parent="Total Upload" priority=4 queue=free_wifi-pcq_upload
/interface bridge port
add bridge=WAN interface=ether7
add bridge=WAN interface=ether8
add bridge=WAN interface=WAN-Out
add bridge=WAN interface=ether1
/ip address
add address=10.1.5.254/23 comment=MGMT interface=LAN network=10.1.4.0
add address=10.121.255.254/16 comment="Office WiFi" interface="Office WiFi" network=10.121.0.0
add address=10.122.255.254/16 comment="Free WiFi" interface="Free WiFi" network=10.122.0.0
add address=xxx.xxx.xxx.xxx/30 comment="ISP 1" interface=WAN network=xxx.xxx.xxx.xxx
add address=192.168.100.2/24 comment="Free Wifi to Firewall" interface="MK to FW" network=192.168.100.0
add address=192.168.101.1/24 comment="Firewall to WAN" interface="FW to MK" network=192.168.101.0
add address=xxx.xxx.xxx.xxx.37.97/29 comment="ISP 1 - Public LAN" interface=WAN network=xxx.xxx.xxx.xxx
/ip cloud
set ddns-enabled=yes
/ip dhcp-server network
add address=10.1.4.0/23 dhcp-option="EAP Adoption" gateway=10.1.5.254
add address=10.121.0.0/16 dns-server=10.121.255.254 gateway=10.121.255.254 netmask=16
add address=10.122.0.0/16 dns-server=10.122.255.254 gateway=10.122.255.254 netmask=16
/ip dns
set allow-remote-requests=yes servers=8.8.8.8,8.8.4.4
/ip firewall filter
add action=drop chain=input comment="Drop DNS request from WAN interface" dst-port=53 in-interface=WAN protocol=udp
add action=drop chain=input dst-port=53 in-interface=WAN protocol=tcp
add action=accept chain=input comment=IPSec-ESP dst-port=500 protocol=udp src-port=500
add action=accept chain=input comment=IPSec-ESP protocol=ipsec-esp
add action=accept chain=output comment=IPSec-ESP protocol=ipsec-esp
add action=jump chain=forward comment="Detect DDos Attack and Drop DDoS Traffic" connection-state=new disabled=yes jump-target=detect-ddos
add action=return chain=detect-ddos disabled=yes dst-limit=32,32,src-and-dst-addresses/10s
add action=add-dst-to-address-list address-list=ddosed address-list-timeout=10m chain=detect-ddos disabled=yes
add action=add-src-to-address-list address-list=ddoser address-list-timeout=10m chain=detect-ddos disabled=yes
add action=drop chain=forward connection-state=new disabled=yes dst-address-list=ddosed src-address-list=ddoser
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment=" Port scanners to list" in-interface=WAN protocol=tcp psd=21,3s,3,1
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="NMAP FIN Stealth scan" in-interface=WAN protocol=tcp tcp-flags=fin,!syn,!rst,!psh,!ack,!urg
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="SYN/FIN scan" in-interface=WAN protocol=tcp tcp-flags=fin,syn
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="SYN/RST scan" in-interface=WAN protocol=tcp tcp-flags=syn,rst
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="FIN/PSH/URG scan" in-interface=WAN protocol=tcp tcp-flags=fin,psh,urg,!syn,!rst,!ack
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="ALL/ALL scan" in-interface=WAN protocol=tcp tcp-flags=fin,syn,rst,psh,ack,urg
add action=add-src-to-address-list address-list="port scanners" address-list-timeout=2w chain=input comment="NMAP NULL scan" in-interface=WAN protocol=tcp tcp-flags=!fin,!syn,!rst,!psh,!ack,!urg
add action=drop chain=input comment="Drop port scanners" in-interface=WAN src-address-list="port scanners"
add action=passthrough chain=unused-hs-chain comment="place hotspot rules here" disabled=yes
/ip firewall mangle
add action=mark-routing chain=prerouting comment="Free WiFi route to Firewall (Content Filtering)" new-routing-mark=to_FW passthrough=no src-address=10.122.0.0/16

inactive time

add action=mark-packet chain=postrouting comment="Mark Free WiFi Download traffic (Sunday)" new-packet-mark=Limit_Free_WiFi out-interface="Free WiFi" passthrough=no time=0s-1d,sun

inactive time

add action=mark-packet chain=prerouting comment="Mark Free WiFi Upload traffic (Sunday) " in-interface="Free WiFi" new-packet-mark=Limit_Free_WiFi_Upload passthrough=no time=0s-1d,sun

inactive time

add action=mark-packet chain=postrouting comment="Mark Free WiFi Download traffic (Monday-Saturday 00:00 to 09:00)" new-packet-mark=Limit_Free_WiFi out-interface="Free WiFi" passthrough=no time=
0s-9h,mon,tue,wed,thu,fri,sat

inactive time

add action=mark-packet chain=prerouting comment="Mark Free WiFi Upload traffic (Monday-Saturday 00:00 to 09:00)" in-interface="Free WiFi" new-packet-mark=Limit_Free_WiFi_Upload passthrough=no time=
0s-9h,mon,tue,wed,thu,fri,sat
add action=mark-packet chain=postrouting comment="Mark Free WiFi Download traffic (Monday-Saturday 09:00 to 19:00)" new-packet-mark=Free_WiFi out-interface="Free WiFi" passthrough=no time=
9h-19h,mon,tue,wed,thu,fri,sat
add action=mark-packet chain=prerouting comment="Mark Free WiFi Upload traffic (Monday-Saturday 09:00 to 19:00)" in-interface="Free WiFi" new-packet-mark=Free_WiFi_Upload passthrough=no time=
9h-19h,mon,tue,wed,thu,fri,sat

inactive time

add action=mark-packet chain=postrouting comment="Mark Free WiFi Download traffic (Monday-Saturday 19:00 to 24:00)" new-packet-mark=Limit_Free_WiFi out-interface="Free WiFi" passthrough=no time=
19h-1d,mon,tue,wed,thu,fri,sat

inactive time

add action=mark-packet chain=prerouting comment="Mark Free WiFi Upload traffic (Monday-Saturday 19:00 to 24:00)" in-interface="Free WiFi" new-packet-mark=Limit_Free_WiFi_Upload passthrough=no time=
19h-1d,mon,tue,wed,thu,fri,sat
add action=mark-packet chain=postrouting comment="Mark Office WiFi Download traffic" new-packet-mark=Office_WiFi out-interface="Office WiFi" passthrough=no
add action=mark-packet chain=prerouting comment="Mark Office WiFi Upoload traffic" in-interface="Office WiFi" new-packet-mark=Office_WiFi_Upload passthrough=no
/ip firewall nat
add action=accept chain=srcnat comment="No NAT for ISP1 Public LAN addresses" src-address=xxx.xxx.xxx.xxx/29
add action=accept chain=srcnat dst-address=xxx.xxx.xxx.xxx/29
add action=accept chain=srcnat comment="IPSec VPN - From WEMA-MGMT to Kiwire Network" dst-address=10.20.10.0/24 out-interface=WAN src-address=10.1.4.0/23
add action=accept chain=srcnat comment="IPSec VPN - From WEMA-MGMT to EAP Controller Network" dst-address=10.30.10.0/24 out-interface=WAN src-address=10.1.4.0/23
add action=dst-nat chain=dstnat comment="SSH port forward to network switches" dst-address=xxx.xxx.xxx.xxx dst-port=10200 protocol=tcp to-addresses=10.1.5.200 to-ports=22
add action=dst-nat chain=dstnat dst-address=xxx.xxx.xxx.xxx dst-port=10201 protocol=tcp to-addresses=10.1.5.201 to-ports=22
add action=dst-nat chain=dstnat dst-address=xxx.xxx.xxx.xxx dst-port=10202 protocol=tcp to-addresses=10.1.5.202 to-ports=22
add action=dst-nat chain=dstnat dst-address=xxx.xxx.xxx.xxx dst-port=10203 protocol=tcp to-addresses=10.1.5.203 to-ports=22
add action=dst-nat chain=dstnat dst-address=xxx.xxx.xxx.xxx dst-port=10204 protocol=tcp to-addresses=10.1.5.204 to-ports=22
add action=dst-nat chain=dstnat dst-address=xxx.xxx.xxx.xxx dst-port=10205 protocol=tcp to-addresses=10.1.5.205 to-ports=22
add action=dst-nat chain=dstnat dst-address=xxx.xxx.xxx.xxx dst-port=10206 protocol=tcp to-addresses=10.1.5.206 to-ports=22
add action=dst-nat chain=dstnat dst-address=xxx.xxx.xxx.xxx dst-port=10207 protocol=tcp to-addresses=10.1.5.207 to-ports=22
add action=dst-nat chain=dstnat dst-address=xxx.xxx.xxx.xxx dst-port=10208 protocol=tcp to-addresses=10.1.5.208 to-ports=22
add action=dst-nat chain=dstnat dst-address=xxx.xxx.xxx.xxx dst-port=10209 protocol=tcp to-addresses=10.1.5.209 to-ports=22
add action=dst-nat chain=dstnat dst-address=xxx.xxx.xxx.xxx dst-port=10210 protocol=tcp to-addresses=10.1.5.210 to-ports=22
add action=dst-nat chain=dstnat dst-address=xxx.xxx.xxx.xxx dst-port=10211 protocol=tcp to-addresses=10.1.5.211 to-ports=22
add action=masquerade chain=srcnat out-interface=WAN
add action=passthrough chain=unused-hs-chain comment="place hotspot rules here" disabled=yes
add action=masquerade chain=srcnat comment="masquerade hotspot network" src-address=10.121.0.0/16
add action=masquerade chain=srcnat comment="masquerade hotspot network" src-address=10.122.0.0/16
add action=masquerade chain=srcnat comment="Allow FW to MK traffic go to internet" src-address=192.168.101.0/24
/ip hotspot ip-binding
add address=10.121.0.0/16 server="hs-Office WiFi"
add address=10.122.0.0/16 server="hs-Free WiFi"
add address=0.0.0.0/0 type=blocked
/ip hotspot walled-garden
add comment="place hotspot rules here" disabled=yes
/ip hotspot walled-garden ip
add action=accept disabled=no dst-address=10.20.10.203
add action=accept disabled=no src-address=10.20.10.203
/ip ipsec peer

Unsafe configuration, suggestion to use certificates

add address=xxx.xxx.xxx.xxx/32 dpd-interval=10s enc-algorithm=aes-128 exchange-mode=aggressive hash-algorithm=sha256 nat-traversal=no secret=xxxxxxxxxx
/ip ipsec policy
add dst-address=10.20.10.0/24 level=unique proposal=IPSec_to_DC sa-dst-address=xxx.xxx.xxx.xxx sa-src-address=xxx.xxx.xxx.xxx src-address=10.1.4.0/23 tunnel=yes
add dst-address=10.30.10.0/24 level=unique proposal=IPSec_to_DC sa-dst-address=xxx.xxx.xxx.xxx sa-src-address=xxx.xxx.xxx.xxx src-address=10.1.4.0/23 tunnel=yes
/ip route
add check-gateway=ping distance=2 gateway=192.168.100.1 routing-mark=to_FW
add check-gateway=ping distance=1 gateway=xxx.xxx.xxx.xxx
add distance=1 dst-address=10.20.10.0/24 gateway=LAN
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www port=10080
set ssh port=4101
/lcd
set enabled=no touch-screen=disabled
/radius
add address=10.20.10.203 secret=xxxxxxx service=hotspot timeout=3s
/radius incoming
set accept=yes
/system clock
set time-zone-name=Asia/Singapore
/system identity
set name=WEMA-MK
/system ntp client
set enabled=yes primary-ntp=218.186.3.36 secondary-ntp=203.174.83.202
/tool graphing interface
add interface=WAN
add
add interface="Office WiFi"
add interface="Free WiFi"
/tool graphing resource
add

16

Looking at your config, What is the purpose of bridging your wan interfaces?
With the traffic you have you have 2 seperate vlans. Free Wifi and Office wifi. Are all users with the vlans treated the same as far as bandwidth? do the speeds alter at different times of the day or night?

17

Hey bro I am not sure why the bridge was configured. It was there when I took over this router. We only alternate the free wifi day and night. For the office wifi they are treated with the same bandwidth for all.

18

Is traffic flowing through all 3 links currently?

19

Traffic flowing 2 links at any point of time.

20

Are the 2 links the 512 links through the same isp.