Community discussions

MikroTik App
 
User avatar
benesm1
just joined
Topic Author
Posts: 18
Joined: Fri Oct 08, 2010 11:37 am
Location: Europe/Prague

CCR1016-12G packet loss

Thu Mar 22, 2018 9:20 pm

Hello everybody,
I'm being haunted by a curious bug. I'm using a CCR1016-12G as a transparent firewall, which is inserted between a Cisco WS-3560E-24TD (Cisco IOS Software, C3560E Software (C3560E-UNIVERSALK9-M), Version 15.0(2)SE9, RELEASE SOFTWARE (fc1)) on one side (connected to ether1) and a VC comprising of two Juniper EX4600s (with JUNOS 14.1X53-D44.3) on the other side, connected to the bonding1 interface. I'm plagued with intermittent packet loss. I've set up several measurement machines and I'm using mtr to collect ICMP data, aggregated at 60 seconds (for example: (for i in `seq 1 10080`; do mtr -rwc 60 147.32.82.56 >> mtr.kn.log.2.txt; done).
I can prove without any doubt that the packet loss occurs on the CCR1016. The packet loss can be as bad as 10% in one minute, then everything is OK for several hours... Last week I replaced the CCR with a pfSense based firewall with similar configuration and the packet loss is gone. I took my measurements in a course of two weeks. Today I swapped my CCR1016-12G with my colleague's and it is as bad as the first one. So there must be either be an error in my config, or there is some systemic trouble with the entire CCR series. I'm using ROS 6.40.6. I'm disposed to returning both of my CCRs for a full refund. Please see my config below:

/interface ethernet
set [ find default-name=ether1 ] comment="CCR1016 <-> VIC Cisco" mac-address=00:0C:42:C7:39:BF
set [ find default-name=ether2 ] disabled=yes mac-address=00:0C:42:C7:39:C0
set [ find default-name=ether3 ] disabled=yes mac-address=00:0C:42:C7:39:C1
set [ find default-name=ether4 ] disabled=yes mac-address=00:0C:42:C7:39:C2
set [ find default-name=ether5 ] disabled=yes mac-address=00:0C:42:C7:39:C3
set [ find default-name=ether6 ] disabled=yes mac-address=00:0C:42:C7:39:C4
set [ find default-name=ether7 ] disabled=yes mac-address=00:0C:42:C7:39:C5
set [ find default-name=ether8 ] disabled=yes mac-address=00:0C:42:C7:39:C6
set [ find default-name=ether9 ] disabled=yes mac-address=00:0C:42:C7:39:C7
set [ find default-name=ether10 ] mac-address=00:0C:42:C7:39:C8
set [ find default-name=ether11 ] mac-address=00:0C:42:C7:39:C8
set [ find default-name=ether12 ] disabled=yes mac-address=00:0C:42:C7:39:CA
/interface vlan
add comment=management interface=ether1 name=vlan_209_mgmt vlan-id=209
add comment="spojka VIC" interface=ether1 name=vlan_534_vic_spojka vlan-id=534
/interface bonding
add comment="CCR1016 <-> EX4600" lacp-rate=1sec min-links=1 mode=802.3ad name=bonding1 slaves=ether10,ether11 transmit-hash-policy=layer-3-and-4
/interface list
add name=wan
/ip firewall layer7-protocol
add comment="Block Bit Torrent" name=layer7-bittorrent-exp regexp=\
    "^(\\x13bittorrent protocol|azver\\x01\$|get /scrape\\\?info_hash=get /announce\\\?info_hash=|get /client/bitcomet/|GET /data\\\?fid=)|d1:ad2:id20:|\\x08'7P\\)[RP]"
/queue tree
add disabled=yes name=1_voip packet-mark=voip parent=bonding1 priority=1
add disabled=yes name=1_out_voip packet-mark=voip parent=vlan_534_vic_spojka priority=1
/queue type
add kind=sfq name=sfq
/queue tree
add disabled=yes name=8_dmz_in packet-mark=no-mark parent=bonding1 queue=sfq
add disabled=yes name=8_out_dmz packet-mark=no-mark parent=vlan_534_vic_spojka queue=sfq
/snmp community
set [ find default=yes ] addresses=0.0.0.0/0 read-access=no
add addresses=10.9.0.0/16 authentication-password=XXX authentication-protocol=SHA1 encryption-password=XXX name=XXX security=private
/system logging action
set 3 remote=10.9.248.250
/ip firewall connection tracking
set enabled=yes
/ipv6 settings
set max-neighbor-entries=1024
/interface list member
add interface=vlan_534_vic_spojka list=wan
/ip address
add address=147.32.252.98/30 interface=vlan_534_vic_spojka network=147.32.252.96
add address=192.168.255.1/24 interface=bonding1 network=192.168.255.0
add address=10.9.0.1/16 interface=vlan_209_mgmt network=10.9.0.0
/ip dns
set servers=10.9.252.1
/ip firewall address-list
add address=5.145.104.0/21 comment="Statni pokladna Centrum sdilenych sluzeb, s.p. (EET)" disabled=yes list=whitelist
add address=147.32.32.2 comment=b01-rtr-nat list=natboxes
add address=147.32.32.6 comment=mk-rtr-nat list=natboxes
add address=147.32.32.9 comment=b01-nat-kofax list=natboxes
add address=147.32.32.71 comment="Strahov autodoprava RCVUT" list=natboxes
add address=147.32.32.72 comment="Najemnik Podoli - Lysak" list=natboxes
add address=147.32.32.99 comment=hk-rtr-rb750-3psrv.suz.cvut.cz list=natboxes
add address=147.32.32.100 comment=nh-rtr-nat.suz.cvut.cz list=natboxes
add address=147.32.32.101 comment=snat-menzy-vlan202.suz.cvut.cz list=natboxes
add address=147.32.32.102 comment=snat-mk-vlan203.suz.cvut.cz list=natboxes
add address=147.32.32.103 comment=snat-adm-vlan204.suz.cvut.cz list=natboxes
add address=147.32.32.104 comment=eduroam1.suz.cvut.cz list=natboxes
add address=147.32.32.105 comment="Najemnik Podoli - Holy" list=natboxes
add address=147.32.32.128/28 comment="netmap voip" list=natboxes
add address=147.32.32.168 comment=nat-podmen-matusin.suz.cvut.cz list=natboxes
/ip firewall filter
add action=accept chain=input comment="input accept vse, co je established, related, untracked" connection-state=established,related,untracked
add action=drop chain=input comment="input drop vse, co je invalid" connection-state=invalid
add action=jump chain=input in-interface-list=wan jump-target=input_from_wan
add action=drop chain=input_from_wan comment="input dropni vsechno, co jde z blacklistu na Joshaven" src-address-list=blacklist
add action=fasttrack-connection chain=forward comment="forward fasttrack vse, co je established, related" connection-mark=!voip connection-state=established,related
add action=accept chain=forward comment="forward accept vse, co je established, related, untracked" connection-state=established,related,untracked
add action=jump chain=forward in-interface-list=wan jump-target=forward_from_wan
add action=drop chain=forward_from_wan comment="dropni vsechno, co jde z blacklistu na Joshaven" src-address-list=blacklist
add action=drop chain=forward_from_wan comment="dropni vsechno, co je na manual_blacklist" src-address-list=manual_blacklist
add action=reject chain=forward_from_wan comment="SNMP prijimame pouze z CVUT" dst-port=161,162 out-interface=bonding1 protocol=udp reject-with=icmp-network-unreachable src-address=!147.32.0.0/16
add action=reject chain=forward_from_wan comment="syslog prijimame pouze z CVUT" dst-port=514 out-interface=bonding1 protocol=udp reject-with=icmp-network-unreachable src-address=!147.32.0.0/16
add action=drop chain=forward_from_wan comment="Docasne NTP zvenku na b01-switch-srv-1" dst-address=147.32.32.3 dst-port=123 protocol=udp
add action=drop chain=forward_from_wan comment="Docasne NTP zvenku na b01-switch-srv-2" dst-address=147.32.32.4 dst-port=123 protocol=udp
add action=add-src-to-address-list address-list=ban_rdp address-list-timeout=1d chain=forward_from_wan comment="RDP zvenku na bananovnik" dst-address-list=!natboxes dst-port=3389 protocol=tcp
add action=drop chain=forward_from_wan comment="Blokuj komplet provoz od vsech adres na ban_rdp" src-address-list=ban_rdp
/ip firewall mangle
add action=fasttrack-connection chain=forward disabled=yes in-interface=vlan_534_vic_spojka out-interface=bonding1 src-address=!81.92.155.128/25
add action=accept chain=forward disabled=yes in-interface=vlan_534_vic_spojka out-interface=bonding1 src-address=!81.92.155.128/25
add action=fasttrack-connection chain=forward connection-state="" disabled=yes dst-address=!81.92.155.128/25 in-interface=bonding1 out-interface=vlan_534_vic_spojka
add action=accept chain=forward connection-state="" disabled=yes dst-address=!81.92.155.128/25 in-interface=bonding1 out-interface=vlan_534_vic_spojka
add action=mark-connection chain=forward connection-state=new disabled=yes in-interface=vlan_534_vic_spojka new-connection-mark=voip out-interface=bonding1 passthrough=yes src-address=81.92.155.128/25
add action=mark-connection chain=forward connection-state=new disabled=yes dst-address=81.92.155.128/25 in-interface=bonding1 new-connection-mark=voip out-interface=vlan_534_vic_spojka passthrough=yes
add action=mark-packet chain=forward connection-mark=voip disabled=yes new-packet-mark=voip passthrough=no
/ip firewall service-port
set sip disabled=yes
/ip route
add distance=1 gateway=147.32.252.97
add distance=1 dst-address=147.32.32.0/24 gateway=192.168.255.2
/ip service
set telnet disabled=yes
set www disabled=yes
set api disabled=yes
set api-ssl disabled=yes
/ipv6 address
add address=2001:718:2:1ffe::51/127 advertise=no interface=vlan_534_vic_spojka
add address=fdd1:c8a3:aaeb::1 advertise=no interface=bonding1
/ipv6 route
add distance=1 gateway=fe80::221:55ff:feb1:7bc4%vlan_534_vic_spojka
add distance=1 dst-address=2001:718:2:1d00::/56 gateway=fdd1:c8a3:aaeb::2
/snmp
set contact=admin@suz.cvut.cz enabled=yes location="CZ;Praha;Vanickova 7;068---p01-s127-"
/system clock
set time-zone-name=Europe/Prague
/system identity
set name=b01-rtr-transfw
/system logging
set 0 action=remote
set 1 action=remote
set 2 action=remote
set 3 action=remote
/system ntp client
set enabled=yes primary-ntp=10.9.252.2 secondary-ntp=10.9.252.2
/system package update
set channel=bugfix
/system scheduler
add comment="Download openbl list" interval=1w name=DownloadOpenBL_List on-event=DownloadOpenBL policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=jan/01/1970 start-time=\
    01:05:00
add comment="Apply openbl List" interval=1w name=InstallOpenBL_List on-event=ReplaceOpenBL policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=jan/01/1970 start-time=01:15:00
add comment="Download spamnaus list" interval=1w name=DownloadSpamhausList on-event=DownloadSpamhaus policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=jan/01/1970 start-time=\
    02:02:00
add comment="Apply spamnaus List" interval=1w name=InstallSpamhausList on-event=ReplaceSpamhaus policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=jan/01/1970 start-time=\
    02:12:00
add comment="Download dshield list" interval=1w name=DownloadDShieldList on-event=Download_dshield policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=jan/01/1970 start-time=\
    02:42:00
add comment="Apply dshield List" interval=1w name=InstallDShieldList on-event=Replace_dshield policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=jan/01/1970 start-time=02:52:00
/system script
add name=DownloadOpenBL owner=admin policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive source=\
    "\
    \n/tool fetch url=\"http://joshaven.com/openbl.rsc\" mode=http;\
    \n:log info \"Downloaded openbl.rsc from Joshaven.com\";\
    \n"
add name=ReplaceOpenBL owner=admin policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive source=\
    "\
    \n/ip firewall address-list remove [find where comment=\"OpenBL\"]\
    \n/import file-name=openbl.rsc;\
    \n:log info \"Removed old OpenBL records and imported new list\";\
    \n"
add name=DownloadSpamhaus owner=admin policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive source=\
    "\
    \n/tool fetch url=\"http://joshaven.com/spamhaus.rsc\" mode=http;\
    \n:log info \"Downloaded spamhaus.rsc from Joshaven.com\";\
    \n"
add name=ReplaceSpamhaus owner=admin policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive source=\
    "\
    \n/ip firewall address-list remove [find where comment=\"SpamHaus\"]\
    \n/import file-name=spamhaus.rsc;\
    \n:log info \"Removed old Spamhaus records and imported new list\";\
    \n"
add name=Download_dshield owner=admin policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive source=\
    "\
    \n/tool fetch url=\"http://joshaven.com/dshield.rsc\" mode=http;\
    \n:log info \"Downloaded dshield.rsc from Joshaven.com\";\
    \n"
add name=Replace_dshield owner=admin policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive source=\
    "\
    \n/ip firewall address-list remove [find where comment=\"DShield\"]\
    \n/import file-name=dshield.rsc;\
    \n:log info \"Removed old dshield records and imported new list\";\
    \n"

Any ideas of what is going on are most wellcome. I can add that disabling the bond doesn't help, using STP instead of it or running on just one plain cable to just one EX4600 doesn't help also. There is even no influence if I disable the firewall alltogether.
 
User avatar
benesm1
just joined
Topic Author
Posts: 18
Joined: Fri Oct 08, 2010 11:37 am
Location: Europe/Prague

Re: CCR1016-12G packet loss

Fri Mar 30, 2018 10:46 pm

I've replaced the CCR1016-12G with borrowed CCR1036-8G-2S+. I've configured bonding over the SFP+ interfaces and connected it to the Juniper EX4600 VC. The Cisco side remains at 1Gbps. I've begun a new round of testing and if everything goes well I'll consider trying the CCR1072-1G-8S+. The interesting part is, that the export from the CCR1036-8G-2S+ is slightly different than the one from the CCR1016-12G (of course I mean besides the obvious hardware dependent parts).

By the way - is anybody from Mikrotik reading this forum? I coul'd have possibly found a bug in one of their top-end devices and there seems to be a very little interest.
 
User avatar
Maggiore81
Trainer
Trainer
Posts: 562
Joined: Sun Apr 15, 2012 12:10 pm
Location: Italy
Contact:

Re: CCR1016-12G packet loss

Wed Jan 18, 2023 8:55 am

Sorry to resurrect an old thread. Do you have any updates for that?
If you want Mikrotik to check that, you have to open a support ticket to them providing all the infos.

Who is online

Users browsing this forum: Amazon [Bot], astelsrl, CGGXANNX, en1gm4, eworm, h3x00r, Kanzler and 88 guests