Community discussions

MikroTik App
 
Dilvish5
just joined
Topic Author
Posts: 2
Joined: Fri Aug 24, 2018 3:08 pm

Small outgoing syn attack causing losses on tcp packets in the whole network

Wed Mar 25, 2020 9:58 pm

We have a CCR1036 (updated to latest RouterOS) and today we've noticed a degradation in tcp connections.
Basically, a lot of syn packets were being lost or had a very high delay. The cpu usage was around 3% and no indication what was causing the issue.
This is a hping towards yahoo.com:
# hping3 -S -p 80 98.137.246.8
HPING 98.137.246.8 (vmbr0 98.137.246.8): S set, 40 headers + 0 data bytes
len=44 ip=98.137.246.8 ttl=114 DF id=8224 sport=80 flags=SA seq=0 win=512 rtt=147.8 ms
len=44 ip=98.137.246.8 ttl=114 DF id=4400 sport=80 flags=SA seq=1 win=512 rtt=151.7 ms
len=44 ip=98.137.246.8 ttl=114 DF id=24791 sport=80 flags=SA seq=2 win=512 rtt=151.7 ms
len=44 ip=98.137.246.8 ttl=114 DF id=53332 sport=80 flags=SA seq=3 win=512 rtt=151.6 ms
len=44 ip=98.137.246.8 ttl=114 DF id=17011 sport=80 flags=SA seq=5 win=512 rtt=151.5 ms
len=44 ip=98.137.246.8 ttl=114 DF id=42890 sport=80 flags=SA seq=7 win=512 rtt=151.4 ms
len=44 ip=98.137.246.8 ttl=114 DF id=54953 sport=80 flags=SA seq=9 win=512 rtt=147.2 ms
len=44 ip=98.137.246.8 ttl=114 DF id=37269 sport=80 flags=SA seq=10 win=512 rtt=147.2 ms
len=44 ip=98.137.246.8 ttl=114 DF id=2146 sport=80 flags=SA seq=11 win=512 rtt=147.1 ms
len=44 ip=98.137.246.8 ttl=114 DF id=52506 sport=80 flags=SA seq=12 win=512 rtt=147.0 ms
len=44 ip=98.137.246.8 ttl=114 DF id=37543 sport=80 flags=SA seq=13 win=512 rtt=147.0 ms
len=44 ip=98.137.246.8 ttl=114 DF id=64224 sport=80 flags=SA seq=16 win=512 rtt=142.8 ms
len=44 ip=98.137.246.8 ttl=114 DF id=64714 sport=80 flags=SA seq=17 win=512 rtt=146.7 ms
len=44 ip=98.137.246.8 ttl=114 DF id=15448 sport=80 flags=SA seq=20 win=512 rtt=146.5 ms
len=44 ip=98.137.246.8 ttl=114 DF id=2808 sport=80 flags=SA seq=22 win=512 rtt=146.3 ms
len=44 ip=98.137.246.8 ttl=114 DF id=29931 sport=80 flags=SA seq=24 win=512 rtt=154.2 ms
len=44 ip=98.137.246.8 ttl=114 DF id=58923 sport=80 flags=SA seq=27 win=512 rtt=145.9 ms
len=44 ip=98.137.246.8 ttl=114 DF id=12196 sport=80 flags=SA seq=29 win=512 rtt=145.8 ms
len=44 ip=98.137.246.8 ttl=114 DF id=61845 sport=80 flags=SA seq=32 win=512 rtt=153.6 ms
len=44 ip=98.137.246.8 ttl=114 DF id=48560 sport=80 flags=SA seq=33 win=512 rtt=145.5 ms
len=44 ip=98.137.246.8 ttl=114 DF id=21284 sport=80 flags=SA seq=34 win=512 rtt=145.4 ms
len=44 ip=98.137.246.8 ttl=114 DF id=53032 sport=80 flags=SA seq=35 win=512 rtt=145.3 ms
len=44 ip=98.137.246.8 ttl=114 DF id=29744 sport=80 flags=SA seq=36 win=512 rtt=145.2 ms
len=44 ip=98.137.246.8 ttl=114 DF id=9080 sport=80 flags=SA seq=37 win=512 rtt=153.2 ms
len=44 ip=98.137.246.8 ttl=114 DF id=33930 sport=80 flags=SA seq=38 win=512 rtt=145.1 ms
len=44 ip=98.137.246.8 ttl=114 DF id=26648 sport=80 flags=SA seq=39 win=512 rtt=149.0 ms
len=44 ip=98.137.246.8 ttl=114 DF id=42187 sport=80 flags=SA seq=40 win=512 rtt=141.0 ms
len=44 ip=98.137.246.8 ttl=114 DF id=52279 sport=80 flags=SA seq=41 win=512 rtt=144.9 ms
len=44 ip=98.137.246.8 ttl=114 DF id=49533 sport=80 flags=SA seq=43 win=512 rtt=148.8 ms
len=44 ip=98.137.246.8 ttl=114 DF id=47941 sport=80 flags=SA seq=46 win=512 rtt=148.6 ms
len=44 ip=98.137.246.8 ttl=114 DF id=58020 sport=80 flags=SA seq=47 win=512 rtt=148.5 ms
len=44 ip=98.137.246.8 ttl=114 DF id=56137 sport=80 flags=SA seq=48 win=512 rtt=152.4 ms
len=44 ip=98.137.246.8 ttl=114 DF id=36939 sport=80 flags=SA seq=49 win=512 rtt=156.3 ms
len=44 ip=98.137.246.8 ttl=114 DF id=55343 sport=80 flags=SA seq=50 win=512 rtt=148.3 ms
len=44 ip=98.137.246.8 ttl=114 DF id=43028 sport=80 flags=SA seq=52 win=512 rtt=148.1 ms
len=44 ip=98.137.246.8 ttl=114 DF id=3454 sport=80 flags=SA seq=53 win=512 rtt=152.0 ms
len=44 ip=98.137.246.8 ttl=114 DF id=56267 sport=80 flags=SA seq=54 win=512 rtt=151.9 ms
len=44 ip=98.137.246.8 ttl=114 DF id=17186 sport=80 flags=SA seq=60 win=512 rtt=155.4 ms
len=44 ip=98.137.246.8 ttl=114 DF id=28346 sport=80 flags=SA seq=62 win=512 rtt=143.3 ms
len=44 ip=98.137.246.8 ttl=114 DF id=51240 sport=80 flags=SA seq=66 win=512 rtt=147.0 ms
len=44 ip=98.137.246.8 ttl=114 DF id=10720 sport=80 flags=SA seq=68 win=512 rtt=146.8 ms
len=44 ip=98.137.246.8 ttl=114 DF id=59413 sport=80 flags=SA seq=69 win=512 rtt=146.8 ms
^C
--- 98.137.246.8 hping statistic ---
70 packets transmitted, 42 packets received, 40% packet loss
round-trip min/avg/max = 141.0/148.4/156.3 ms
A similar result was received when trying to hping port 80 on the router. 1/3 of the packets were having between 1000 and 15000ms delay, while the normal was 30.

After a lot of investigations, disabling all firewall rules and banging my head to the wall for the whole day, i've discovered one customer sending a rather small attack (12-13k pps).
Attack logs look like this:
19:33:31.240229 IP source-ip.8201 > target-ip.80: Flags [S], seq 537486444:537487326, win 62336, length 882: HTTP
19:33:31.240315 IP source-ip.59796 > target-ip.80: Flags [S], seq 3918803235:3918804086, win 64911, length 851: HTTP
19:33:31.240431 IP source-ip.22627 > target-ip.80: Flags [S], seq 1482935325:1482936181, win 63292, length 856: HTTP
19:33:31.240482 IP source-ip.28719 > target-ip.80: Flags [S], seq 1882128903:1882129771, win 61074, length 868: HTTP
19:33:31.240646 IP source-ip.38839 > target-ip.80: Flags [S], seq 2545371210:2545372081, win 62719, length 871: HTTP
19:33:31.240673 IP source-ip.16607 > target-ip.80: Flags [S], seq 1088386150:1088387036, win 61802, length 886: HTTP
19:33:31.240756 IP source-ip.25510 > target-ip.80: Flags [S], seq 1671887128:1671887981, win 63850, length 853: HTTP
19:33:31.240833 IP source-ip.11372 > target-ip.80: Flags [S], seq 745296430:745297296, win 60017, length 866: HTTP
19:33:31.240924 IP source-ip.28841 > target-ip.80: Flags [S], seq 1890159177:1890160061, win 64631, length 884: HTTP
19:33:31.241004 IP source-ip.23008 > target-ip.80: Flags [S], seq 1507869800:1507870651, win 62083, length 851: HTTP
19:33:31.241110 IP source-ip.13463 > target-ip.80: Flags [S], seq 882340131:882340991, win 65498, length 860: HTTP
19:33:31.241178 IP source-ip.55826 > target-ip.80: Flags [S], seq 3658656782:3658657636, win 64826, length 854: HTTP
19:33:31.241271 IP source-ip.63105 > target-ip.80: Flags [S], seq 4135714613:4135715461, win 60060, length 848: HTTP
19:33:31.241345 IP source-ip.36972 > target-ip.80: Flags [S], seq 2423055921:2423056811, win 60117, length 890: HTTP
19:33:31.241434 IP source-ip.55590 > target-ip.80: Flags [S], seq 3643178513:3643179406, win 64410, length 893: HTTP
19:33:31.241533 IP source-ip.2287 > target-ip.80: Flags [S], seq 149938043:149938901, win 60307, length 858: HTTP
19:33:31.241617 IP source-ip.61127 > target-ip.80: Flags [S], seq 4006041408:4006042266, win 61367, length 858: HTTP
19:33:31.241700 IP source-ip.10015 > target-ip.80: Flags [S], seq 656366659:656367546, win 62440, length 887: HTTP
19:33:31.241824 IP source-ip.2129 > target-ip.80: Flags [S], seq 139527295:139528171, win 64139, length 876: HTTP
19:33:31.241872 IP source-ip.48014 > target-ip.80: Flags [S], seq 3146703389:3146704246, win 63329, length 857: HTTP
19:33:31.241958 IP source-ip.60608 > target-ip.80: Flags [S], seq 3972028710:3972029606, win 65108, length 896: HTTP
19:33:31.242045 IP source-ip.53718 > target-ip.80: Flags [S], seq 3520482843:3520483706, win 65178, length 863: HTTP
19:33:31.242126 IP source-ip.9193 > target-ip.80: Flags [S], seq 602493242:602494111, win 64333, length 869: HTTP
19:33:31.242219 IP source-ip.17464 > target-ip.80: Flags [S], seq 1144554320:1144555216, win 63334, length 896: HTTP
19:33:31.242319 IP source-ip.50095 > target-ip.80: Flags [S], seq 3283047198:3283048081, win 60715, length 883: HTTP
19:33:31.242401 IP source-ip.29370 > target-ip.80: Flags [S], seq 1924835609:1924836491, win 63450, length 882: HTTP
19:33:31.242487 IP source-ip.19283 > target-ip.80: Flags [S], seq 1263785520:1263786371, win 64718, length 851: HTTP
19:33:31.242575 IP source-ip.38097 > target-ip.80: Flags [S], seq 2496753683:2496754541, win 63837, length 858: HTTP
19:33:31.242655 IP source-ip.22280 > target-ip.80: Flags [S], seq 1460206884:1460207761, win 62135, length 877: HTTP
19:33:31.242743 IP source-ip.15451 > target-ip.80: Flags [S], seq 1012619124:1012619976, win 61216, length 852: HTTP
19:33:31.242853 IP source-ip.62021 > target-ip.80: Flags [S], seq 4064641081:4064641951, win 61301, length 870: HTTP
19:33:31.242908 IP source-ip.58772 > target-ip.80: Flags [S], seq 3851725434:3851726291, win 64907, length 857: HTTP
Blocking the customer in raw tables resolved the problem and no losses could be detected on tcp traffic.


However, i am looking for a long term solution to both detect and block such attacks.

Also, i do not understand why it managed to cause such a big impact on our router, even though the cpu usage was only 3%.

Does anyone have any thoughts?

Who is online

Users browsing this forum: jamrobe and 141 guests