Dear Gentle Folk,
We have a Spectrum light fiber 200 megabit link from 184.74.74.251 on our end to our default gateway at 184.74.74.249.
The edge router is a mik RB1100 AHx2 pulling 130 Mbps at 8:00pm netflix time every night.
This is a wireless network, connected to two other identical miks out in the boonies via various Ubi Powerbeams ectc that are NOT full.
Most of the outgoing internet traffic from our edte router is NATed as we use 10.16.x.x for our customers, a very few get public statics.
At this time time of night traffic going out from both remote miks crosses our edge IP number at about 15ms, but
comes back to us from our default gateway at 300ms or more, generating complaints from gamers.
This shows on both IP an UDP pings and traceroutes which are included below.
The high pings show up clearly in the traceroute from either of the two remote miks, EXCEPT when the source address at the remote mik is a public static IP such as 64.57.177.115.
We give some customers public statics if they pay for it and need it, also when the their IP phones are not clean or their games are latency logged. Now we know why.
I have set the contract tables to about 5000 entries on the edge routers by setting the TCP time out to 1 hour rather than 1 day.
Below are the trace routes showing the problem.
Homer
Script started on Wed Oct 3 20:50:36 2018
Diagram of two way packet travel.
Internet → Spectrum → Edge router → 2nd edge router → Ubi
wireless → HT MIK → HT linux box → HT MIK → Ubi wireless → 2nd edge
→ 1st edge → Spectrum → Internet
Both of these traceroutes are done from HT linux box at
ht.lightlink.com at 10.16.31.1 and 64.57.178.115 running the standard
linux traceroute command using UDP while selecting the source IP.
Tests using traceroute with ICMP show better results overall, but
the anomolous behavior is still present and obvious. Somewhere around
10pm the pings on the line 4 (our default gateway) go to 350 or higher for private IP’s only,
As an aside the cross over from rr to zayo at around line 10 has a definite router suffering 40 percent packet loss per mik traceroute, and high pings, I trust this an unrelated problem which I have reported to spectrum.
superoot ht/root: traceroute rahul.net
traceroute to rahul.net (192.160.13.1), 30 hops max, 38 byte packets
1 10.16.31.1 (10.16.31.1) 0.395 ms 0.246 ms 0.224 ms
2 10.16.0.100 (10.16.0.100) 24.839 ms 15.937 ms 36.098 ms
3 10.16.0.81 (10.16.0.81) 30.600 ms 34.418 ms 33.268 ms
4 rrcs-184-74-74-249.nys.biz.rr.com (184.74.74.249) 129.170 ms 120.818 ms 118.282 ms
5 agg15.ithcnycy01h.northeast.rr.com (24.58.29.37) 117.790 ms 88.738 ms 74.498 ms
6 agg79.esyrnydr02r.northeast.rr.com (24.58.52.166) 69.661 ms 109.944 ms 92.866 ms
7 agg27.albynyyf01r.northeast.rr.com (24.58.32.80) 82.507 ms 106.408 ms 85.682 ms
8 bu-ether16.nycmny837aw-bcr00.tbone.rr.com (66.109.6.74) 60.466 ms 131.979 ms 123.717 ms
9 0.ae0.pr1.nyc20.tbone.rr.com (107.14.17.216) 130.737 ms 77.329 ms 0.ae1.pr1.nyc20.tbone.rr.com (107.14.17.218) 61.152 ms
10 66.109.7.42 (66.109.7.42) 111.025 ms 139.414 ms 180.333 ms
11 ae15.cs1.lga5.us.zip.zayo.com (64.125.29.220) 212.784 ms 215.930 ms 199.601 ms
12 ae3.cs1.ord2.us.eth.zayo.com (64.125.29.209) 222.214 ms 272.746 ms 275.365 ms
13 ae5.cs1.den5.us.eth.zayo.com (64.125.29.19) 223.801 ms 232.370 ms 232.057 ms
14 ae6.cs1.sjc2.us.eth.zayo.com (64.125.31.219) 208.124 ms 228.739 ms 200.035 ms
15 ae0.cs2.sjc2.us.eth.zayo.com (64.125.28.141) 225.244 ms 285.400 ms 269.445 ms
16 ae27.cr2.sjc2.us.zip.zayo.com (64.125.30.233) 112.362 ms 151.617 ms 144.536 ms
17 ae16.mpr4.sjc7.us.zip.zayo.com (64.125.31.15) 169.256 ms 181.409 ms
18 208.185.155.94.ipyx-086177-910-zyo.above.net (208.185.155.94) 140.333 ms 225.998 ms 226.146 ms
19 cr1-55smarket-hu-0-3-0-20-0.bb.as11404.net (192.175.28.137) 256.466 ms 284.031 ms 254.554 ms
20 cr2-scl-hu-0-7-0-20-0.bb.as11404.net (192.175.28.241) 260.415 ms 317.565 ms 239.203 ms
21 xe5-2.core1.svr.layer42.net (65.50.198.90) 132.247 ms 115.430 ms 167.405 ms
22 64.13.169.202 (64.13.169.202) 136.793 ms 135.717 ms 131.533 ms
23 bolero.rahul.net (192.160.13.1) 134.289 ms 157.570 ms 208.625 ms
traceroute to rahul.net (192.160.13.1) from 64.57.178.115, 30 hops max, 38 byte packets
1 64.57.178.113.lightlink.com (64.57.178.113) 0.592 ms 0.486 ms 0.448 ms
2 10.16.0.100 (10.16.0.100) 30.508 ms 88.624 ms 37.632 ms
3 10.16.0.81 (10.16.0.81) 52.858 ms 61.148 ms 53.518 ms
4 rrcs-184-74-74-249.nys.biz.rr.com (184.74.74.249) 21.483 ms 14.191 ms 22.508 ms
5 agg15.ithcnycy01h.northeast.rr.com (24.58.29.37) 17.730 ms 33.402 ms 31.861 ms
6 agg79.esyrnydr02r.northeast.rr.com (24.58.52.166) 43.449 ms 33.915 ms 44.695 ms
7 agg27.albynyyf01r.northeast.rr.com (24.58.32.80) 42.757 ms 32.067 ms 45.372 ms
8 bu-ether16.nycmny837aw-bcr00.tbone.rr.com (66.109.6.74) 53.228 ms 49.165 ms 61.743 ms
9 0.ae1.pr1.nyc20.tbone.rr.com (107.14.17.218) 40.529 ms 66.109.1.59 (66.109.1.59) 100.964 ms 30.580 ms
10 66.109.7.42 (66.109.7.42) 57.723 ms 50.161 ms 39.246 ms
11 ae15.cs1.lga5.us.zip.zayo.com (64.125.29.220) 98.350 ms 99.198 ms 104.741 ms
12 ae3.cs1.ord2.us.eth.zayo.com (64.125.29.209) 98.932 ms 121.668 ms 94.530 ms
13 ae5.cs1.den5.us.eth.zayo.com (64.125.29.19) 112.785 ms 115.158 ms 141.876 ms
14 ae6.cs1.sjc2.us.eth.zayo.com (64.125.31.219) 103.530 ms 104.580 ms 100.072 ms
15 ae0.cs2.sjc2.us.eth.zayo.com (64.125.28.141) 135.108 ms 107.203 ms 130.627 ms
16 ae27.cr2.sjc2.us.zip.zayo.com (64.125.30.233) 113.110 ms 106.224 ms 114.708 ms
17 ae16.mpr4.sjc7.us.zip.zayo.com (64.125.31.15) 100.607 ms 117.650 ms 100.785 ms
18 208.185.155.94.ipyx-086177-910-zyo.above.net (208.185.155.94) 133.364 ms 122.496 ms 122.088 ms
19 cr1-55smarket-hu-0-3-0-20-0.bb.as11404.net (192.175.28.137) 119.796 ms 132.738 ms 103.842 ms
20 cr2-scl-hu-0-7-0-20-0.bb.as11404.net (192.175.28.241) 144.844 ms 124.806 ms 113.562 ms
21 xe5-2.core1.svr.layer42.net (65.50.198.90) 100.260 ms 126.175 ms 108.887 ms
22 64.13.169.202 (64.13.169.202) 99.793 ms 126.827 ms 101.207 ms
23 bolero.rahul.net (192.160.13.1) 101.102 ms 117.595 ms 113.608 ms
Homer