Greetings,

I have an RB2011UiAS-RM running ROS 6.32.2 and firmware 3.24. Very basic setup using a PPPoE connection to my
Internet provider CenturyLink. Internet access works but is very slow to the tune of ~400ms ping time to any place on the internet. CenturyLink support indicates the circuit side is trained up where it should be 12Mbps down / 2.5Mbps up.
CenturyLink modem is running in Transparent Bridge. This is the first time I have ever tried Mikrotik with a PPPoE WAN connection and mostly looking for other forum members experiences with this type of config and if there is anything special
I need to do with respect to connecting to CenturyLink.

As always, thanks for any insight anyone can provide.

Aaron

I use Mikrotik PPPoE with CenturyLink 60/30 vdsl service but have had no problem.

The config is simple.
You mention 400ms latency but what kind throughput are you getting?
Are you pinging directly from the RB2011 or are you pinging from a host connected to it?

Is the DSL Modem in "Bridge" mode?
Are you doing any "Layer 7" filtering?
Just checking as you shouldn't see latency that long.
You might post your configuration, editing out sensitive information for us to take a look at.

-tp

Thanks for the replies. I am pinging from the Mikrotik itself. There are no mangle rules or layer 7 protocol changes made to the default firewall. The modem is a CenturyLink C1000A that is running in "Transparent Bridge" mode. I am getting ~400ms ping times into the gateway IP address provided by CenturyLink. Roughly ~400ms ping times into places like 8.8.8.8 or 205.171.3.65 which is a CenturyLink DNS server. Configuration is below:

#oct/11/2015 17:26:16 by RouterOS 6.32.2
# software id = 7NM0-48YS
#
/interface bridge
add admin-mac=E4:8D:8C:**:**:** auto-mac=no name=bridge-local
/interface ethernet
set [ find default-name=ether1 ] auto-negotiation=no name=ether1-gateway
set [ find default-name=ether2 ] mtu=1492
set [ find default-name=ether6 ] name=ether6-master-local
set [ find default-name=ether7 ] master-port=ether6-master-local name=ether7-slave-local
set [ find default-name=ether8 ] master-port=ether6-master-local name=ether8-slave-local
set [ find default-name=ether9 ] master-port=ether6-master-local name=ether9-slave-local
set [ find default-name=ether10 ] master-port=ether6-master-local name=ether10-slave-local
/interface pppoe-client
add add-default-route=yes dial-on-demand=yes disabled=no interface=ether1-gateway max-mru=1492 max-mtu=1492 mrru=1492 name=\
"CenturyLink ISP" password=****** use-peer-dns=yes user=********@qwest.net
/ip neighbor discovery
set ether1-gateway discover=no
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip ipsec proposal
set [ find default=yes ] enc-algorithms=aes-128-cbc
/ip pool
add name=dhcp ranges=192.168.49.100-192.168.49.199
/ip dhcp-server
add add-arp=yes address-pool=dhcp disabled=no interface=bridge-local name="******"
/interface bridge port
add bridge=bridge-local interface=ether2
add bridge=bridge-local interface=ether3
add bridge=bridge-local interface=ether4
add bridge=bridge-local interface=ether5
add bridge=bridge-local interface=ether6-master-local
add bridge=bridge-local interface=sfp1
/ip address
add address=192.168.49.254/24 comment="default configuration" interface=ether2 network=192.168.49.0
/ip dhcp-server network
add address=192.168.49.0/24 gateway=192.168.49.254 netmask=24
/ip dns
set allow-remote-requests=yes servers=205.171.3.65,205.171.2.65,8.8.8.8,8.8.4.4
/ip dns static
add address=192.168.49.254 name=router
/ip firewall filter
add chain=input comment="Remote Winbox" dst-port=8291 protocol=tcp
add chain=input comment=Ping protocol=icmp
add chain=input protocol=ipsec-esp
add chain=input port=500 protocol=udp
add chain=input protocol=ipsec-ah
add chain=input comment=Related connection-state=related
add chain=input comment=Established connection-state=established
add action=drop chain=input comment="Drop Unsolicited" in-interface=ether1-gateway
add chain=forward comment=Established connection-state=established
add chain=forward comment=Related connection-state=related
add action=drop chain=forward comment=Invalid connection-state=invalid
/ip firewall nat
add chain=srcnat dst-address=192.168.0.0/24 src-address=192.168.49.0/24
add action=masquerade chain=srcnat comment="default configuration" out-interface="CenturyLink ISP"
/ip ipsec peer
add address=96.93.***.***/32 secret=*********
/ip ipsec policy
add dst-address=192.168.0.0/24 sa-dst-address=96.93.***.*** sa-src-address=65.102.**.*** src-address=192.168.49.0/24 tunnel=yes
/ip service
set telnet disabled=yes
set ssh disabled=yes
/system clock
set time-zone-autodetect=no time-zone-name=America/Denver
/system identity
set name=*********-RTR1
/system scheduler
add interval=30m name="Update DNS Table" on-event="DNS Update" policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=\
oct/06/2015 start-time=02:10:49
/system script
add name="DNS Update" owner=admin policy=ftp,reboot,read,write,policy,test,password,sniff,sensitive source="# Creates static DNS entres for DHCP cli\
ents in the named DHCP server.\r\
\n# Hostnames passed to DHCP are appended with the zone.\r\
\n \r\
\n# Set the first two variables according to your installation.\r\
\n:local dhcpserver \"************\"\r\
\n:local zone \"workgroup\"\r\
\n \r\
\n# Set the TTL to the scheduler frequency for this script.\r\
\n:local ttl \"00:05:00\"\r\
\n \r\
\n# Clear old static DNS entries matching the zone and TTL.\r\
\n/ip dns static\r\
\n:foreach dnsrecord in=[find where name ~ (\".*\\\\.\".\$zone) ] do={\r\
\n\t:local fqdn [ get \$dnsrecord name ]\r\
\n\t:local hostname [ :pick \$fqdn 0 ( [ :len \$fqdn ] - ( [ :len \$zone ] + 1 ) ) ]\r\
\n\t:local recordttl [get \$dnsrecord ttl]\r\
\n\t:if ( \$recordttl != \$ttl ) do={\r\
\n\t\t:log debug (\"Ignoring DNS record \$fqdn with TTL \$recordttl\")\r\
\n\t} else={\r\
\n\t} else={\r\
\n\t\t/ip dhcp-server lease\r\
\n\t\t:local dhcplease [ find where host-name=\$hostname and server=\"\$dhcpserver\"]\r\
\n\t\t:if ( [ :len \$dhcplease ] > 0) do={\r\
\n\t\t\t:log debug (\"DHCP lease exists for \$hostname in \$dhcpserver, keeping DNS record \$fqdn\")\r\
\n\t\t} else={\r\
\n\t\t\t:log info (\"DHCP lease expired for \$hostname, deleting DNS record \$fqdn\")\r\
\n\t\t\t/ip dns static remove \$dnsrecord\r\
\n\t\t}\r\
\n\t}\r\
\n}\r\
\n \r\
\n# Create or update static DNS entries from DHCP server leases.\r\
\n/ip dhcp-server lease\r\
\n:foreach dhcplease in=[find where server ~ (\"\$dhcpserver\")] do={\r\
\n\t:local hostname [ get \$dhcplease host-name ]\r\
\n\t:if ( [ :len \$hostname ] > 0) do={\r\
\n\t\t:local dhcpip [ get \$dhcplease address ]\r\
\n\t\t:local fqdn ( \$hostname . \".\" . \$zone )\r\
\n\t\t/ip dns static\r\
\n\t\t:local dnsrecord [ find where name=\$fqdn ]\r\
\n\t\t:if ( [ :len \$dnsrecord ] > 0 ) do={\r\
\n\t\t\t:local dnsip [ get \$dnsrecord address ]\r\
\n\t\t\t:if ( \$dnsip = \$dhcpip ) do={\r\
\n\t\t\t\t:log debug (\"DNS record for \$fqdn to \$dhcpip is up to date\")\r\
\n\t\t\t} else={\r\
\n\t\t\t\t:log info (\"Updating DNS record for \$fqdn to \$dhcpip\")\r\
\n\t\t\t\t/ip dns static remove \$dnsrecord\r\
\n\t\t\t\t/ip dns static add name=\$fqdn address=\$dhcpip ttl=\$ttl\r\
\n\t\t\t}\r\
\n\t\t} else={\r\
\n\t\t\t:log info (\"Creating DNS record for \$fqdn to \$dhcpip\")\r\
\n\t\t\t/ip dns static add name=\$fqdn address=\$dhcpip ttl=\$ttl\r\
\n\t\t}\r\
\n\t}\r\
\n}"
/tool mac-server
set [ find default=yes ] disabled=yes
add interface=ether2
add interface=ether3
add interface=ether4
add interface=ether5
add interface=ether6-master-local
add interface=ether7-slave-local
add interface=ether8-slave-local
add interface=ether9-slave-local
add interface=ether10-slave-local
add interface=sfp1
add interface=bridge-local
/tool mac-server mac-winbox
set [ find default=yes ] disabled=yes
add interface=ether2
add interface=ether3
add interface=ether4
add interface=ether5
add interface=ether6-master-local
add interface=ether7-slave-local
add interface=ether8-slave-local
add interface=ether9-slave-local
add interface=ether10-slave-local
add interface=sfp1
add interface=bridge-local

Offhand, the thing that concerns me is "dial-on-demand=yes". Set it to no and see if that helps.

Turned off the "Dial on Demand" in the PPPoE Interface settings. No Change. I went ahead and put the CenturyLink Modem into router mode and configured the same PPPoE credentials to make sure the basic circuit works. I get a consistent 40ms roundtrip to the same sites as before. I have now reset the Mikrotik back to its factory configuration
and made only the changes to require it to connect to the internet via the PPPoE. Same situation. A continuous ping
will start at roughly 40ms then shoot up to ~400ms and stay there. I have now changed the configuration to use Ethernet ports 9 and 10 to see if maybe 1 or 2 were defective. No change. I also swapped out the power supply and tried putting an Ethernet switch between the WAN side and the cable modem as I read somewhere in another post that solved a similar problem for somebody else although it didn't make any sense to do so. Not sure where to go from here other than scrap the unit and try another one. As always, thanks for the quick insight from the other Mikrotik forum users.

Aaron

The pppoe and physical ethernet ports aren't in a bridge by any chance are they?

How about the internal switches? If you're using ether1 for the modem for instance, make sure that it's master-port is set to none and that none of the ports in that group (ports 2-5 and the spf port) have master-port set to ether1.

When you had the modem in routed mode, were you still pinging from the rb2011?

When I had the modem configured in router mode, I was pinging with a laptop via Ethernet and running in NAT mode. Pings were in the 40ms range. The modem is now back in Transparent bridge and connected to the Mikrotik port 10 via PPPoE. The master for port 10 is "None". Is there anything special about the way the "Arp" settings should be set on the Ethernet ports? Should the link speed and duplex be set a certain way for the port connected to the modem?

Thanks,

Aaron

Very strange. I'm looking at my own config and it's about as simple as it gets.

The ppp profile is set at all defaults right?
Which market are you in?
Which modem do you have?

The ethernet port settings should be left at default except for master port. It must be set to none and no other port should reference it as a master port.

What happens if you traceroute instead of ping?

What happens if you put the modem in routed mode and use the rb2011 as a switch?

Yeah, that's what I said. VERY strange. The PPP was initially setup with no changes to the defaults except username
and PPPoE password. This device is located in Pueblo, Colorado. It was intended to provide an IPsec tunnel to another Mikrotik 2011 in Colorado Springs that is connected to a Comcast circuit. The IPsec does work today but, as you can image, very slow with 400+ round trip times. The modem is a Zyxel 5000Z resold by CenturyLink. I had CenturyLink bring me another unit and it made no difference in the ping times. When the CenturyLink modem is providing the NAT/Routed/PPPoe connection I can get roughly 40ms round trips consistently. Trace route below.

[******@MikroTik] /tool> traceroute 8.8.8.8
# ADDRESS LOSS SENT LAST AVG BEST WORST STD-DEV STATUS
1 184.99.0.12 0% 3 274.6ms 361.5 274.6 426.7 64
2 184.99.1.89 0% 3 431.8ms 421 358.9 472.4 47
3 67.14.24.89 33.. 3 486.9ms 487.1 486.9 487.2 0.2
4 67.134.166.226 33.. 3 timeout 391.6 312.1 471.1 79.5
5 216.239.42.247 0% 3 464.3ms 320.6 135.7 464.3137.3
6 216.239.42.235 33.. 3 353.6ms 369.4 353.6 385.2 15.8
7 8.8.8.8 0% 3 465.1ms 379.6 294.1 465.1 85.5

Thanks a bunch for the help.

Aaron

I'm up in Golden so we're in the same legacy Qwest territory. 8.8.8.8 is anycasted and we both hit the same one, in Denver.

You're not routed over the ipsec link by accident are you? The trace doesn't look like it but can you disable ipsec altogether as a test?

I'm on a bonded VDSL with a Technicolor C2000T modem but I do have a few RB2011's hanging around so I'm going to give this a try this afternoon. I may actually have a 5000Z somewhere so if I can find it, I'll try that as well.

What happens when you run the setup in "double NAT"? Take the modem out of transparent mode and and connect your MT, keeping NAT enabled on it as well. We can then rule out if it's PPPoE related.

I've seen strange instances where ports on the 2011 will either refuse to link with another device, or auto-negotiation will fail and I'll have to manually set speed and duplex. Ports 6-10 (the second switch group) were always the most problematic for me.

Thanks for the replies. I am going to try the double NAT idea. As a temporary test I am going to configure the CenturyLink modem in its basic PPPoE with NAT configuration and 192.168.100.254 for the LAN side.
I will assign the address of 192.168.100.253 to the WAN interface on the Mikrotik and set a default route to 192.168.100.254. I was unable to find a setting in the modem documentation for IPSEC pass through so I will create
a forward for UDP 500 and 4500 and see how it goes. I suspect I will need to reconfigure the IPSEC endpoints for NAT
Traversal and run over UDP 4500.

Does this sound like I have covered the bases?

Thanks,

Aaron

What modem do you have? A simple google search on the modem and "IPSec passthrough" might yield any additional info you may need. Otherwise I think you've got the right idea.

Thanks for the replies. I am going to try the double NAT idea. As a temporary test I am going to configure the CenturyLink modem in its basic PPPoE with NAT configuration and 192.168.100.254 for the LAN side.
I will assign the address of 192.168.100.253 to the WAN interface on the Mikrotik and set a default route to 192.168.100.254. I was unable to find a setting in the modem documentation for IPSEC pass through so I will create
a forward for UDP 500 and 4500 and see how it goes. I suspect I will need to reconfigure the IPSEC endpoints for NAT
Traversal and run over UDP 4500.

Does this sound like I have covered the bases?

Thanks,

Aaron

I would suggest not doing anything with ipsec at first. Create the minimum config to get internet connectivity and test latency first.

Greetings,

Putting the C1000a in router/Nat mode cured the high latency issues. Running in the 24ms range now when pinging from the Mikrotik. I have reconfigured the ipsec tunnel for NAT traversal and it comes up and shows established on both ends. Problem now is, I can't seem to ping the remote router on either end or hosts in the remote subnet. I have verified that the NAT bypass rule is still in place above the NAT masquerade rule. Hosts on each subnet are able to access the internet.

Thanks in advance for any insight.

Aaron

Maybe try putting your Mikrotik at the DMZ address on your modem: http://internethelp.centurylink.com/int ... v-dmz.html
This would (hopefully) eliminate any potential port mapping issues that may still be present.

If you want to try some additional steps on the transparent bridging side of things, you could dial the PPPoE connection directly from your laptop and see if you see a latency spike. I suspect it's probably the modem causing the issue and nothing you did wrong.

Success, I think. Got the ping situation sorted. Turns out, even though the ipsec tunnel says it is "Established", that doesn't mean its going to work. In the ipsec policy configuration on the Mikrotik that is behind the NAT'd interface in the CenturyLink modem, I needed to change the source address to the NAT'd WAN IP I had set on the Mikrotik.. In this particular case it is 192.168.100.253. Self inflicted problem solved. This whole configuration still has some oddities.

If I put the WAN IP of the Mikrotik (192.168.100.253) in the DMZ of the CenturyLink modem, I am immediately back to the high latency issue. Actually worse to the tune of ~700ms round trip to 8.8.8.8. Bazaar.

If I reboot the Mikrotik and start a ping from the terminal window to 8.8.8.8, I get round trip times of about 100ms.
If I then go and run a ping from a workstation on the local subnet to 8.8.8.8 I get 40ms round trip.
If I then go back to the terminal window on the Mikrotik and run the same ping I now get 40ms round trip time there as well.

If I wait, say thirty minutes or so, with no traffic coming from the local subnet, the ping times from the mikrotik termial window go back up to the 100ms range. Any traffic coming from the local subnet will bring them back down.

As always, many thanks to the Mikrotik forum.

Aaron

Glad you have something workable. Sure sounds like a crappy modem is to blame for the latency issues.