TiVo => EoIP => TiVo ... fail

RouterOS General
1

SUMMARY

  • TiVo DVR supports multiple devices on the same LAN, allowing one to access content that was recorded on another device
  • I successfully set up an EoIP bridge (across the internet) between 2 locations with 2x MikroTik hEX RB750Gr3
  • Using a laptop it looks to me like everything works
  • Nevertheless, my SE southeast TiVo is not recognizing my NE northeast TiVo
  • suggestions?

DETAIL

TiVo is an old school DVR that I use to record OTA Over The Air television broadcasts.

My NE NorthEast location gets many TV stations broadcast over the air.
My SE SouthEast location doesn’t get any.
My plan was/is to try to access recorded TV content from NE => SE.

TiVo supports two devices on the same LAN … content recorded on one unit can be viewed and/or replicated to another unit.
They say it is for 2 different rooms in the same house.
I got a second TiVo, added it to my existing NE LAN … and it worked fine.
Content that was recorded on my existing NE TiVo could be viewed on my second unit.

TiVo explicitly says that your devices must be on the same LAN with the same subnet and mask.
I saw something that said that TiVo uses mDNS broadcasts for discovery.
Web searches showed that some people claim to have been successful across bridges or VPNs, but I did not find a cookbook/textbook solution.
Several said that the TiVo units are fragile during playback and want/need very reliable connections.

I got 2x MikroTik hEX RB750Gr3 routers running v6.49.17
I successfully set up an EoIP tunnel between locations NE and SE.
2 RJ45 ports on the SE hEX are assigned to the EoIP-bridge.

While in SE, if I plug in a laptop to one of the EoIP ports then I successfully get assigned an NE IP addr through DHCP.
I can discover and connect to the printer at the NE location.
Running speedtest.net in a browser across the EoIP bridge I get >20 Mbps down and >15 Mbps up (but I don’t think streaming bandwidth is my current problem)

I was very confident and pumped up when I turned on my SE TiVo … and then very deflated when it didnt work.
Says that it can’t communicate with my NE TiVo.

I mirrored a hEX port and set up WireShark to capture activity.
I see plenty of regular mDNS broadcasts from my SE TiVo advertising its presence, but none from my NE TiVo.
I did capture my NE TiVo sending a broadcast ARP request for the NE gateway, but no mDNS broadcasts and no communication with my SE TiVo.

POSSIBLE ISSUES

I admit that I didn’t plug NE TiVo directly into NE hEX … There is a ubiquiti core switch and a linksys unmanaged switch in between.
I didn’t think that would be an issue, but now I am wondering.
But, the two TiVo units weren’t plugged into the same switch when they were on the same NE LAN, and they saw each other.

The MikroTik hEX units are behind ubiquiti gateways and are using a pre-existing OpenVPN tunnel between NE and SE sites.
So the tunnel is not the most efficient and is not 100% MikroTik, but the EoIP tunnel is providing transport and it should be fine.

QUESTIONS

I only have WireShark on one end, so I can’t see what is going on at the other end of the EoIP tunnel.
This is my first MikroTik project, so I haven’t yet figured out how to use Torch …
Q: Can I use Torch to see what is going on in my remote NE location?

Everything I read said that EoIP replicates all packets. But maybe I failed to configure something properly …
Q: Is it possible that the EoIP tunnel is filtering out some types of packets?

Q: Any suggestions?

¡Gracias!
Miguel

2

The easiest thing to try is change MTU on the EoIP interface. If it’s not 1500, that be worth a try. Now that will cause fragmentation over the VPN, but TiVo UDP packets may be too big to fit when a compressed MTU. You may also want to make sure “Don’t Fragment” is unchecked (i.e. allow fragments), but this is the default.

Now, assuming your WAN has a MTU of 1500 and OpenVPN overhead is 50 bytes… the EoIP interface’s MTU actually should be 1412. Now if the WAN has a lower MTU, say with PPPoE that using 1492, it be 8 lower. And for TCP, MTU being right is helpful/faster… But for UDP packets, like TiVo UDP broadcast used to discover things, if the TiVi packet does not fit, it may be dropped…so using the “wrong” MTU may help for TiVo discovery.

And since possible your UBNT are dropping broadcasts (but that is not typically a default), that be the other place where TiVo’s UDP broadcast messages be dropped. So if easy, remove the switch in a test… Since obviously the MTU is not the issue if it works without UBNT.

3

Sure, you’d do it on the EoIP interface. In Torch you can filter for UDP port 2190 traffic, which is TiVo’s broadcast. And also you want to increase the “Entry Timeout” since by default only packets from last 3 seconds are shown… but TiVo broadcasts might be once a minute or something way longer than default 3 seconds. But you won’t get the packet data, which wireshark would, (and for TiVo I think it’s plain text with device info, but been a while since I’ve seen a TiVo).

4

A: You can but I’d strongly advise against that. /tool sniffer is much more useful in terms that it both shows the actual direction of individual packets if used alone like Torch and it saves the captured packets in pcap format into files you can download and open in Wireshark. I.e. you effectively have Wireshark on the remote end, except that not in real time.

For both torch and sniffer, make sure that hw is set to no for the Ethernet port you bridge with the EoIP tunnel interface on the /interface bridge port row if you want to torch/sniff on that Ethernet, otherwise the sniffer doesn’t have access to one direction of the packets (although it does not seem logical for packets that have to pass through the CPU anyway).


A: not intentionally, but there were cases here on the forum asking for help with some mysteries; some of them got explained as unrelated to EoIP, some remained unresolved. Just based on previous experience, haven’t you made a backup on one hEX and restore it on the other one in order to “save time” configuring the second one? The thing is that one of the purposes of backup is to allow to create a “clone” the device if it dies, so a restore of a backup file overwrites even the interface MAC addresses on the device where you run it, which is not good if both the template and the clone are connected to the same network, especially if bridging is involved.

5

Wasn’t 1500. Now set to 1500.

You may also want to make sure “Don’t Fragment” is unchecked (i.e. allow fragments), but this is the default.

confirmed

Now, assuming your WAN has a MTU of 1500 and OpenVPN overhead is 50 bytes… the EoIP interface’s MTU actually should be 1412. Now if the WAN has a lower MTU, say with PPPoE that using 1492, it be 8 lower. And for TCP, MTU being right is helpful/faster… But for UDP packets, like TiVo UDP broadcast used to discover things, if the TiVi packet does not fit, it may be dropped…so using the “wrong” MTU may help for TiVo discovery.

Looking at the TiVo mDNS packets that I can see on my side, most packets are a few hundred bytes. But the largest are 1399 bytes.

And since possible your UBNT are dropping broadcasts (but that is not typically a default), that be the other place where TiVo’s UDP broadcast messages be dropped. So if easy, remove the switch in a test… Since obviously the MTU is not the issue if it works without UBNT.

I can’t get to the NE location for a few weeks, so I am stuck with the UBNT core switch for now.

6

upping MTU changed behavior.
Wireshark shows that NE TiVo and SE TiVo are now communicating via UDP.
However, playback still not operational.
mDNS packets still not coming across EoIP bridge.
Research on /tool sniffer tomorrow.
Will advise

7

SUMMARY

  • I have spent a lot of time on trying to get my 2 TiVo boxes to communicate over MikroTik EoIP, but still no joy.
  • packet sniffer reveals that the two TiVo boxes are communicating, but still not operational
  • attempts to run the TiVo iOS app on iPad also fail

DETAIL

Per previous message, upping MTU changed behavior.

Reference to the MikroTik packet sniffer was very helpful.
I have been able to capture packets and can see that the devices are communicating with each other, yet something is missing.
My sniffer pcap files from the two locations indicate that all packets are properly making it across the EoIP tunnel.

TiVo has an iOS app for iPad.
I thought it might use similar auth and streaming mechanisms.
On TiVo iPad the setup process for LAN streaming (across the EoIP tunnel) fails to complete properly.

Will advise.

8

It may TiVo is looking for bridged/routed connections. One way be the packet’s TTL, although I’m not sure TTL be changed since bridged… but if it’s not use TTL = 65, you can “reset” the TTL on the far end using /ip/firewall/mangle. The Mikrotik LTE docs show the commands for T-Mobile, but TiVo may be doing something similar (in way different context but config be similar):
https://help.mikrotik.com/docs/spaces/ROS/pages/30146563/LTE#LTE-Avoidingtetheringspeedthrottling

9

Rest assured it is not, bridging does not touch the contents of the Ethernet frame being transported.

But another point, what is the value of dont-fragment in the EoIP settings? Does Wireshark show any packets between the devices that has this IP header bit set?

10

Could latency be an issue, like their protocol requires very low delays so it works over physical LAN but not when tunneled over the Internet (or even very long fiber - can’t go faster than speed of light in glass, that alone adds about 1ms per 100km to the RTT).

11

But another point what is the value of > dont-fragment > in the EoIP settings?

dont-fragment is set to no on both ends of the EoIP tunnel.
MTU is set to 1500 on both ends of the EoIP tunnel.


Does Wireshark show any packets between the devices that has this IP header bit set?

Yes, many packets have the DF Don’t Fragment flag set in the IP header
I observe this using the following wireshark display filter: ip.flags.df == 1


I think there is no fragmentation or packet loss occurring across the tunnel when MTU == 1500
I used the following wireshark display filter: ip.flags.mf == 1 || ip.frag_offset > 0

To cross-check to see that I am doing it correctly, I did the following test:

  • unplug ethernet jack on SE (southeast) TiVo
  • started MikroTik packet sniffers on the ethernet ports on both ends of the EoIP tunnel, with filtering to only the SE TiVo MAC addr
  • plug in ethernet jack on SE TiVo
  • observe on television screen that SE TiVo cannot fully connect to NE (northeast) TiVo
  • unplug the the ethernet jack on SE TiVo
  • stop both sniffers
  • download pcap files and display in 2 different instances of wireshark

When I run this test with lower MTU==300 then I see (in wireshark) many fragmented packets on both ends of the EoIP tunnel.
When I run this test with restored MTU==1500 then I do not see any fragmented packets on either end of the EoIP tunnel.
So, I do not think fragmentation is occurring when MTU==1500


I do not think any packets are being dropped when MTU==1500

After going through the above sequence, I compare the pcap files from the ports on each end of the tunnel.
The packet counts of the 2 pcap files are 1595 vs 1597.
The difference is 2 extra ARP packets at the end, caused by the time delay in shutting down the 2 sniffers.
Otherwise, the packet sequence is exactly the same.

12

Yes, I am beginning to fear that they are explicitly checking for latency.

The tunnel is over a site-to-site OpenVPN from two locations more than 1500km apart.
Ping times (right now) are very stable … here is the result of 100 pings from SE MikroTik hEX <=> OpenVPN <=> NE MikroTik hEX:
sent=100 received=100 packet-loss=0% min-rtt=59ms avg-rtt=61ms max-rtt=76ms

13

Perhaps, TiVo were pretty aggressive long ago at trying to enforce DRM stuff. I don’t got anything … if you’re looking at the packet traces and see same ones on both end…

14

I guess one long-shot option MIGHT be use /tool/traffic-gen to generate the broadcast packets. Since you have the traces, that might not be so difficult to try. Where it work/help IDK.

15

In trying to reconfigure port mirroring on remote UniFi switch I did something to bring the network down.
I had to get a friend to go over and move a patch cable.
Now I am afraid to continue experimentation remotely.

Will advise

16

You can try the following setup to experiment locally using one RouterOS device:

  1. Assign 2 IP addresses at /ip/address to the lo interface
  2. Set up a EoIP tunnel using [1] addresses, set MTU to something absurdly high, e.g. 65000; this tunnel will simulate Internet
  3. Assign 2 more IP address at /ip/address, this time one per [2] EoIP interface
  4. Set up one more EoIP tunnel using [3] addresses, set MTU to 1500; this tunnel will simulate your EoIP tunnel
  5. Make two bridges, one bridge with TiVO interface and one of the [4] EoIP interfaces and another bridge with iPad WiFi AP interface and the remaining [4] EoIP interface

Make sure it works in this “ideal” conditions. Then you can try degrading network conditions by changing MTU of either of the EoIP tunnels and /queue to limit bandwidth.