RTSP Helper

Regarding RTSP helper which is the actual connection tracking module. We will look into this and this one is the one I was talking about there must be a very good reason why after 15 years it is still not a part of the Linux kernel.



An RTSP helper is no more appropriate for inclusion in the kernel than is an HTTP proxy. This is very much userland type code.

Most of the commercial video broadcasting services does include optional packages for RTSP for the aim of configuration purposes like bandwidth management etc. It’s a typical service domain ie does not fit natural for use in a router.

https://github.com/aler9/rtsp-simple-server

How do you deal with NAT then, if it is not on the router itself? Unless something can translate this into UPnP requests mapping ports dynamically (which will be in fact a very nice and clean solution) I cannot see how these will things can operate NAT (see previous perfectly-elaborated response from @tangent)

Regards!

It would be saddening if my response to your request for more information provided the reason why you aren’t going to address this real and widespread problem.

The thing is, I don’t need this feature myself. I’ve been using RTSP for decades now, but purely inside the LAN, so the NAT problem doesn’t affect me. I was responding only to give you more detail about the scope of the problem and to give you a possible solution to it.

I was surprised to be pointed at a conntrack module for this. I haven’t dug into its code, but I suspect it can cover only a subset of the cases. Again, RTSP is complicated enough that complete handling requires more code than is appropriate for a kernel module. What I can’t answer for you is whether that subset covers a large enough proportion of those wanting this feature to be worth integrating that kernel module anyway. It may well be a case of “good enough is good enough.”

Up-thread, we see people using the acryonym “ALG”, which I assume means “application level gateway”, another name for a proxy on a layer 7 service. If that read is correct, some of MikroTik’s competitors are solving this with one of these userspace level proxies.

This is not a niche corner case. RTSP is an old, well-established Internet standard protocol, with RFCs and everything. It is being eclipsed by the likes of HLS and simple HTTP download-and-playback type “streaming”, but it will only dwindle on the time scale of infrastructure replacement. I don’t need to lecture MikroTik about how long it takes to turn whole infrastructures over, now do I? :wink:


RTSP proxy will not be able to solve the NAT problem anyway.

Sure it can. Firewall rules redirect the outbound RTSP requests to the proxy, the proxy on the gateway makes the media request, then forwards the replies back to the client that originally asked for them.

The only question in my mind is whether the more lightweight conntrack method suffices for a sufficient percentage of real-world cases as to be good enough to not bother solving the full complexity that is RTSP and its associated file formats and protocols.


there must be a very good reason why after 15 years it is still not a part of the Linux kernel.

Linux is largely driven by big business’s needs these days, not by random end users. Which big business would be served by making it easier for people to consume their media the way they want to?

Now ask this: which collection of big businesses have a deep and broad history of preventing every little technological advance in consumption convenience until they work out a way to lock it down first?

I do not suggest that RTSP proxying/NAT conntracking is any useful step toward movie piracy or anything like that, but to show that the companies poised to be the biggest advocates in this problem space instead have a vested interest in maintaining their closed systems. Why would they help anyone escape them, no matter how legal the use case?

The same goes for the IP cam people: every one of the big companies either sells a security appliance or partners with companies that provide them. If a NAT gateway in the middle of the LAN breaks that appliance, that means the company can sell two appliances! Again, why would the company advocate for getting an RTSP conntracking module into the kernel?

And the same goes for the likes of AirPlay receivers: Apple wants you to buy another HomePod and connect to it through their $/month cloud service, not redirect the RTSP stream through a NAT to play back on a Raspberry Pi.

This sort of problem is right in line with MikroTik’s mission: someone set up an annoying barrier, and we need technology that will help us to build a bridge over it.

Yeah, mr Tangent’s post was well formulated and exactly in line with my opinion.

RTSP (and its related pals) shows up in many different shapes and forms thus a build-in protocol helper would have to implement all the different variants. That’s why many commercial streaming broadcast software are tailored for a specific echo system often with specific CPEs (using tailor made fw). Same goes for surveillance software that need to support old cams with special configurations to work with NAT/CGNAT using techniques like hole punching (eg STUN) and/or specifying open ports.

I can’t thank enough Mr/Mrs. @tangent for the kind of replies he/she is giving, and the pretty well knowledge about RTSP demonstrated. And I do agree with you @Larsa that RTSP has many different shapes or implementation variants. But let’s be clear on what we need, that is fairly simple: something (call it ALG, or the magic man in the middle) that intercepts an RTSP session opening request (very well known TCP 554 connection) and, reading the response of this request, dynamically map the UPD data ports related to this connection to IP that initiates the connection. That’s it, nothing else.

In fact, we have been able to workaround this behavior ourselves (search in the forum link I shared before a user called “Toniros”, and the script he built) with a pretty nasty solution: you mark the connection with mangle for any new TCP on 554 and add the source IP to a list. And program a script that is constantly reading that list and creating a DMZ mapping in NAT for that address.
When you change TV channel and ask for a new stream, process will start again, and so on. As the rest of connections are related, you don’t even need the DMZ rule to be on your source IP forever, you can swap it with another tv box at home.
Problem? In my opinion, even when it works, the solution is not elegant at all (to be honest, it is rubbish). And it does comes with problems when you have more than one tv box at home, where you need to wait your turn when you change TV channel for the script to run and to identify you as source IP to update the DMZ rule… and you may even have a collision with other tv boxes, if both of them request different streams at the same time.

All of this will be solved by this kind of ALG, that map whatever port the server is telling you it will deliver the data flow, in this particular moment, for this particular channel and this particular source IP (tv box). And this is totally transparent for you, that is how the internet provider CPE works. It is something a router should be doing? Probably not, but the problem was there and other brands have already sorted out. We can’t we?

As @tangent wisely said: for me it is clearly in line with Mikrotik philosophy, sort out barriers using technology in a smart way.

Kind regards, and thanks again both for your contributions.

Just to be clear, are we talking about the end user ie consumer market or the business side?

End users.

Regards.

They should open source routeros and routerboot just like red hat and sell support service.

Routeros should be merged with openwrt project IMHO

But that’s just a dream, because, people are greedy

Mikrotik, could you give me implementation dates?

Like… never?
http://forum.mikrotik.com/t/rtsp-helper/146549/1

I don't see that as a definitive answer. mrz expressed ignorance of what we wanted. I told him, then several people agreed and amplified it. Now MikroTik knows what we want. The question then is, when do we get it, if ever?

This is another instance where the feature can be implemented as a Docker container, so perhaps the question devolves to when we get Docker back. This needn't be a RouterOS core feature: the RTSP helper doesn't need to manipulate routes or operate down at the kernel's conntrack level. It can, but it doesn't have to. It can be implemented as a userspace proxy.

Since the concept “RTSP” and related protocols contains so many different levels and spans all the way from L3 to L7, one has to be very specific about the needs.

For example a standard port opening helper on L3 (aka NAT helper) may only offer a tiny part of the solution that also might be application dependent. Also, on the higher level the term ALG (Application Level Gateway) might mean there is a substantial need for specific intelligence to be implemented for handling various application states (aka RTSP Appliance) that probably is better suited for Docker

General info

Some RFC’s

Is why,
http://forum.mikrotik.com/t/rtsp-helper/146549/1

Just like that.

And even a "pure and simple" L3 helper might need different profiles depending on the purpose ie iptv (manual or preset isp profiles), the ability to change port numbers, enabling RTSPU and so on, etc...

Yes, which is why I keep trying to talk y'all out of this nonstandard kernel module. Not only can this be done in userspace, with a bit of help from standard RouterOS firewall rules, it will have wider scope if done that way.

You will probably be able to create a combination of all imaginable number of RTSP solutions just by using standard fw rules and a container appliance.

Forgot to add that regarding surveillance and similar solutions, professional installations for NVR/DVR never ever rely on “automatic” features like ALG/NAT-helpers since the behavior almost always differs depending on the manufacturer of the network equipment, different models and even different implementations within the same model series.

Btw in this context, I believe that Containers will allow a huge opportunity for various “plug-in functions” that will expand the usage of Mikrotik as specially adapted “appliances” for all sorts of specialized functionallity (USP).

+1 to Tik support on this.

RTSP NAT Traversal needs to be implemented at Layer 7 by the app/device vendor using STUN, TURN and ICE not at the router.

At router level, you should ensure to do NAT correctly using Netmap to ensure STUN works seamlessly as the ports of the internal device IP will match the port of the public facing IP – This way P2P connectivity can be established even behind CGNAT without port forwarding directly.
http://forum.mikrotik.com/t/how-to-edge-router-and-bng-optimization-for-isps/150007/1

The real solution, stop whining and get on IPv6.

With 7.4beta4, they’ve given you the tools to solve it, using Containers.


RTSP NAT Traversal needs to be implemented at Layer 7 by the app/device vendor using STUN, TURN and ICE not at the router.

That’s one way to achieve it, but as long as you have a single public IP, you can also do it with an RTSP proxy (a.k.a. RTSP ALG) at the source NAT point so the proxy can make similar connections outbound and relay the reply streams to the host that asked for them.

This one looks plausible. I tested building it under the Docker Desktop cross-compilation tool chain, and it does build for ARMv7, requiring ~60 MB of storage space. That’s as far as I took it, because I don’t need it here, so I don’t have a good test case for configuring and evaluating it.


This way P2P connectivity can be established even behind CGNAT

If that’s going on, then I think the RTSP proxy linked above will not work without an underlying L3 tunnel out to somewhere sane. (e.g. VPN, SSH SOCKS proxy, etc.)


The real solution, stop whining and get on IPv6.

It isn’t a panacea: big mobile carriers often give you IPv6 and CGNAT. Fun! :face_vomiting:

You totally ignored the main point – Doing NAT correctly using netmap instead of src nat: