Community discussions

MikroTik App
 
User avatar
sgtoartilleria
just joined
Topic Author
Posts: 7
Joined: Sat Oct 03, 2015 11:20 pm

RTSP Helper

Sun Jan 31, 2021 12:19 pm

Hello, I want to ask if you consider incorporating RTSP helper in new versions of RouterOS.

Thank you very much
 
tx6376
just joined
Posts: 12
Joined: Tue Feb 02, 2021 8:35 pm

Re: RTSP Helper

Tue Feb 02, 2021 8:38 pm

+1
Please, provide RTSP ALG
 
EvgeniyV
just joined
Posts: 6
Joined: Sun Oct 28, 2018 5:49 pm

Re: RTSP Helper

Tue Nov 23, 2021 6:22 pm

As a video surveillance system integrator, I am frustrated that this feature is missing.
+1
 
tx6376
just joined
Posts: 12
Joined: Tue Feb 02, 2021 8:35 pm

Re: RTSP Helper

Tue Feb 01, 2022 3:35 pm

News on RouterOS v7 ?
Thanks.
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 4246
Joined: Sun May 01, 2016 7:12 pm
Location: California
Contact:

Re: RTSP Helper

Tue Feb 01, 2022 10:45 pm

News on RouterOS v7 ?
Thanks.
It's not directly listed in IP>Firewall>Service Ports under 7.2rc3.

But have you tried using the "sip" ALG there with your RTSP port listed? ALG really deals with SDP, so the "sip" ALG may not care it's actually RTSP. Hard to know, but might be possible.

Alternatively, Mikrotik does support UPnP which some/most cameras support, that combined with scripts/firewall rules might also work to deal with multiple subnets of cameras. In UPnP, you can control the "internal"/"external" interface – and nothing says the "external" UPnP interface has to be a WAN.
 
jhbarrantes
Frequent Visitor
Frequent Visitor
Posts: 56
Joined: Wed Aug 21, 2019 2:56 pm

Re: RTSP Helper

Tue Feb 08, 2022 11:45 pm

We definitely need this to be implemented in RouterOS v7. The lack of this feature is negatively affecting to all MikroTik users in Spain, under the major ISP provider in this country, Movistar. This ISP uses this helper for mapping UDP ports for IPTV on demand traffic. Flow starts with a tcp packet to 554 that can identify in mangle and mark accordingly, but all we can do later is to DMZ all traffic to the source IP opening this communication, due to the lack of this helper, with the help of a script.
But, if you have more than one tv box (very common), you need script constantly running for doing this tricky NAT mapping, alternating between destination IPs. With the helper, the random negotiated ports for UDP RTSP traffic will be automatically mapped to the source IP opening this connection, but, as we don’t know these port range in advance, we cannot use, as far as I could understand from the last comment, the SIP ALG.

Correct me if I’m wrong, we may be totally wrong and missing some magic hidden feature, but after squeezing my brain a lot, I can’t figure out how to workaround the lack of this very basic feature you can find, in other hand, in cheap crappy routers, but not in MikroTik. My only guess is this is already implemented in a way we don’t understand, and we are just stupid not finding it out. Or it is just a totally forgotten feature, because nobody else uses.

Guys from MikroTik, please take this request seriously. You have a very nice community of users loving your products in Spain. But the lack of this feature is a very annoying thing we have been suffering from quite a long time already. So please, if it is in your hands, do implement it, for Gods shake. And, if it is already implemented in a weird obscure manner, please let us know, we will be glad to keep learning.

All the best guys!
 
volkirik
Member Candidate
Member Candidate
Posts: 212
Joined: Sat Jul 23, 2016 2:03 pm

Re: RTSP Helper

Mon Feb 14, 2022 3:31 pm

indeed

+1 for rtsp helper (alg)
 
tx6376
just joined
Posts: 12
Joined: Tue Feb 02, 2021 8:35 pm

Re: RTSP Helper

Thu Feb 17, 2022 10:21 am

+1 rtsp helper (alg)
Thanks @jhbarrantes Gracias
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 4246
Joined: Sun May 01, 2016 7:12 pm
Location: California
Contact:

Re: RTSP Helper

Fri Feb 18, 2022 2:29 am

News on RouterOS v7 ?
Thanks.
It's not directly listed in IP>Firewall>Service Ports under 7.2rc3.

But have you tried using the "sip" ALG there with your RTSP port listed? ALG really deals with SDP, so the "sip" ALG may not care it's actually RTSP.
Nope, it SIP ALG needs at least some SIP BEFORE fixing the SDP fields (which is used by BOTH SIP and RTSP) – so overloading the SIP ALG is NOT a workaround here.

I don't deal with cameras that often, but avoid ever needing ALG for anything... But get that not always possible.

What I do know is OpenWRT does support an ALG for RTSP. Seems it's a kernel patch that's likely missing in the kernel that RouterOS uses: https://openwrt.org/packages/pkgdata/km ... elper-rtsp But obviously a missing kernel patch isn't fixable by an MT admin.
 
jhbarrantes
Frequent Visitor
Frequent Visitor
Posts: 56
Joined: Wed Aug 21, 2019 2:56 pm

Re: RTSP Helper

Sat Feb 19, 2022 5:14 pm



It's not directly listed in IP>Firewall>Service Ports under 7.2rc3.

But have you tried using the "sip" ALG there with your RTSP port listed? ALG really deals with SDP, so the "sip" ALG may not care it's actually RTSP.
Nope, it SIP ALG needs at least some SIP BEFORE fixing the SDP fields (which is used by BOTH SIP and RTSP) – so overloading the SIP ALG is NOT a workaround here.

I don't deal with cameras that often, but avoid ever needing ALG for anything... But get that not always possible.

What I do know is OpenWRT does support an ALG for RTSP. Seems it's a kernel patch that's likely missing in the kernel that RouterOS uses: https://openwrt.org/packages/pkgdata/km ... elper-rtsp But obviously a missing kernel patch isn't fixable by an MT admin.

I hope they will include this soon, now that v7 brings a new kernel and, as you mention, it is already supported at kernel level, such as in OpenWRT. All spaniard colleagues will be very happy if this ever happen.

Hope MKT admins could read that @normis, @emils, @sergejs

Thanks!
 
ToniRos
just joined
Posts: 2
Joined: Mon Dec 11, 2017 12:20 am

Re: RTSP Helper

Wed Mar 23, 2022 11:16 pm

+1 for rtsp helper (alg)

Thanks.
 
volkirik
Member Candidate
Member Candidate
Posts: 212
Joined: Sat Jul 23, 2016 2:03 pm

Re: RTSP Helper

Mon Mar 28, 2022 4:44 pm

MikroTik support #[SUP-78171]: hello there (regarding RTSP helper)
-------- Original message --------
Subject: hello there (regarding RTSP helper)
Date: Mon, 28 Mar 2022 16:39:41 +0300
To: support@mikrotik.com


hello there

we definitely need RTSP helper (alg)

openwrt has it. but we love routerOS

could you please add it to ROS?

viewtopic.php?p=920991#p920991


thanks and regards
Last edited by volkirik on Mon Mar 28, 2022 8:59 pm, edited 1 time in total.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 21759
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: RTSP Helper

Mon Mar 28, 2022 5:09 pm

Not that its likely or a solution but it moviestar doing something non-standard, why is this only Spain?
How should they provide their service? Perhaps that is also another angle to tackle?
In other words, I can see this as more likely if its a global phenomena.......
 
jhbarrantes
Frequent Visitor
Frequent Visitor
Posts: 56
Joined: Wed Aug 21, 2019 2:56 pm

Re: RTSP Helper

Mon Mar 28, 2022 8:49 pm

Is not just about Movistar using RTSP for their VOD solution (by the way, about 3,8 million customers only in Spain.. if only tiny percent currently uses or think on using Mikrotik for their setups... these are LOTS of routers). Many CCTV camera system also uses the same protocol and will highly benefit from having this kind of helper implemented in our routers. What I can't really guess to understand is how this helper is implemented in many crappy routers that I will never buy, but not yet in Mikrotik.

Hope they finally pay attention to this, think they don't really imagine how many potential customers they could have in Spain if this is properly implemented.

Regards!
 
volkirik
Member Candidate
Member Candidate
Posts: 212
Joined: Sat Jul 23, 2016 2:03 pm

Re: RTSP Helper

Mon Mar 28, 2022 8:54 pm

MikroTik support #[SUP-78171]: hello there (regarding RTSP helper)
Mon, 28 Mar 2022 17:59:00 +0300 (EEST)
Artūrs C. (Jira) <support@mikrotik.com>


—-—-—-—
Please REPLY ABOVE THIS LINE ^ (for faster response, use our support portal).

Hello,

Thank you for contacting MikroTik Support.

We do not have any plans to add such a feature at the moment, but if more users will request it, we will see how this can be implemented.

Best regards,
Artūrs C.
 
jhbarrantes
Frequent Visitor
Frequent Visitor
Posts: 56
Joined: Wed Aug 21, 2019 2:56 pm

Re: RTSP Helper

Mon Mar 28, 2022 9:17 pm

MikroTik support #[SUP-78171]: hello there (regarding RTSP helper)
Mon, 28 Mar 2022 17:59:00 +0300 (EEST)
Artūrs C. (Jira) <support@mikrotik.com>


—-—-—-—
Please REPLY ABOVE THIS LINE ^ (for faster response, use our support portal).

Hello,

Thank you for contacting MikroTik Support.

We do not have any plans to add such a feature at the moment, but if more users will request it, we will see how this can be implemented.

Best regards,
Artūrs C.

That's exactly what I mean. How can we actually increase that number? What is the correct way of requesting this feature, using the appropriated channel? I can spread out this if correct instructions are given, so many people can actually come and request it.

Kind regards!
 
volkirik
Member Candidate
Member Candidate
Posts: 212
Joined: Sat Jul 23, 2016 2:03 pm

Re: RTSP Helper

Mon Mar 28, 2022 9:20 pm

they seem to be counting posts in forum imho

so your users/friends can register and post their request in this forum topic

thats all i know.
 
jhbarrantes
Frequent Visitor
Frequent Visitor
Posts: 56
Joined: Wed Aug 21, 2019 2:56 pm

Re: RTSP Helper

Mon Mar 28, 2022 9:30 pm

they seem to be counting posts in forum imho

so your users/friends can register and post their request in this forum topic

thats all i know.

Will still be valid if we keep a single post like this top high in comments? I mean, for not having too many replicas flowing throughout the forum, asking for the same.

Thanks!
 
volkirik
Member Candidate
Member Candidate
Posts: 212
Joined: Sat Jul 23, 2016 2:03 pm

Re: RTSP Helper

Mon Mar 28, 2022 9:41 pm

customer services do always count calls, tickets, emails, forum posts etc.

and report to their supervisors... you get the idea...

the more demand they receive from customers for a feature, the more priority it receives in implementation.

dunno about forum rules, i try to be nice and friendly, thats all.

they also should be tolerating unless it becomes spam.
 
jhbarrantes
Frequent Visitor
Frequent Visitor
Posts: 56
Joined: Wed Aug 21, 2019 2:56 pm

Re: RTSP Helper

Mon Mar 28, 2022 9:57 pm

customer services do always count calls, tickets, emails, forum posts etc.

and report to their supervisors... you get the idea...

the more demand they receive from customers for a feature, the more priority it receives in implementation.

dunno about forum rules, i try to be nice and friendly, thats all.

they also should be tolerating unless it becomes spam.

OK; let's try first by inviting people to comment on this topic, and if it doesn't have the expected result, give them the ability of opening their own support tickets.

Thanks!
 
volkirik
Member Candidate
Member Candidate
Posts: 212
Joined: Sat Jul 23, 2016 2:03 pm

Re: RTSP Helper

Mon Mar 28, 2022 10:25 pm

thank you, too, friend!
 
dalemk
just joined
Posts: 1
Joined: Mon Mar 28, 2022 11:07 pm

Re: RTSP Helper

Mon Mar 28, 2022 11:10 pm

Hi,

I want RTSP helper implemented in future versions of RouterOS, too.

Please!
 
tx6376
just joined
Posts: 12
Joined: Tue Feb 02, 2021 8:35 pm

Re: RTSP Helper

Mon Mar 28, 2022 11:19 pm

+1 for rtsp helper (alg)

Thanks.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 12522
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: RTSP Helper

Mon Mar 28, 2022 11:25 pm

Don't waste time, instead of making useless posts, contact your sellers, who buy MikroTik hardware to ask for that feature,
this is a user forum and the request must be written to support@mikrotik.com

They do matter, not you.
 
jagarciam
just joined
Posts: 2
Joined: Fri Apr 04, 2014 3:19 pm

Re: RTSP Helper

Tue Mar 29, 2022 9:45 am

+1
Please, provide RTSP ALG
 
volkirik
Member Candidate
Member Candidate
Posts: 212
Joined: Sat Jul 23, 2016 2:03 pm

Re: RTSP Helper

Tue Mar 29, 2022 11:08 am

I think we need to be more optimist, here..
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 12522
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: RTSP Helper

Tue Mar 29, 2022 12:57 pm

Just realist... Count only who buy hardware...
 
volkirik
Member Candidate
Member Candidate
Posts: 212
Joined: Sat Jul 23, 2016 2:03 pm

Re: RTSP Helper

Tue Mar 29, 2022 2:09 pm

We are real buyers , they are traders

Just register your license into your Mt account
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 12522
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: RTSP Helper

Tue Mar 29, 2022 3:41 pm

I have more than 10000 devices...
But when I post something no one go to see how many devices I have on account..........
 
volkirik
Member Candidate
Member Candidate
Posts: 212
Joined: Sat Jul 23, 2016 2:03 pm

Re: RTSP Helper

Tue Mar 29, 2022 4:03 pm

Sad to hear 😔
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 7174
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: RTSP Helper

Tue Mar 29, 2022 5:25 pm

A little bit in general about requests like: Please implement "four_letter_combination" feature. Without a single word on what exactly is assumed by this letter combination, followed by several +1 posts.
It is immediately a red flag and most likely will be ignored.
And stating that this "few letter combination" feature is needed by thousands of users, but at the same time agitating to ask your grandma to make an account and add +1 post adds another red flag.
There must be at least some description of when and how the requested feature is expected to be used, in most cases solution is already possible with existing RouterOS functionality.

Regarding this particular RTSP helper, which is supposedly such a basic feature.
By doing a quick search it was first implemented in 2003 since then there is very little information. There are some forks in GitHub, but that's it. There must be a very good reason why this "basic" feature is not added to the Linux kernel for over 15 years. So there is no simple toggle "enable RTSP helper".

So should we waste the resources to research this protocol and write the helper just for a few grandmas +1?

From this topic, there is only one genuine post that can be taken into account and that is by jhbarrantes about the actual use case by Movistar.
 
User avatar
jbl42
Member Candidate
Member Candidate
Posts: 223
Joined: Sun Jun 21, 2020 12:58 pm

Re: RTSP Helper

Tue Mar 29, 2022 9:23 pm

There must be a very good reason why this "basic" feature is not added to the Linux kernel for over 15 years. So there is no simple toggle "enable RTSP helper".
While I personaly do not have a need for a RTSP proxy in ROS, there still is a very good reason there is none in the Linux kernel: This is a propblem that can and should be solved by a user space application. So the kernel maintainers will hardly add such a thing.

What we used with success is rtsp-simple-server running in a docker image on Linux boxes or on Synology NAS.
Once ROS 7 gets back docker support, it can also be run on ROS acting as a rtsp proxy
 
jhbarrantes
Frequent Visitor
Frequent Visitor
Posts: 56
Joined: Wed Aug 21, 2019 2:56 pm

Re: RTSP Helper

Tue Mar 29, 2022 10:13 pm

A little bit in general about requests like: Please implement "four_letter_combination" feature. Without a single word on what exactly is assumed by this letter combination, followed by several +1 posts.
It is immediately a red flag and most likely will be ignored.
And stating that this "few letter combination" feature is needed by thousands of users, but at the same time agitating to ask your grandma to make an account and add +1 post adds another red flag.
There must be at least some description of when and how the requested feature is expected to be used, in most cases solution is already possible with existing RouterOS functionality.

Regarding this particular RTSP helper, which is supposedly such a basic feature.
By doing a quick search it was first implemented in 2003 since then there is very little information. There are some forks in GitHub, but that's it. There must be a very good reason why this "basic" feature is not added to the Linux kernel for over 15 years. So there is no simple toggle "enable RTSP helper".

So should we waste the resources to research this protocol and write the helper just for a few grandmas +1?

From this topic, there is only one genuine post that can be taken into account and that is by jhbarrantes about the actual use case by Movistar.

Hi @mrz, thanks for the reply,

While I do agree some "+1" comments on for posts requesting "Implement ABDC feature" could be a red flag for you, I don't think this is the kind of feature deserving such an absolute lack of attention. I agree this is probably something most people won't use, but definitely is a stopper for someone having to work with video streams. And I know it has been requested here for ages now (you can check by doing a quick search at the forum). And, don't get me wrong, but if I compare the kind of responses this request has received in this forum with this response from one of your competitors, the difference here is HUGE. For just a minute, I would like to be that Ubnt user, reading the first response, even if the feature finally came three years later.

The feature is already implemented in may other brands, some of them with their own patches, but most of them relying on this implementation. It is incorporated in all brands using OpenWRT as base image, so, why can't we do the same? I know there will be tons of reasons for not doing it, but what I would like to understand from you and your wide view from the product (obviously wider than a final user's view) why is that so difficult to add? Or even more, is this already implemented in such a way that we haven't explore it yet how to use it properly?

I also would really like to see this embebed into linux kernel, for sure! But if that is not the case, what is stopping us for just adding it as an extra module, as you do with others? Is there any major technical issue why this cannot be addressed? Or it is just a matter of "just a few will use it, so don't waste time on that". I'm telling you that because there are 3,8 million customers only in Spain with crappy Movistar's CPEs, and a huge number of these want to use your products. Specially because the kind of support we find out for your brand in forums like this is massive, and it is something you (as Mikrotik) should be proud of. Do you want to see Spain demanding Mikrotik equipment as ever before? Just pay attention to this feature and buy a big piggy bank, because you will fill it up.

From my side, I have already sent a support ticket explaining why I think this could be a really nice to have, considering both scenarios with VOD streamings and RTP video transmissions in general (live recording cameras). And the response was not that bad: As we see RTSP helper is not available in the Linux kernel so the implementation could be more complicated. We take a look if we can support such an RTSP helper in the future

But it will be great if we do have a clear guidance on how to request this such kind of features, for not falling into that red flag grandma's kind ignored posts.

All the best,

J.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 12522
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: RTSP Helper

Wed Mar 30, 2022 2:34 am

Who guarantees that if that change is applied, sales of Routers in Spain will increase dramatically?
And without causing further problems to other users who don't use it?
And delays in the development of v7?

Too many times I have heard similar phrases, all bullshit.

Several times, at the beginning, when we had to cover an area it seemed who knows how many users then subscribed...
You spend a lot of money and subscribe in 4 instead of 400.

Now, instead, I put the radio links where I want, instead of satisfying those two or three users who want the connection,
that make it seem like "who knows how many later"...

Usually those who ask for something saying "who knows how many then", "you'll see how many are waiting", and similar bullshit,
do it only for themselves because they want it,
and usually then nothing "miraculous" happens as promised, and any interest is lost once obtained...
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 4246
Joined: Sun May 01, 2016 7:12 pm
Location: California
Contact:

Re: RTSP Helper

Wed Mar 30, 2022 3:54 am

So should we waste the resources to research this protocol and write the helper just for a few grandmas +1?
Nope. But container support would provide a more self-service solution to these kinda problems. Hopefully that comes back soon.
 
User avatar
tangent
Forum Guru
Forum Guru
Posts: 1655
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: RTSP Helper

Wed Mar 30, 2022 5:38 am

Without a single word on what exactly is assumed by this letter combination

RTSP is one of those protocols designed in the early days of the commercial Internet — 1996 — before NAT became a common thing. Its creators therefore felt free to continue the old assumption of a symmetric network: all clients are potentially servers, and vice versa. NAT breaks RTSP. Without some kind of help to get the port numbers and such sorted out, a client can make an RTSP connection, but the returning media streams cannot transit the router.

This is because RTSP behaves much like classic active-mode FTP: the client connects out to the server, tells it what it wants, and the server then connects back to the client on a separate port number — or pair of port numbers in the case of separate video + audio streams — to send the requested media streams. In the case of FTP, it's only one file at a time, but with media streams, you could have a security appliance pulling a dozen of them in parallel, so there isn't a simple rule you can hard-code like "forward port 20 back to the client when it connects out on port 21" as there was with active-mode FTP.

There were more examples of this type of protocol decades ago, but most have been replaced by something designed to work with NAT interference. Classic example: FTP first went passive-only, then got replaced in virtually all functional cases by either SFTP for read/write use cases or HTTP for read-only cases. Result? Now we don't need "FTP helpers" on our NAT routers.

The thing about RTSP is that it's Ⓐ used in a lot of broadcast-grade IPTV type applications, where there's tremendous inertia against moving to something that doesn't have RTSP's problems (e.g. HLS streaming); and Ⓑ used in a lot of embedded devices like IP cams, AirPlay streaming receivers, etc, where it can't be reasonably replaced at all. If you want to wait another decade or so, the need for RTSP will dwindle to about the popularity of fountain pens and AM radio, but like all tech that was once widespread, it will never go away entirely.

A big problem with implementing an RTSP helper is that it's a fairly complicated protocol. For one, it's basically an extension of HTTP/1.0; not nearly so bad as HTTP/1.1 or its follow-ons, but still nontrivial. For another, you can think of RTSP as the "VCR buttons" protocol for controlling other protocols, so some knowledge of SDP, RTP, and other streaming media formats and protocols is required in order to work out how to perform the desired "helper" actions.

The only practical advice I have for y'all at MikroTik is to look into Live555. The library is LGPL, but if that's a non-starter, I do believe the library's author — co-inventor of RARP + TFTP network booting and prior owner of "live.com" before Microsoft bought it off him — will be happy to offer you a buy-out for closed-source commercial use. If you can use it, it'll save you a tremendous amount of time in building the feature. They've got a pre-built RTSP proxy server which may be what your customers actually need.

Or, get containers working again so we can spin up a Live555 Proxy Server on our own. :)

is already possible with existing RouterOS functionality.

Sadly, I do believe that is not the case here. RTSP proxying requires digging deep into the protocol and rewriting things.

not added to the Linux kernel for over 15 years

An RTSP helper is no more appropriate for inclusion in the kernel than is an HTTP proxy. This is very much userland type code.
 
volkirik
Member Candidate
Member Candidate
Posts: 212
Joined: Sat Jul 23, 2016 2:03 pm

Re: RTSP Helper

Wed Mar 30, 2022 1:07 pm

They should open source routeros just like red hat and sell support service.

Routeros should be merged with openwrt project IMHO

But that's just a dream, people are greedy
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 7174
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: RTSP Helper

Wed Mar 30, 2022 3:33 pm

but most of them relying on this implementation.
Yes, that is the one we were looking at and one I was talking about.

Yet again we come to the conclusion that by "implement RTSP helper" people mean different things. That is what I was talking about initially because a post "implement xyz" can mean one thing but +1 for such a post could mean something completely different.
RTSP proxy is the user space application and not something that we should be focusing on. When KVM, docker, or whatever other virtualization methods will be added back to the RouterOS, then all these proxies, servers, etc could be running there. AFAIK RTSP proxy will not be able to solve the NAT problem anyway.

Regarding RTSP helper which is the actual connection tracking module. We will look into this and this one is the one I was talking about there must be a very good reason why after 15 years it is still not a part of the Linux kernel.
 
jhbarrantes
Frequent Visitor
Frequent Visitor
Posts: 56
Joined: Wed Aug 21, 2019 2:56 pm

Re: RTSP Helper

Wed Mar 30, 2022 4:22 pm

Without a single word on what exactly is assumed by this letter combination

RTSP is one of those protocols designed in the early days of the commercial Internet — 1996 — before NAT became a common thing. Its creators therefore felt free to continue the old assumption of a symmetric network: all clients are potentially servers, and vice versa. NAT breaks RTSP. Without some kind of help to get the port numbers and such sorted out, a client can make an RTSP connection, but the returning media streams cannot transit the router.

This is because RTSP behaves much like classic active-mode FTP: the client connects out to the server, tells it what it wants, and the server then connects back to the client on a separate port number — or pair of port numbers in the case of separate video + audio streams — to send the requested media streams. In the case of FTP, it's only one file at a time, but with media streams, you could have a security appliance pulling a dozen of them in parallel, so there isn't a simple rule you can hard-code like "forward port 20 back to the client when it connects out on port 21" as there was with active-mode FTP.

There were more examples of this type of protocol decades ago, but most have been replaced by something designed to work with NAT interference. Classic example: FTP first went passive-only, then got replaced in virtually all functional cases by either SFTP for read/write use cases or HTTP for read-only cases. Result? Now we don't need "FTP helpers" on our NAT routers.

The thing about RTSP is that it's Ⓐ used in a lot of broadcast-grade IPTV type applications, where there's tremendous inertia against moving to something that doesn't have RTSP's problems (e.g. HLS streaming); and Ⓑ used in a lot of embedded devices like IP cams, AirPlay streaming receivers, etc, where it can't be reasonably replaced at all. If you want to wait another decade or so, the need for RTSP will dwindle to about the popularity of fountain pens and AM radio, but like all tech that was once widespread, it will never go away entirely.

A big problem with implementing an RTSP helper is that it's a fairly complicated protocol. For one, it's basically an extension of HTTP/1.0; not nearly so bad as HTTP/1.1 or its follow-ons, but still nontrivial. For another, you can think of RTSP as the "VCR buttons" protocol for controlling other protocols, so some knowledge of SDP, RTP, and other streaming media formats and protocols is required in order to work out how to perform the desired "helper" actions.

The only practical advice I have for y'all at MikroTik is to look into Live555. The library is LGPL, but if that's a non-starter, I do believe the library's author — co-inventor of RARP + TFTP network booting and prior owner of "live.com" before Microsoft bought it off him — will be happy to offer you a buy-out for closed-source commercial use. If you can use it, it'll save you a tremendous amount of time in building the feature. They've got a pre-built RTSP proxy server which may be what your customers actually need.

Or, get containers working again so we can spin up a Live555 Proxy Server on our own. :)

is already possible with existing RouterOS functionality.

Sadly, I do believe that is not the case here. RTSP proxying requires digging deep into the protocol and rewriting things.

not added to the Linux kernel for over 15 years

An RTSP helper is no more appropriate for inclusion in the kernel than is an HTTP proxy. This is very much userland type code.

I just logged in to say big THANK YOU. I hope all responses in forums were like yours.

Kind regards.
 
jhbarrantes
Frequent Visitor
Frequent Visitor
Posts: 56
Joined: Wed Aug 21, 2019 2:56 pm

Re: RTSP Helper

Wed Mar 30, 2022 4:26 pm

but most of them relying on this implementation.
Yes, that is the one we were looking at and one I was talking about.

Yet again we come to the conclusion that by "implement RTSP helper" people mean different things. That is what I was talking about initially because a post "implement xyz" can mean one thing but +1 for such a post could mean something completely different.
RTSP proxy is the user space application and not something that we should be focusing on. When KVM, docker, or whatever other virtualization methods will be added back to the RouterOS, then all these proxies, servers, etc could be running there. AFAIK RTSP proxy will not be able to solve the NAT problem anyway.

Regarding RTSP helper which is the actual connection tracking module. We will look into this and this one is the one I was talking about there must be a very good reason why after 15 years it is still not a part of the Linux kernel.

Thanks a lot for considering this @mrz. Hope to hear back form you (Mikrotik) at some point, either way to give me good news for this been implemented or just to explain what is the reason for this not going through. Anyway, thanks a lot in advance.

Kind regards.
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1604
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: RTSP Helper

Wed Mar 30, 2022 4:39 pm

Regarding RTSP helper which is the actual connection tracking module. We will look into this and this one is the one I was talking about there must be a very good reason why after 15 years it is still not a part of the Linux kernel.

An RTSP helper is no more appropriate for inclusion in the kernel than is an HTTP proxy. This is very much userland type code.

Most of the commercial video broadcasting services does include optional packages for RTSP for the aim of configuration purposes like bandwidth management etc. It's a typical service domain ie does not fit natural for use in a router.

https://github.com/aler9/rtsp-simple-server
 
jhbarrantes
Frequent Visitor
Frequent Visitor
Posts: 56
Joined: Wed Aug 21, 2019 2:56 pm

Re: RTSP Helper

Wed Mar 30, 2022 6:36 pm

Regarding RTSP helper which is the actual connection tracking module. We will look into this and this one is the one I was talking about there must be a very good reason why after 15 years it is still not a part of the Linux kernel.

An RTSP helper is no more appropriate for inclusion in the kernel than is an HTTP proxy. This is very much userland type code.

Most of the commercial video broadcasting services does include optional packages for RTSP for the aim of configuration purposes like bandwidth management etc. It's a typical service domain ie does not fit natural for use in a router.

https://github.com/aler9/rtsp-simple-server

How do you deal with NAT then, if it is not on the router itself? Unless something can translate this into UPnP requests mapping ports dynamically (which will be in fact a very nice and clean solution) I cannot see how these will things can operate NAT (see previous perfectly-elaborated response from @tangent)

Regards!
 
User avatar
tangent
Forum Guru
Forum Guru
Posts: 1655
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: RTSP Helper

Wed Mar 30, 2022 7:09 pm

Yet again we come to the conclusion that by "implement RTSP helper" people mean different things.

It would be saddening if my response to your request for more information provided the reason why you aren't going to address this real and widespread problem.

The thing is, I don't need this feature myself. I've been using RTSP for decades now, but purely inside the LAN, so the NAT problem doesn't affect me. I was responding only to give you more detail about the scope of the problem and to give you a possible solution to it.

I was surprised to be pointed at a conntrack module for this. I haven't dug into its code, but I suspect it can cover only a subset of the cases. Again, RTSP is complicated enough that complete handling requires more code than is appropriate for a kernel module. What I can't answer for you is whether that subset covers a large enough proportion of those wanting this feature to be worth integrating that kernel module anyway. It may well be a case of "good enough is good enough."

Up-thread, we see people using the acryonym "ALG", which I assume means "application level gateway", another name for a proxy on a layer 7 service. If that read is correct, some of MikroTik's competitors are solving this with one of these userspace level proxies.

This is not a niche corner case. RTSP is an old, well-established Internet standard protocol, with RFCs and everything. It is being eclipsed by the likes of HLS and simple HTTP download-and-playback type "streaming", but it will only dwindle on the time scale of infrastructure replacement. I don't need to lecture MikroTik about how long it takes to turn whole infrastructures over, now do I? 😉

RTSP proxy will not be able to solve the NAT problem anyway.

Sure it can. Firewall rules redirect the outbound RTSP requests to the proxy, the proxy on the gateway makes the media request, then forwards the replies back to the client that originally asked for them.

The only question in my mind is whether the more lightweight conntrack method suffices for a sufficient percentage of real-world cases as to be good enough to not bother solving the full complexity that is RTSP and its associated file formats and protocols.

there must be a very good reason why after 15 years it is still not a part of the Linux kernel.

Linux is largely driven by big business's needs these days, not by random end users. Which big business would be served by making it easier for people to consume their media the way they want to?

Now ask this: which collection of big businesses have a deep and broad history of preventing every little technological advance in consumption convenience until they work out a way to lock it down first?

I do not suggest that RTSP proxying/NAT conntracking is any useful step toward movie piracy or anything like that, but to show that the companies poised to be the biggest advocates in this problem space instead have a vested interest in maintaining their closed systems. Why would they help anyone escape them, no matter how legal the use case?

The same goes for the IP cam people: every one of the big companies either sells a security appliance or partners with companies that provide them. If a NAT gateway in the middle of the LAN breaks that appliance, that means the company can sell two appliances! Again, why would the company advocate for getting an RTSP conntracking module into the kernel?

And the same goes for the likes of AirPlay receivers: Apple wants you to buy another HomePod and connect to it through their $/month cloud service, not redirect the RTSP stream through a NAT to play back on a Raspberry Pi.

This sort of problem is right in line with MikroTik's mission: someone set up an annoying barrier, and we need technology that will help us to build a bridge over it.
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1604
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: RTSP Helper

Wed Mar 30, 2022 7:52 pm

How do you deal with NAT then, if it is not on the router itself? Unless something can translate this into UPnP requests mapping ports dynamically (which will be in fact a very nice and clean solution) I cannot see how these will things can operate NAT (see previous perfectly-elaborated response from @tangent) Regards!

Yeah, mr Tangent's post was well formulated and exactly in line with my opinion.

RTSP (and its related pals) shows up in many different shapes and forms thus a build-in protocol helper would have to implement all the different variants. That's why many commercial streaming broadcast software are tailored for a specific echo system often with specific CPEs (using tailor made fw). Same goes for surveillance software that need to support old cams with special configurations to work with NAT/CGNAT using techniques like hole punching (eg STUN) and/or specifying open ports.
 
jhbarrantes
Frequent Visitor
Frequent Visitor
Posts: 56
Joined: Wed Aug 21, 2019 2:56 pm

Re: RTSP Helper

Wed Mar 30, 2022 8:37 pm

I can't thank enough Mr/Mrs. @tangent for the kind of replies he/she is giving, and the pretty well knowledge about RTSP demonstrated. And I do agree with you @Larsa that RTSP has many different shapes or implementation variants. But let's be clear on what we need, that is fairly simple: something (call it ALG, or the magic man in the middle) that intercepts an RTSP session opening request (very well known TCP 554 connection) and, reading the response of this request, dynamically map the UPD data ports related to this connection to IP that initiates the connection. That's it, nothing else.

In fact, we have been able to workaround this behavior ourselves (search in the forum link I shared before a user called "Toniros", and the script he built) with a pretty nasty solution: you mark the connection with mangle for any new TCP on 554 and add the source IP to a list. And program a script that is constantly reading that list and creating a DMZ mapping in NAT for that address.
When you change TV channel and ask for a new stream, process will start again, and so on. As the rest of connections are related, you don't even need the DMZ rule to be on your source IP forever, you can swap it with another tv box at home.
Problem? In my opinion, even when it works, the solution is not elegant at all (to be honest, it is rubbish). And it does comes with problems when you have more than one tv box at home, where you need to wait your turn when you change TV channel for the script to run and to identify you as source IP to update the DMZ rule... and you may even have a collision with other tv boxes, if both of them request different streams at the same time.

All of this will be solved by this kind of ALG, that map whatever port the server is telling you it will deliver the data flow, in this particular moment, for this particular channel and this particular source IP (tv box). And this is totally transparent for you, that is how the internet provider CPE works. It is something a router should be doing? Probably not, but the problem was there and other brands have already sorted out. We can't we?

As @tangent wisely said: for me it is clearly in line with Mikrotik philosophy, sort out barriers using technology in a smart way.

Kind regards, and thanks again both for your contributions.
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1604
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: RTSP Helper

Thu Mar 31, 2022 7:06 am

Just to be clear, are we talking about the end user ie consumer market or the business side?
 
jhbarrantes
Frequent Visitor
Frequent Visitor
Posts: 56
Joined: Wed Aug 21, 2019 2:56 pm

Re: RTSP Helper

Thu Mar 31, 2022 9:32 am

Just to be clear, are we talking about the end user ie consumer market or the business side?

End users.

Regards.
 
volkirik
Member Candidate
Member Candidate
Posts: 212
Joined: Sat Jul 23, 2016 2:03 pm

Re: RTSP Helper

Fri Apr 01, 2022 7:16 pm

They should open source routeros and routerboot just like red hat and sell support service.

Routeros should be merged with openwrt project IMHO

But that's just a dream, because, people are greedy
 
User avatar
Cascuda
newbie
Posts: 34
Joined: Mon Jan 02, 2017 4:34 pm
Location: Spain
Contact:

Re: RTSP Helper

Wed May 11, 2022 2:27 pm

Mikrotik, could you give me implementation dates?
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 12522
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: RTSP Helper

Wed May 11, 2022 2:34 pm

 
User avatar
tangent
Forum Guru
Forum Guru
Posts: 1655
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: RTSP Helper

Wed May 11, 2022 2:40 pm


I don't see that as a definitive answer. mrz expressed ignorance of what we wanted. I told him, then several people agreed and amplified it. Now MikroTik knows what we want. The question then is, when do we get it, if ever?

This is another instance where the feature can be implemented as a Docker container, so perhaps the question devolves to when we get Docker back. This needn't be a RouterOS core feature: the RTSP helper doesn't need to manipulate routes or operate down at the kernel's conntrack level. It can, but it doesn't have to. It can be implemented as a userspace proxy.
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1604
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: RTSP Helper

Wed May 11, 2022 3:56 pm

Since the concept "RTSP" and related protocols contains so many different levels and spans all the way from L3 to L7, one has to be very specific about the needs.

For example a standard port opening helper on L3 (aka NAT helper) may only offer a tiny part of the solution that also might be application dependent. Also, on the higher level the term ALG (Application Level Gateway) might mean there is a substantial need for specific intelligence to be implemented for handling various application states (aka RTSP Appliance) that probably is better suited for Docker

General info
Some RFC's
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 12522
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: RTSP Helper

Wed May 11, 2022 6:17 pm

 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1604
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: RTSP Helper

Wed May 11, 2022 6:40 pm


Just like that.

And even a "pure and simple" L3 helper might need different profiles depending on the purpose ie iptv (manual or preset isp profiles), the ability to change port numbers, enabling RTSPU and so on, etc...
 
User avatar
tangent
Forum Guru
Forum Guru
Posts: 1655
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: RTSP Helper

Wed May 11, 2022 8:25 pm


And even a "pure and simple" L3 helper might need different profiles depending on the purpose ie iptv (manual or preset isp profiles), the ability to change port numbers, enabling RTSPU and so on, etc...

Yes, which is why I keep trying to talk y'all out of this nonstandard kernel module. Not only can this be done in userspace, with a bit of help from standard RouterOS firewall rules, it will have wider scope if done that way.
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1604
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: RTSP Helper

Wed May 11, 2022 8:59 pm

Yes, which is why I keep trying to talk y'all out of this nonstandard kernel module. Not only can this be done in userspace, with a bit of help from standard RouterOS firewall rules, it will have wider scope if done that way.

You will probably be able to create a combination of all imaginable number of RTSP solutions just by using standard fw rules and a container appliance.
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1604
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: RTSP Helper

Thu May 12, 2022 12:53 pm

Forgot to add that regarding surveillance and similar solutions, professional installations for NVR/DVR never ever rely on "automatic" features like ALG/NAT-helpers since the behavior almost always differs depending on the manufacturer of the network equipment, different models and even different implementations within the same model series.

Btw in this context, I believe that Containers will allow a huge opportunity for various "plug-in functions" that will expand the usage of Mikrotik as specially adapted "appliances" for all sorts of specialized functionallity (USP).
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: RTSP Helper

Sat Jun 18, 2022 3:32 pm

+1 to Tik support on this.

RTSP NAT Traversal needs to be implemented at Layer 7 by the app/device vendor using STUN, TURN and ICE not at the router.

At router level, you should ensure to do NAT correctly using Netmap to ensure STUN works seamlessly as the ports of the internal device IP will match the port of the public facing IP – This way P2P connectivity can be established even behind CGNAT without port forwarding directly.
viewtopic.php?t=176358

The real solution, stop whining and get on IPv6.
 
User avatar
tangent
Forum Guru
Forum Guru
Posts: 1655
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: RTSP Helper

Sat Jun 18, 2022 4:20 pm

+1 to Tik support on this.

With 7.4beta4, they've given you the tools to solve it, using Containers.

RTSP NAT Traversal needs to be implemented at Layer 7 by the app/device vendor using STUN, TURN and ICE not at the router.

That's one way to achieve it, but as long as you have a single public IP, you can also do it with an RTSP proxy (a.k.a. RTSP ALG) at the source NAT point so the proxy can make similar connections outbound and relay the reply streams to the host that asked for them.

This one looks plausible. I tested building it under the Docker Desktop cross-compilation tool chain, and it does build for ARMv7, requiring ~60 MB of storage space. That's as far as I took it, because I don't need it here, so I don't have a good test case for configuring and evaluating it.

This way P2P connectivity can be established even behind CGNAT

If that's going on, then I think the RTSP proxy linked above will not work without an underlying L3 tunnel out to somewhere sane. (e.g. VPN, SSH SOCKS proxy, etc.)

The real solution, stop whining and get on IPv6.

It isn't a panacea: big mobile carriers often give you IPv6 and CGNAT. Fun! 🤮
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: RTSP Helper

Sat Jun 18, 2022 7:17 pm

+1 to Tik support on this.

With 7.4beta4, they've given you the tools to solve it, using Containers.

RTSP NAT Traversal needs to be implemented at Layer 7 by the app/device vendor using STUN, TURN and ICE not at the router.

That's one way to achieve it, but as long as you have a single public IP, you can also do it with an RTSP proxy (a.k.a. RTSP ALG) at the source NAT point so the proxy can make similar connections outbound and relay the reply streams to the host that asked for them.

This one looks plausible. I tested building it under the Docker Desktop cross-compilation tool chain, and it does build for ARMv7, requiring ~60 MB of storage space. That's as far as I took it, because I don't need it here, so I don't have a good test case for configuring and evaluating it.

This way P2P connectivity can be established even behind CGNAT

If that's going on, then I think the RTSP proxy linked above will not work without an underlying L3 tunnel out to somewhere sane. (e.g. VPN, SSH SOCKS proxy, etc.)

The real solution, stop whining and get on IPv6.

It isn't a panacea: big mobile carriers often give you IPv6 and CGNAT. Fun! 🤮
You totally ignored the main point – Doing NAT correctly using netmap instead of src nat:
viewtopic.php?t=176358
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1604
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: RTSP Helper

Sat Jun 18, 2022 9:53 pm

RTSP NAT Traversal needs to be implemented at Layer 7 by the app/device vendor using STUN, TURN and ICE not at the router. At router level, you should ensure to do NAT correctly using Netmap to ensure STUN works seamlessly as the ports of the internal device IP will match the port of the public facing IP – This way P2P connectivity can be established even behind CGNAT without port forwarding directly.
viewtopic.php?t=176358

Claiming it only will take a "simple" NAT or ALG helper to solve it is unfortunately not the whole truth. RTSP is a complete echo system that span all the way from L3 to L7 thus it usually requires a lot more.

Please read:
viewtopic.php?t=172168#p922606
viewtopic.php?p=940575&sid=21a1f1cc1ad5 ... c9#p922681
viewtopic.php?p=940575&sid=21a1f1cc1ad5 ... c9#p932486
viewtopic.php?p=940575&sid=21a1f1cc1ad5 ... c9#p932706
--

Have a nice weekend!
 
User avatar
tangent
Forum Guru
Forum Guru
Posts: 1655
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: RTSP Helper

Sun Jun 19, 2022 6:01 am

Doing NAT correctly using netmap instead of src nat:
viewtopic.php?t=176358

It's rather arrogant to say netmap is the "correct" way to implement NAT, full stop. It is a useful tool in some contexts, not all.

That aside, I don't see how iptables' netmap NAT method solves the RTSP problem at all. RTSP is not a typical NAT traversal problem where the responding host has to work out a way to talk back to the host that sent the request. There are many ways RTSP can work. Indeed, the core of the protocol is to negotiate precisely how the two hosts will communicate henceforth, matching desired behaviors to available services. The thing is, since RTSP was invented before NAT, there isn't a part of the protocol to signal how the remote hosts should do NAT traversal.

RTSP is a TCP-based protocol (normally) that merely sets the reply stream up, which is often UDP. It is possible to tunnel replies back over the same RTSP connection, in part to get around problems like the class discussed in this thread, but it's uncommon. We can ignore RTSP-over-HTTP for the same reason.

What may happen in normal RTSP is that it negotiates two UDP streams in reply: one for audio, and one for video. (Contrast the muxing case, where both come back inside a single MPEG-TS stream or similar.) Does netmap + STUN/TURN/etc. handle that case?

You should now see the value in having a proxy at the border: the internal client talks to the proxy, and the proxy talks to the remote service to set the stream up, which it then relays back inside to the requesting client.
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: RTSP Helper

Sun Jun 19, 2022 11:24 am

RTSP NAT Traversal needs to be implemented at Layer 7 by the app/device vendor using STUN, TURN and ICE not at the router. At router level, you should ensure to do NAT correctly using Netmap to ensure STUN works seamlessly as the ports of the internal device IP will match the port of the public facing IP – This way P2P connectivity can be established even behind CGNAT without port forwarding directly.
viewtopic.php?t=176358

Claiming it only will take a "simple" NAT or ALG helper to solve it is unfortunately not the whole truth. RTSP is a complete echo system that span all the way from L3 to L7 thus it usually requires a lot more.

Please read:
viewtopic.php?t=172168#p922606
viewtopic.php?p=940575&sid=21a1f1cc1ad5 ... c9#p922681
viewtopic.php?p=940575&sid=21a1f1cc1ad5 ... c9#p932486
viewtopic.php?p=940575&sid=21a1f1cc1ad5 ... c9#p932706
--

Have a nice weekend!
Did you read my reply right? I never said we need a NAT Helper aka ALG. What I said is to ensure 1:1 port mapping when performing NAT, so if an RTSP app is using port 5000 on PC1, the netmap will ensure it is port 5000 on public IP mapping. And hence the server/client/peer can directly talk on port 5000 WITHOUT the need for ALG.
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: RTSP Helper

Sun Jun 19, 2022 11:26 am

Doing NAT correctly using netmap instead of src nat:
viewtopic.php?t=176358

It's rather arrogant to say netmap is the "correct" way to implement NAT, full stop. It is a useful tool in some contexts, not all.

That aside, I don't see how iptables' netmap NAT method solves the RTSP problem at all. RTSP is not a typical NAT traversal problem where the responding host has to work out a way to talk back to the host that sent the request. There are many ways RTSP can work. Indeed, the core of the protocol is to negotiate precisely how the two hosts will communicate henceforth, matching desired behaviors to available services. The thing is, since RTSP was invented before NAT, there isn't a part of the protocol to signal how the remote hosts should do NAT traversal.

RTSP is a TCP-based protocol (normally) that merely sets the reply stream up, which is often UDP. It is possible to tunnel replies back over the same RTSP connection, in part to get around problems like the class discussed in this thread, but it's uncommon. We can ignore RTSP-over-HTTP for the same reason.

What may happen in normal RTSP is that it negotiates two UDP streams in reply: one for audio, and one for video. (Contrast the muxing case, where both come back inside a single MPEG-TS stream or similar.) Does netmap + STUN/TURN/etc. handle that case?

You should now see the value in having a proxy at the border: the internal client talks to the proxy, and the proxy talks to the remote service to set the stream up, which it then relays back inside to the requesting client.
Arrogant? Have you actually deep dived into the Linux kernel? All modern Linux kernel implementations mimics netmap even for src nat action. There's no differentiation between the two except on MikroTik. I take it you're not Linux savvy?

And the effects of netmap for all traffic types including RTSP is straightforward:
viewtopic.php?p=940652#p940652
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: RTSP Helper

Sun Jun 19, 2022 11:57 am

The good news however is MT fixed the src nat problem partially in 7.3.1. So port mapping is now 1:1, however, the address will still get randomised for every outbound NATted traffic, which breaks P2P connectivity even though the port is predictable and persistent and hence netmap is still needed.

Still, this is an improvement from ROS v6 Kernel 3.x.x.

Just tested a few public RTSP streams with a PC behind NAT44 and NAT444 - Src NAT vs netmap in, both using ROS v7. When using netmap I am able to connect to public RTSP streams from VLC without any issues ALG or Proxy crap. The moment I switch to src NAT, it either fails to connect to the stream or delay+no video only audio shows up.

Example:
rtsp://rtsp.stream/pattern
rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mp4
Others:
https://g33ktricks.blogspot.com/p/the-r ... tocol.html

So where's that Linux engineering expert claiming netmap is arrogance? My advice is, instead of trolling me, spin up WireShark and inspect per packet when using src nat and netmap and then derive a conclusion on which one works best for P2P networking behind NAT (44 and 444).

Screenshot showing the TCP session and multiple UDP streams with no problem when using netmap, from conn_track:
Image

Note:
I have deployed NAT44/NAT444 for a few clients using Tik and netmap, nobody complained that their IP Cameras etc were failing, they all worked fine.
 
User avatar
tangent
Forum Guru
Forum Guru
Posts: 1655
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: RTSP Helper

Sun Jun 19, 2022 1:29 pm

Have you actually deep dived into the Linux kernel?

Is that necessary to hold an opinion? Only kernel hackers need reply? No userland experience is relevant?

There's no differentiation between the two except on MikroTik.

Since that's the tool I have, and that's the owner of forum of the OS I'm replying on, I'm going to reply in that context. There's little point offering solutions for other OSes here, now is there?

I take it you're not Linux savvy?

The first Linux distro I used was distributed on 5¼" floppies. It ran on kernel 0.99pl14.

That wasn't my first *ix system. The first popular one I used was DEC Ultrix on a VAX, based on 4.3BSD. Surrounding that were assorted SVR4 variants from companies long since gone, and less well remembered than DEC.

Am I all-knowing? Of course not. But if you aren't getting your point across to one such as me, who do you suppose you're effective in communicating to?

And the effects of netmap for all traffic types including RTSP is straightforward:

Like the article you linked above — here, section "CGNAT" — this is blind assertion. The article states, "netmap algorithm + state tracking will handle what gets mapped where." Okay, but how? Everything I can pull up with web searches talks about trivial cases like mapping a public /24 to a private /24. That doesn't tell me how the article's rules work in this more complicated case, which then leads me to ask questions like the above.

Another case: Can I use netmap in place of traditional masquerading, with one public IP in front of a /24 private LAN?

And another: I have a LAN with two levels of VLAN, a restricted internal VLAN (e.g. IoT) behind a less restricted user VLAN. The restricted VLAN needs to get through the user VLAN's NAT firewall to get out to the Internet. In that situation, does netmap allow clients on the restricted VLAN subnet to pull RTSP streams from the user VLAN? (1 hop.) Does it let the restricted VLAN pull RTSP streams from the Internet? (2 hops.) If so, how?

The article you linked is entirely unlike the networks I work with. Without some understanding of how much of its voluminous advice I must take and how much I can ignore, I'm left treating it all as provisionally irrelevant. The only other option is cargo-cult imitation.

If you want me to understand, you're going to have to get beyond blind assertion.

I am able to connect to public RTSP streams from VLC without any issues ALG or Proxy crap.

I analyzed your first link, and it behaves as I said above: it negotiates separate RTP audio and video streams, apparently without using any explicit NAT traversal mechanism. (If it does, I missed it.) This is what you're showing in your screen shots, host 23.88.67.97 sending RTP on UDP 8000 and 8001. (RTP ≠ RTSP)

I'm not using RouterOS on the border of my NAT44 network here — my RouterOS boxes are all relegated to smart switch roles — so what I don't get is, why does it work? The RTSP TCP outbound is on port 554, whereas the negotiated return streams come from two different UDP ports. How does my border router know to send those returning streams back inside? Surely it doesn't just let everything from that host come back in without regard to connection tracking.

Can I infer that my border router has an RTSP helper built into it? There is no visible setting for it in the configuration GUI, but that isn't diagnostic; the router's from a company that's famous for candy-coating all its internal power, presenting little through its GUI.

(Yes, one of these days I need to replace it with a RouterOS device. There are reasons I can't, but yes, one day.)

As for netmap on RouterOS, I still don't see why it's any solution. The client (VLC) doesn't send anything out to the server on UDP ports 8000-8001, and the client-side ports are random besides. At what point is the NAT mapping established?

Here's one of the SETUP requests I captured:

SETUP rtsp://rtsp.stream:554/pattern/trackID=0 RTSP/1.0
CSeq: 4
User-Agent: LibVLC/3.0.17.3 (LIVE555 Streaming Media v2016.11.28)
Transport: RTP/AVP;unicast;client_port=50140-50141

What is it about netmap that lets the router know my client computer is listening on UDP ports 50140 and 50141, so it should send UDP packets on those ports to my computer? Surely learning that fact requires some sort of deep packet inspection and/or a proxy?
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: RTSP Helper

Sun Jun 19, 2022 1:56 pm

Have you actually deep dived into the Linux kernel?

Is that necessary to hold an opinion? Only kernel hackers need reply? No userland experience is relevant?

There's no differentiation between the two except on MikroTik.

Since that's the tool I have, and that's the owner of forum of the OS I'm replying on, I'm going to reply in that context. There's little point offering solutions for other OSes here, now is there?

I take it you're not Linux savvy?

The first Linux distro I used was distributed on 5¼" floppies. It ran on kernel 0.99pl14.

That wasn't my first *ix system. The first popular one I used was DEC Ultrix on a VAX, based on 4.3BSD. Surrounding that were assorted SVR4 variants from companies long since gone, and less well remembered than DEC.

Am I all-knowing? Of course not. But if you aren't getting your point across to one such as me, who do you suppose you're effective in communicating to?

And the effects of netmap for all traffic types including RTSP is straightforward:

Like the article you linked above — here, section "CGNAT" — this is blind assertion. The article states, "netmap algorithm + state tracking will handle what gets mapped where." Okay, but how? Everything I can pull up with web searches talks about trivial cases like mapping a public /24 to a private /24. That doesn't tell me how the article's rules work in this more complicated case, which then leads me to ask questions like the above.

Another case: Can I use netmap in place of traditional masquerading, with one public IP in front of a /24 private LAN?

And another: I have a LAN with two levels of VLAN, a restricted internal VLAN (e.g. IoT) behind a less restricted user VLAN. The restricted VLAN needs to get through the user VLAN's NAT firewall to get out to the Internet. In that situation, does netmap allow clients on the restricted VLAN subnet to pull RTSP streams from the user VLAN? (1 hop.) Does it let the restricted VLAN pull RTSP streams from the Internet? (2 hops.) If so, how?

The article you linked is entirely unlike the networks I work with. Without some understanding of how much of its voluminous advice I must take and how much I can ignore, I'm left treating it all as provisionally irrelevant. The only other option is cargo-cult imitation.

If you want me to understand, you're going to have to get beyond blind assertion.

I am able to connect to public RTSP streams from VLC without any issues ALG or Proxy crap.

I analyzed your first link, and it behaves as I said above: it negotiates separate RTP audio and video streams, apparently without using any explicit NAT traversal mechanism. (If it does, I missed it.) This is what you're showing in your screen shots, host 23.88.67.97 sending RTP on UDP 8000 and 8001. (RTP ≠ RTSP)

I'm not using RouterOS on the border of my NAT44 network here — my RouterOS boxes are all relegated to smart switch roles — so what I don't get is, why does it work? The RTSP TCP outbound is on port 554, whereas the negotiated return streams come from two different UDP ports. How does my border router know to send those returning streams back inside? Surely it doesn't just let everything from that host come back in without regard to connection tracking.

Can I infer that my border router has an RTSP helper built into it? There is no visible setting for it in the configuration GUI, but that isn't diagnostic; the router's from a company that's famous for candy-coating all its internal power, presenting little through its GUI.

(Yes, one of these days I need to replace it with a RouterOS device. There are reasons I can't, but yes, one day.)

As for netmap on RouterOS, I still don't see why it's any solution. The client (VLC) doesn't send anything out to the server on UDP ports 8000-8001, and the client-side ports are random besides. At what point is the NAT mapping established?

Here's one of the SETUP requests I captured:

SETUP rtsp://rtsp.stream:554/pattern/trackID=0 RTSP/1.0
CSeq: 4
User-Agent: LibVLC/3.0.17.3 (LIVE555 Streaming Media v2016.11.28)
Transport: RTP/AVP;unicast;client_port=50140-50141

What is it about netmap that lets the router know my client computer is listening on UDP ports 50140 and 50141, so it should send UDP packets on those ports to my computer? Surely learning that fact requires some sort of deep packet inspection and/or a proxy?
1. You don't need the dst nat rules specified in the article. Just src nat with netmap as action. I am not using the dst nat rules either, except for port forwarding when required like say hosting a web server or very well an RTSP server/device.

2. If you have a /32 public IPv4 address, you can replace masquerade with src nat (in all latest LTS Kernel mainline) or netmap (in MikroTik) - I've been doing this for many clients who aren't ISPs/Telcos aka home users and SMEs, no complaints arrived at my door and no refunds asked – I have a not working = me giving them full refund policy

3. Your “border router” sounds like a Chinese router from TP-Link or some other bullcrap - I cannot comment on how they work unless they expose the kernel configuration like Tik does. That is for you to figure out. Chances are, they likely use a fairly modern variant of the Linux kernel where src nat and netmap does the same behaviour aka 1:1 persistent mapping.

4. You “analysed” my first link? Check all the links, when using netmap instead of src nat mapping RTSP works perfectly fine behind 44 and 444

5. Of course, the client will initiate outbound on a random port (not listen, listening is a completely different state), nobody's saying otherwise. What netmap (and modern day Linux src nat) does is ensure 1:1 mapping to help make STUN work WITHOUT TURN/ICE. This is a completely L3 concept, the L4-L7 protocol is irrelevant excluding IPv4 literals like SIP where of course, you need ALG no matter what. If the ports are random on both ends of the NAT, then you (the apps) will be forced to rely on TURN/ICE etc.

6. NAT Traversal doesn't mean just ALG, I take it you aren't Linux savvy (Oh did I say it twice? I think not), NAT traversal at best happens on L3 (netmap or newer src nat along with STUN) and at worse it happens on L7 (ALG, TURN, ICE)

So instead of trolling me again with your ego and pride, perhaps learn something from shared info in this thread, test it on your own network and stop whining. Either way, the real solution is end-to-end native IPv6.

For your info I have end-to-end visibility on the traffic as I have authorised access to my ISPs' edge (inter-AS router) and their BNG (that is used for delegating public prefixes to me on both v4 and v6 stack), where we thoroughly tested whatever was pointed out in that article end-to-end and not just CGNAT as usual with WireShark.

The simplest way to confirm deny src nat vs netmap is BitTorent, try seeding/download open source Ubuntu torrent on either configuration, run WireShark and see which one is more efficient for establishing P2P connectivity without port forwarding or dynamic port forward like PCP/UPnP etc.
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: RTSP Helper

Sun Jun 19, 2022 2:07 pm

Further to note

RTSP is a combination protocol:

RTP is used for streaming the audio/video/whatever over UDP.
RTSP itself is used for pausing/rewinding the stream over TCP.

https://en.wikipedia.org/wiki/Real_Time ... g_Protocol

This is exactly what is shown in my screenshot, RTSP is the established TCP connection, the remaining are RTP over UDP.
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: RTSP Helper

Sun Jun 19, 2022 2:14 pm

The reply is too long and hence I'm breaking this into threads @mods, this is not a spam.

For inter-VLAN it does not matter, VLAN is a L2 concept.

You can have 4000+ VLANs, use netmap and the same logic works as long as you have sufficient public IP addresses available for handling 10k+ hosts on all the VLANs to prevent port exhaustions.
That is similar to CGNAT subscription ratio.

Your replies suggest you're an enterprise network guy/woman/they/them, likely never touched traffic from ISP edge all the way to the customer's end-point, and that is unfortunately one of the problems I observed with enterprise folks, they get all wild and crazy with little to show for it on how L1 to L4 actually works.

tl;dr for our Linux expert here:
1. Use netmap or modern src nat for 1:1 port mapping
2. Ensure sufficient amount of public IPs to avoid port exhaustion. If you have only a /32, at best in my observations try to avoid having more than 100 hosts trying to establish P2P for any application not just RTSP.
3. You're seeing a smaller picture of network traffic when the highest you can go is the CPE device on your side. When you get exposed all the way to the eBGP router of the ISP, you'll find a lot of crazy shit and broken traffic due to lack of BCPs/BCOP in the various ISPs. And yes, NAT/CGNAT is a leading factor in broken traffic – Even if you're doing it right on your end, if you're behind a CGNAT, you're at their mercy.
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: RTSP Helper

Sun Jun 19, 2022 2:30 pm

Somewhere in the history of the internet, engineers started mistaking NAT Traversal (including ALG, STUN etc) as a “Firewall” intelligence feature.

Let's say all your devices have public IPv4 addresses, there's no NAT involved. And you accept established, related, icmp (PMTUD) but you drop the rest – How will P2P work in this case? There's no NAT, meaning no ALG, there's no STUN/TURN/ICE/WebRTC because it's all public IPs. In this case it's simple, the host behind the restrictive firewall will need to establish a session with the remote destination that is open to accepting inbound connections and from there it can directly reply to the source IP:Port since the stateful firewall now sees it as “established, related”

In this case it “just works” because the port on the end host is persistent, same and binds directly to the public IP, there's no NAT middle box interfering and changing that port number.

But I understand some folks have a hard time digesting the fact, that someone else knows certain things better than they do, I'm done on this thread, take my advice/knowledge/information and verify it yourself or don't, couldn't care less. I'm not the one getting calls from clients complaining about broken P2P traffic on IPv4.
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1604
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: RTSP Helper

Mon Jun 20, 2022 6:53 pm

@DarkNate, I was merely try to explain that the RTSP echo system is implemented differently depending on type of usage eg iptv, surveillance system etc thus one has to be very specific what "RTSP" application we are talking about. Also, it's not enough to mention some random port numbers as it depends on the streaming protocol used and furthermore possible need to implement application states.

Or as the wiki states pretty well: "The transmission of streaming data itself is not a task of RTSP.

Most [but not all] RTSP servers use the Real-time Transport Protocol (RTP) in conjunction with Real-time Control Protocol (RTCP) for media stream delivery. However, some vendors implement proprietary transport protocols [please note]. The RTSP server software from RealNetworks, for example, also used RealNetworks' proprietary Real Data Transport (RDT). [and there are a lot others]

Protocol directives: While similar in some ways to HTTP, RTSP defines control sequences useful in controlling multimedia playback. While HTTP is stateless, RTSP has state; an identifier is used when needed to track concurrent sessions. Like HTTP, RTSP uses TCP to maintain an end-to-end connection and, while most RTSP control messages are sent by the client to the server, some commands travel in the other direction (i.e. from server to client).

Presented here are the basic RTSP requests [among many others on level L7]. Some typical HTTP requests, like the OPTIONS request, are also available. The default transport layer port number is 554[7] for both TCP and UDP, the latter being rarely used for the control requests." [and there are a lot of other combinations depending of what RTSP Stream Server Appliance and possible tailor made RTSP client that is being used]..."

Ref: en.wikipedia.org/wiki/Real_Time_Streaming_Protocol

Bottom line, it's virtually impossible to implement a general RTSP "helper" since there isn't just one "standard". Quite the opposite there are many different ones including proprietary solutions and they all differ depending of intended application.
 
User avatar
jbl42
Member Candidate
Member Candidate
Posts: 223
Joined: Sun Jun 21, 2020 12:58 pm

Re: RTSP Helper

Mon Jun 20, 2022 10:59 pm

Bottom line, it's virtually impossible to implement a general RTSP "helper" since there isn't just one "standard". Quite the opposite there are many different ones including proprietary solutions and they all differ depending of intended application.
While I agree on this, I think the basic request is more along enabling RTSP based IPTV solutions as used by several big providers in Europe (MoviStar/Telefonica in Spain, Orange in France, ...) with NAT/masqerade. Where RTSP is used by the client to tell the server on what ports it expects incoming content UDP steams from the server.
Many commercial SOHO routers provide an ALG/helper to sniff UDP port numbers from outgoing RTSP and forward incoming streams to said ports from WAN to the respective LAN client. Kind of a dynamic WAN->LAN dstNAT for a specific UDP stream.

This will for sure not cover anything more advanced like CCTV/surveillance, customer routing / CGNAT at ISP level or enterprise firewalls.
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1604
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: RTSP Helper

Mon Jun 20, 2022 11:46 pm

Well, that's exactly what I'm trying to say all the time ie you can't just say I want "RTSP" but have to be very specific about the details like: I want IPTV support for MoviStar/Telefonica in Spain, XXX in country YYY and ZZZ in country XXX etc.

And since most of them use different implementations, the sugar-coating (ie gui settings) need to be adjusted according to the specific iptv supplier. In other words just a simple iptv "helper" may require a lot of sugar-coating since there might be different settings for different implementations. Since there is no universal standard one has to choose what suppliers you want to support and adopt the settings accordingly. To get a grip have look at some of the consumer devices that sugar-coates (implements) iptv.

Repeat after me: There Are No Standards, Not Even for IPTV. :D

And as for CCTV/surveillance and similar solutions, professional installations for NVR/DVR never ever relies on "automatic helpers" since the behavior almost always differs depending on the manufacturer of the network equipment, different models and even different implementations within the same model series.
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: RTSP Helper

Mon Jun 20, 2022 11:56 pm

Bottom line, it's virtually impossible to implement a general RTSP "helper" since there isn't just one "standard". Quite the opposite there are many different ones including proprietary solutions and they all differ depending of intended application.
with NAT/masqerade. Where RTSP is used by the client to tell the server on what ports it expects incoming content UDP steams from the server.
Many commercial SOHO routers provide an ALG/helper to sniff UDP port numbers from outgoing RTSP and forward incoming streams to said ports from WAN to the respective LAN client. Kind of a dynamic WAN->LAN dstNAT for a specific UDP stream.
That's the whole point, stop using masquerade and src nat and replace it with netmap, and you're good to go for 8/10 RTSP cases. Re-read my previous replies.
 
User avatar
jbl42
Member Candidate
Member Candidate
Posts: 223
Joined: Sun Jun 21, 2020 12:58 pm

Re: RTSP Helper

Tue Jun 21, 2022 12:16 am

Repeat after me: There Are No Standards, Not Even for IPTV. :D

Sniffing the Transport field in the outgoing RTSP request as defined in RFC2326 is enough to have all those IPTV solutions working.
Transport: RTP;unicast;client_port=12345
OpenWRT manages to handle all those IPTV services with just that.
So I'm happy to to agree to disagree ;-)

The main reason some SOHO boxes have specific GUI settings for specific providers like MoveStar,Orange etc, is they use special VLANs and/or DHCP options for IPTV and some require specific client MACs on the IPTV interfaces, require additional IGMP settings, etc. This of course would have to be setup additionaly at the appropriate layers on MikroTiks.
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1604
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: RTSP Helper

Tue Jun 21, 2022 12:31 am

That's the whole point, stop using masquerade and src nat and replace it with netmap, and you're good to go for 8/10 RTSP cases. Re-read my previous replies.

Unfortunately not since there are no standard "RTSP cases" (but I get your point). In case if IPTV, netmap whould just be one type of "helper" (for transport) thus you will need other settings to get it to work. As I said earlier, have look at some of the consumer devices that implements iptv "helpers"

https://www.asus.com/us/support/FAQ/1011708/
http://files.dlink.com.au/products/DSL- ... r_IPTV.pdf
http://files.dlink.com.au/products/dsl- ... _Guide.pdf
https://www.tp-link.com/us/support/faq/1546/
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1604
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: RTSP Helper

Tue Jun 21, 2022 12:43 am

The main reason some SOHO boxes have specific GUI settings for specific providers like MoveStar,Orange etc, is they use special VLANs and/or DHCP options for IPTV and some require specific client MACs on the IPTV interfaces, require additional IGMP settings, etc. This of course would have to be setup additionaly at the appropriate layers on MikroTiks.

Exactly, there you go! :-D

And every freakin distributor loves to lock in the end user with their specific solution aka "triple play" and likes. And besides VLAN and other related stuff, all using specific port numbers, transport initialization, multiple streams using different transports and sometimes even proprietary application level protocols, So not much of a standard...

Regarding iptv on openwrt it often requires a lot of hacking (configuration) to implement a specific distributor. Here is just a basic example: openwrt.org/docs/guide-user/network/wan/udp_multicast
Last edited by Larsa on Tue Jun 21, 2022 1:04 am, edited 1 time in total.
 
User avatar
jbl42
Member Candidate
Member Candidate
Posts: 223
Joined: Sun Jun 21, 2020 12:58 pm

Re: RTSP Helper

Tue Jun 21, 2022 1:04 am

And every freakin distributor loves to lock in the end user with their specific solution aka "triple play" and likes. And besides VLAN and other related stuff, all using specific port numbers, transport initialization, multiple streams using different transports and sometimes even proprietary application level protocols, So not much of a standard...
This is why @darknets endlessly repeated solution of bypassing connection tracking by using netmap (which was specfically designed for being stateless for stateless routing) works... for an ISP border router were connection tracking is disabled, all routing stateless and hence nothing else than netmap is possible. And also not desirable as the ISP customers would like the get all packets coming from the public Internet.

In a CPE/SOHO router almost everone else than @darkent likes to have connection tracking enabled for NAT, so no netmap. Who knows (or cares) if he ever gets the difference...
 
jhbarrantes
Frequent Visitor
Frequent Visitor
Posts: 56
Joined: Wed Aug 21, 2019 2:56 pm

Re: RTSP Helper

Tue Jun 21, 2022 10:22 am

Repeat after me: There Are No Standards, Not Even for IPTV. :D
Sniffing the Transport field in the outgoing RTSP request as defined in RFC2326 is enough to have all those IPTV solutions working.
Transport: RTP;unicast;client_port=12345
OpenWRT manages to handle all those IPTV services with just that.
So I'm happy to to agree to disagree ;-)

Exactly. That's the only thing we're missing on Mikrotik CPEs, exactly that bit. And, for all people collaborating in the post: what we're looking for is for a solution from the end user perspective, not from the provider point of view. Is this a wrong way of addressing this? Probably, but I'm no one to tell my provider how to implement their network. And trust me when I say this provider has a very well known video platform which is the envy for rest of providers. And the video solution they deliver work flawlesly with their equipment. What's the problem then?: The rest (dhcp, routing, firewall, etc) doesn't work that well and we love to use our own Mikoritik equipment. And, if other vendors has already resolved the RTSP ALG, why can't we? I was even thinking on any kind of L7 RegExp (IP -> Firewall -> Layer 7 Protocols) that could intercept this, but I cannot figure out how to play with this and mangle to achieve what we need.

The main reason some SOHO boxes have specific GUI settings for specific providers like MoveStar,Orange etc, is they use special VLANs and/or DHCP options for IPTV and some require specific client MACs on the IPTV interfaces, require additional IGMP settings, etc. This of course would have to be setup additionaly at the appropriate layers on MikroTiks.

The rest is already covered, all other bits. We have a conditional DHCP based on device vendor (DHCP Option), delivering specific settings for DNS and video server address. We also have routing resolved by RIP and the static address they deliver for each IPTV client, plus the VLAN setup. The only thing that is still missing is this ALG for mapping UDP steam ports back to the CPE "on the fly" when TV box select a channel and hit play, for on demand content. One guy even did a very rude (in my opinion) solution, which is apply a DMZ to the TV box, but obviously this get more and more complicated when you have more than on box at home (having to run a script periodically that change this DMZ host based on the last TV box that send a packet to 554)

In summary: I really appreciate all the passion shown with responses at the post, I have learn a lot from that. But please don't loose track of what the end user is looking for: what we really need is a kind of solution / workaround / magic trick... etc that can do exactly what other vendors do when intercepting RTSP messages to do a dynamic port mapping for UDP ports delivering audio/video streams. Is DarkNate solution valid? I don't know, but I'm glad to try, if this could be some how applicable to then end user CPE, aka the mikrotik box we all have at home.

Thanks all!
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: RTSP Helper

Tue Jun 21, 2022 12:59 pm

And every freakin distributor loves to lock in the end user with their specific solution aka "triple play" and likes. And besides VLAN and other related stuff, all using specific port numbers, transport initialization, multiple streams using different transports and sometimes even proprietary application level protocols, So not much of a standard...
This is why @darknets endlessly repeated solution of bypassing connection tracking by using netmap (which was specfically designed for being stateless for stateless routing) works... for an ISP border router were connection tracking is disabled, all routing stateless and hence nothing else than netmap is possible. And also not desirable as the ISP customers would like the get all packets coming from the public Internet.

In a CPE/SOHO router almost everone else than @darkent likes to have connection tracking enabled for NAT, so no netmap. Who knows (or cares) if he ever gets the difference...
Bro, what are you smoking? Netmap is NOT stateless. I use it on ISP BNGs and also in my personal home router for /32s and the same thing for normal home users who are my clients.

Src nat and netmap = same thing in Linux LTS latest mainline. The difference is older versions src nat is randomised outbound port mapping.

Stateless NAT does not exist on IPv4, it only exists for IPv6 via NPTv6.
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: RTSP Helper

Tue Jun 21, 2022 1:01 pm

Instead of whining here about broken P2P protocols on NATted IPv4, switch to netmap, remove src nat on both src nat and dst nat chain in the Linux based router OSes.

ALG is irrelevant for most protocols if you know how to do NATting properly on Layer 3/4 aka netmap.
 
User avatar
Larsa
Forum Guru
Forum Guru
Posts: 1604
Joined: Sat Aug 29, 2015 7:40 pm
Location: The North Pole, Santa's Workshop

Re: RTSP Helper

Tue Jun 21, 2022 4:44 pm

@Jhbarrantes, if you or anyone else has the skillset to offer a complete configuration for a particular iptv distributor one would be able to create reference implementation using Containers.

However, I'm pretty sure we won't see a NAT/ALG helper in Ros not at least until somebody at MT decides to build a specific interface dedicated to the consumer market. Until then, most things can be done using standard fw rules, vlan settings etc, but you still need all the nitty gritty details for a specific iptv distributor to create a working configuration.
 
jhbarrantes
Frequent Visitor
Frequent Visitor
Posts: 56
Joined: Wed Aug 21, 2019 2:56 pm

Re: RTSP Helper

Tue Jun 21, 2022 7:06 pm

I'd take the offer. Give me some time and I will prepare this for you.

Thanks Larsa.
 
User avatar
jbl42
Member Candidate
Member Candidate
Posts: 223
Joined: Sun Jun 21, 2020 12:58 pm

Re: RTSP Helper

Wed Jun 22, 2022 1:25 am

Bro, what are you smoking?
Grow up.
Netmap is NOT stateless. I use it on ISP BNGs and also in my personal home router for /32s and the same thing for normal home users who are my clients.
I have given up to ask you how iptables netmap statefuly accepts incoming UDP content streams to a port requested by the internal client inside an RTSP request and originating from a server IP never used as outgoing dest before. Let's put that to rest.

If you however are interested in Linux stateless native address translation for 1:1 NAT netmapping, as aften used for routing through transport networks:
https://man7.org/linux/man-pages/man8/tc-nat.8.html
 
DarkNate
Forum Guru
Forum Guru
Posts: 1065
Joined: Fri Jun 26, 2020 4:37 pm

Re: RTSP Helper

Wed Jun 22, 2022 2:27 pm

Netmap isn't stateless, did you even try this on your router at all? It clearly loads conn_track and stateful tracking. Did you not see how RTSP works smoothly for clients behind netmap instead of src nat? Did you not read my previous replies that explains it?

The link you share isn't NETMAP Mr Expert, that's a completely different mechanism and it's not even available on RouterOS.
 
User avatar
anav
Forum Guru
Forum Guru
Posts: 21759
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada
Contact:

Re: RTSP Helper

Tue Jun 28, 2022 5:48 pm

Regardless of the back and forth, is there a viable solution for moviestar spain?
Now that containers are back in vogue (round two beta 5).....................!!
 
dksoft
Member Candidate
Member Candidate
Posts: 152
Joined: Thu Dec 06, 2012 8:56 am
Location: Germany

Re: RTSP Helper

Wed Jul 27, 2022 1:11 pm

The new RTSP helper in 7.5b4 works fine for me and solved my problems with Reolink cameras.
 
jhbarrantes
Frequent Visitor
Frequent Visitor
Posts: 56
Joined: Wed Aug 21, 2019 2:56 pm

Re: RTSP Helper

Wed Jul 27, 2022 2:19 pm

The new RTSP helper in 7.5b4 works fine for me and solved my problems with Reolink cameras.

Great news. I am in conversations to get it tested for movistar iptv. Let’s see if we can end up this post with very good news.

Question for you, did the helper create any dynamic nat rule when it is in use, such as the UPnP does?

Thanks!
 
User avatar
skylark
Member Candidate
Member Candidate
Posts: 144
Joined: Wed Feb 10, 2016 3:55 pm

Re: RTSP Helper

Wed Jul 27, 2022 4:54 pm

No, a helper will act as all other helpers (under ip/firewall/service-port/) without creating dynamic NAT rules.
 
jhbarrantes
Frequent Visitor
Frequent Visitor
Posts: 56
Joined: Wed Aug 21, 2019 2:56 pm

Re: RTSP Helper

Wed Jul 27, 2022 7:13 pm

Perfect! Thanks a lot for clarifying.
 
jhbarrantes
Frequent Visitor
Frequent Visitor
Posts: 56
Joined: Wed Aug 21, 2019 2:56 pm

Re: RTSP Helper

Thu Jul 28, 2022 11:54 am

It seems to be working fine with Movistar IPTV (Spain) too! Very good news for all we were waiting for this to be implemented. You will get tons of happy customers when releasing this to the stable version.

@Mikrotik Support, just to satisfy my own curiosity... What was the final implementation you followed for adding this helper as a firewall/service ALG helper in v7? Was it hard to define or implement? Was this much more easy to add to v7 than previous v6 versions, where this was never considered, due to kernel migration?

Thanks a lot!
 
volkirik
Member Candidate
Member Candidate
Posts: 212
Joined: Sat Jul 23, 2016 2:03 pm

Re: RTSP Helper

Thu Jul 28, 2022 6:38 pm

set rtsp disabled=no ports=554,8554,7236
 
jhbarrantes
Frequent Visitor
Frequent Visitor
Posts: 56
Joined: Wed Aug 21, 2019 2:56 pm

Re: RTSP Helper

Fri Jul 29, 2022 11:25 am

set rtsp disabled=no ports=554,8554,7236

Was this setup for any specific application? If so, please share. Just enabling this, with 554 by default, works for most use cases.

Kind regards.
 
volkirik
Member Candidate
Member Candidate
Posts: 212
Joined: Sat Jul 23, 2016 2:03 pm

Re: RTSP Helper

Fri Jul 29, 2022 7:47 pm

found on web. most common 3 ports i would guess.
 
josepc
just joined
Posts: 7
Joined: Thu Feb 10, 2022 12:31 am

Re: RTSP Helper

Thu Aug 04, 2022 10:49 am

RTSP helper working fine with IPTV Movistar Spain. Thank you very much!
 
ToniRos
just joined
Posts: 2
Joined: Mon Dec 11, 2017 12:20 am

Re: RTSP Helper

Thu Sep 01, 2022 8:31 pm

RTSP Helper work as a charm in v7.5.
Tanks a lot.
 
Rox169
Member
Member
Posts: 467
Joined: Sat Sep 04, 2021 1:47 am

Re: RTSP Helper

Fri Sep 02, 2022 7:47 am

Hi,

how does RTSP Helper work ? Simple explanation please...
 
rb9999
newbie
Posts: 28
Joined: Thu Dec 06, 2018 3:09 pm

Re: RTSP Helper

Fri Sep 02, 2022 10:01 am

Can confirm, it works!
@Rox169 would you like to know how rtsp helper in mikrotik implementation works or overall? RTSP is one of those non-layer3-agnostic protocols (like FTP, SIP, ...) that kinda messes things up if the client is behind a NAT.
Let's assume RTSP server is on 10.1.1.1/24, router wan is on 10.2.2.2/24, router lan is 192.168.1.1/24 and RTSP client is on 192.168.1.222/24. Router also preforms SNAT/MASQUERADE for all 192.168.1.0/24 traffic. When client would like to play some rtsp stuff from 10.1.1.1, it opens up a random port that's for accepting RTSP traffic (and notifies the server in the SETUP state), but as the client is behind NAT, without RTSP helper (also called RTSP ALG) that port is closed on the router. With enabled RTSP helper, router scans RTSP traffic for that kind of messages and forward traffic to the client accordingly. In some cases (like active FTP) the control traffic is also modified (like PORT 192,168,1,222,5,6 > PORT 10,2,2,2,5,6)...
 
User avatar
atari
just joined
Posts: 11
Joined: Sat Mar 16, 2013 7:15 pm
Location: Paracin
Contact:

Re: RTSP Helper

Sat Oct 22, 2022 3:52 am

Still not work whith fritzbox 6940 rtsp
 
jagarciam
just joined
Posts: 2
Joined: Fri Apr 04, 2014 3:19 pm

Re: RTSP Helper

Fri Aug 02, 2024 11:24 am

It works perfectly!!

Who is online

Users browsing this forum: No registered users and 2 guests