QoS for broadcast audio stream

tommyd · June 4, 2025, 12:43pm

Hi,

I am trying to configure QoS for a WAN connected office which needs to send and receive an UDP audio stream (a transport stream, few channels about 400Kb/s each direction). The application using those streams is very sensitive for delays, jitter, and packet loss. The location has also few dozens of local users, so they are generating some significant traffic. The stream quality is not good enough.
I hope you can give me some advice what is wrong with my configuration and how to fix it

I’ve tired few ways but did not get any significant improvement. My last idea was to use DSCP and PCP values which are used by the rest of the network and the WAN Provider. My configuration should:

mark traffic with required DSCP valuses
remark DSCP to priority
add packet marks according to set priority

The WAN interface is using tagged frames, so the provider can apply priority to the traffic and the rest of our network trusts DSCP headers.
The diagram below shows simplified setup, the BO1 location is using an advanced L3 switch and performs way better (however it is smaller with fewer users)

My configiration:

/interface bridge
add name=bridge1-lan
/interface bridge port
add bridge=bridge1-lan interface=ether3-lan 


/interface vlan
add interface=bridge1-lan name=vlan170 vlan-id=170
add interface=bridge1-lan name=vlan171 vlan-id=171
add interface=bridge1-lan name=vlan172 vlan-id=172
add interface=ether1-wan1 name=vlan3012 vlan-id=3012


/queue type
add kind=cake name=cakeUP
add kind=cake name=cakeDOWN
 
/queue interface
set ether1-wan1 queue=multi-queue-ethernet-default
set ether3-lan queue=multi-queue-ethernet-default


/queue tree
add comment="Uplink QoS" max-limit=100M name=QoS_UP parent=vlan3012 queue=ckeUP
add comment="Downlink QoS" max-limit=100M name=QoS_DOWN parent=bridge1-lan queue=cakDOWN
add comment="Queue Priority 1" name="UP IP Precedence 7. Network Control (Top Priority)" packet-mark=up_ip_precedence_7 parent=QoS_UP priority=1 queue=ckeUP
add comment="Queue Priority 2" name="UP IP Precedence 6. Internetwork Control (High Priority) " packet-mark=up_ip_precedence_6 parent=QoS_UP priority=2 queue=ckeUP
add comment="Queue Priority 3" limit-at=5M max-limit=5M name="UP IP Precedence 5. Voice (Medium-High Priority) " packet-mark=up_ip_precedence_5 parent=QoS_UP priority=3 queue=ckeUP
add comment="Queue Priority 4" name="UP IP Precedence 4. Interactive Video (Medium Priority)" packet-mark=up_ip_precedence_4 parent=QoS_UP priority=4 queue=ckeUP
add comment="Queue Priority 5" limit-at=2M max-limit=2M name="UP IP Precedence 3. Critical Data or Call Signaling (Medium-Low Priority)" packet-mark=up_ip_precedence_3 parent=QoS_UP priority=5 queue=ckeUP
add comment="Queue Priority 6" limit-at=85M max-limit=85M name="UP IP Precedence 0. Best Effort (Low Priority)" packet-mark=no-mark parent=QoS_UP priority=6 queue=ckeUP
add comment="Queue Priority 7" name="UP IP Precedence 2. Background (Very Low Priority)" packet-mark=up_ip_precedence_2 parent=QoS_UP priority=7 queue=ckeUP
add comment="Queue Priority 8" name="UP IP Precedence 1. Scavenger (Bottom Priority) " packet-mark=up_ip_precedence_1 parent=QoS_UP queue=ckeUP
add comment="Queue Priority 1" name="DN IP Precedence 7. Network Control (Top Priority)" packet-mark=dn_ip_precedence_7 parent=QoS_DOWN priority=1 queue=cakDOWN
add comment="Queue Priority 2" name="DN IP Precedence 6. Internetwork Control (High Priority)" packet-mark=dn_ip_precedence_6 parent=QoS_DOWN priority=2 queue=cakDOWN
add comment="Queue Priority 3" limit-at=2M max-limit=2M name="DN IP Precedence 5. Voice (Medium-High Priority)" packet-mark=dn_ip_precedence_5 parent=QoS_DOWN priority=3 queue=cakDOWN
add comment="Queue Priority 4" name="DN IP Precedence 4. Interactive Video (Medium Priority)" packet-mark=dn_ip_precedence_4 parent=QoS_DOWN priority=4 queue=cakDOWN
add comment="Queue Priority 5" limit-at=5M max-limit=5M name="DN IP Precedence 3. Critical Data or Call Signaling (Medium-Low Priority)" packet-mark=dn_ip_precedence_3 parent=QoS_DOWN priority=5 queue=cakDOWN
add comment="Queue Priority 7" name="DN IP Precedence 2. Background (Very Low Priority) " packet-mark=dn_ip_precedence_2 parent=QoS_DOWN priority=7 queue=cakDOWN
add comment="Queue Priority 8" name="DN IP Precedence 1. Scavenger (Bottom Priority)" packet-mark=dn_ip_precedence_1 parent=QoS_DOWN queue=cakeDOWN
add comment="Queue Priority 6" limit-at=85M max-limit=85M name="DN IP Precedence 0. Best Effort (Low Priority)" packet-mark=no-mark parent=QoS_DOWN priority=6 queue=cakeDOWN


#
# firewall mangle

# * 

/ip firewall mangle
#  mark local traffic with DSCP
add action=set-priority chain=postrouting comment="Prioritize ACKs" new-priority=6 packet-size=0-123 protocol=tcp tcp-flags=ack
add action=set-priority chain=postrouting comment="Respect DSCP tagging" new-priority=from-dscp-high-3-bits
add action=change-dscp chain=input comment="BGP Top Priority" new-dscp=46 protocol=tcp src-port=179
add action=change-dscp chain=output comment="BGP Top Priority" dst-port=179 new-dscp=46 protocol=tcp
add action=change-dscp chain=input comment="SSH Top Priority" dst-port=22 new-dscp=26 protocol=tcp
add action=change-dscp chain=output comment="SSH Top Priority" dst-port=22 new-dscp=26 protocol=tcp
add action=change-dscp chain=input comment="WinBox Top Priority" dst-port=8291 new-dscp=26 protocol=tcp

# remark outgoing traffic with DSCP
add action=change-dscp chain=postrouting comment="Mark users BE(0)" dst-address-list=qos-users-dst new-dscp=0 protocol=tcp
add action=change-dscp chain=postrouting comment="Mark LOW AF13(14)" new-dscp=14 protocol=tcp src-address-list=qos-users-src
add action=change-dscp chain=postrouting comment="Mark MEDIUM AF31(26)" dst-address-list=qos-medium-dst dst-port=7000-7999 log-prefix=MARKAF31 new-dscp=26 protocol=udp
add action=change-dscp chain=postrouting comment="Mark MEDIUM AF31(26)" dst-port=7000-7999 new-dscp=26 protocol=udp src-address-list=qos-medium-src
add action=change-dscp chain=postrouting comment="Mark LIVESTREAM EF(46)" dst-address-list=qos-livestream-dst dst-port=8000-8999 new-dscp=46 protocol=udp
add action=change-dscp chain=postrouting comment="Mark LIVESTREAM EF(46)" dst-port=8000-8999 new-dscp=46 protocol=udp src-address-list=qos-livestream-src

# set priority based on DSCP
add action=set-priority chain=postrouting comment="DSCP 0 CoS 0" dscp=0 new-priority=0
add action=set-priority chain=postrouting comment="DSCP 8 CoS 1" dscp=8 new-priority=1
add action=set-priority chain=postrouting comment="DSCP 16 CoS 2" dscp=16 new-priority=2
add action=set-priority chain=postrouting comment="DSCP 24 CoS 3" dscp=24 new-priority=3
add action=set-priority chain=postrouting comment="DSCP 26 CoS 3" dscp=26 log-prefix=DSCP26 new-priority=3
add action=set-priority chain=postrouting comment="DSCP 32 CoS 4" dscp=31 new-priority=3
add action=set-priority chain=postrouting comment="DSCP 40 CoS 5" dscp=40 new-priority=5
add action=set-priority chain=postrouting comment="DSCP 46 CoS 5" dscp=46 new-priority=5
add action=set-priority chain=postrouting comment="DSCP 48 CoS 6" dscp=48 log-prefix=DSCP48 new-priority=6
add action=set-priority chain=postrouting comment="DSCP 56 CoS 7" dscp=56 new-priority=7

# mark packets 
# WAN
add action=accept chain=postrouting comment="UP IP Precedence (aka Packet Priority) 0 - Best Effort (Low Priority) (default)" out-interface=vlan3012 priority=0
add action=mark-packet chain=postrouting comment="UP IP Precedence (aka Packet Priority) 1 - Scavenger (Bottom Priority) (apply packet mark ip_precedence_1)" new-packet-mark=up_ip_precedence_1 out-interface=vlan3012 passthrough=no priority=1
add action=mark-packet chain=postrouting comment="UP IP Precedence (aka Packet Priority) 2 - Background (Very Low Priority) (apply packet mark ip_precedence_2)" new-packet-mark=up_ip_precedence_2 out-interface=vlan3012 passthrough=no priority=2
add action=mark-packet chain=postrouting comment="UP IP Precedence (aka Packet Priority) 3 - Critical Data or Call Signaling (Medium-Low Priority) (apply packet mark ip_precedence_3)" new-packet-mark=up_ip_precedence_3 out-interface=vlan3012 passthrough=no priority=3
add action=mark-packet chain=postrouting comment="UP IP Precedence (aka Packet Priority) 4 - Interactive Video (Medium Priority) (apply packet mark ip_precedence_4)" new-packet-mark=up_ip_precedence_4 out-interface=vlan3012 passthrough=no priority=4
add action=mark-packet chain=postrouting comment="UP IP Precedence (aka Packet Priority) 5 - Voice (Medium-High Priority) (apply packet mark ip_precedence_5)" log-prefix=UP5 new-packet-mark=up_ip_precedence_5 out-interface=vlan3012 passthrough=no priority=5
add action=mark-packet chain=postrouting comment="UP IP Precedence (aka Packet Priority) 6 - Internetwork Control (High Priority) (apply packet mark ip_precedence_6)" new-packet-mark=up_ip_precedence_6 out-interface=vlan3012 passthrough=no priority=6
add action=mark-packet chain=postrouting comment="UP IP Precedence (aka Packet Priority) 7 - Network Control (Top Priority) (apply packet mark ip_precedence_7)" new-packet-mark=up_ip_precedence_7 out-interface=vlan3012 passthrough=no priority=7

# streaming devices vlan
add action=accept chain=postrouting comment="DN IP Precedence (aka Packet Priority) 0 - Best Effort (Low Priority) (default)" out-interface=vlan171 priority=0
add action=mark-packet chain=postrouting comment="DN IP Precedence (aka Packet Priority) 1 - Scavenger (Bottom Priority) (apply packet mark ip_precedence_1)" new-packet-mark=dn_ip_precedence_1 out-interface=vlan171 passthrough=no priority=1
add action=mark-packet chain=postrouting comment="DN IP Precedence (aka Packet Priority) 2 - Background (Very Low Priority) (apply packet mark ip_precedence_2)" new-packet-mark=dn_ip_precedence_2 out-interface=vlan171 passthrough=no priority=2
add action=mark-packet chain=postrouting comment="DN IP Precedence (aka Packet Priority) 3 - Critical Data or Call Signaling (Medium-Low Priority) (apply packet mark ip_precedence_3)" new-packet-mark=dn_ip_precedence_3 out-interface=vlan171 passthrough=no priority=3
add action=mark-packet chain=postrouting comment="DN IP Precedence (aka Packet Priority) 4 - Interactive Video (Medium Priority) (apply packet mark ip_precedence_4)" new-packet-mark=dn_ip_precedence_4 out-interface=vlan171 passthrough=no priority=4
add action=mark-packet chain=postrouting comment="DN IP Precedence (aka Packet Priority) 5 - Voice (Medium-High Priority) (apply packet mark ip_precedence_5)" log-prefix=DN5 new-packet-mark=dn_ip_precedence_5 out-interface=vlan171 passthrough=no priority=5
add action=mark-packet chain=postrouting comment="DN IP Precedence (aka Packet Priority) 6 - Internetwork Control (High Priority) (apply packet mark ip_precedence_6)" new-packet-mark=dn_ip_precedence_6 out-interface=vlan171 passthrough=no priority=6
add action=mark-packet chain=postrouting comment="DN IP Precedence (aka Packet Priority) 7 - Network Control (Top Priority) (apply packet mark ip_precedence_7)" new-packet-mark=dn_ip_precedence_7 out-interface=vlan171 passthrough=no priority=7

# users vlan
add action=accept chain=postrouting comment="DN IP Precedence (aka Packet Priority) 0 - Best Effort (Low Priority) (default)" out-interface=vlan170 priority=0
add action=mark-packet chain=postrouting comment="DN IP Precedence (aka Packet Priority) 1 - Scavenger (Bottom Priority) (apply packet mark ip_precedence_1)" new-packet-mark=dn_ip_precedence_1 out-interface=vlan170 passthrough=no priority=1
add action=mark-packet chain=postrouting comment="DN IP Precedence (aka Packet Priority) 2 - Background (Very Low Priority) (apply packet mark ip_precedence_2)" new-packet-mark=dn_ip_precedence_2 out-interface=vlan170 passthrough=no priority=2
add action=mark-packet chain=postrouting comment="DN IP Precedence (aka Packet Priority) 3 - Critical Data or Call Signaling (Medium-Low Priority) (apply packet mark ip_precedence_3)" new-packet-mark=dn_ip_precedence_3 out-interface=vlan170 passthrough=no priority=3
add action=mark-packet chain=postrouting comment="DN IP Precedence (aka Packet Priority) 4 - Interactive Video (Medium Priority) (apply packet mark ip_precedence_4)" new-packet-mark=dn_ip_precedence_4 out-interface=vlan170 passthrough=no priority=4
add action=mark-packet chain=postrouting comment="DN IP Precedence (aka Packet Priority) 5 - Voice (Medium-High Priority) (apply packet mark ip_precedence_5)" log-prefix=DN5 new-packet-mark=dn_ip_precedence_5 out-interface=vlan170 passthrough=no priority=5
add action=mark-packet chain=postrouting comment="DN IP Precedence (aka Packet Priority) 6 - Internetwork Control (High Priority) (apply packet mark ip_precedence_6)" new-packet-mark=dn_ip_precedence_6 out-interface=vlan170 passthrough=no priority=6
add action=mark-packet chain=postrouting comment="DN IP Precedence (aka Packet Priority) 7 - Network Control (Top Priority) (apply packet mark ip_precedence_7)" new-packet-mark=dn_ip_precedence_7 out-interface=vlan170 passthrough=no priority=7

wiseroute · June 4, 2025, 2:26pm

hello,

before going in depth with dscp and friends,

can you be more specific for this audio? is it ip radio or ip pbx or something else?

what layer 2 link did you use?

how about local vlan?

tommyd · June 4, 2025, 2:42pm

As I wrote the audio is some application specific audio stream for radio broadcast. It is some proprietary protocol for radio broadcast software with quite poor error correction. A single stream can contain one or more audio streams. It is nothing like RTMP or Icecast. The layer 2 link is a VPLS service from a national telecom. It is similar to metro ethernet

What do you mean about local vlan? Every branch office has couple of local vlans: one for users, one for streaming devices and sometimes third vlan for something else. The provider CPE devices do accept tagged traffic and tagged frames are transmitted unmodified trough the provider network. They also use PCP/CoS field in our Ethernet frames to apply priorities in their network and accept priority values of 3 a 5. Thats why we use tagged frames on WAN

wiseroute · June 4, 2025, 3:06pm

i see. it’s a radio broadcast.

and what kind of visible poor quality did you have? crosstalk or something else?

was this radio just for intranet? or to internet broadcast? do you use custom device for listening to this radio? or just basic any network connected browser/app?

I’m sorry for long assessment)

Amm0 · June 5, 2025, 12:47am

I’m not 100% sure CAKE is helping you in this case…or at least adding complexity since CAKE does have some important params. Have you tried a similar queue, like fq_codel or WFQ or even just using no queue?

Also if can you enable FEC on your MPEG-TS streams? That often helps with raw UDP media streams.

Finally, I’d check with your service provider on their QoS DSCP strategy… since not everyone uses same values… They’d likely know what their highest priority DSCP value is.

tommyd · June 5, 2025, 7:37am

The stream contains some control data, which the receiver is checking (i.e. timestamps, order ). If the packet delay exceed some threshold it is dropped. The receiver produces statistics of packet that are dropped, duplicated, delayed, etc.

tommyd · June 5, 2025, 7:51am

I took CAKE from some forum post also, earlier i was using the default.

Also if can you enable FEC on your MPEG-TS streams? That often helps with raw UDP media streams.

The production stream is using some light FEC but it is not effective enough if the WAN link is under heavy load

Finally, I’d check with your service provider on their QoS DSCP strategy… since not everyone uses same values… They’d likely know what their highest priority DSCP value is.

They say they do, actually they look only for the PCP header in The Ethernet frames. I did some packet capture and Mikrotik is setting it correctly. There is another similar location with the same operator, but they have Aruba 6300f switch as a WAN router and the transmission quality is clearly better. QoS configuration on the switch is pretty basic classifier + policy, and switch uses a default scheduler dwrr

wiseroute · June 5, 2025, 3:32pm

hello tommy,

looks like you have way too many problem to work on)

on the wan side, vpls can be considered as passenger protocols.
dscp trust means additional paid service. it is usually written in the contract - because not everyone can order the isp to accommodate their qos needs. do check your sla.
your firewall mangle - it has way too many 46 other than the actual protocol who desperately need it.
what is the significance of ack? other than let the udp flow faster.
did you know that queueing can make things worst if not set correctly?
bandwidth capping. loosen it. or have bigger bandwidth available for better users experience.
what your app doing with timestamp etc - is no difference to layer 7 filters. cpu hog make everything twisted.
one device for one app is better for critical mission.
can go on but i prefer the simplest things first.