Same issue here, the dynamic policy is created (as it should) from setting the l2tp server to use-ipsec and assigning a pre-shared key. however once the client disconnects from the l2tp/ipsec tunnel, the dynamic plolicy should be removed (and if the user re-connects from the same source IP, then it should again be recreated).
the issue is that the dynamic policy is not removed, and then there is no way to manually remove it (it gives error, can not remove dynamically created policy). If another l2tp (non ipsec) device needs to connect on udp 1701 udp, then they cannot until the prior rule is removed (which there is no way to do so).
should be a somewhat simple bug fix for mikrotik.
EDIT: CORRECTION, you can "flush" any dynamically created policies by going to "installed SAs" and clicking flush then choose ALL. I still think some kinda of idel timer should be set on dynamically created ipsec policys
:beep :beep :beep