If these were a "site to site" VPN, you can then make use of firewall rules to only allow from certain IPs, but as this is typically used for people to work remotely, i.e. today from home, tomorrow from coffee shop, etc. it is difficult to limit who can connect from where, etc.
So best solution is to Use strong passwords
Yes I know this to limit by IP address, but your example with connection today, tomorrow from different location will be hard to limit by IP, but we can't secure by user ?
In my log on router I can see a lot connection who trying to connect at my VPN L2TP, but how I can reduce this trying ?
I create input chain input 17udp port 500 1701 4500 WAN interface with connection state established and new and what more need to do to secure L2TP ?