I have setup an IPSEC tunnel between my RB and a Bintecrouter.
It works fine, pings are running, but when it try to connect to the server on the other side by rdp i get a timeout.
I tested se same situation with another router and everything works.
Has anyone a clue?

Track the packets of the RDP connection attempt with Tools > Torch and find out where the packet gets lost.

Check firewall of windows, maybe remote desktop is only allowed for local subnet.