Concentrate on working with ether1, other ports aren’t used for netinstall process.
Then follow this sequence (it worked most of times on all of my devices):
connect cable between ether1 and PC
setup PC appropriately (e.g. disable firewall, excess network interfaces, …)
start netinstall executable and appropriately set it up
optionally start wireshark and start capturing traffic on netinstall interface
press reset button on RB device and kerp it pressed until further
apply power to RB
wait until you see BOOTP traffic in wireshark (or for netinstall to show RB on list of detected devices)
stop pressing the reset button. It’s often hard to keep it pressed all this time (can be a minute) without momentarily letting it off, to me this is the hardest part of whole netinstall process …
It sounds more an issue of some kind with netinstall, that stuff is tricky, there are more than a few reports that it is easier on Linux and very finnicky on Windows 10/11.
I have higher chance of doing netinstall in linux VS windows 10/11 but if your hands are tied make it sure that you disable all lan adapters in windows except for the wired LAN and disable windows firewall to make it sure, ether1 is the port you are going to use for netinstall, In my experience you have to hold the reset button until to the very end for the netinstall to work specially in RB5009, just my 0.2$
If nothing else, reading that will give you insights into the necessary complexities in this process, as opposed to the unnecessary ones imposed by pressing a general-purpose Windows laptop into this role instead.
It is the “Default Router Configuration Script” given immediately below the “-s reset.scr” bit that references it. If you don’t like mine, write one that configures the router as you please.