Normally the internet is working perfectly with load balancing on Ether5 via DHCP and clients are able to access internet without any issue.
But when I create PPPoE Server on Ether5 it breaks the internet and PPPoE ISP connection are not getting connected they just show link established and internet not working what can be the issue in this configuration with PPPoE server.
Note: When PPPoE created when router is running all working but after reouter restart this problem happen what needs to be changed in this configuration.
Sounds like a hard to spot bug to me. A wild guess is that the pppoe-client interfaces somehow get connected to the local pppoe-server rather than to the external ones. Is there any chance that they could eventually see each other via an external switch?
What happens if, in the state where everything runs, you disable one of the pppoe clients and then enable it again? Does it come up allright or does it end up in that wrong state you’ve otherwise seen after the reboot?
2 are connected with their respective PPPoE servers but no data transfer happening and one is just showing link Established and nothing happening after that I restored the old config and reboot the router then restored to old config DHCP on port5 old setup without PPPoE.
You have responded none of my two questions. While answering the second one requires to create the failing configuration again, so I understand it cannot be done anytime, the one regarding the physical interconnection can be answered without changing a single bit.
All 4 ISP connections coming from their respective Fiber ONT (In Bride Mode) and plugged into Ether 1, 2,3 and 4 so there is no chance that they can communicate or see each other. And PPPoE clients are configured into ppp section (4 PPPoE Out ISP)
And PPPoE Server is configured on Ether5 so as per the router configuration there is no way that (4 PPPoE Out) can connect to PPPoE server on port 5 as its not in bridge mode.
If you need any other information then let me know.
Given the architecture, even if your device had a switch chip, a frame sent by the CPU cannot get back to the CPU without an external loopback connection, that’s why I’ve asked about it. Regarding the possibility of ports in the ether1..ether4 group, it is actually not excluded that they do receive each other’s traffic unless the ONTs come from different upstream ISPs (normally subscriber-facing ports should be isolated at the ISP end but you never know), but that doesn’t bother us, the problem would be a leak back to ether5 which is not in that group.
So it looks like a software issue. Hence the only thing I can suggest as a quick remedy is a workaround - a script scheduled to run at startup, that will disable both the PPPoE client interfaces and the PPPoE server one, wait a while, enable only the client ones, then wait another while, and enable the server:
(the initial delay before disabling the interfaces is there to give the system time to apply the configuration if eventually the startup job got started too early).
The reason for my second question was to find out what happens if one of the PPPoE clients loses connection while everything is running - if the issue is systematic, that script workaround above will resolve only the startup but not the recovery from an outage.
So it does have a switch chip, but that changes nothing - the CPU is connected to a single port of it, so even if the chip was initialized later than the PPPoE client and server interfaces got up (which is not very likely), there is still no way how the switch chip could return frames sent by the CPU back to it.
So try the workaround script, and also try what happens if you disconnect and re-connect one of ether1..ether4 cables during runtime.