This not an easy one. Many things can be wrong, and you have no access, not even a view on the uplink network topology.
What the ISP says, is correct, or is not. I have had many “words” with ISP support, disagreeing with the helpdesk.
There was the risk of being terminated as subscription, while trying to get to the second or third level support.
Its a balancing act. Even if you have the proof in your hands, they may not agree, and see it differently.
So being careful in the communication, try to find out what the real topology is.
What can be wrong? As @sindy already explained, it is common practice to “disable port forwarding” in a switch between clients.
But there is also the IP routing protocol, that optimises on its own, based on assumptions, that may not be there;
IP routing assumes, that IP traffic, in the same subnet, coming in and going out through the same interface, could communicate directly without involving that router.
May be true, may be not true! (eg. the switch port forwarding cfr @sindy, but also the “horizon” set on routers or on bridges. (name for this is Vendor dependent)
Interfaces on a MT bridge with the same “horizon” value , will not communicate. (I do use this quite often, to make the network tree-like)
The uplink has a different horizon value, all downlink ports have the same horizon value. Nodes on the downlink ports cannot communicate with nodes on the other downlink ports. Eliminates broadcast storms, or bonjour storms, or NBT master browser elections in windows, etc etc.
The clients can only communicate over the uplink router. Who optimises, and eg. sends an ICMP redirect to the clients : “don’t use this way, you can go directly without me” , what is actually not true.
How to find out? Name your connection A, your neighbors on the same subnet B.
You probably can PING A from the internet. Idem dito for B. Test it. If A cannot PING B , and B cannot PING A, then that direct communication is broken.
Well you can check in A if you get a MAC address for B , in the ARP table. You can try to send ARP requests (who has IP xxxxx?), or reverse ARP requests (what IP has yy:yy:yy:yy:yy:yy)
If no ARP can be found, then direct communication is a problem. I would not be surprised. (Cable modem ISP is all on the same cable, No problem there)
You can sniff the WAN links for ICMP messages. (redirects and others) Also then you could see broadcasts, and packets send to A or B.
Let’s hope there is no assymetric flow (A->B is not the B>A path) Statefull inspection firewalls will not allow for this, they even randomise TCP packet numbers on the fly. What prevents session spoofing.
How to solve.
As @sindy already suggested, try to get both routers with their addresses A and B , to be in different subnets, at least for what A and B thinks the subnets are.
So give them a smaller subnet than the ISP subnet. A must see B in a different subnet, and will always use the uplink router. Idem dito for B.
You may have to disable DHCP client, and copy that information manually in the WAN interface of A and B. (Let’s hope the gateay does not change IP addresses over time)
You can also disable “accept-redirects (yes | no; Default: no) Whether to accept ICMP redirect messages.” on A and B. But A and B must be in separate subnets , to use the gateway in the first attempt.
Proxy ARP on the gateway is in the hands of the ISP.
Static ARP in A and B using the gateway MAC for B and A IP addresses , is just trying to make A and B comm go over the gateway.
No luck? Quite possible.
Then the ICE (STUN and TURN) techniques may be required to reach each other. This techniques are using a public IP server to initiate the connection, and gradually run down the need for the IP server to be in the flow. STUN is using UDP hole punching to get through a firewall and resolve NAT. TURN will permanently use the cloud IP server. (And this works almost always)
Known providers to me are “Zerotier” and “Hamachi Logmein”. “Teamviewer” is similar, just as “GoToMyPC”, “VNC” be it for defeating NAT and firewalls, their initial setup should work, until the explained ‘network split’ has to be crossed, there it will fail. There are alternatives … as softwares .
Maybe use Traceroute to find the sequence of routing steps for A and for B. What is common? (probably all) What steps work if they point to each other?
If not, I will contact my ISP once more.