Bridging From QinQ to trunk port with pppoE clients

I am trying to Bridge from a QinQ port interconnected to a Wireless SP. The Wireless SP links a few customers of ours which includes wireless access as well as Backhaul for FTTH customers. We then interconnect to an ISP aggregator as well as another SP to handoff the relevant end customers to these two SP’s. Diagram is attached. We have bridged the VLAN’s as shown in the diagram. If we have a client 1 (pppoE) at Complex 1 connecting to ISP 1 via the aggregator everything works. As soon as I add Complex 2 to the aggregator bridge Client 1 pppoE negotiation does not work. We have tried VLAN filtering but then it does not work at all.
Any help welcome. Thanks
architecture (1).jpg

Something is telling me that vlan-filtering doesn’t work with S-tags, but that should not be the key here.

If I get the picture correctly, you only want the C-vlans to be forwarded between ports, and only ether2 and ether4 use QinQ whereas ether3 holds only C-vlans, right? If so, you can roughly do the following:

/interface vlan
add name=QinQ-ether2 vlan-id=1800 use-service-tag=yes interface=ether2
add name=QinQ-ether4 vlan-id=1800 use-service-tag=yes interface=ether4

/interface bridge
add name=cvlan-bridge vlan-filtering=no protocol-mode=none

/interface bridge port
add bridge=cvlan-bridge interface=QinQ-ether2
add bridge=cvlan-bridge interface=QinQ-ether4
add bridge=cvlan-bridge interface=ether3

And, if you need vlan-filtering, the following should do:

/interface bridge vlan
add bridge=cvlan-bridge vlan-ids=1729,1730 tagged=cvlan-bridge,QinQ-ether2,ether3
add bridge=cvlan-bridge vlan-ids=1735,1736 tagged=cvlan-bridge,QinQ-ether2,QinQ-ether4

/interface bridge set cvlan-bridge vlan-filtering=yes

If the above is about what you’ve done (my automated signature suggests the best form of the answer), the next directions I would dig in are the following:

  • independent instances of STP protocol running at the machines which you don’t manage, interfering with each other’s operation and causing ghostly effects
  • a device in Complex 2 has the same MAC address like Client 1’s device (I know MAC addresses are supposed to be globally unique but people have various ideas and assigning a user MAC address from the globally coordinated pool is not the worst one I’ve come across)

Hi Sindy,
Thanks for the reply, I may have not explained this very clearly. Our C-Vlans are also stripped when forwarded to Port 3. What remains is the ISP S-Vlan and the client C-Vlan which we transport transparently across the network. For example we would have S-Vlan 3602 and C-Vlan 501 which would be passed to Port 3. Hope this makes it a bit clearer.

No, it makes it a bit more foggy as you say that you strip “your” C-vlans but keep the “client” C-vlans but the picture doesn’t give a clue that there are actually two layers of C-vlans and how they are layered. And the picture clearly states that VLAN IDs 1729 and 1730 should be forwarded between ether2 and ether3. Plus path from ether3 to ISPs is clearly marked with “802.1q”, unlike ether2 and ether4 which are marked with QinQ.

So maybe a more detailed picture could explain it better?

Nevertheless, can you exclude that MSP BPDUs are leaking between the various companies’ networks and causing a mess?

Ok, Let me redo diagram.

Updated diagram. Hope this is clearer. Just to iterate, if FTTH Complex 1 is connected and bridged everything works perfectly. As soon as we bridge FTTH Complex 2 then the pppoe discovery does not work (ie the PADI,PADR,PADO negotiation does not happen)
architecture (1).jpg

So when the frame of an ISP1 customer arrives from a user in Complex 1 to your ether2, it has four VLAN tags, in particular (top to bottom order is outermost to innermost):

s-tag 1800
c-tag 1730
s-tag 3627
c-tag 502

and when you forward that frame to ISP1, you send it only with

s-tag 3627
c-tag 502

Correct?

correct

I’d recommend you to fetch some coolware before reading.

Frames for ISP1 from Complex 1 come with your C-tag 1730, and frames from Complex 2 come with your C-tag 1729. Both have the “inner S-tag” 3627, so in this direction (customer to ISP), it would be enough to strip the outer s-tag and outer c-tag and send them to ether3. But in the opposite direction (ISP to customer), the information you need to properly assign your (outer) c-tag is the VID in the inner (ISP’s) c-tag.

So to properly map the inner c-tag VID to the outer c-tag VID, there is no other way than to create one local bridge per each inner c-tag as per the following diagram, remove all four tags on the way from the ether2 and re-add the two inner ones as forwarding the frames towards ether3, and vice versa in the opposite direction (U.xxxx.T represents an /interface vlan with vlan-id value, tag type (S/C) and Untagged and Tagged ends):

. ether2 --- T.S1800.U --|-- T.C1730.U --- T.S3627.U --- T.C502.U --- bridge-c502 --- U.C502.T -|- U.S3627.T --- ether3
.                        '-- T.C1729.U --- T.S3627.U --- T.C501.U --- bridge-c501 --- U.C501.T -'

So the complete configuration for just two inner C-vlans, and if we ignore ether4 for the moment for simplicity sake, would be

/interface bridge
add name=b-s3627-c501 protocol-mode=none vlan-filtering=no
add name=b-s3627-c502 protocol-mode=none vlan-filtering=no

/interface vlan
add name=os1800 use-service-tag=yes vlan-id=1800 interface=ether2
   add name=oc1729 vlan-id=1729 interface=os1800
      add name=is3627-oc1729 use-service-tag=yes vlan-id=3627 interface=oc1729
         add name=ic501-oc1729 vlan-id=501 interface=is3627-oc1729
   add name=oc1730 vlan-id=1730 interface=os1800
      add name=is3627-oc1730 use-service-tag=yes vlan-id=3627 interface=oc1730
         add name=ic502-oc1730 vlan-id=502 interface=is3627-oc1730

add name=is3627-e3 use-service-tag=yes vlan-id=3627 interface=ether3
   add name=ic501-e3 vlan-id=501 interface=is3627-e3
   add name=ic502-e3 vlan-id=502 interface=is3627-e3

/interface bridge port
add bridge=b-s3627-c501 interface=ic501-e3
add bridge=b-s3627-c501 interface=ic501-oc1729
add bridge=b-s3627-c502 interface=ic502-e3
add bridge=b-s3627-c502 interface=ic502-oc1730

In order to add ether4 to the scheme, you’ll have to insert an additional bridge with members ether2, ether4 and hook the tagged side of /interface vlan name=os1800 to that new bridge instead of to ether2 if you don’t care about filtering of outer c-vlans; if you do, this additional bridge will move one layer below, so ether2 and ether4 will each have its own /interface vlan (named os-1800-e2 and os-1800-e4, the “tagless” ends of these two /interface vlan will be member ports of this additional bridge like it was in the previous example, and the tagged sides of oc1729 and oc1730 will be hooked to this additional bridge.

I assume that the inner c-tag VIDs are not coordinated between the ISPs, that’s why I keep the ISP’s s-tag part of the name of the inner bridges.

Thanks Sindy, will test and post update.

During the night, a variant which may require slightly less configuration items came to my mind, which replaces untagging and retagging of the innner (ISP’s) c-vlans and using one bridge per each internal c-vlan to interconnect their ISP direction and access direction by using a single bridge per ISP (inner s-vlan) and using vlan-filtering on that common bridge:

.                                                       bridge-s3627
.                                                            |
. ether2 --- T.S1800.U --|-- T.C1730.U --- T.S3627(C801).U --|-- U.S3627.T --- ether3
.                        '-- T.C1729.U --- T.S3627(C802).U --|

The configuration would be

/interface bridge
#add name=b-s3627-c501 protocol-mode=none vlan-filtering=no
#add name=b-s3627-c502 protocol-mode=none vlan-filtering=no
add name=b-s3627 protocol-mode=none vlan-filtering=yes #+

/interface vlan
add name=os1800 use-service-tag=yes vlan-id=1800 interface=ether2
   add name=oc1729 vlan-id=1729 interface=os1800
      add name=is3627-oc1729 use-service-tag=yes vlan-id=3627 interface=oc1729
         #add name=ic501-oc1729 vlan-id=501 interface=is3627-oc1729
   add name=oc1730 vlan-id=1730 interface=os1800
      add name=is3627-oc1730 use-service-tag=yes vlan-id=3627 interface=oc1730
         #add name=ic502-oc1730 vlan-id=502 interface=is3627-oc1730

add name=is3627-e3 use-service-tag=yes vlan-id=3627 interface=ether3
   #add name=ic501-e3 vlan-id=501 interface=is3627-e3
   #add name=ic502-e3 vlan-id=502 interface=is3627-e3

/interface bridge port
#add bridge=b-s3627-c501 interface=ic501-e3
#add bridge=b-s3627-c501 interface=ic501-oc1729
#add bridge=b-s3627-c502 interface=ic502-e3
#add bridge=b-s3627-c502 interface=ic502-oc1730
add bridge=b-s3627 inteface=is3627-e3     #+
add bridge=b-s3627 inteface=is3627-oc1729 #+
add bridge=b-s3627 inteface=is3627-oc1730 #+

/interface bridge vlan #+
add bridge=b-s3627 vlan-ids=501 tagged=b-s3627,is3627-e3,is3627-oc1729 #+
add bridge=b-s3627 vlan-ids=502 tagged=b-s3627,is3627-e3,is3627-oc1730 #+