Traffic is being sent to BGP Peer but received from IP Transit provider

honeyfairy · November 29, 2021, 8:47pm

[Disclaimer, below ISP names and prefixes are only for example, and does not refer to any real ISP or Transit provider]
Hi
I buy IP Transit from LemonISP, and they give me a default gateway to send my traffic to. They peer with me via BGP. I advertise my /22 prefix to them.
I also peer with PineappleISP and many others at the IXP that my router is racked at.

So, I receive prefixes belonging to LemonISP from PineappleISP, because PineappleISP and LemonISP are peering, so traffic to LemonISP prefixes are free.
Lemon ISP prefix is 45.45.0.0/16.
I have that in my routing table, next hop is Pineapple ISP.

The problem is, traffic from my router destined for that prefix leaves my router and does go to PineappleISP, then to LemonISP. But LemonISP sends traffic back via the IP Transit Port, not via PineappleISP.

Is there a BGP setting that I can change to have the traffic flow To and From those prefixes come back to my router via PineappleISP
Will it help if i ask them to send me a full table instead of a default gateway?

I have a made a diagram to explain. The red is how the traffic is currently flowing. The green is how i need it to flow. (for the specific prefix)

joegoldman · November 29, 2021, 9:35pm

Ultimately you can not control how other networks route traffic from your network.

In general, a provider will always prioritise customer routes highest, then IX, then transit.

In this case, you are customer, so you would be higher priority than the IX routes. It also commercially makes sense because the more of the link you are buying off them you use, the more likely you are to upgrade it to handle the increased traffic.

There are some scenarios where this might be fixable.

You are advertising a /22 you say, if this is true, then keep the /22 with Lemon, but advertise 2x/23’s or 4x/24’s to Pineapple / IXP - this should hopefully mean a more specific route is installed at LemonISP, which route length tends to take priority over local pref - I have seem some providers overwrite this into summary routes to still prefer customer paths/direct paths.

If that doesn’t work, then the other scenario is simply asking LemonISP to prefer the route via IX when available, but then you are essentially asking them to provide you transit via the IX link, so when traffic hits them from non IX paths it would still also take the IX path to get to you. They might not be happy to do this. The IX might not be happy with this.

honeyfairy · November 29, 2021, 9:50pm

Ultimately you can not control how other networks route traffic from your network.

In general, a provider will always prioritise customer routes highest, then IX, then transit.

In this case, you are customer, so you would be higher priority than the IX routes. It also commercially makes sense because the more of the link you are buying off them you use, the more likely you are to upgrade it to handle the increased traffic.

There are some scenarios where this might be fixable.

You are advertising a /22 you say, if this is true, then keep the /22 with Lemon, but advertise 2x/23’s or 4x/24’s to Pineapple / IXP - this should hopefully mean a more specific route is installed at LemonISP, which route length tends to take priority over local pref - I have seem some providers overwrite this into summary routes to still prefer customer paths/direct paths.

If that doesn’t work, then the other scenario is simply asking LemonISP to prefer the route via IX when available, but then you are essentially asking them to provide you transit via the IX link, so when traffic hits them from non IX paths it would still also take the IX path to get to you. They might not be happy to do this. The IX might not be happy with this.

Thank you for this suggestion and understanding exactly what I wrote.
I will try this and see

honeyfairy · November 29, 2021, 11:33pm

Please correct me if I am doing this wrong. But I tried it via Filters. I added a /24 like this:

Print
11   chain=EBGP_V4_OUT-Test prefix=45.45.45.0/24 prefix-length=16-22 protocol=bgp invert-match=no action=accept set-bgp-prepend-path="" 
12   chain=EBGP_V4_OUT-Test protocol=bgp invert-match=no action=discard set-bgp-prepend-path="" 

/routing bgp
add in-filter=EBGP_V4_BILATEL_IN max-prefix-limit=10000 name=Pineapple out-filter=EBGP_V4_OUT-Test remote-address=\
    171.1.1.1 remote-as=123451 ttl=default

joegoldman · November 29, 2021, 11:41pm

No.

you said 45.45.0.0/16 is Lemon so not sure why you are filtering them.

If you are 46.46.46.0/22, you’ll want to have:

Filter to lemon, that only allows your 46.46.46.0/22 route at prefix length 22 (or don’t define prefix length)
Filter to pineapple, that allows your 46.46.46.0/22, with prefix length 24
Make sure you have either an active route or bgp network entry for the 46.46.46.0/22, and the 46.46.46/47/48/49.0/24’s - This is important as the filters just define what ROUTES or NETWORKS are allowed to filter through, they still have to exist somewhere for the filters to process them.

THEN you should see just your 4x /24 routes going to Pineapple, and just your /22 route going to Lemon.

(note i didn’t actually check that 46.0/22 was a valid subnet so my examples may be technically incorrect but its more to convey the logic)

honeyfairy · November 30, 2021, 12:51am

So I tried this. I am now advertising a /24 to Pineapple and the full /22 to Lemon.
Yet, Upstream traffic goes via Pineapple, Downstream traffic comes in via Lemon.

honeyfairy · November 30, 2021, 12:52am

joegoldman · November 30, 2021, 10:01am

It is entirely possible that Lemon summary route back to customer routes, or don’t accept customer prefixes in at all via Pineapple/IX

Like I said, you can not control how other people egress traffic.
Last step is to ask them if they support doing it somehow and hope they are nice.