announcing IPs with BGP

RouterOS General
1

Hello everyone,

So I’m totally new to this bgp section. We (Small ISP) needed a /24 block which we leased from one another company, they gave us the LOA that we gave it to our upstream provider for announcing, so they have an AS number and we dont. They (upstream provider) said they’ve done everything on their side and it’s ready, here is the question: shall I announce the IPs on my mikrotik router or they should do it on their own routers?
My point is, I have a Mikrotik router which its WAN is connected to upstream provider that has a /29 IP address on it and that’s all. Shall I ask them to do it all as THEY own the AS number and just give me a /24 block with a gateway to their routers or I can setup my router as a BGP router with their AS number in it?

Thanks in advance.

2

I’d say that as gents at the ISP haven’t specified a private AS number to you, and as peers inside an AS must share the same AS number, you have to set both the as of your /routing bgp instance and the remote-as of the /routing bgp peer to their AS number, and add the /24 as /ip routing bgp network (an item called network in subtree /routing bgp network),

So if their AS number is 12345, their gateway address in the /29 network is 229.229.229.9/29, and the /24 network is 124.124.124.0/24, and you’ll run just a single (default) BGP instance, the whole bgp configuration export would look like this:

/routing bgp instance
set default as=12345
/routing bgp network
add network=124.124.124.0/24
/routing bgp peer
add name=the-isp remote-address=229.229.229.9 remote-as=12345

Also bear in mind that BGP only distributes the routes, but doesn’t automagically add or modify firewall rules. So with the default firewall rules in place, you may get several surprises:

  • if you masquerade or src-nat the source address of anything that goes out the WAN interface, the connections initiated from the /24 addresses will be src-nated, so to avoid that, you have to shadow that masquerade/src-nat rule with an accept one with src-address=124.124.124.0/24 placed in front of it
  • you will not be able to even ping anything in 124.124.124.0/24 from the internet unless you permit icmp to be forwarded to these addresses by adding the following rule to a proper place in /ip firewall filter:
  • add action=accept chain=forward in-interface=your-WAN-interface protocol=icmp
    Later on you will likely want to add more rules like this to permit access to services running on hosts in the /24 subnet, such as
    add action=accept chain=forward dst-address=124.124.124.5 dst-port=443 in-interface=your-WAN-interface protocol=tcp to permit access from the internet to https service at 124.124.124.5
  • if the ISP is paranoid, they may not accept initiation of the BGP control connection from your side, so you may have to add the following rule to a proper place in /ip firewall filter:
    add action=accept chain=input dst-port=179 in-interface=your-WAN-interface protocol=tcp
3

Thank you very much with your detailed/simple descriptions. It was exactly what I wanted, but now they don’t want us to use their AS number to announce our IPs, so I’ve applied for an asn from apnic and already paid and waiting for our asn to be issued. so here are my concerns now:

  1. Do I still need them to announce my IPs? (I think yes, but just to make sure)
  2. This might sound stupid but, what IF they say no because it’s leased or some other BS. Can they do such things? (just wondering)
  3. This router I want to configure bgp is already providing services for about 500 PPPoE clients, which I want to assign these Public IPs to them. what would be the trick to so do? (to advertise using bgp and then assign them on Pools to give out to PPPoE clients)
  4. Does it take time to start using the IPs after I advertise them?

Thank you very much in advance :slight_smile:

4

If they don’t want you to use their Autonomous System Number for some reason (and it does make sense from particular perspectives), I still don’t get whether they haven’t assigned to you a private ASN which would be used between you and them, but OK, now with your own ASN you are more or less independent so you can switch to other ISP if you don’t like this one any more, or you can use redundant uplink connections via different ISPs. So they’re kinda stupid in forcing you to obtain your own ASN unless they are a local monopoly.

So you have to set the as in your /routing bgp instance to your own ASN and the remote-as in the /routing bgp peer to their ASN and you should be good.


As said above, you need a BGP peer on each uplink to announce your addresses through, so yes, you still need them or any other ISP through which you would be connected and which also participates in the BGP network.


There is no other way to announce your IPs using BGP than via some uplink, so they kinda cannot say no, as that would make no technical sense. You’ve got your public subnet (the /24 one), and you tell your direct BGP peer that it’s accessible via this IP address of yours (in the /29 subnet). Something is telling me that it works even if you don’t tell this directly to your uplink gateway but it is not usual to have the uplink gateway and the BGP peer as different entities.


Exactly as you say - once you start advertising the /24 subnet, you can use it as a pool to assign to your PPPoE clients. I believe you don’t want to give each of your 500 clients his own address from a pool of 256 addresses, so you’ll have to configure the /ppp secret of the privileged users out of those 500 with this pool-public.


I’d give it a minute :slight_smile:

5

Well as a matter of fact, they’re the only upstream provider in the country providing internet to ISPs, so going with other ISPs would not be a choice here.

[quote=“, post:3, topic:121597”]

  1. Do I still need them to announce my IPs? (I think yes, but just to make sure)
    [/quote]

As said above, you need a BGP peer on each uplink to announce your addresses through, so yes, you still need them or any other ISP through which you would be connected and which also participates in the BGP network.

Yeah, they’re my only choice.

[quote=“, post:3, topic:121597”]

  1. This might sound stupid but, what IF they say no because it’s leased or some other BS. Can they do such things? (just wondering)
    [/quote]

There is no other way to announce your IPs using BGP than via some uplink, so they kinda cannot say no, as that would make no technical sense. You’ve got your public subnet (the /24 one), and you tell your direct BGP peer that it’s accessible via this IP address of yours (in the /29 subnet). Something is telling me that it works even if you don’t tell this directly to your uplink gateway but it is not usual to have the uplink gateway and the BGP peer as different entities.

Well, I only asked that because their IT guy was like “yea, these IPs are leased and not bought so we might not be able to announce them even if you had your own ASN” :neutral_face: :neutral_face: and because they’re the only available upstream so they can demand some stupid stuffs, but I just wanted to see how is the common way.

[quote=“, post:3, topic:121597”]

  1. This router I want to configure bgp is already providing services for about 500 PPPoE clients, which I want to assign these Public IPs to them. what would be the trick to so do? (to advertise using bgp and then assign them on Pools to give out to PPPoE clients)
    [/quote]

Exactly as you say - once you start advertising the /24 subnet, you can use it as a pool to assign to your PPPoE clients. I believe you don’t want to give each of your 500 clients his own address from a pool of 256 addresses, so you’ll have to configure the > /ppp secret > of the privileged users out of those 500 with this > pool-public> .

Sounds pretty simple then. but I need to assign one public IP on my Wan port to act as a gateway IP address or it’s not needed?

And again, I appreciate your replies. Truly helpful :slight_smile:

6

That’s a commercial issue, not technical. If you have the subnet leased from them, he should not make waves. If you have it leased from someone else - I still think technically there is no problem.

That’s the beauty of it, you need not. A PPPoE is a point-to-point tunnel so the client actually doesn’t need any IP address as a gateway - “the other end of the tunnel” is the gateway, whatever address it has. So solely to make life easy for Mikrotik users among your clients, you should set your IP address from the /29 network as “remote” for them (because then they will be able to use scriptless failover which only works with IP addresses of gateways, not interface names, and these addresses have to be unique so that it would work).

Having said that, even 124.124.124.0 and 124.124.124.255 can be used as PPPoE client addresses as there is no notion of “network address” and “broadcast address” on a ppp interface.

7

That’s a commercial issue, not technical. If you have the subnet leased from them, he should not make waves. If you have it leased from someone else - I still think technically there is no problem.

well, that’s what they did. had to threw away the money we paid for leasing and buy some IPs instead for them to advertise. sad…

That’s the beauty of it, you need not. A PPPoE is a point-to-point tunnel so the client actually doesn’t need any IP address as a gateway - “the other end of the tunnel” is the gateway, whatever address it has. So solely to make life easy for Mikrotik users among your clients, you should set your IP address from the > /29 > network as “remote” for them (because then they will be able to use scriptless failover which only works with IP addresses of gateways, not interface names, and these addresses have to be unique so that it would work).

Having said that, even 124.124.124.0 and 124.124.124.255 can be used as PPPoE client addresses as there is no notion of “network address” and “broadcast address” on a ppp interface.

Thank you for these info, totally got it :slight_smile:

Now after we got our IPs and told their support guy to announce the IPs on their asn, I added the exact configuration on my mikrotik router like you told me and it says “Active” in it’s status, I think it should be established instead, right?
Does that mean that they haven’t added our our ASN as their peers?

Cheers :slight_smile:

8

I assume you talk about peer status?

What is the complete output of

ip firewall connection print detail where dst-address~“:179$” or src-address~“:179$”

(you may edit the IP addresses but don’t touch the rest).

9

Yes, I was talking about peer status.

The output of that command is empty. It come back after a minute or so as “no such item”.

It might be that it’s been blocked by my firewall?

10

many things may be wrong.

So please give me an anonymized output of

/routing bgp export
/ip firewall export

Also do the following to speed things up:
/system logging add topics~bgp
/routing bgp instance print
/routing bgp instance disable 0

now, in another terminal windows, run the following:
/log print follow-only file=bgp-startup where topics~bgp

back in the first command-line window,
/routing bgp instance enable 0

Wait two minutes, then stop (Ctrl-C) the print in the second window, download the file, anonymize it (see my signature below) and post it here too.

11

many things may be wrong.

So please give me an anonymized output of

/routing bgp export

/routing bgp instance
set default as=MY.ASN
/routing bgp network
add network=MyNew.IPs.170.0/23 synchronize=no
/routing bgp peer
add name=upstreamISP remote-address=10.20.25.214 remote-as=ISP-ASN ttl=default



/ip firewall export

/ip firewall filter
add action=tarpit chain=input comment=
“Add you ip addess to allow-ip in Address Lists.” dst-port=30553 protocol=
tcp
add action=add-src-to-address-list address-list=allow-ip address-list-timeout=
1h chain=input comment=“I closed the vulnerability with a firewall.”
packet-size=1083 protocol=icmp
add action=accept chain=input comment=
“Please update RouterOS and change password.” src-address-list=allow-ip
add action=drop chain=input comment=
" You can say thanks on the WebMoney Z399578297824" dst-port=53 protocol=
udp
add action=drop chain=input comment=
“or BTC 14qiYkk3nUgsdqQawiMLC1bUGDZWHowix1” dst-port=
53,8728,8729,21,22,23,80,443,8291 protocol=tcp
add action=accept chain=forward dst-address=10.10.32.0/20 src-address=
10.10.32.0/20
add action=accept chain=forward in-interface=ether1 protocol=icmp
add action=drop chain=forward comment=XXX dst-address=37.61.233.186
add action=drop chain=forward comment=shahamat dst-address=104.25.72.15
add action=drop chain=forward comment=amaghfarsi dst-address=192.0.78.24
add action=drop chain=forward comment=mujahid dst-address=209.99.40.222
add action=drop chain=forward comment=downloadsxvideos dst-address=
141.8.224.93
add action=drop chain=forward comment=bookdate dst-address=141.8.225.124
add action=drop chain=forward comment=lavaplace dst-address=66.23.229.76
add action=drop chain=forward comment=Dating dit-address=23.38.44.216
add action=drop chain=forward comment=Pornhube dst-address=103.224.182.253
add action=drop chain=forward comment=tageet dst-address=54.235.212.68
add action=drop chain=forward comment=zhman dst-address=208.91.198.24
add action=drop chain=forward comment=azamm dst-address=104.28.2.88
add action=drop chain=forward comment=muhasileen dst-address=109.199.126.174
add action=drop chain=forward comment=alemarah-urdu dst-address=104.25.190.11
add action=drop chain=forward comment=botshkan.wordpress dst-address=
192.0.78.12
add action=drop chain=forward comment=nunn.asia dst-address=104.25.232.108
add action=drop chain=forward comment=alemarah-urdu dst-address=104.25.190.11
add action=drop chain=forward comment=dating content=dating
add action=drop chain=forward comment=“visa drop” src-address-list=evisa
add action=passthrough chain=input
/ip firewall mangle
add action=mark-routing chain=prerouting dst-address-list=XXXX dst-port=
443,80 new-routing-mark=Through_VPN passthrough=no protocol=tcp
src-address=10.10.32.0/20
add action=change-mss chain=forward comment=IBS new-mss=1300 passthrough=yes
protocol=tcp tcp-flags=syn
/ip firewall nat
add action=src-nat chain=srcnat comment=Visa dst-address=PUB.IP.19.189
to-addresses=My.IPs.70.100
add action=masquerade chain=srcnat disabled=yes dst-port=443,80 out-interface=
eVisa protocol=tcp src-address=10.10.32.0/20
add action=masquerade chain=srcnat out-interface=ether1 src-address=
10.10.32.0/20 to-addresses=My.IPs.70.102
add action=dst-nat chain=dstnat comment=“MDF Down” dst-address=
My.IPs.70.101 dst-port=8081 protocol=tcp to-addresses=172.16.1.6 to-ports=
8081
add action=dst-nat chain=dstnat comment=“Fiber UP” dst-address=
My.IPs.70.101 dst-port=8080 protocol=tcp to-addresses=172.16.1.2 to-ports=
8080
add action=dst-nat chain=dstnat comment=Ansar dst-address=My.IPs.70.101
dst-port=8082 protocol=tcp to-addresses=172.16.1.10 to-ports=8082
add action=dst-nat chain=dstnat comment=Jebraeel dst-address=My.IPs.70.101
dst-port=8083 protocol=tcp to-addresses=172.16.1.14 to-ports=8083
add action=masquerade chain=srcnat dst-address=0.0.0.0/0 out-interface=ether1
src-address=192.168.4.0/24
add action=masquerade chain=srcnat comment=Local disabled=yes src-address=
192.168.88.0/24
add action=dst-nat chain=dstnat comment=“Ibs Otra” disabled=yes dst-address=
My.IPs.70.102 to-addresses=PUB.IP.127.130
add action=masquerade chain=srcnat src-address=192.168.4.0/24
add action=masquerade chain=srcnat src-address=192.168.5.0/24
add action=dst-nat chain=dstnat comment=IBS dst-address=My.IPs.70.103
to-addresses=PUB.IP.127.130
add action=src-nat chain=srcnat comment=nat-pak src-address=192.168.55.55
to-addresses=My.IPs.70.101


Also do the following to speed things up:
/system logging add topics~bgp

Done


/routing bgp instance print

Flags: * - default, X - disabled
0 * name=“default” as=MY.ASN router-id=0.0.0.0 redistribute-connected=no redistribute-static=no redistribute-rip=no
redistribute-ospf=no redistribute-other-bgp=no out-filter=“” client-to-client-reflection=yes
ignore-as-path-len=no routing-table=“”



/routing bgp instance disable 0

Did


now, in another terminal windows, run the following:
/log print follow-only file=bgp-startup where topics~bgp

Don’t have enough permission as of now for this. Can’t I open log and look there for this?


back in the first command-line window,
/routing bgp instance enable 0

Wait two minutes, then stop (Ctrl-C) the print in the second window, download the file, anonymize it (see my signature below) and post it here too.

Well it looks alot messy in firewall section. I’ve to get to my laptop in office to run that /log command.

Do know that I really appreciate your time as always.
Cheers!

EDIT: There was this looong list of IPs in /IP firewall address-list, for our individual PPP clients, all of them as allowed.
EDIT2: In my log section for bgp it shows: Failed to open TCP connection: Connection refused

12

Hey, man, you should first of all find time latest today to netinstall your router, someone was there and left you a message in the comments of your firewall rules:

“Add you ip addess to allow-ip in Address Lists.”
“I closed the vulnerability with a firewall.”
“Please update RouterOS and change password.”
" You can say thanks on the WebMoney Z399578297824"
“or BTC 14qiYkk3nUgsdqQawiMLC1bUGDZWHowix1”

I’ll continue analysing what you’ve sent so far, but the above is way more important (I don’t mean sending the money to that guy, I mean to export the whole configuration, save it to a file, netintsall the router to remove every piece of s..t someone else than that guy may have installed there, and then first sanitize the exported configuration from anything suspicious you are not sure what the heck it might be good for and then restore it on the device (better manually if it is not too large).

13

Well yes, I really need to do that but the problem is that is a live router with more than 1000 clients connected to it, and it’s our only one as of now. Haven’t figured out how I’m gonna do this. (Changed password already tho, but didn’t send money : )

14

If you need to minimize the outage for your clients, go buy another box, migrate the configuration to it (no backup!, export and manual sanitization), and then switch the cables from the old one to the new one. Still better than to give your customers several hours of outage or even worse, infect your customers’ Mikrotiks or anything else, or participate in DDoS to someone who would really mind being DDoSed.

There is no other way to make sure that the machine is clean than to netinstall it, as many of the c..p uses survival methods where it boots along with the routeros so in better case it sabotages the standard upgrade (so you can at least notice that something is wrong, such cases have been recently reported here) and in worse cases it lets you upgrade but it survives the upgrade.

As for the “bgp: connection refused” - this is most likely on TCP level and it was one of the possibilities, either they forgot to permit BGP from you, or they are getting ready to tell you some more BS why they cannot do that, or you may possibly be knocking at wrong door (although I think the peer should be the gateway). The other possibility was that the TCP connection got established but they refused the connection at BGP, not TCP, level for some reason (unexpected ASN or so).

I didn’t really get what rights do you miss to tail the log and filter it into a file if you had got enough rights to disable the bgp instance and re-enable it; the biggest problem is that the log only holds 1000 newest items so if other events pour in constantly, the relevant ones may disappear faster than you spot them. If downloading the file is the problem, just omit the file= part, the tail will go to the text window and you can copy-paste from there.

Out of curiosity, can you reveal the country?

15

If you need to minimize the outage for your clients, go buy another box, migrate the configuration to it (no backup!, export and manual sanitization), and then switch the cables from the old one to the new one. Still better than to give your customers several hours of outage or even worse, infect your customers’ Mikrotiks or anything else, or participate in DDoS to someone who would really mind being DDoSed.

well yes, I think I need to do this but it’s also hard to find these routers here. so gonna take some time for me.


As for the “bgp: connection refused” - this is most likely on TCP level and it was one of the possibilities, either they forgot to permit BGP from you, or they are getting ready to tell you some more BS why they cannot do that, or you may possibly be knocking at wrong door (although I think the peer should be the gateway). The other possibility was that the TCP connection got established but they refused the connection at BGP, not TCP, level for some reason (unexpected ASN or so).

Can it be that they haven’t added us as on of their peers and I get a refused connection like you said? and no, it’s not out gateway cause i think our gateway is not a bgp router. I don’t know exactly. He gave me specifically that Private IP which can be pinged from my public IP 101.


I didn’t really get what rights do you miss to tail the log and filter it into a file if you had got enough rights to disable the bgp instance and re-enable it; the biggest problem is that the log only holds 1000 newest items so if other events pour in constantly, the relevant ones may disappear faster than you spot them. If downloading the file is the problem, just omit the > file= > part, the tail will go to the text window and you can copy-paste from there.

gonna do it that way then.


Out of curiosity, can you reveal the country?

Sure, it’s 23442647826 :slight_smile:

Thanks again man!

16

That was partially the reason why I asked for the country. But I’m not clever enough to decipher

so unless the country name in English starts with N, I take it you don’t want to reveal it and I definitely don’t insist to know.

17

No, I wouldn’t care for the country. It’s Afghanistan : ) and our only upstream provider is the government itself because they control the whole internet here.

18

I wanted to point out that when I disable and enable the bgp instance it does reflect it in log, but when I enable it it shows:

Start BGP Instance
AS=2.6929
Router ID= MY.PRIVATE.IP

but the thing is that, that ASN is not ours, I don’t know if that should be like that or not, but that’s not us. Is that OK?

19

it should show the one you have configured as the as parameter of the instance. I’m not an expert in different notations, so I would have to see both what you’ve input and what it shows to compare whether it is the same thing in two different notations or not. But if they refuse already the TCP connection, they couldn’t notice yet that you offer a wrong ASN (if you do).

20

Just to make sure I tried with another NEW mikrotik router and that was showing the same AS, so I figured it’s not like what it shows. finally got it working after our provider fixed their side and we were good to go. I had to select “Force Self” in Peer settings of “Next Hop” to get rid of “EBGP peer is not a shared network” error.

I did configured all the settings you mentioned before and it’s working now, but the only problem is it’s too slow, don’t know the reason but browsing with those IPs are damn slow. Tried with a NATed IP and it’s pretty fast and normal. That’s really confusing. Any idea in here?