Community discussions

MikroTik App
 
mellow
just joined
Topic Author
Posts: 10
Joined: Mon Aug 08, 2016 12:46 pm

CCR RAM SCAM!?

Mon Aug 08, 2016 1:08 pm

Please prove me wrong.

I found this old thread. It looks like RouterOS is only using 2GB RAM regardless the platform. Even the official hardware from Mikrotik cannot utilise more than 2GB of RAM. Does that mean RouterOS is not meant to be used in large scale? Do I miss anything here?
Interesting post, I am looking for same answer, How to increase max-entries: I have recently bought CCR-1036 Hardware. I am doing lots of firewall stuff and i badly need that setting bump. Document lying about it, that it will increase base on available RAM. We have 16GB RAM and 14GB Free. What i should do now???
[admin@MikroTik] > /ip firewall connection tracking print
                   enabled: auto
      tcp-syn-sent-timeout: 5s
  tcp-syn-received-timeout: 5s
   tcp-established-timeout: 1d
      tcp-fin-wait-timeout: 10s
    tcp-close-wait-timeout: 10s
      tcp-last-ack-timeout: 10s
     tcp-time-wait-timeout: 10s
         tcp-close-timeout: 10s
   tcp-max-retrans-timeout: 5m
       tcp-unacked-timeout: 5m
               udp-timeout: 10s
        udp-stream-timeout: 3m
              icmp-timeout: 10s
           generic-timeout: 10m
               max-entries: 524288
             total-entries: 524316

Free Memory:
[admin@MikroTik] > /system resource print
             uptime: 4h6m43s
            version: 6.35rc3 (testing)
         build-time: Feb/03/2016 07:32:45
        free-memory: 14.6GiB
       total-memory: 15.9GiB
                cpu: tilegx
          cpu-count: 36
      cpu-frequency: 1200MHz
           cpu-load: 100%
     free-hdd-space: 883.3MiB
    total-hdd-space: 1024.0MiB
  architecture-name: tile
         board-name: CCR1036-8G-2S+
           platform: MikroTik
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 24651
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: CCR RAM SCAM!?

Mon Aug 08, 2016 1:12 pm

This number in "max-entries" will increase only when needed. When you will have 60'000 entries, the max entries will increase to accommodate that. I know it may be confusing, but it works that way. It will not reserve RAM for connections that don't exist. It will increase when you will hit the limit for some period of time. It will use 16GB, there is no scam ;)
No answer to your question? How to write posts
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1734
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Re: CCR RAM SCAM!?

Mon Aug 08, 2016 1:20 pm

What does it have to do with RAM??? It looks like amount of connections in conntrack is simply limited to half a million, and together with RAW table (firewall filter that happens before connection tracking) added to latest versions, i do not see the reason why there should be more.

memory is utilized quite well especially when you using BGP, so there are no limit of RAM usage.
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
mellow
just joined
Topic Author
Posts: 10
Joined: Mon Aug 08, 2016 12:46 pm

Re: CCR RAM SCAM!?

Mon Aug 08, 2016 1:21 pm

This number in "max-entries" will increase only when needed. When you will have 60'000 entries, the max entries will increase to accommodate that. I know it may be confusing, but it works that way. It will not reserve RAM for connections that don't exist. It will increase when you will hit the limit for some period of time. It will use 16GB, there is no scam ;)
Hi normis. Thank you for the quick reply. But please read carefully. The total-entries is already greater than max-entries. I don't see max-entries increased anyhow. The original post of the thread even kept it running for hours on x86 platform and didn't see the value changed.
max-entries: 524288
total-entries: 524316
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 24651
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: CCR RAM SCAM!?

Mon Aug 08, 2016 1:24 pm

Original post artificially increased connections for experiment sake. Do you really need half a million connections? In what real-life scenario?
No answer to your question? How to write posts
 
mellow
just joined
Topic Author
Posts: 10
Joined: Mon Aug 08, 2016 12:46 pm

Re: CCR RAM SCAM!?

Mon Aug 08, 2016 1:28 pm

What does it have to do with RAM??? It looks like amount of connections in conntrack is simply limited to half a million, and together with RAW table (firewall filter that happens before connection tracking) added to latest versions, i do not see the reason why there should be more.

memory is utilized quite well especially when you using BGP, so there are no limit of RAM usage.
Thank you for your reply. But the original post already quoted on normis. normis said the following in this thread so I assume connection tracking has something to do with RAM.
and what about conntrack max-entries?


this number doesn't automatically increase just by adding RAM, it increases based on remaining RAM, when you use most of the entries given in the number. So if your default max is 500000 with 4GB, it will be also 500000 with 16GB until you use 500000, then it will increase based on free RAM
 
mellow
just joined
Topic Author
Posts: 10
Joined: Mon Aug 08, 2016 12:46 pm

Re: CCR RAM SCAM!?

Mon Aug 08, 2016 1:42 pm

Original post artificially increased connections for experiment sake. Do you really need half a million connections? In what real-life scenario?
Well... What is wrong with "artificially inscresed connections for experiment sake"? People need to test the limit of the hardware/software. How are people suppose to test without artificially generating data? We all need to test the equipment before we deploy to production environment. At least I would've done my feasibility study first which is what I am doing now.

Well... I plan to start a small ISP for my neighbourhood. I happen to live around college campus. Here has entire village of college students. As we all know, college students do whatever they want. Let's say for example they all use BitTorrent. It's very easy to reach 10000 connections for 1 PC with BitTorrent running. Let's say 1 house has 10 PCs and I would start small with just 10 houses. That's already 1 million connections.

Based on the math, 1 CCR can only server 5 clients max...
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1734
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Re: CCR RAM SCAM!?

Mon Aug 08, 2016 1:52 pm

Sorry, but what planet are you from?

i have /22 network full with clients, and only time i run into problems with amount of connections was when i was under DDoS attack(to be more precise - some of the clients were infected to be a part of DDoS attack to somewhere).

it is 500k, it is 500 connections per /22 client. atm they use 83 connections on average.
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
mellow
just joined
Topic Author
Posts: 10
Joined: Mon Aug 08, 2016 12:46 pm

Re: CCR RAM SCAM!?

Mon Aug 08, 2016 2:04 pm

Sorry, but what planet are you living on?

i have /22 network full with clients, and only time i run into problems with amount of connections was when i was under DDoS attack(to be more precise - some of the clients were infected to be a part of DDoS attack to somewhere).
Haha. I'm living on planet Earth. I believe it's the same planet as yours.

Anyway, joking aside.

Thank you very much for your valuable experience. Do you use connection tracking for your firewall? If so, how many connections do you normally get for presumably 1000 clients? Do you have SNMP set up to track your connections? Do you ever see your max-entries greater than 500k?

Thank you very much in advance if you can share your more about your real life situation.


Edit: Thank you. I see you added your comment. How do you manage to have only 83 connections on average? Do your clients not use BitTorrent? Don't they download anything from Internet? Can you please share a bit more. Thank you.
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1734
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Re: CCR RAM SCAM!?

Mon Aug 08, 2016 2:14 pm

First of all putting a state-full firewall on such large network is stupid. on almost all my routers that work with 100+ clients i have connection tracking off.

But after i got a call that i'm source of DDoS attack, i had to turn it on for some "special" clients.

Now with new RAW table - especially action=no-track, all problems are solved, i can leave firewall on form some clients, and rest of the clients go "no-track" mode. So i can't give you number for whole /22 network, i can just say that those "special" clients are using 83 connections on average right now.
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
mellow
just joined
Topic Author
Posts: 10
Joined: Mon Aug 08, 2016 12:46 pm

Re: CCR RAM SCAM!?

Mon Aug 08, 2016 2:27 pm

First of all putting a state-full firewall on such large network is stupid. on almost all my routers that work with 100+ clients i have connection tracking off.

But after i got a call that i'm source of DDoS attack, i had to turn it on for some "special" clients.

Now with new RAW table - especially action=no-track, all problems are solved, i can leave firewall on form some clients, and rest of the clients go "no-track" mode. So i can't give you number for whole /22 network, i can just say that those "special" clients are using 83 connections on average right now.
Thank you for your reply once again.

I don't really want to use connection tracking, but I see an example from wiki that marks connections to handle load balancing. Is there a better way to load balance without using connection tracking? How do you manage your WAN connections in such scale? This is my bottleneck right now.
 
mellow
just joined
Topic Author
Posts: 10
Joined: Mon Aug 08, 2016 12:46 pm

Re: CCR RAM SCAM!?

Mon Aug 08, 2016 2:32 pm

normis!!! Please don't disappear on me.

Can you please please elaborate on how max-entries for connection tracking works? Is it NOT properly implemented?

Is it just like what macgaiver said that we shouldn't use connection tracking in large scale?
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1734
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Re: CCR RAM SCAM!?

Mon Aug 08, 2016 2:36 pm

BGP is managing it for me :)

PCC is fine for client's network if you use NAT.
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
mellow
just joined
Topic Author
Posts: 10
Joined: Mon Aug 08, 2016 12:46 pm

Re: CCR RAM SCAM!?

Mon Aug 08, 2016 2:56 pm

BGP is managing it for me :)

PCC is fine for client's network if you use NAT.
OTZ...

Thank you, but I guess BGP doesn't work for me. I honestly have no clue what BGP is besides it's a routing protocol like OSPF. I have never ever had my hands on BGP in my life. I don't think my local ISPs would ever let me use BGP or any other routing protocol.

Like I said, I would like to start small to subscribe a better ADSL/Fibre line and resale it.

I just had a quick look at my local ISP (Bell Canada). It's Business level doesn't say anything about BGP and I don't think I can afford Enterprise level of Internet. In other words, I need connection tracking with more than 500k.
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1734
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Re: CCR RAM SCAM!?

Mon Aug 08, 2016 3:08 pm

I'm 99,999% sure you will be fine with 500k
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
mellow
just joined
Topic Author
Posts: 10
Joined: Mon Aug 08, 2016 12:46 pm

Re: CCR RAM SCAM!?

Mon Aug 08, 2016 3:14 pm

I'm 99,999% sure you will be fine with 500k
Hahaha!!! Thank you very much. It's nice to have someone in the field saying so.



normis! Can you please confirm if it's a bug that can be fixed regarding to the 500k? Although I may not (0.001%) need it, it's still good to know it's there when I need it.
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 24651
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: CCR RAM SCAM!?

Mon Aug 08, 2016 3:52 pm

we use a hash function in our connection tracking table, bigger the table, bigger the size of the hash necessary to map all connections. Bigger hash size, slower the searches, more resources used etc.

So there is no point having bigger hash than you can possibly need. From our experience in support, number of connections is rarely the bottleneck, but all those cases were fixed with firewall raw table and "no-track" action.

I strongy suggest to test everything out before jumping to theoretical conclusions.
No answer to your question? How to write posts
 
mellow
just joined
Topic Author
Posts: 10
Joined: Mon Aug 08, 2016 12:46 pm

Re: CCR RAM SCAM!?

Mon Aug 08, 2016 4:19 pm

we use a hash function in our connection tracking table, bigger the table, bigger the size of the hash necessary to map all connections. Bigger hash size, slower the searches, more resources used etc.

So there is no point having bigger hash than you can possibly need. From our experience in support, number of connections is rarely the bottleneck, but all those cases were fixed with firewall raw table and "no-track" action.

I strongy suggest to test everything out before jumping to theoretical conclusions.

normis! Thank you very VERY MUCH. You answer has been very helpful. :)

My idea is just to prepare for the very worst case scenario. I will try my best to come up the most efficient firewall rules for my needs, and play with RAW table.
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8420
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: CCR RAM SCAM!?

Tue Sep 27, 2016 2:43 pm

I'm 99,999% sure you will be fine with 500k
originally it was something like "I'm 99,999% sure you will be fine with 640k" :lol:
Russian-speaking forum: https://forum.mikrotik.by/. Welcome!

For every complex problem, there is a solution that is simple, neat, and wrong.

MikroTik. Your life. Your routing.
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8420
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: CCR RAM SCAM!?

Wed Oct 12, 2016 2:51 pm

Well, now it's official. There's hard limit on the number of entries in connection tracking table: http://forum.mikrotik.com/viewtopic.php ... 10#p562410

So, no 16G can be used =)
Russian-speaking forum: https://forum.mikrotik.by/. Welcome!

For every complex problem, there is a solution that is simple, neat, and wrong.

MikroTik. Your life. Your routing.
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1734
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Re: CCR RAM SCAM!?

Wed Oct 12, 2016 3:40 pm

Well, now it's official. There's hard limit on the number of entries in connection tracking table: http://forum.mikrotik.com/viewtopic.php ... 10#p562410

So, no 16G can be used =)
1M connections with conntrack on - amount of used RAM will be the less of your problems :)

there are many other ways, to use RAM, not only conntrack.... I still don't understand point of this topic...
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8420
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: CCR RAM SCAM!?

Wed Oct 12, 2016 4:38 pm

I still don't understand point of this topic...
well, I know people who add new routers just because they approach 500k limit, not because the router is overloaded by CPU or bandwidth
Russian-speaking forum: https://forum.mikrotik.by/. Welcome!

For every complex problem, there is a solution that is simple, neat, and wrong.

MikroTik. Your life. Your routing.
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1734
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Re: CCR RAM SCAM!?

Wed Oct 12, 2016 4:56 pm

well, I know people who add new routers just because they approach 500k limit, not because the router is overloaded by CPU or bandwidth
What is the source of such amount? DDoS?
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
User avatar
Chupaka
Forum Guru
Forum Guru
Posts: 8420
Joined: Mon Jun 19, 2006 11:15 pm
Location: Minsk, Belarus
Contact:

Re: CCR RAM SCAM!?

Wed Oct 12, 2016 5:10 pm

popular p2p, I think
Russian-speaking forum: https://forum.mikrotik.by/. Welcome!

For every complex problem, there is a solution that is simple, neat, and wrong.

MikroTik. Your life. Your routing.
 
User avatar
macgaiver
Forum Guru
Forum Guru
Posts: 1734
Joined: Wed May 18, 2005 5:57 pm
Location: Sol III, Sol system, Sector 001, Alpha Quadrant

Re: CCR RAM SCAM!?

Thu Oct 13, 2016 8:29 am

popular p2p, I think
You have to really try to get even in 5 digit number of connections per client using any current p2p.
With great knowledge comes great responsibility, because of ability to recognize id... incompetent people much faster.
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 24651
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: CCR RAM SCAM!?

Thu Oct 13, 2016 9:02 am

popular p2p, I think
You have to really try to get even in 5 digit number of connections per client using any current p2p.
I have never reached 4 digits, even when trying hard.
No answer to your question? How to write posts
 
jarda
Forum Guru
Forum Guru
Posts: 7603
Joined: Mon Oct 22, 2012 4:46 pm

Re: CCR RAM SCAM!?

Thu Oct 13, 2016 10:40 am

Small home networks with one u torrent running machine easily reach 3-5 thousands connections. You definitely were not trying so hard.
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 24651
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: CCR RAM SCAM!?

Thu Oct 13, 2016 11:11 am

Small home networks with one u torrent running machine easily reach 3-5 thousands connections. You definitely were not trying so hard.
OK, I have no idea how, but anyway, if you reach 1 million connections with connection tracking ON, you have other problems before the connection limit. If this is torrent, your bandwidth ran out long time ago anyway
No answer to your question? How to write posts
 
jarda
Forum Guru
Forum Guru
Posts: 7603
Joined: Mon Oct 22, 2012 4:46 pm

Re: CCR RAM SCAM!?

Thu Oct 13, 2016 11:58 am

That's true of course. But it is another topic... And anyway, normis, isn't here a rule that forbids repeating of the last post by citation?
 
User avatar
normis
MikroTik Support
MikroTik Support
Posts: 24651
Joined: Fri May 28, 2004 11:04 am
Location: Riga, Latvia

Re: CCR RAM SCAM!?

Thu Oct 13, 2016 12:26 pm

Is there? If there are multiple posts above mine, I like to quote to be more specific, who I am responding to. Never know if somebody will post, while I am writing.
No answer to your question? How to write posts
 
User avatar
ZeroByte
Forum Guru
Forum Guru
Posts: 4051
Joined: Wed May 11, 2011 6:08 pm

Re: CCR RAM SCAM!?

Thu Oct 13, 2016 9:35 pm

And anyway, normis, isn't here a rule that forbids repeating of the last post by citation?
I didn't know that was a rule - it does annoy me whenever someone quotes an entire long post over just a sentence or two....

Your typical posts tend to be one liners anyway, so Normis had no choice but to quote your entire post anyway. :lol:
When given a spoon,
you should not cling to your fork.
The soup will get cold.
 
jarda
Forum Guru
Forum Guru
Posts: 7603
Joined: Mon Oct 22, 2012 4:46 pm

Re: CCR RAM SCAM!?

Thu Oct 13, 2016 11:23 pm

You know that tapping in tapatalk on the phone is not so convenient therefore I am trying to be as short as possible. And I believe I saw a rule saying do not citate the last post. Well, maybe it was somewhere else, if it is not good behavior here, then never mind. But like zerobyte, I don't like it too.

Who is online

Users browsing this forum: rufusbrand8869 and 19 guests