Community discussions

MikroTik App
 
jay22
just joined
Topic Author
Posts: 15
Joined: Fri May 29, 2020 11:33 pm

Download traffic is not showing on Interface!

Tue May 10, 2022 4:23 pm

Hey,

I'm passing a few VMs(Windows) traffic through a Mikrotik, on interface, the upload traffic is showing correctly when I upload something through VMs, but when I download, it does not show the actual traffic.
For example, I'm downloading with 400Mbps on VM, but it shows like 10Mbps on Mikrotik Interface!

Any idea what is happening there?
 
jay22
just joined
Topic Author
Posts: 15
Joined: Fri May 29, 2020 11:33 pm

Re: Download traffic is not showing on Interface!

Tue May 10, 2022 4:30 pm

Update:::

I have attached the results.
Also, I wonder Why Rx and Tx increase to the same range when I just download or upload at the same time!!! both show same amount!
You do not have the required permissions to view the files attached to this post.
 
kraal
Member Candidate
Member Candidate
Posts: 142
Joined: Tue Jan 19, 2021 10:24 pm

Re: Download traffic is not showing on Interface!

Tue May 10, 2022 5:04 pm

From what I see on your screenshot you only have a single interface (and the filter button does not seem to be enabled). Is that correct ? If that's the case and if you're using your device as a router which ony has a single port connected (and does intervlan routing for instance) this may explain the situation: traffic comes in from ether1 on vlanA is routed to vlanB and goes out through ether1 as well. As a result you would observe then 2x ~718Mbps on a full duplex link.

As for the 2nd result (download) we could guess various reasons. But the best would be to have some more information about your network topology and the config file of your device (is it a device or a VM with RouterOS ?).
 
jay22
just joined
Topic Author
Posts: 15
Joined: Fri May 29, 2020 11:33 pm

Re: Download traffic is not showing on Interface!

Tue May 10, 2022 6:29 pm

Thank you for your attention.

Yes, there is only one interface. this is a RouterOS CHR VM that is running on an ESXi.
I have like 50 VMs(Windows and Linux) and want to pass all these VMs traffic through this VM( Mikrotik CHR) as an internal gateway and sort of a firewall!

The purpose was/is to use this VM to limit the speed through the queue, but the limit only worked for upload speed, and download did not!
I did not know that download traffic is not showing on the interface, so I thought something was wrong with the queue. (so I created a topic for that: viewtopic.php?t=185771) check this topic for more info of network topology, please.

Now, I see, that download traffic is not showing on the interface at all! so I guess the problem is here.

I appreciate your ideas.
 
jay22
just joined
Topic Author
Posts: 15
Joined: Fri May 29, 2020 11:33 pm

Re: Download traffic is not showing on Interface!

Thu May 12, 2022 2:32 am

Any Ideas?
($30 gift as crypto for who gives the solution)
I need this really bad.


PS: I'm not sure If I'm allowed to motivate to get a solution with $$ ! if not, admins let me know and remove this reply.
 
User avatar
Jotne
Forum Guru
Forum Guru
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: Download traffic is not showing on Interface!

Thu May 12, 2022 10:21 am

Can you not setup a Router in VM with at least 2 interface. It will then be simpler to separate download/upload and simpler to make queues to limit the bandwidth etc.
 
jay22
just joined
Topic Author
Posts: 15
Joined: Fri May 29, 2020 11:33 pm

Re: Download traffic is not showing on Interface!

Thu May 12, 2022 4:24 pm

Hi,

Yes, I can add another interface to VM, but what will be the solution? how should I configure the second interface?

Here is a diagram of the network that I have:(attached)
You do not have the required permissions to view the files attached to this post.
 
kraal
Member Candidate
Member Candidate
Posts: 142
Joined: Tue Jan 19, 2021 10:24 pm

Re: Download traffic is not showing on Interface!

Thu May 12, 2022 5:11 pm

Are you running the speedtest from one of the Windows or Linux VMs hosted on the ESXi ?

Are you sure that all the traffic goes through this CHR (i.e is not load balanced in any way by the ESXi between a) the CHR and b) directly to the main gateway of the server ) ? (this would explain the results you see as the CHR would limit throughput, but loadbalanced traffic to server gateway instead of CHR would allow full throughput.)

As you have 7 links going from VMs and to the main gateway of server, and only a single interface on the router I guess that there is some vswitching done on ESXi (or virtual bridging, but as all devices seem to be on the same subnet...) But as guessing does not bring anything good, what is the configuration of the CHR (it would help others help you) ?

And I would also say that having at least 2 interfaces on the CHR would make things easier to read and understand (1 for all VMs connected to the CHR: the VMs could be on a virtual switch, 1 outgoing from the CHR to the main gateway to represent connection to the outside of the ESXi)

(but again I do not know much about ESXi and CHR, I just try to analyse the symptoms you encouter, so maybe I should keep it shut instead of maybe putting you on a wrong track)
 
jay22
just joined
Topic Author
Posts: 15
Joined: Fri May 29, 2020 11:33 pm

Re: Download traffic is not showing on Interface!

Thu May 12, 2022 6:43 pm

I apprciate any help Kraal.

Yes, test speed is running on one of the Windows VMs in the same subnet. The gateway for this Windows VM(I use static public IP for each VM) is set as CHR's main IP, so I'm sure all traffic is/should pass from CHR.
Anyway, as you see, the upload is showing and can be limited without any issue.

There is no weird configuration on ESXi, it is just ALL defaults(No bridges, no switching). as well as CHR(v6.49), which I just set up IP on the interface, with no further configurations.

The idea for 2 interfaces, what will be IP configuration for the second interface? How should I pass traffic between these two to connect VMs to main gateway(internet)
If you are suggesting a LAN configuration, it won't work for us.


Thanks
 
akakua
newbie
Posts: 49
Joined: Mon Apr 06, 2020 4:52 pm

Re: Download traffic is not showing on Interface!

Thu May 12, 2022 8:19 pm

What netmasks is used for ip interfaces of mikrotik chr and main gateway of server?
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Download traffic is not showing on Interface!

Thu May 12, 2022 8:26 pm

Nothing to suggest without a configuration export.

What I assume is that you misinterpret what the upload and download means in the vernacular of simple queues - since you use the CHR to hairpin the traffic, you cannot use the interface name as target. Maybe you'll even have to use queue tree rather than queue simple.

But first let's try to modify the current configuration once you post it.
 
akakua
newbie
Posts: 49
Joined: Mon Apr 06, 2020 4:52 pm

Re: Download traffic is not showing on Interface!

Thu May 12, 2022 8:56 pm

I think all devices are on the same network. when the server sends data to its gateway on .3, then chr forwards it to its gateway .1 (this explains the rx tx rate in the upload test) the response is returned directly via .1 to the server (this explains the rx tx rate in the download test)
 
jay22
just joined
Topic Author
Posts: 15
Joined: Fri May 29, 2020 11:33 pm

Re: Download traffic is not showing on Interface!

Thu May 12, 2022 8:58 pm

Here are CHR configurations:
#
/ip neighbor discovery
set ether1 discover=no
/tool user-manager customer
set admin access=\
    own-routers,own-users,own-profiles,own-limits,config-payment-gw
/ip address
add address=56.56.56.3/27 comment="added by setup" interface=ether1 \
    network=56.56.56.0
/ip route
add comment="added by setup" distance=1 gateway=56.56.56.1
/system lcd
set contrast=0 enabled=no port=parallel type=24x4
/system lcd page
set time disabled=yes display-time=5s
set resources disabled=yes display-time=5s
set uptime disabled=yes display-time=5s
set packets disabled=yes display-time=5s
set bits disabled=yes display-time=5s
set version disabled=yes display-time=5s
set identity disabled=yes display-time=5s
set ether1 disabled=yes display-time=5s
/tool user-manager database
set db-path=user-manager

And here is the Windows VM network details:
PS C:\Users\Administrator> ipconfig
Ethernet adapter Ethernet0:

   Connection-specific DNS Suffix  . : 
   IPv4 Address. . . . . . . . . . . : 56.56.56.4
   Subnet Mask . . . . . . . . . . . : 255.255.255.224
   Default Gateway . . . . . . . . . : 56.56.56.3

Tunnel adapter isatap.{DE903337-A67E-4D58-9112-65D087D5A7CD}:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . : 

Tunnel adapter 6TO4 Adapter:

   Connection-specific DNS Suffix  . : 
   IPv6 Address. . . . . . . . . . . : 2002:9246:6a56::9246:6a56
   Default Gateway . . . . . . . . . : 2002:c058:6301::c058:6301
                                       2002:c058:6301::1
 
jay22
just joined
Topic Author
Posts: 15
Joined: Fri May 29, 2020 11:33 pm

Re: Download traffic is not showing on Interface!

Thu May 12, 2022 9:29 pm

Yes akakua, that makes sence!
I saw some other ideas that incoming traffic(download) can be routed/passed through a different gateway than the original one!

But still, I'm not sure about this, just looking for a solution.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Download traffic is not showing on Interface!

Thu May 12, 2022 9:48 pm

I was expecting to see the queue configuration in the export, but never mind, the issue seems to be clear, but please confirm my understanding.

There is a gateway in the subnet (.1); you expect that if you put the CHR to the same subnet with an address .3, and tell the PC (address .4) that its gateway is the .3 instead of the .1, all traffic from the PC will flow to .3 which will forward it to .1, and the .1 will send all the responses to .3 which will forward them to .4?

If so, this will not work unless you configure both the PC(s) and the actual gateway in a very special way. Normally, if a router finds out that when forwarding a packet, the IP address of the gateway is in the same subnet like the source address of the packet, it forwards the packet but it also informs the sender that there is a better gateway in the subnet; the sender normally caches this information and sends the subsequent packets to the same destination via the new gateway. You have to disable this either by setting send-redirects under /ip settings to no on the CHR, or by configuring all the other hosts in that subnet to ignore the redirects.

For the responses for .4 coming from the outside, there is no way to convince the real gateway (.1) to send them to .3 rather than directly to .4, because no gateway is necessary to send a packet to an address in your own subnet - the .1 sends an ARP request for .4, and the .4 will respond with its own MAC address.

The clean way would be to use another subnet to connect the CHR with the actual gateway - you can do that even without creating a separate port group for that subnet, but this requires that you can configure the .1 to use the address of the .3 in that other subnet as a gateway to the original subnet.

A dirty way would be to use src-nat at the CHR, making the .1 see the traffic from anywhere (e.g. .4) as coming from the .3 itself. If there is src-nat somewhere further in the network anyway, it is not a big deal; if the addresses of the hosts in the subnet are public ones and you want the remote servers to see the actual addresses of the clients, you cannot do this.

So the proper solution depends on multiple factors as you can see. Specify these factors to get a more specific advice.
 
User avatar
chechito
Forum Guru
Forum Guru
Posts: 2989
Joined: Sun Aug 24, 2014 3:14 am
Location: Bogota Colombia
Contact:

Re: Download traffic is not showing on Interface!

Thu May 12, 2022 10:01 pm

i think maybe you have some mistake on virtual network configuration or design on hypervisor and you are bypassing your traffic (not passign accross MikroTik CHR)

also on CHR you have only one subnet so no routing to do by CHR

traffic is not showing because is not passing by the CHR
 
jay22
just joined
Topic Author
Posts: 15
Joined: Fri May 29, 2020 11:33 pm

Re: Download traffic is not showing on Interface!

Thu May 12, 2022 11:53 pm

Hi, thanks for your attention.
I was expecting to see the queue configuration in the export, but never mind, the issue seems to be clear, but please confirm my understanding.

There is a gateway in the subnet (.1); you expect that if you put the CHR to the same subnet with an address .3, and tell the PC (address .4) that its gateway is the .3 instead of the .1, all traffic from the PC will flow to .3 which will forward it to .1, and the .1 will send all the responses to .3 which will forward them to .4?
Yes, the purpose will be Queue to limit the download and upload speed, but currently, as you see, the traffic is not passing through my CHR yet!
The scenario that you explained is exactly what I did, I thought it work!
If so, this will not work unless you configure both the PC(s) and the actual gateway in a very special way. Normally, if a router finds out that when forwarding a packet, the IP address of the gateway is in the same subnet like the source address of the packet, it forwards the packet but it also informs the sender that there is a better gateway in the subnet; the sender normally caches this information and sends the subsequent packets to the same destination via the new gateway. You have to disable this either by setting send-redirects under /ip settings to no on the CHR, or by configuring all the other hosts in that subnet to ignore the redirects.
I have disabled the "send-redirects" on CHR, but same results! we may have like 500 VMs that are using by customers, so we can not make changes on those, one by one.
For the responses for .4 coming from the outside, there is no way to convince the real gateway (.1) to send them to .3 rather than directly to .4, because no gateway is necessary to send a packet to an address in your own subnet - the .1 sends an ARP request for .4, and the .4 will respond with its own MAC address.

The clean way would be to use another subnet to connect the CHR with the actual gateway - you can do that even without creating a separate port group for that subnet, but this requires that you can configure the .1 to use the address of the .3 in that other subnet as a gateway to the original subnet.
This solution makes sense, even though we do not have direct access to (.1) since it is under datacenter/provider control. we may be able to contact them to configure it for us.
So, please explain this solution in details:
we should ask the provider to add a small IP subnet (/29 or /30) to this server. and what configuration should we do on CHR(.3) & on the main gateway(.1)?
Do we need to change the gateway address for each VMs(.4) in the network?
------------------------------------

Also, I wonder, is there any other way like using MAC address or etc ? or Am I doing the worst solution(CHR in same subnet) to see BW and limit speed? what do other people do for the same scenario in first place?

Btw, The dirty way won't work for us since the remote client should have their own IP and see the actual/public IP totally.
 
akakua
newbie
Posts: 49
Joined: Mon Apr 06, 2020 4:52 pm

Re: Download traffic is not showing on Interface!

Fri May 13, 2022 9:22 am

You can ask them to give you control of the entire 56.56.56.0/27 network and create second /29 network between ESXi and their router. You must set second IP on interface ether1 of CHR from the /29 network and set a default route via IP address from the /29 network of their router. They must set the route to 56.56.56.0/27 via /29 IP address of the CHR, on their router. This is the simplest and "dirty" solution for you.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Download traffic is not showing on Interface!

Fri May 13, 2022 9:22 pm

In fact, if the ESXi is totally under your control, you can make use of proxy-arp functionality.

If you create another interface (ether2) at the CHR, and assign and IP address to it the following way:
/ip address add interface=ether2 address=56.56.56.30/32 network=56.56.56.1
and set
/interface ethernet set ether2 arp=proxy-arp
the CHR will respond with MAC address of ether2 to ARP requests coming from the gateway to any address from 56.56.56.0/27, so the gateway will send IP packets for all these addresses to it. And the CHR will route them further via ether1.

But there is a catch - if you don't have any other access to the ESXi, and something goes wrong, you'll lose access to the ESXi management once you move the physical NIC to the new vswitch. And no "safe mode", reverting the configuration in case the management server is lost, is available on ESXi.

If you have physical access to the ESXi, and can assign another management address and link it via a separate vswitch to another physical ethernet, the above requires zero changes to the gateway configuration.
 
jay22
just joined
Topic Author
Posts: 15
Joined: Fri May 29, 2020 11:33 pm

Re: Download traffic is not showing on Interface!

Thu May 19, 2022 6:17 pm

Hello guys,

First of all, thank you all for help, as I told, there is a little gift(30$) for the solution, And we are sure Sindy helped more.
Sindy, Give us a cryptocurrency wallet to send gift. (any coin you wish)

Also, we need more help on this and need a private consultant on skype/telegram/email for implementing this. this will be a paid conversation at least 200-300$ for your help. I'm sure your time is more valuable than this, but we can discuss it privately later.
How can we contact you?( I have attached a PGP in case you don't want to share your contact info in public)

-------------------------------------------------
We have discussed this issue with data center tech team and they gave us the below solution after checking this topic.
This can be done, but it will generate a short downtime for your services.
We can allocate a new /29, you will configure this new IP on your server, and when you are ready we will shut the gateways on our side, and will route everything directly to your server.
After that, you have to create the gateway and configure the IPs as you wish on your side.
To summarize, yes we can do what was suggested in that thread (Sindy) with the gateway on your side for all the current IPs, and only a /29 for connectivity between your server and our switch.
and gave us a topology that how this can be done, and the first step is making vswitches.(attached).
The only change is moving the gateway from our side to yours.
We think you should prepare your vswitches first and we can allocate a /30 to configure on your side.
After this, we can statically route your IPs towards the IP from the /30.
I will share the results after Sindy comes on board to help and implement.
You do not have the required permissions to view the files attached to this post.
 
sindy
Forum Guru
Forum Guru
Posts: 10205
Joined: Mon Dec 04, 2017 9:19 pm

Re: Download traffic is not showing on Interface!

Fri May 20, 2022 8:47 am

I don't want to install PGP for this single case - can we use just openssl instead? My public key for openssl is

public key code

-----BEGIN PUBLIC KEY-----
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEArauOfFEn3Grb2WhnlKe1
hEG9bwq+7fJiDhFyCK+CcmXwQyGWl4LIop5VCTNUYq/++PSFOAmkoLA1+TMTAN6s
f7ukyVErCxpfUMy+xhutVzsVDHIaAScJFr3QUPmI+544EMplZn70DOWIWgv4Xtac
jeiuqJZ2mExt/8amZTQ2dJ3MHmRaD20gUI+ByJwkQopxUZTOfKAXkhkL/HTcksyO
ORSh+EzFN+ll674084RpbXGKnaDYysAObLSib9UgnHnQ98WLbh+4XZqDc6T82mFW
0BcxLI8pUeITwBaexX8YbDYOFnxskLIlGcRb0F9sRGV8as4H5+zuwlO14K1LDb4Y
suNttQSKYFOZ91G2SOBlGiejO2xf7t3spdv0LurOkvvKJo+wOwAhfz/8zBubUBqd
0jXQyzkQ+WkBMzKBYwOl39YQXUHYHxxGUXMupslSyw1qcyW0QeC91hfM+/GNk+j/
+w+lqLV0So2gc8vD/9mVAcMv9g8eSffvwEKygnXsqlg4QLvjCVNu1TMtYH9NJAgs
2VoBgDaAZrc5id9dk41Zh19L4mYJ+uTiafZ/g/5jx5AEjPdVcsKBsatQVld+RuS5
B+kLTXbPuk8kXRLhKuSOEEBGOvXaCvwqNPdvghNOQSvydVOkVyS6/SL5I0UWOg8s
2B5HsHfn4Ph9gjxDMmiUl1UCAwEAAQ==
-----END PUBLIC KEY-----
The commandline to encrypt a file jay22.txt with your Telegram name and/or e-mail address and print the result out base64-encoded for posting here is

openssl rsautl -in jay22.txt -out jay22.enc -pubin -inkey sindy.pub -encrypt
openssl base64 -e -in jay22.enc
 
jay22
just joined
Topic Author
Posts: 15
Joined: Fri May 29, 2020 11:33 pm

Re: Download traffic is not showing on Interface!

Fri May 20, 2022 3:03 pm

Sure, here you are:

QmInTc0Gydkx8e06d73cuUHICT19pikZVr3/hSEU7nJ1TEawd+BI15lJ2iq1CICf
lGA64caJV097T5YFhMA9gxpGlCWPwjANJz9IDxIDP6PenrIYo/RDGC9O3P0WbOGv
IT6Zs9ok3YejjnKY4xxlPEY/GudmwYFpAYRhpYh1KMko1RCqVrMVelo6cZA5mLCP
3fADEZGH35QSU0mcegpd07tSbpQRnweOfwhaPKr+lLPbKQgYpwr1r9XBYALu7uf+
Gp6FurBcOYSB6KcUYgq/h/+0ID89Z24cYuA9wRJZz8i2/Wt4ywmx/DKHcLbviBvS
/+Kk4MyJRKzn1VBoPL3CTxC22ia1WzHTdF6xWzMDS4l/jL4NBHTiaN4Flijh4dKX
h2Cihqt9piAq91Kq9Nr9CTeB2f1uhd+Hk4fJyf6ZTgtb4zbWf/8cd60IgocIQ2Hx
1eAf3veqe+9oalqnxLl/JErAyvIu8mlsUbC1cfQs5N+z+pIegXnkxx+UQpi0Tq/R
be17M8WqdRqJquHFAj87DCN8ZUdwt5HeTiD1UZUSRrbAk+FoauxCnqCZQBwWQQNk
6ys2/EprEfDByR98C+1pES6IDJYOWBG1cpS2+O2ncXvYObDaWpEMKRNxIQjhUGFr
DPLP1yMle3zG0ay0oZro+FN7LVsvdSG6ZjqoMBxYpaQ=

Who is online

Users browsing this forum: baragoon, fposavec, keithy, menyarito and 91 guests