I have a weird issue I cannot explain. I only picked it up because I was experimenting with queues to see if I could improve performance.
I have my bacbone vlan 172.17.0.0/24. This has all my servers backup etc. Backbone Router 172.17.0.1. I then have three segments. I have a router at each segment providing various vlans. There is not nat on these 3 routers.
segment 1 172.17.0.2
segment 2 172.17.0.3
segment 3 172.17.0.4
Lets say I am sitting behind router for segment 1 accessing a backup server on the backbone. If i download or upload a file to this server the traffic passes through the backbone router instead of being switched through the switch. If I connect directly to the backbone vlan switches my traffic does not pass through the backbone router.
Why is the traffic not switched directly instead of passing through backbone router when behind one of the segment routers.

Additional information if I upload to a device on backbone from one of the segments traffic does not hit backbone router. Only when I download from server. So traffic from backbone to segment passes through backbone router to segment router. I would have thought only the initial communication would have gone through backbone router then once communications were established coms would be just between the 2 interfaces.

On the router that you want the switching to happen on, do you have the ports in the switch group slaved to a master port? You may also want to try adding these ports to a bridge if you want those ports to be in the same L2 group (broadcast domain). Hope this helps...

Does your network look like this logically?

It appears that hosts on backbone are not receiving ICMP redirects as a result traffic goes directly from one segment to the host. but to return back to that host it passes through the backbone router.

this is a good illustration.
So if remote branch host wants to access a file from Host H. Traffic passes through R2 to Branch H. When traffic returns to remote branch host the traffic passes through R1 to R2 from HOST H.

The way it should work is Host H should send traffic to R1 then R1 sends ICMP redirect telling it that R2 is the one it needs to talk to to get o remote branch host. but this is not happening.

Hi,

relying on ICMP redirect is a bad pratice in my opinion.

You need to configure your devices to accept ICMP redirects.
For the mikrotik routers:
I suggest, you create some sort of transfer network segment and use OSPF in that net, so the correct routing information is passed to the relevant routers.

Regards,
Ape

Yes I am looking at that as an option moving servers into own segment to make traffic flow more direct. for now I am trying to see why redirect is not working as it should.

You could place a static route with DHCP, not sure if this is supported on the client OS or by MikroTik's server. You could also define a static route manually.

An alternative could be to run a routing protocol on Host H and have it form an adjacency with R1 and R2 which tells Host H which networks they are connected to.

Either solution would break the dependence on ICMP Redirects.

I installed a router and pass a vlan to the switch the servers were connected to. Then moved each server over to the new vlan. Took a couple of days but well worth it in the long run. ICMP redirects are unreliable so this was the best solution. I had though about this in the past but did not realize how much traffic was passing through my links to my backbone router that shouldn't have been.

It sounds like you've resolved the issue. I'm not sure I follow your latest post. It sounds like you added a router and have moved the servers onto a VLAN behind it with the router as the default gateway. The router is then aware of where to send the traffic appropriately, correct?

what I have done is moved away from the servers being on the backbone vlan and placed them behind there own router. this means as you say the router is aware of where to send traffic so that traffic that is destined to the internet is sent through backbone router otherwise all other traffic is sent directly to the router the vlan is behind. I use OSPF so it is a simple solution.