How to set different wireless channels on router used as extender?

That cannot be done. The repeater always has both input and output on the same channel.

You should run an ethernet cable from your RB911 to your main router or you should buy something like this instead:
https://mikrotik.com/product/pwr_line_ap

With two of those you can connect them via the powerline.

For best performance I recommend to connect both by Ethernet cable if somehow possible.
Or upgrade to dual band routers and use 2.4GHz only to connect both wirelessly and use the 5GHz for wifi sharing.

Out of curiosiry:


Why do you recommend this way over using 5GHz for PtP between routers and 2.4GHz for clients?

Both options are possible, but this is my point of view:
2.4GHz indoor PtP: will reach longer distance and through more obstacles, only one 2.4GHz channel occupied by the link

5GHz indoor PtP: better throughput, but on longer distance or more walls weak signal, two 2.4GHz channels occupied by APs for clients