AP probably shouldn't reboot when some client connects to it. It is possible though and I can think of a few reasons for it.
One is weak power supply adapter. When client connects, AP will start to transmit at higher (full) power because of improper antenna (with too low gain). Thus it's causing higher power draw from the power adapter. If that one can not supply demanded current, voltage will drop causing AP to restart.
But this might happen even if you use proper antenna ... under full wireless utilization.
Another reason might be some peculiarity of WiFi subsystem - when improper antenna is connected (too high VSWR), WiFi power amplifier might enter into some trouble triggering SoC to bail out.