When we don't modify the focus we aren't risking giving focus to a
container behind the current fullscreen one.
_con_move_to_con can with ignore_focus is called through the swap
command or through con_move_to_workspace for floating containers. This
change shouldn't break the expectations of the callers there.
Fixes issue #3259.
b5f887287a