They are both ok but I’d use “change” if I’m asking a stranger to give me his airplane seat, for example. But if I’m at the movies with my children and one can’t see the screen because there is someone tall in front of her, I’d say “Here, let’s switch seats”. Switching is fast and easy. It would be a little presumptuous to use that with a stranger, since it minimizes the request.
“I have a favor to ask, Sir. Would you mind if we changed seats so that I can sit next to my wife?”
“You don’t mind switching seats do you?” (Implies that it would be unreasonable to say no)