I speak from the point of view of a mathematician. Mathematicians use the words frequently and interchangeably. There is absolutely no difference. When they assign a problem (= question) as homework to a student, they refer to it as an "exercise".
The mathematician uses the word "problem" in a way that no other person does. For most people, "problems" are bad things that you want to avoid. The mathematician, by contrast, is constantly searching for interesting problems. The solution of such problems is the purpose of their lives.
A problem becomes "interesting" if you do not know its answer -- especially if NOBODY knows the answer -- and discovering its solution will prod you to gain valuable insights. Those are the problems you want to think about. You cannot look up the answer because nobody knows it. You cannot ask a machine for an answer. You can only think about it long and hard, talk to other mathematicians about it, and read what others have discovered about related questions. That process is called "research".
Unlike many other people, when mathematicians cannot figure out a solution to a problem, they admit defeat. That is a remarkable characteristic in a world where for so many people not knowing an answer is no impediment to offering one anyway.