They are fairly synonymous, with slight changes in flavor.
If I peer at something, I look intently at it with curiosity. If I look at something, don't know what it is and want to find out, I peer at it.
'Gaze' has a positive tone. If a woman gazed at me (I wish) it would mean that she was interested in me in a positive way.
However, if a woman stares at me, her facial expression is somewhat negative and I might think that I'm wearing some piece of clothing wrong or spilled something disagreeable on my shirt.