Great question, and there are many answers depending on the situation:
Look at the shop window:
This is correct if you are looking at the shop window itself (the glass). Maybe you are looking at the glass to see how clean it is, or perhaps there is a sign on the window. If there is a sign printed on the window, like the name of the shop, then the sign is ON the window.
Look through the window:
This is true if you are looking at something on the OTHER side of the window. You are looking through the glass itself.
Look, a window:
This is true if you see a window and you are shocked or surprised, "oh look, a window!". Imagine you are in a situation where you don't expect to see a window. This isn't a phrase you are likely to hear a lot though.
Look out of the window:
This is true when you are inside a room, looking out through a window. "I was in my bedroom, looking out the window"
Look in the window:
This is said when you are outside a room and looking in through the window. "I was in town looking in shop windows"
"Help! there is a man looking in my bedroom window!"
English Hack:
Prepositions are difficult for non-native speakers. For this example, concentrate on the last two situations. These are the ones you are most likely to come across in everyday life. You will most often be outside looking in, or inside looking out. The phrases to concentrate on are "I'm looking out the window" or "I'm looking in the window" I can't remember the last time I told someone I was looking AT the window itself
Hope this helps!!
ask me if not.