'Listen to' is most generally used. 'Someone is listening at the door', means someone is outside the room trying to hear what is said by having their ear next to the door.

Hello, I am a native English speaker and we do no use "listen at". I'd like to make a small correction (very common for non-native speakers) with your final question. I should read either "Why can't we use 'listen to' in this sentence?" or "Why can we not use 'listen to' in this sentence?" I might not be the best person to explain this correction, however wanted to point that out to help you sound more fluent. You can use "listen to" in that sentence and in fact it sounds better than "listen at". Without knowing more context of the question that is my best response.

hi! hopefully a grammar expert will jump in here and explain this better, but i will also give it a go :) i think this is to do with certain verbs having certain prepositions that go with them, to the exclusion of other prepositions. It's something that probably isn't very logical and just has to be remembered. With practice you will probably notice that "listening at something" sounds strange. You could search the internet for "English verbs with prepositions" to find some lists of these.