It is unfortunate that people slur the pronunciation of these vowels so much.
They are hard to distinguish when spoken fast, but in slow speech or when they are by themselves, it shouldn't be that difficult to pronounce them distinctly.
Their difference is similar to the difference between the first sounds of "end", and (strong) "and" in English ([e] vs [æ]).
ㅔ corresponds to [e], andㅐto [æ], although the Korean and English vowel sounds are not exactly the same.
에 is pronounced with the mouth opened just a little, and the sound is made in the front.
애 requires the mouth to open wider, and the sound comes from further back in the mouth.
에 is one of the easiest sounds to make, while 애 takes a bit more effort to pronounce accurately.
This is one reason why people tend to say 에 for both. It's a case of pronunciation habits converging to what's easier.