A Korean friend kindly showed me the difference...
어 is made quite close to the back of the mouth, like the o in Korean. It's usually transliterated as "eo", as in Seoul (서울).
오 is pushed forward to the lips, as in show.
여 and 요 just add a y sound, of course.