People use this phrase in two circumstance:
1. Physical: It refers to the dizzy feeling you have when you are about to pass out or you're drunk or you have a hangover. "I drank way too much. I was nauseated and my head was swimming."
But the emotional meaning seems more common to me.
2. Emotional: It refers to feeling of being overwhelmed by a lot of new information or emotionally excited in a way that makes it hard to concentrate. "It was my first day on the job, and they were barking orders at me. My head was swimming." Or "she was really pretty and charming. I couldn't think of anything clever to say. I couldn't focus. My head was swimming."