Your " soft spot" can mean something that you are particularly sensitive about or you have a weakness for. It's not as commonly used anymore but you could say "this person's soft spot is that he never went to college." That would imply he tends to feel inferior because of his limited education. You can also say " soft spot " to mean something you like. "I have a soft spot for ice cream." (I have a weakness for ice cream, I cannot resist it).

You hit me in my soft spot. The balls are a soft spot. If you hit someone there it hurts. Now extend that image to just about any soft spot, physical, emotional, etc. The most famous soft spot is Achilles Heel.

In an informal way 'soft spot' could also refer to a feeling that one have for a girl/boy, liking that person so much. You can't think of anything except for that special someone ,so it is a fondness and affection you have for a certain person. When you hit someone in their soft spot in that context, you are confronting them with that person,that they can hardly resist.