I bought this book, it is mine, it belongs to me, I own it.
"belong with" means "should be with". You cannot own another human being. So "you belong with me" means "you should be with me". You can say this with a loved one. Also, if their are kids on a school trip and they are in groups, the teacher could say, "Kevin, what are you doing here? You don't belong with me, you belong with Mr. Smith."
I love Sylvia's comment, by the way, it works perfectly and is a very good example.