This is a common misuse of the expression. In fact, it's so common, it's even found its way into the dictionaries. If you do something on someone's behalf, you are doing it in that person's place. For example, you might make a request on someone's behalf if they are unable to or lack the authority to do this for themselves. In other words, you are that person's representative. You are right to be confused by the expression 'Don't leave on my behalf' - it doesn't really make any sense. The correct expression should be 'Don't leave on my account', or more informally, 'Don't leave for my sake'.

it means 'please don't leave because I have arrived and you feel that you should'. Example you go to visit a friend and they are entertaining a friend of theirs you do not know. This friend then says "I will go now and leave you two alone:" you reply to this friend of your friend "Please don't leave on my behalf." (please do not stop visiting Mary /Jim/ whoever because I have arrived) this general sort of occurrence is how it is used but like su.ki has explained it actually means acting on someones behalf.

Please don't leave because of me. Please don't leave to make me feel better / happy. It is used if the person leaving thinks they are making a situation better by leaving (maybe to make more space or because the person didn't like the other person or is uncomfortable around them).