Yes you are right. You say it when you have to leave to go somewhere else.

It means you are asking a permission to be excused.

Basically it means: "Can I go?". May I = Can I, Could I. To be excused = Have a permission to go. It's very official, like used when you have dinner with your boyfriends snobby parents, and you want to go to the toilet.

May I be excused? May I leave this place, please?; May I leave to use the toilet? Nature calls. May I be excused? The student raised her hand and said, "Teacher, may I be excused?"