Captivity: normally referring to animals, such as animals in a zoo. I would never use this when referring to people actually.
Detention: Refers to being bound in one place, could be an official sentence, "home detention", or even kids in school detention.
Custody: Being in 'custody' refers to being held/imprisoned by someone. You are "in their custody".