If you're talking about a person or animal, I would say that the distinguishing point is whether it's entirely submerged, especially the head.
Like if you were standing in a pool and the water was up to your neck, then you would be in the water. If you dunked your head under the water, then you'd be under water.
Sometimes food is labeled as 'in water.' In that case, you wouldn't refer to it as under water.
For general objects, 'under water' typically means entirely submerged. Like a pier is in the water, whereas if the pier was under water, then it would be entirely submerged.