Usually the phrase is "he's got my balls in a vice."
It has the image of someone (a male) with his testicles caught in a vice and someone else turning the vice tighter.
It means to have someone in a situation where they must do what another person wants, with no choice.
I think you'll agree that if some poor male had his private parts in such a situation, he would readily agree to anything demanded by the person tightening the vice.
It is not outdated - it is, however, certainly rude and possibly offensive. I would not use it. Use the phrase "He's got me over a barrel" instead. It means the same thing and is not offensive.