I would be more likely to say "you're pulling that off" or "you pulled that off"
The thing is "pulling something off" means that you are doing something that is hard to do. So when it is said about clothing it means basically "Most people would look very ugly or silly in that item of clothing. But you look nice when wearing it."
If someone asks you "am I pulling this shirt off?" What they're really asking is "I think that I might look really stupid wearing this shirt. I doubt it looks nice. Do you think it looks good or should I wear a different shirt?"
It is a compliment, but it has a lot of hidden meaning.