You would bring it up in conversation rather than use a negative descriptive adjective. “He’s not my type” “He’s not as handsome as her last boyfriend”.
In fact, even labeling in a positive way, for example “Who is that gorgeous guy she’s dating?”, comes across as rather shallow and judgmental.
In general, you don’t want to sound as if your opinion of someone’s attractiveness is important.