In my personal experience I use often the expression in response to someone telling me about some difficulty or unfortunate experience they have had. for example:
friend: "This morning I dropped my phone into the toilet."
Me: "oh man, what a drag! that sucks."
Friend: "I missed my flight to Cancun."
Me: "What a drag!"