Technically it's a curse, but it's a pretty weak one. "Damn you" would mean 'to damn someone to hell'. In your example, "damn girl" is actually used as an exclamation, like "Damn girl, you's real fine!" I usually hear "damn girl" spoken as Black English (AAVE) but it's been appropriated by other English speakers as well. If your friends have a good sense of humor and don't mind swears, feel free to let your tongue loose!