This means that you disagree with something. You are saying that the speaker may have that opinion or that condition but you (the second person) do not, so it makes just one. The opposite is if you say that it makes two of you. In this case, you agree or have the same condition as the speaker. It is neither rude nor polite on its own. It varies on your tone when you say it. There are examples when you are simply saying that you differ in some way than the speaker. Polite example: Speaker- Today, I got home and my kids cleaned the house. You- That makes one of us. My house was extra dirty when I got home. But even this example can be said in a way that it is rude. I started to write a rude example, but each one sounded polite if the person changes their attitude and tone. Because it can be both ways, I would advise against using it in writing, and also careful when using it when speaking.

I suppose that it is the opposite of a colloquial expression "That makes two of us.", which means "I totally agree with you.". ／ one ⇔ two

It could be said in response to a statement of opinion. For example, "I can't wait to go on my next Princess cruise!" Other person responds quietly to a bystander, "That makes one of us..."

It's a sarcastic response. I don't use it because it sounds a bit offensive.

This sentence is only used in a sarcastic way. It follows the same structure as "That makes two of us," which is a way of agreeing on something, but not sarcastically.