That's a really interesting question, Julien, and there is no clear answer.
One thing worth noting, perhaps, is the fact that the English 'you' is actually the equivalent of the French 'vous'. In the past, we had a set of tutoiement pronouns (thou, thee, thy, thine), but these are no longer used. I remember the first time I went to France, a French person said to me 'Is it true that you vouvoie EVERYBODY in English? How formal!'
So there is nothing inherently disrespectful about 'you'. 'You' is a neutral English, and if you use standard polite language and standard correct grammar, then that is the equivalent of vouvoiement. You show respect by avoiding informal language, slang and colloquialisms.
Likewise, you add informality and signal tutoiement by using more colloquial phrasing and choice of language. As Phil says, if you say 'What's up?' it's a informal 'ça va?' to a friend, whereas 'How are you?' would be the equivalent of a formal 'Comment allez-vous?'.
As for the use of polite terms of address, we use 'Sir' and 'Madam' far less than you say 'Monsieur' and 'Madame' in French. I'm an older woman, but the only people who ever say 'Madam' to me are airline and hotel staff, simply because they have to do this as part of their job. In the UK, these terms of address are only ever used in service industries, never in daily life.