"To" is correct, but "be respectful of others" is also sometimes heard .... I feel there's a subtle difference:
- "To" implies that one should actively try to demonstrate respect to others through his/her words and actions.
- "Of" implies a more neutral state - simply being conscious of other peoples' feelings, time, etc. It's more of a double negation of not showing disrespect to others than it is of directly showing respect (i.e., do NOT NOT be respectful to others vs. be respectful to others).
Small difference, but that's the feeling both phrases give, in my experience. It is probably safer to stick with "to". :)