Hi Mae,
Thanks for your question. I don't think that is a rude question at all. I would say that is a perfectly acceptable question, especially if you are in an English speaking country. The only time I could think of that it might be rude is if you ask the wrong person in a place like France!
My only suggestion, and maybe it is because I am half South African and we say this ALL the time haha, would be to say, 'I'm sorry, but do/can you speak English?' or 'Sorry to bother you, but do you speak English?'
I hope that helps (",)
Nick Teacher