No, the words "Of course" are not rude in themselves. In fact, there are many situations where 'Of course' is a polite answer. Here are some examples:

"It's hot in here. Can I open the window?"      "Of course. Please do."

"May I borrow one of your pencils?"                "Of course. Help yourself."

However, there are other situations where "Of course" does sound rude. Here are some examples:

"Do you visit your family during the holidays?"      "Of course."

"Do you understand English TV?"                            "Of course".

These very abrupt answers will surprise and even offend a native speaker listener, because they imply that you are thinking "What a stupid question! Isn't it obvious that I visit my family?' Or "Of course" could even suggest that you are offended by the question: "What a rude question! Of course I understand English!" 

This is what your teacher meant by saying that 'Of course' was rude. Not the words themselves - but the implication of giving them as an answer.

Jack is right, of course. But let me tell you of another tricky situation. Some English learners from certain backgrounds use "of course" way too often; I would call it incorrect word selection as a native US English speaker. For example, they might use it instead of really common phrases like "well", "you see" etc. 

Of course I'm not sure about that sir. = Well I'm not sure about that sir.

Of course, it goes something like this.... = You see, it goes something like this....

If your teacher always corrects you and tells you to say "sure", they must be teaching you American English.

In British English, we say "of course". 

In American English, they say "sure".

If you would prefer to learn to speak like a British person, then you should say "of course".

However, if you wish to speak American English, you should say "sure".

You can make them sound rude, or you can make them sound polite.

Su.Ki. has, OF COURSE, answered your question with her usual thoroughness, so I don't need to say more!