No, 'yet' isn't more formal. It's commonly used. 'yet' is often used together with 'but' in conversations and writing to avoid repetition of 'but'. But 'yet' has it's own purpose in conversation, too. It can have a subtle difference in meaning to 'but' (as a conjunction). Some will use it on occasion when they want to stress more the idea that follows 'yet'. Example. "I was sick, yet I still went to work" versus "I was sick, but I still went to work". Personally, I would choose the 'yet' version if I really wanted to make a point that I went to work despite being so sick. This is a subtle difference and not all would follow it.

It certainly is a little more formal - you are correct. However, we still use it in everyday speech - even in informal contexts on occasion.

"Yet" is more formal in certain contexts. It's not formal to use it at the ends of sentences: "I haven't found my car keys yet." When it's used in the middle of sentences instead of "but," it's slightly more formal.

Yet also often implies that the 2nd clause is a bit unexpected, given the first. ("He has failed the licensing exam 7 times, yet he has not given up his dream of becoming a doctor")