1. Steadfast means the same thing as "unyielding." In the sentence you gave, it just means that the man stared at you without looking away, almost like there was a kind of determination behind it.
2. As a native speaker, "steadfastly" is the kind of word that sounds clunky, and I will never use it in conversation. I'm sure there might be situations in writing where I might decide it's the best word to use, but I don't think I ever have. Its root word, "steadfast," is fairly common, and I use it every now and then, but it is very uncommon in my experience to see it turned into an adverb like this.
3. Synonyms for "steadfastly" include: solidly, securely, firmly, deeply, soundly.
4. I think you should add "steadfast" to your vocabulary. It's not very common in conversation, but it does occasionally come up. And it's common enough in literature to justify being familiar with it.