The phrase is “stunned by.” “Stunned at” is incorrect.

You can also read "stunned at her beauty" as a direction like stunned towards her beauty. Kind of similar to に or で particle. By in "by her beauty" would be similar to because, "he was stunned because of her beauty". Like the を particle. This might be wrong since I'm not a teacher but I think it's a funny alternative meaning.

The second one is correct

"He is stunned by her beauty" is the correct one.

"He is stunned by her beauty" is how a native speaker would normally say that. However, if you said, "He is stunned at her beauty," you would still be understood and the meaning would not change. In short, "by" is the right word choice, but if you did it the other way it would have no ill effect on the meaning.