First off, "gained the better of" is not the most common form of this expression. The more common modern expression is "get the better of". It means "to overwhelm or overcome", "to gain an advantage over", or "to defeat in a competition". If a feeling gets the better of you, you cannot stop yourself from doing something, despite knowing that what you are doing is wrong.
In this context, it means that his want to be "secretive temperament" overwhelmed him. It forced him to stop keeping journals and get rid of everything he had written.
Some examples:
"Her curiosity got the better of her and she opened the door and looked inside." Her curiosity forces her to open the door. It's implied that she shouldn't do this.
"He fought fiercely, but his opponent easily got the better of him." Despite fighting hard, his opponent easily defeats him.