If you have money burning a hole in your pocket, it means that you have just received money that you want very much to spend - usually on something impractical. It does not necessarily meant that someone is reckless with money all the time - just that the current situation tempts a person to spend the money they have.
Likewise, I would interpret a secret burning a hole in the pocket as a person having a secret that they want to tell very badly. It would not necessarily mean that the person can't keep a secret - just that he or she wants very much to tell current secret they are keeping right now.
I must say, though, that Brian is correct in that you would hear this in reference to money, not a secret.