"That's a load off my chest" has to do with a worry or anxiety that has disappeared. "That's a load off my shoulders" is related to a burden or chore that you've been relieved of.

I would say a load off one's mind or a weight off one's mind I am feeling better after I get a load off your mind.