"Muster up the courage" is the most natural one by far

They are both grammatical; however, "muster up courage" or "muster the courage" are far more natural than "muster courage."