I can definitely see the confusion, since these would both translate to "(no) me importa" in Spanish. There is no difference in meaning between these two expressions, but there is a difference in structure. While you would say "I (don't) mind X", you must say that "X does(n't) matter to me". I hope that makes sense.