"Would you like to have dinner with us" is a lot better than "would you mind having a dinner with us".

Actually, for US English "Do you mind" is more formal than just bluntly asking for whatever it is you want. However, as other people have mentioned, "Do you mind" is usually used when there's a probability that the person being asked won't want to do the thing they're being asked to do. So "do you mind having dinner with us..." would imply that the person really doesn't want to have dinner with you. for example: "I know you really wanted to go to that new restaurant with your friends - but it's snowing like crazy and they might not be able to drive there safely. Do you mind having dinner with us instead?"

Thank you guys ! Now it is clear to me ! :)

"Do you mind" is used 1.) to ask permission do so something or 2.) to request someone do something for you. It's not used to invite someone to do something, or to ask if someone wants to do something. When I hear "Do you mind having dinner with us tomorrow?", I imagine we already have plans to have dinner another day. For example: "Sorry, I forgot that I have to work next Monday night. Do you mind having dinner with us tomorrow instead?" So the best way to invite someone is to say, "Would you like to have dinner with us?" or "Do you want to have dinner with us?" Don't forget to say "to have dinner" and not just "to dinner." I hope that was helpful!

As a native English speaker, I’d say that “do you mind” tends to give a preconception that what someone is about to ask is something that the other will find inconvenient. Being asked to have dinner is usually a pleasant invitation, so I’d recommend “Would you like to have dinner with us” instead.