It depends on the context, "lunch" refers to a meal eaten in the middle of the day, usually around the beginning of the afternoon between breakfast and dinner. Meanwhile, "dinner" refers to your last meal later on in the evening time, or closer to night. If you want to invite someone out, you can say "let's eat out" and it would be the most correct. You have to understand that "lunch" and "dinner" are both nouns and "eat" is a verb. So, saying "let's lunch/dinner out" wouldn't make sense, because a verb has to go in the middle of the sentence. Meanwhile if you are talking about eating out in the past tense, you can say "I went out for lunch" or "I went out for dinner" depending on what time of the day it is, because "went out" is the verb that is being used in the middle of the sentence.

"eat out" is very natural, when you do not eat at home. "dinner out" and "lunch out" are just more specific to the actual meal