They are both correct. If you say
'I'd like to invite you for dinner.'
I would understand this to mean this as an invitation to my home. It is a reduced form of 'I'd like to invite you [to my house] for dinner'. This is quite acceptable, and quite understandable.
If you say 'I'd like to invite you to dinner', this could mean either at home or at a restaurant.