'Very much' and 'a lot' have a similar meaning, but the usage is different.
'A lot' is used more for things you can count (e.g. A lot of people), or to indicate that an action is carried out to a high degree of frequency (e.g. he shouts a lot). 'Very much' is more used to intensify the degree of a single action.
So 'thank you a lot' does not sound grammatically correct to me, as it's a single act of thanking, so adding 'a lot' does not work.
'Thank you very much' is perfectly OK as 'very much' is used to indicate that your act of thanking is meant to a high degree.
'Thanks a lot' is also perfectly OK as 'thanks' is now a countable noun, rather than a single action verb.
Hope that helps,
Tim