Thanks a lot to all of you for coming to visit me.
Thanks a lot, all of you, for coming to visit me.
I would use commas in the second sentence because (all of you) is a parenthetical element that can be removed without altering the essential meaning of the sentence.
I think both sentences are fine, when written correctly.