No, no difference.
You're pretty much right but it usually means that the person couldn't wait for it to happen, and has wanted it to happen for a long time.
It works also when talking about someone else. "For John, the start of the football season can't/couldn't come soon enough." Can't/couldn't depending on what tense you're talking in as the narrator.