I can't give you a good explanation. I can only give you examples of (US) usage.
When I get off the train, it takes me less than five minutes to walk home.
When I get off the train, it takes me less than five minutes to walk to my house.
I'm leaving now, I should get home by six.
I'm leaving now, I should get to my home by six.
In the examples without "to," it feels to me as if perhaps "home" is working as a modifier of "walk" rather than as a noun, although it's curious that if follows "walk." We can and do use "home" as an adjective in phrases like "the home team" or "home cooking."