I came home from downtown, and passed the place where I work. passing through my work

I came home from downtown, passing by  through my company premises/office (or "my work". Which is natural? I learnt that "work" can also mean the place where I work). I didn't stop and just passed by.

When I go downtown, I have to pass my company, which is on the way downtown. I am not sure "pass through " used in the above sentence is natural or accurate. I believe there is a better phrase for native speakers.