I've never heard that expression before. I'm gathering you have City A, which is located ON some hill somewhere. And someone is trying to give you a description of it in relation to City B. They might say, "Markstown--that place is just down the hill from Jamestown" (these are fictitious cities I made up). Though I've never personally heard someone use a hill as a point of reference, it is possible.