'By' in this sense means 'next to' or 'close to'. If someone lives by the desert, it would mean that they live close to the edge of a desert. I don't see a problem with this expression.

"by the desert" sounds weird. We'd usually say "in the desert".