In most situations, I would say to use the construction, "What's the number for X?"
What's the number for the library?
What's the number for Domino's?
I don't see any reason to say 'phone number' because context will be clear enough. If I were to ask that question, that's how I would ask it. If you want to be more formal or polite, you can say, "Could you tell me the number for the library?"