"Wee" is a Scottish word and it means small. When you say, "we partied into the wee hours of the morning", it means "we partied into the small hours of the morning". It depends on the context but 1-3am are "wee hours" of the morning. 4-6am are also wee hours. All it really means is that you were doing something till very late!!
5am would be more likely, however, if someone said: "I woke up in the wee hours of the morning." As I say, it's all context... however people aren't likely to wake up before 5am unless they have a flight etc.
As for your sentences:
1.She plans on entertaining him into the wee hours.
=She's gonna spend time with him (doing stuff) until very late.
"She's going to spend time entertaining him until it's very late". Correct ... of course don't use "gonna" in writing unless it's informal.
2. The dance went on into the wee hours
=The dance went on til late
"The dance went on till late". Yes.
3. Is it always plural? Do you every say "the wee hour"?
Generally you would always see this in plural. But it could be in the singular.
eg.
Mr. X: "We were dancing until the wee hours!"
Mr Y: "When did you finish?"
Mr. X: "3am!"
My. Y: "Wow! That is a wee hour!"
Bear in mind that "wee" is just a word for "small" in Scotland. Generally speaking, you'd see wee used in the singular a lot. For instance, "she's a wee lass" = "she's a small girl".
Or
"It's a lovely wee house" = "It's a lovely little house".
I hope this helps.
----
"Staying through the wee hours" <== This is right
"Staying in the wee hours" <== This can be right but would be less common. I will give you an example using the same preposition but a different verb.
Eg. Mr X: "When did she leave?"
Mr Y: "She left in the wee hours of the morning".
You could use the past continuous as well:
Mr Y: "She was leaving in the wee hours of the morning. It took her hours to finally pack her luggage."
You can also say "From/during the wee hours of the morning".