They mean the same thing. However, if the person you are speaking with was not there himself, you would not say "I wish I had been there too, because neither one of you were there. If he was just telling you what he read or heard it was like, you could say "Oh, I wish I had been there, don't you?"