In this case, "nii" means "兄" (older brother). Maybe it is nickname what you heard. For example, there is a man whoes name is Hideo and he had a younger brother. His younger brother call like "Hide-nii". Other case, a person whoes face look like older than same age or a person who is relyed by his freinds like real brother, is called "Aniki" or "○○○-nii" in the classroom.
You should not call "○○○-nii" to a person who looks like older than you and meet a few times. In this case, you should use "○○○○-san".
You can only call a person who is a close friend.
Can you understand what I want to say?