Yes, it is correct. The pronouns on both sides of the verb “to be” should be in the same case. That is the technical, traditional, and correct rule.
But most English speakers (at least, American English speakers) now say “He is taller than me,” even though this is not the traditionally correct way to say it. If you say “He is taller than I,” many American English speakers will think that you made a mistake. Or they will think that you are trying to sound fancy.
Curiously, if you add “am”. — “He is taller than I am” — then it sounds correct and natural to everyone.