Grammatically, it is correct to say "he is taller than I". This is because you are comparing "he is tall" to "I am tall" so "he is taller than I am tall" is what you're meaning, but when spoken, you leave off the "am tall" from the "I". This is the same construction as saying "she has more money than I." You can remember it by thinking it in two shorter sentences "she has money" and "I have money" (not "me has money").
So if you're taking a test, this is the way to figure it out.
But now that you know that, I must admit that in spoken English, you will often hear "he is taller than me," even with native speakers. This is grammatically incorrect, but is often used when the speaker wants to be intentionally extra casual, or else they just don't know the correct grammar. So you will definitely be understood by native speakers if you say "than me" although technically it is not correct.
Panda's comments above about "further" and "farther" are a good explanation.