You have good ears! Actually, most native Japanese speaker doesn't care
whether it's pronounced or not. There is a general rule, however, if you care.
Vowels [u] and [ i ] are tend to become silent or very weak
1) between voiceless consonants
For example, "hyaku-sai(100 years old)" pronounced like [hyaksai]
2) after voiceless consonants and at the end of a word(or a group of words)
For example, "....desu." pronounced like [....des]
Voiceless consonants are [k], [t], [p], [ch], [ts], [s], [sh], and[h].
This is just a general tendency, so it doesn't mean you have to follow it.