This is the easiest way to remember it, even if some people have a different idea:
'a few' means '3'. So, if you can have '3' of something, then you can have 'a few' of them. In grammar terms, 'a few' is applied to countable nouns ONLY. So you CANNOT have 'a few time' or 'a few water'.
'a little', 'a little bit', and 'a bit' refer to things you can't count, like water or time. If you CANNOT have '3' of something, you CAN have 'a little' of it.
So, here are some examples:
I have a few apples.
I have a little time.
I want a bit of water.
I want a few things from the store.
Just for the record, "a few" is often used to mean any number of things, and people have very different ideas of what "a few" means, although we all agree that it means "not many". I was always taught that 'few' means '3'.