Yes, your guess is right. It can mean "It is okay/all right. Don't worry"
It can be confusing. 'Cool' has many different meanings.
If you are confused by a meaning, check with an English/English dictionary, to see what other meanings a word may have.
Here are all meanings for cool (The one in your sentence is #6):
1 fairly cold; not hot or warm
2 making you feel pleasantly cool
a room painted in cool greens and blues
3 calm; not excited, angry or emotional
4 not friendly, interested or enthusiastic
5 (informal) used to show that you admire or approve of something because it is fashionable, attractive and often different
6 (informal) people say Cool! or That's cool to show that they approve of something or agree to a suggestion
7 (informal) calm and confident in a way that lacks respect for other people, but makes people admire you as well as disapprove
8 [only before noun] (informal) used about a sum of money to emphasize how large it is
IDIOMS
(as) cool as a cucumber
very calm and controlled, especially in a difficult situation
play it cool (informal) to deal with a situation in a calm way and not show what you are really feeling