In the movie example, your friend doesn't feel strongly about the movie. He might not even be sure if he liked it or disliked it, so he picks someplace in the middle (it was okay) and adds "I guess" to express extra uncertainty. Further examples of this use would be: [regarding a person I've just met, who was nice to me but made me feel strange] "Well, he's nice, I guess." [regarding something which bothers me a little bit, but I can easily tolerate] "I guess I don't mind." [if someone asks me to help them, and I don't really want to but I'm going to do it, but I want them to know that I don't want to do it] "I guess I can help you." In the work example, it's kind of an acknowledgement that the situation is awkward. (I'm not sure how to explain this, but I'm going to try.) Maybe your coworker doesn't know you well, or maybe he doesn't want to seem like he's telling you what to do, or maybe he's shy, or maybe he doesn't have any ideas and wants you to take charge. So he's using "I guess" to sort of soften his statement. Further examples would be: [meeting a new roommate for the first time] "So I guess we're going to be living together." [my friend Anna brought her new boyfriend to my house, then went to the bathroom and left us together, and I have nothing to say to him] "So I guess you and Anna are a thing, huh?"

'I guess' means 'I suppose'. We use it to indicate a lack of certainty.

Yes, it is a habit of speaking. It is a "disclaimer". It means that I suggest something, but not with complete conviction. It also invites disagreement. If I write to you, and conclude with the phrase "I guess" it means that I am open to disagreement from you. You may want to say; "No, that's not right." Regarding the movie dialogue, if the person says "It's okay, I guess." It means that the person is "half-hearted" in the answer given. It was not genuinely liked as a movie. The person mostly "tolerated" the movie, without being very excited about it. .