It's not negative at all. It's a fairly neutral word, and if you hear someone calling their superior 'boss', this is likely to suggest that they have a friendly or even informal relationship.
The only negative association is the adjective 'bossy', used to refer to someone who has a pushy, domineering manner. But this is only because the person who is behaving in a bossy manner ISN'T your boss - they might be your sister or brother, wife or husband, for example. That's why it's negative.
What image does 'Boss' suggest? Bruce Springsteen, of course.