I use both words (as an American), but in different ways. There are people that I work with at my office, they are my co-workers. There are people that work in the same field as me, that do not work at my office, but I know them, and we have work-related things in common. I often have to work with, or attend conferences with them. These are my colleagues.
Lorraine is correct that most English speakers would understand you if you used either word, but their meaning is not exactly the same.