I do consider this incorrect, because 'a couple of cups of tea' can also mean more than 2 cups.
Grammatically it is not incorrect, but it does not really make sense to use this sentence. Definitely not if you want people to know that you mean 2 cups.
You can indeed say (Like Evelin said) "I drank a couple of cups of tea and went to work". But that still doesn't sound natural to me. "I drank some tea and went to work" is what I would say.
I hope this helps