It depends on the context. Someone could text message you at 6 o'clock asking, "What are you doing?" You could say, "I'm leaving work." That would just mean, the workday is over and it's time to go home.
For someone who wants to say they are quitting their job...
"I'm leaving the company."
"I'm leaving my job."