The most natural answer for me as a Frenchman would be the same as Alexis. Most common is the negative form of this expression "je n'en ai rien à faire" meaning "i really don't care", "it doesn't interest me". It doesn't mean then "i have something to do" which would be translated by "j'ai quelque chose à faire" (without "en").