To piggyback, in this case, is to use an existing thing (in your case, an answer) as support.
I wouldn't call it a foul if you said 'I am piggybacking on my previous answer,' but I will say it sounds a little odd (because I don't hear it often.)
I would say something around the lines of, "Referring to my previous answer, there is extra info I would like to add."