There is no real difference. Response and reply are synonyms in this context. I'd probably default to using "sorry for the late reply", particularly in emails.

They mean the same thing.

Hi, Angela! You could use the two interchangeably.