It is a somewhat archaic saying to do with handshaking etiquette. It means "forgive me/pardon me for not removing my glove," as it is/was expected for one to remove his or her glove when shaking hands with someone who is not also wearing gloves. The phrase is not very likely to come up today.