Grammatically, it is an adverbial phrase that modifies the adjective "glad" to explain *WHY* you are glad. You are glad because you are done.
Infinitives can act as adverbs, nouns, or adjectives. Here are examples:
"To be done with my project is something I am glad about." (noun)
"That is the project to be done by us." (adjective. It means we need to do that project.)