It is extremely formal. You'll find it in legal documents mostly. I use it in conversation, but only when I want to sound overly pretentious and stuffy. You wouldn't use it in normal conversation.

Just to give you a real-life example, I have an actual contract in front of me and here’s an extract: “… in such circumstances the Landlord may forthwith sell, lease, assign, or otherwise dispose of the Property …”

The word “forthwith” means immediately, without delay. You’re right about it being used principally in formal written English, such as in legal documents. For example, when renting a flat in the UK, you will be asked to sign an agreement with the landlord. As part of the contract, you will promise to compensate the landlord for any damage to the flat when you move out. You will see words like “… the Tenant will reimburse the Landlord forthwith …”

It's formal. I can give you an example: The agreement between us is terminated forthwith.