The rule in the UK used to be that if you start a letter with Dear Sir/Madam, you end it with Yours Faithfully, and if you start with Dear Mr X you end it with Yours Sincerely. I believe that in the US Yours Truly was used instead of Yours Faithfully, but perhaps someone else can confirm that.
These rules are for quite formal business correspondence, but if you follow them you are unlikely to offend anyone.
If you are writing to a friend, or sending an email, there are other more informal ways to finish, such as "kind regards" or "best wishes".