As intransitive verbs, they mean almost the same thing: to struggle or compete. For example, "John and I contended with each other" means the same thing as "John and I contested with each other".
However, as transitive verbs they're opposites. Contend means to claim that something is true, while contest means to claim that something is false. For example:
I contend that apples are a kind of fruit = I assert it to be true that apples are fruit.
I contest that potatoes are a kind of fruit = I assert that potatoes are not a kind of fruit.
The transitive use is uncommon in informal speech. Also, keep in mind when you say the noun "contest", meaning "competition", you emphasize the first syllable. The verb form, which you asked about, always has the emphasis on the second syllable.