Sadly, there is no perfect rule.
Here are some general rules:
1. If an adjective has one syllable, we usually use -er: rich / richer.
2. Some two-syllable words also use -er: thrifty [careful with spending money] / thriftier.
3. If an adjective has three syllables, we usually use "more": beautiful / more beautiful.
4. Also some two-syllable words can take "more": candid [very honest]; more candid.
5. Finally, there are some adjectives that can take "-er" or "more": polite / politer; polite / more polite.
6. I suggest that you do two things:
a. Check a good dictionary.
b. Check the Web for lists of comparative adjectives.
7. Please remember: If a writer can use "-er" or "more," h/she will often make a choice based on sound. Some people think that "You are more handsome than I am" sounds smoother than "You are handsomer than I am." And, of course, some writers think that "handsomer" is better!
* I wish to credit a book called English Review Grammar by Walter Kay Smart.