Well,In order to make a sentence with "would rather" you have two options.
1.I would rather go
2.I would rather I went.(if "would rather" percedes a pronoun,the verb after the pronoun should be in subjunctive form which means "went" in this case instead of "go".
2."Would prefer" is followed by infinitive.
I would prefer( I'd prefer) to have tea in the morning.
however,there are better ways to express the same thing instead of using= "would prefer" such as I prefer to, I want to,I like to.