As with love, like, dislike and a few others, the verb prefer can be followed by either the gerund or the infinitive. In most cases, the meaning is exactly the same. Sometimes, the gerund may subtly suggest that the person actually engages in the said activity*, whereas the infinitive may suggest something more hypothetical. Along these lines, in the conditional mood, these verbs are almost always followed by the infinitive, not the gerund. Bottom line: When in doubt, you’re safest if you always follow these verbs with the infinitive.
*Note: I believe that since the gerund and present participle are identical in form, native speakers may, rightly or wrongly, associate the gerund with an action in progress, thus explaining the slight perceived difference in meaning.