In general, use "for" if you are to receive the note. The note is left for you.
The only way "to" can work is if you are to write the note: "You make the phone call, and leave the note to me" (ie. leave the writing of the note up to me; that'll be my job).