The 'to' form is the more common of the two, but you can also use 'with'
You can say:
Speak with you
Talk with you
Speak to you
Talk to you
All of these are correct. In some situations, the 'to' form suggests a one-way communication and the 'with' form a two-way communication, but that isn't always the case.