SH_O_BO is correct about the formality of chat vs. talk. Typically chat implies a quick discussion about something not very serious. You could chat with family, friends, coworkers, even your boss. Chat also indicates there will be back and forth dialogue in the conversation. Another common term that conveys the same tone as chat is the word visit. "Come by and we'll visit about that." means a pretty unimportant topic is being discussed.
When someone says they need to talk to you or speak to you, there is a more formal tone to the conversation. Talking with someone can be casual, but tends to be more formal and business oriented.
If someone says they need to speak to you, the topic is usually urgent and serious. Also, talk and speak are both used to convey a one way conversation, such as a lecture. Example: The professor spoke about the condition of trade relations between the two countries. No one would ever use chat in this scenario.