You can say both but they have different meanings.
"carry on" means to continue, to conduct, to keep going with... in this case a conversation. "Boy, he can really carry on a conversation! We were there all night!" Sometimes you'll hear, "Don't let me carry on" or "Look at me carry on..." if they think they are talking too much. When you get distracted from a person talking, and they politely stop, you can tell them to continue with "carry on" though this probably sounds a bit formal.
"carry out" means to finish or at least the intention to finish. It doesn't mean to finish early or to finish now... just to finish the full ... in this case conversation... until its end. Like "carry on" this extends far beyond conversations to any task. "Carry out this assignment!" is a formal command to finish a task.
"We carried out a conversation" means we completed a full conversation. It sounds formal and official, like you are writing a report for the police.
"We carried on a conversation (late into the night)" means essentially the same thing but is more a reference to its continuance and length than that it was firmly finished and is a bit more formal than casual speech but would be natural in writing and in polite semi-formal speech.