Both are correct in real life conversation and no one will ever notice.
However, "graduated from xy university" is the grammatically correct form, while "graduated xy university" is incorrect.
In writing, especially where it's important that your grammar is accurate, always use "from".