There are many situations in which “chart” and “diagram” can both be used. For example when we talk about a kind of visual representation of data or the relationships between things or concepts.
However I’ve seen “chart” used in multiple contexts where “diagram” doesn’t work. For example, in a hospital a doctor may look at a patient’s “chart”, the sheets of paper that explain the patient’s condition and treatment (something along those lines). And in jazz I’ve heard musicians referring to the sheets of paper representing the music as charts. You can’t use “diagram” in those contexts.
There may be other times when diagram is appropriate and chart isn’t, but none come to mind immediately.