Generally when you use "located in" it is referring to a specific location within a larger setting.
So, if you were in a library and you wanted to know which section you would find a certain book, you would ask, "what section is this book located in?".
"Germany is located in Europe."
"Located on" is usually used for directions or far away locations. "That house is located on main street."