You may be confusing the words "capital" and "capitol."
A "capital" is a city. It's that contains the center of government for a country or a state. It is always the whole city.
A "capitol" is a building or group of buildings within the capital city, usually housing the main government offices.
Every country has a capital. Not every capital city has a capitol. Whether the national offices are called "the Capitol" is a matter of tradition and history. The name comes from ancient Rome where the center of government was on the Capitoline Hill. I don't know why, but capitol buildings are usually impressive monuments featuring a dome.
Within the United States many, but not every state capital has a building called the "Capitol." For example, I live in Massachusetts where the main state government building is simply called "the State House." So Massachusetts has a capital (the city of Boston) but not no Capitol.
Below, the capital of Cuba is the city of Havana. The capitol of Cuba is El Capitolio, a building within the city of Havana.